JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
1
Defending Against Backdoor Attacks via Alignment Checking in Model-Contrastive Federated Learning
arXiv:2607.26933v1 [cs.CR] 29 Jul 2026
Hongliang Zhang, Zhongyuan Yu, Guijuan Wang, Tianqing He, Wenshuo Ma, Xiaosong Zhang, Jiguo Yu, Fellow, IEEE
Abstract—Federated Learning (FL) is vulnerable to backdoor attacks because of its distributed nature in edge computing scenarios. Existing defense methods show limited efficacy as they overlook the deviations among benign local updates caused by statistical heterogeneity and the stealthiness of backdoor attacks. To tackle these issues, we propose FedDAB, a two-phase method that combines local contrastive regularization with alignment checking, to defend against backdoor attacks. In the first phase, FedDAB introduces a novel model-contrastive term into the local objective to enhance direction and magnitude consistency among benign updates. In the second phase, FedDAB employs an alignment checking strategy to evaluate each local update in terms of overall-direction alignment and parameter-level alignment with historical information, excluding updates that exhibit abnormal alignment patterns from global aggregation. We theoretically prove FedDAB’s robustness with a convergence rate of O(1/T ). Extensive experiments show that FedDAB outperforms existing defense methods against backdoor attacks. Index Terms—Federated learning, backdoor attacks, statistical heterogeneity, edge computing.
I. INTRODUCTION Edge Computing (EC) pushes computation and storage resources from a centralized server closer to edge nodes, thereby enabling tasks to be processed locally. Building on this paradigm, federated learning allows multiple edge nodes to collaboratively train a high-performance global model without sharing their raw data [1]. However, the distributed nature of FL inherently makes it vulnerable to backdoor attacks in EC [2]. Such attacks maintain the global model’s accuracy on clean samples (i.e., main task) while inducing samples with specific features to be misclassified into an attackerchosen target class (i.e., backdoor task). This vulnerability stems from the server’s inability to directly monitor each node’s data or training process. Thus, an attacker may hijack This work was partially supported by NSF of China under Grants 62272256 and 62202250, and the Shandong Province Youth Innovation Team Project under Grant 2024KJH032. (Corresponding author: Jiguo Yu) H. Zhang, W. Ma, and G. Wang are with the Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences), Jinan, 250353, China, Email: [email protected], guijuan [email protected], [email protected]. Z. Yu is with the College of computer science and technology, China University of Petroleum, Qingdao, 266580, China, Email: [email protected]. T. He, and X. Zhang are with School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China, Email: [email protected], [email protected]. J. Yu is with School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China, and also with the Big Data Institute, Qilu University of Technology, Jinan, 250353, China, Email: [email protected]; [email protected].
a subset of nodes to poison their data or manipulate their training processes, causing them to submit malicious updates that compromise the optimization of the global model. To identify malicious updates, existing defense methods primarily focus on magnitude-based or direction-based strategies. Specifically, the methods in [3]–[6] employ Euclidean distance to quantify the magnitude deviations among local updates, whereas those in [7]–[10] adopt cosine similarity to capture their directional relationships. Nevertheless, these methods only capture the overall information of each local update, while overlooking the parameter-level details. Concretely, since each local update is high-dimensional, malicious nodes can launch stealthy poisoning attacks by perturbing a small number of parameters (e.g., flipping signs or amplifying magnitudes) without noticeably changing the overall direction or magnitude of their updates, thereby bypassing the aforementioned defense methods. Furthermore, since data in FL exhibit statistical heterogeneity, i.e., the data is Non-Independent and Identically Distributed (Non-IID), the effectiveness of these methods is further limited. This is because benign updates exhibit significant deviations under Non-IID data [11] [12] [13], thereby blurring the boundary between malicious and benign updates [8] [14] [15]. Since each update is inherently a vector characterized by both direction and magnitude, the deviations among updates are also reflected in both aspects. Thus, improving the consistency of benign updates in both direction and magnitude is crucial for defending against stealthy poisoning attacks in FL under Non-IID data. To improve update consistency, several studies [16]–[21] incorporate contrastive learning [22] into the local training objective as a regularization term, thereby reducing the directional deviation among local updates in FL. However, these studies overlook the magnitude deviation, resulting in residual inconsistency among benign updates. This limitation motivates us to pose the question: Is there a solution that uses contrastive learning to enhance the directional and magnitude consistency, while checking local updates from both overall-direction and parameter-level perspectives? In this paper, we propose FedDAB, a Federated learning defense method designed to Defend Against Backdoor attacks. The method integrates local contrastive regularization to enhance the consistency of local updates among benign nodes, and leverages alignment checking to exclude suspicious updates from nodes. Its novelties are as follows: (i) A novel model-contrastive term is incorporated into the local optimization objective based on contrastive learning. It regularizes benign local updates in both direction and magnitude, mitigating
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
the inconsistency among benign updates caused by Non-IID data. (ii) We examine local updates through both overalldirection checking and parameter-level checking to filter out updates exhibiting abnormal alignment patterns. (iii) We incorporate each node’s historical behaviors into the parameterlevel checking, instead of using only the information from the current round. Our contributions are three-fold. • To the best of our knowledge, FedDAB is the first FL defense method that combines local contrastive regularization with alignment checking, to defend against backdoor attacks under varying data distributions. • We prove that FedDAB satisfies a convergence rate of O(1/T ). Furthermore, we provide a theoretical analysis of the robustness and propagation error of FedDAB. • We conduct extensive experiments on multiple datasets under Non-IID settings to evaluate the performance of FedDAB against various backdoor attacks. Compared to existing defense methods, FedDAB exhibits superior effectiveness and robustness. II. R ELATED W ORKS Federated Learning: A typical FL system consists of a central server and K nodes indexed by k ∈ K, each of which holds its private training dataset Dk . The nodes can cooperatively learn a global model W ∈ Rd for the main task. The global optimization problem of FL can be formulated as: P min F (W ) = k∈K ak Fk (W, Dk ), W
P
where ak := |Dk |/ k∈K |Dk | is the data proportion of node k, and Fk (·) is the local objective function. However, FL is vulnerable to backdoor attacks, where an attacker can poison the training data by embedding backdoor triggers into training samples with specific features. To improve the attack stealthiness, the works in [23] [24] propose the variants of backdoor attacks that align malicious updates with historical global updates, making them less distinguishable from benign updates and enabling them to evade detection. These stealthy backdoor attacks further increase the difficulty of identifying malicious updates in FL. Defending against Backdoor Attacks: Regardless of whether existing defense works are magnitude-based or directionbased, they generally mitigate the adverse impact of malicious nodes by assigning lower weights to suspicious updates or filtering them out before aggregation. Specifically, the works in [7] [9] [25] [26] retain all local updates but assign lower aggregation weights to those identified as malicious, thereby limiting their influence on the global model. For example, Yang et al. observe that updates from malicious nodes exhibit higher pairwise similarity than those from benign nodes [9]. Leveraging this property, they use adaptive clustering to group highly similar updates, aggregates updates within each cluster, and applies projection-based dimensionality reduction to extract a plausible clean update, thereby reducing the contribution of malicious updates. In contrast, the works in [3]–[5], [10] detect and remove malicious updates before updating the global model. Notably, the aforementioned rely on evaluating the deviations among updates in terms of magnitude
2
or direction. However, FL inevitably suffers from statistical heterogeneity, i.e., Non-IID data, which leads to significant deviations in both direction and magnitude among benign updates [27]. Thus, these defense works [3]–[5], [7], [9], [10], [25], [26] struggle to distinguish whether the deviations stem from Non-IID data or backdoor attacks, thereby undermining their effectiveness. Moreover, since backdoor attacks aim to maintain high accuracy on the main task while maximizing the success rate of the backdoor task, malicious updates are crafted to appear plausible in both magnitude and direction. Thus, the works in [7]–[10] provide limited robustness because they capture only overall-direction information of local updates while overlooking parameter-level details. To address these limitations, our FedDAB incorporates both overall-direction and parameter-level information into the aggregation process to detect malicious updates. Contrastive Learning in FL: Contrastive learning is a selfsupervised representation learning that learns effective data representations by contrasting positive and negative pairs [22]. Its core idea is to reduce the distance between the representations of different augmented views of the same sample (i.e., positive pairs), and increase the distance between the representations of augmented views of different samples (i.e., negative pairs). To improve FL performance under Non-IID data, Yang et al. first introduce a model-contrastive term into the FL objective to regularize local updates via contrastive learning [16]. Subsequently, several variants based on the work in [16] have been proposed [28]–[30]. These works enhance the directional consistency among local updates while neglecting the magnitude deviation, leaving residual inconsistency among benign nodes. To this end, we design a novel modelcontrastive term within the local objective to enhance both directional and magnitude consistency among benign updates, thereby making malicious updates easier to detect. III. T HE D ETAILED D ESIGN OF F ED DAB Since FedDAB combines local regularization with alignment checking, the design of both phases is presented. A. Local Contrastive Regularization Due to the statistical heterogeneity of data, benign nodes optimize their local models toward their respective optima [11] [12], resulting in deviations among benign local updates in both direction and magnitude. These deviations cause defense methods [3]–[5], [9], [10] struggle to distinguish malicious updates from benign ones. To eliminate the deviations caused by Non-IID data, the works in [16], [20], [28], [30] incorporate contrastive learning [22] into the local objective to improve the directional consistency among local updates. However, they overlook the magnitude deviation among updates. Thus, we introduce a magnitude regularization into the model-contrastive term, jointly aligning the direction and magnitude among benign updates. In the following, we present the network architecture and local objective. Network Architecture: The network consists of two components: an encoder and a classifier. The encoder extracts a representation vector from an input, and the classifier maps this
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
representation vector to a probability distribution over classes. For simplicity, for any model W , we use ExtW (·) to denote the encoder output produced by model W . Local Objective: In FedDAB, the local optimization function consists of a classification term LCla (·) and a modelcontrastive term LCon (·). The former is used to compute the main task loss, while the latter is used to compute the contrastive loss between the feature representations learned by the local and global models. Specifically, in the t-th round, for every input x, given the global model W t , the current local model wkt , and the previous local model wkt−1 , we extract the global representation vector zgt := ExtW t (x), the current local representation zkt := Extwkt (x), and the previous local representation zkt−1 := Extwkt−1 (x). Since the global model provides better representations than local models, we aim to decrease the distance between zkt and zgt , and increase the distance between zkt and zkt−1 . Thus, similar to works [16] [20], the model-contrastive term LCon (·) is defined as: LCon (W t , wkt−1 , wkt , x) = t sim(zgt ,zk )
− log exp(
t−1 t ∥zk −zk ∥1
3
Algorithm 1: Alignment Checking Input: {∆tk }k∈K , r, H, λDSS , λSAS ▷ {∆tk }k∈K is the local update set, r is the selection ratio, H is the buffer length, λDSS and λSAS are thresholds. Output: W t+1 t 1 Initialize retained node set K ← ∅; 2 // Update Evaluation // t t 3 {αk }k∈K ← {Direction Check(∆k )}k∈K ; P P H t−h t t ; 4 P ← Sgn k∈K Sgn ∆k + h=1 P PH t−h t t p ); 5 pk ← Sgn(Sgn(∆k ) + h=1 k t t t t 6 {βk }k∈K ← {Param Check(∆k , P , pk , r)}k∈K ; 7 for each node k ∈ K do 8 mtk,1 ← MZ score({αkt }k∈K ); 9 mtk,2 ← MZ score({βkt }k∈K ); 10 if mtk,1 <λDSS and mtk,2 <λSAS then 11 Kt ← Kt ∪ {k}; // Update Aggregation // c ← Med({∥∆tk ∥2 } k∈Kt ); n o P t t c 1 ; 14 ∆g ← |Kt | k∈Kt ∆k · min 1, ∥∆t ∥ 12 13
15
W t+1 ← W t − ηg ∆tg ;
k 2
) exp( ) exp( q1 q2 − log , t−1 t t−1 t) t∥ t∥ sim(zgt ,zk ∥zgt −zk sim(zk ,zk ) ∥zk −zk 1 1 ) + exp( ) ) + exp( ) exp( q1 q1 q2 q2
where sim(·) is the cosine similarity function, ∥ · ∥1 is the ℓ1 norm, and q1 and q2 are used to rescale the distances. In the above formula, the first term, termed the direction-alignment term, encourages the current local representation to match the global representation in direction, rather than the previous local representation. The second term, termed the magnitudealignment term, enforces the current local representation to be closer to the global representation and pushes it away from the previous local representation in magnitude. Combining the terms LCla (·) and LCon (·), the loss for an input sample (x, y) is given by: L = LCla (wkt , (x, y)) + µLCon (W t , wkt−1 , wkt , x), where µ is the trade-off parameter. Thus, the local objective of node k is minwk E(x,y)∼Dk [L], which minimizes the classification loss while aligning the current local representation with the global representation. After completing E local SGD iterations, benign node k obtains wkt+1 and uploads the local update ∆tk := wkt+1 − W t to the server. In contrast, malicious nodes violate the predefined training rules and submit manipulated updates to the server. B. Alignment Checking The alignment checking phase consists of two key modules: Update Evaluation and Update Aggregation, and its core steps are shown in Algorithm 1. Concretely, Update Evaluation module is used to evaluate and filter the suspicious updates submitted by nodes. Update Aggregation module is applied to compute the global model. Each module is presented in below. 1) Update Evaluation: Existing defense methods mainly inspect local updates using magnitude-based metrics (e.g., Euclidean/Manhattan distance) or direction-based metrics (e.g., cosine similarity). As the global model tends to converge, the magnitude of local updates gradually decreases, making the magnitude gap between benign and malicious updates less pronounced. Thus, magnitude-based detection is unreliable for
identifying malicious updates. To this end, we evaluate the alignment patterns of local updates via overall-direction checking and parameter-level checking, and then filter out updates with abnormal patterns. Therefore, the Update Evaluation module consists of Overall-Direction Checking, ParameterLevel Checking, and Anomaly Score Detection. Overall-Direction Checking: The local objective of FedDAB is designed to enhance the consistency of local updates among benign nodes in both direction and magnitude. However, in typical backdoor attacks [31] [32], malicious nodes optimize both the main task and the backdoor task, which may cause their updates to deviate in direction from those of benign nodes [33]. Thus, the overall-direction information of each local update is informative for identifying malicious updates. Motivated by this insight, the Overall-Direction Checking step evaluates the Direction Similarity Score (DSS) of each node based on its update direction. Formally, the DSS value αkt of local update ∆tk is defined as: P t αkt := (1/(K − 1)) j∈K,j̸=k Skj , t where Skj := ⟨∆tk , ∆tj ⟩/(∥∆tk ∥ · ∥∆tj ∥). The DSS value αkt captures the average cosine similarity between ∆tk and the other local updates, reflecting its overall-direction alignment. Benign nodes typically have consistent DSS values, whereas malicious nodes exhibit DSS values that are markedly offset from those of benign nodes. Notably, since DSS value is computed from update directions rather than magnitudes, its variability during federated training continues to be utilized for identifying malicious updates. Parameter-Level Checking: In stealthy backdoor attacks [23] [24], malicious nodes can manipulate the signs and magnitudes of certain parameters within their local updates without changing the update direction. Although the DSS value captures the directional information, its detection ability is weakened under such attacks. This motivates us to examine parameter-wise signs and magnitudes for fine-grained
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
4
evaluation. Notably, the works in [26] [34] [35] leverage parameter-sign information to identify malicious updates. Although they utilize the sign information, parameters with very small magnitudes (i.e., unimportant parameters) may weaken their defense effectiveness. In addition, since benign nodes honestly follow the FL protocol in each round, their historical behaviors can provide reliable evidence for detecting malicious updates. However, the above defense methods only utilize local updates from the current round, ignoring the node’s historical behaviors. Thus, the checking step evaluates local updates by combining the signs and magnitudes of important parameters from both the current and historical updates of each node. To effectively resist stealthy backdoor attacks, we examine sign alignment between each node’s update and the global sign vector at the coordinates with large absolute magnitudes. The global sign vector represents the dominant sign at each coordinate across all local updates. Thus, the global sign vector is defined as: P P t = Sgn( k∈K Sgn(∆tk )),
that counts non-zero elements. A larger β means stronger coordinate-wise sign alignment between Y1 and Y2 .
where Sgn(·) denotes the element-wise sign operator that returns −1, 0, or +1 for negative, zero, and positive elements, respectively. Since benign nodes follow the FL protocol across all rounds, incorporating historical information can better reflect their stable behavior. Thus, the server maintains a global sign buffer Sg that stores the most recent H global sign vectors for computing P t , which is reformulated as: P PH P t = Sgn( k∈K Sgn(∆tk ) + h=1 P t−h ),
where ⊙ is the Hadamard product, ∥ · ∥0 is the ℓ0 -norm, (ptk − P t ) captures the coordinate-wise discrepancy between the local and the global sign vectors, and ⌈r × d⌉ (with ⌈·⌉ being the ceiling operator) denotes the number of selected coordinates. After computing each node’s SAS value, the server stores the global sign vector P t in Sg and each node’s local sign vector ptk in Sl . Once the buffer reaches its capacity, the oldest sign vector is replaced by the newly computed one. Anomaly Score Detection: After Overall Direction Checking and Parameter-Level Checking, a Median-based Z-score is introduced to remove the local updates with abnormal DSS and SAS values. The Anomaly Score Detection step is detailed below. Since malicious nodes perturb their local updates to launch backdoor tasks, their DSS and SAS values exhibit noticeable offsets from those of benign nodes. Assuming that most nodes are benign, nodes whose DSS and SAS values lie close to the medians of {αkt }k∈K and {βkt }k∈K are regarded as benign, while those that are far from the medians are considered suspicious. Thus, the median offsets of node k are defined as mtk,1 := |αkt − Med({αkt }k∈K )|, and mtk,2 := |βkt − Med({βkt }k∈K )|, where Med(·) is the median operator. We use the thresholds λDSS and λSAS to identify nodes whose median offsets exceed these bounds. However, since the ranges of median offsets vary substantially across rounds, fixed thresholds λDSS and λSAS may cause all nodes’ offsets to fall within these bounds, thereby failing to filter out anomalous updates. To address this, inspired by the standardization method Z-score, we introduce a robust variant named median-based Z-score MZ score(·) to recalculate the median offsets, adapting to changes in the DSS and SAS ranges during federated training.
where H denotes the length of buffer Sg . Based on the global sign vector P t , the sign alignment degree of each node can be evaluated. To reduce interference from unimportant parameters, we evaluate the sign alignment on the top-r% of coordinates ranked by absolute value within each local update, where r denotes the selection ratio. To identify these coordinates, we use the indicator Topr (·) to mark them, which is defined as follows. Definition 1. (Indicator Topr (·)). Given a vector Y ∈ Rd , and a selection ratio r ∈ [0, 1], we define the function Topr (·) : Rd → {0, 1}d as follows: ( 1, if |Yj | ∈ {|Yπ(1) |, . . . , |Yπ(⌈r×d⌉) |}, [Topr (Y )]j = 0, otherwise, where [Topr (Y )]j denotes the j-th element of the indicator vector Topr (Y ), and π(·) is a permutation of the index set {1, 2, · · · , d} such that: |Yπ(1) | ≥ |Yπ(2) | ≥ · · · ≥ |Yπ(d) |, where π(d) is the index of the element in vector Y that ranks the d-th largest in absolute value. The indicator Topr (∆tk ) outputs a binary vector of the same size as the local update ∆tk , where each element is either 1 or 0. The coordinates with a value 1 indicate the top-r% coordinates in ∆tk with the largest absolute magnitudes. Using these coordinates, the checking step measures each node’s Sign Alignment Score (SAS), which is defined as follows. Definition 2. (Sign Alignment Score). For vectors Y1 , Y2 ∈ Rd , the sign alignment score value β of Y1 and Y2 is defined 2 )∥0 as: β = 1 − ∥ Sgn(Y1 )−Sgn(Y , where ∥ · ∥0 denotes the ℓ0 -norm d
According to Definition 2, each node’s SAS value βkt is computed at the server by comparing its local sign vector ptk with the global sign vector P t . Hence, the server constructs ptk for each node before computing its SAS value. Since P t incorporates historical information, each node’s local sign vector likewise incorporates its own historical information. Thus, the server maintains a local sign buffer Sl that stores the most recent H local sign vectors for each node, based on which ptk is defined as: PH ptk = Sgn(Sgn(∆tk ) + h=1 pt−h k ), where H denotes the length of buffer Sl , set equal to that of buffer Sg . To focus on the coordinates with the largest absolute magnitudes, we use the Topr (·) indicator to select the top-r% coordinates of each local update. Thus, the SAS value βkt for node k is computed as follows: βkt = 1 − ∥(ptk − P t ) ⊙ Topr (∆tk )∥0 /⌈r × d⌉,
Definition 3. (Median-based Z-score MZ score(·)). Let X be a set of scalars with median Med(X) and standard deviation Std(X). For any s ∈ X, its offset is defined by: MZ score(s, X) = (|s − Med(X)|)/ Std(X), which measures the standardized distance of scalar s from Med(X). By applying MZ score(·) function to the sets {αkt }k∈K and
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
{βkt }k∈K , we recompute each node’s median offsets mtk,1 and mtk,2 . Based on these offsets, nodes whose offsets exceed the thresholds are filtered out. Specifically, if mtk,1 ≥ λDSS or mtk,2 ≥ λSAS , its update ∆tk is identified as suspicious, and node
k is excluded from aggregation in the t-th round. Thus, the retained node set Kt is defined as: Kt = {k ∈ K|(mtk,1 <λDSS ) ∧ (mtk,2 <λSAS )}. Notably, λDSS and λSAS are fixed thresholds. The server aggregates updates from Kt to compute the global model. 2) Update Aggregation: After filtering, the remaining nodes are treated as benign, and their updates are used to calculate the global update ∆tg . However, the Update Evaluation module is unable to detect malicious updates with abnormally large magnitudes. If update magnitudes are not considered, malicious updates may bypass Update Evaluation, enabling malicious nodes to compromise the global model via amplifying update magnitudes. To this end, FedDAB employs the median ℓ2 -norm of the retained updates in Kt as the clipping threshold, rescaling these updates. Formally, the aggregation process is expressed as: n o P , ∆tg = |K1t | k∈Kt ∆tk · min 1, ∥∆ct ∥2
5
Beyond the above assumptions, to facilitate the theoretical analysis of FedDAB, we define the robustness concept, namely κ-secure aggregation, as presented in Definition 4. The definition is commonly adopted in the analysis of FL defense methods [38] [39]. Definition 4. (κ-Secure Aggregation) Given the set of model updates {∆k }k∈K and benign node set KB ⊂ K, the aggregation rule G(·): Rd×K → Rd is called κ-secure aggregation when its aggregated global update ∆g := G({∆k }P k∈K ) satisfies ∥∆g − ∆KB ∥22 ≤ κ, where ∆KB := (1/|KB |) k∈KB ∆k , K is the number of nodes, and κ ≥ 0 denotes the robustness coefficient of G(·). This definition quantifies a constant bound, denoted by κ, that constrains the deviation between the global update output by FedDAB and the average of all benign updates. A smaller κ indicates that the aggregation function G(·) produces the global update closer to the average of benign updates, which represents the optimal output. In other words, when G(·) identifies and removes all malicious updates while preserving all benign ones, we have κ = 0, indicating that FedDAB achieves the highest level of κ-secure aggregation.
k
where
c
denotes
the clipping bound, defined as c := Med({∥∆tk ∥2 }k∈Kt ). The global model W t is updated as W t+1 := W t − ηg ∆tg , where ηg is the global learning rate. IV. ROBUSTNESS A NALYSIS OF F ED DAB This section provides the theoretical analysis of FedDAB. Before presenting the theoretical results, we state the following assumptions. Note that Assumption 1 and 2 have been widely adopted in the theoretical analysis of FL [34] [36]. Assumption 3 provides a standard measure for the deviations caused by data heterogeneity among nodes [37] [38]. Such heterogeneity complicates the problem of FL with backdoor attacks, as it may cause the server to confuse malicious updates with benign ones. The proofs of all theorems are provided in Appendix. Assumption 1. (L1 -smoothness). Let KB be the set of benign nodes. For each benign node k ∈ KB , its objective function Fk (·) is assumed to be L1 -Lipschitz smooth with L1 >0. Formally, for any Y1 , Y2 ∈ Rd , the following inequality holds: ∥∇Fk (Y1 ) − ∇Fk (Y2 )∥2 ≤ L1 ∥Y1 − Y2 ∥2 . Moreover, this condition implies the following inequality: Fk (Y1 ) − Fk (Y2 ) ≤ ∇Fk (Y1 )⊤ (Y2 − Y1 ) + L21 ∥Y1 − Y2 ∥22 . Assumption 2. (Bounded Gradient and Variance). For each benign node k ∈ KB , the stochastic gradient gk is an unbiased and bounded-variance estimator of the gradient. Specifically, for any Y ∈ Rd , it holds that E[gk ] = ∇Fk (Y ), ∥∇Fk (Y )∥22 ≤ Z 2 , and E∥gk −∇Fk (Y )∥22 ≤ ξk2 for all benign nodes, where the expectation is taken over local mini-batches. For convenience, the average variance bound is denoted as P ξ¯ := |K1B | k∈KB ξk2 . Assumption 3. (Bounded Heterogeneity). For a constant P L2 ≥0, it holds that |K1B | k∈KB ∥∇Fk (Y ) − ∇FKB (Y )∥22 ≤ L2 for all Y ∈Rd , where the mean gradient over the benign node set P is defined as ∇FKB (Y ) := |K1B | k∈KB ∇Fk (Y ).
Theorem 1 (Bounded Local Deviation). In the t-th round, let P t k∈KB ∆k be the average of benign updates. If 1 local learning rate ηl and local iterations E satisfy ηl ≤ 2E , under Assumption 2 and 3, the local deviation is bounded as: P 2 1 E ∆t − ∆ t ≤ 2ξ¯ + L2 . ∆tKB := (1/|KB |)
|KB |
k∈KB
k
KB 2
Remark 1. When the condition ηl ≤ 1/(2E) holds, mitigating the deviation L2 induced by data heterogeneity can constrain the local deviation (2ξ¯ + L2 ). Thus, in FedDAB, the modelcontrastive term is incorporated into the local objective to align both the magnitude and direction of benign updates, thereby reducing the deviation L2 . Theorem 2 (κ-robustness of FedDAB). Let Assumption 2 and Assumption 3 hold. Suppose that the total number of nodes is K > 1, and that there are Km malicious nodes satisfying 0 ≤ Km < K/(3 + ϵ) for a positive constant ϵ. In the t-th 1 round, if the relations ηl ≤ 2E and |Kt | ≥ K − 2Km holds, FedDAB achieves κ-secure aggregation, where κ is defined by: κ = (1 + Km / (K − 2Km )) (2/ϵ + 1) 2ξ¯ + L2 + 8c2 = O (1 + Km / (K − 2Km )) . Remark 2. The relation emphasizes the importance of preserving a sufficient number of remaining nodes. Moreover, the clipping bound c can mitigate the deviation caused by overly large-magnitude local updates. In addition, [34] [40] have demonstrated the effectiveness of the clipping in mitigating the impact of malicious updates. We argue that FedDAB achieves comparable robustness to defense methods, i.e., RFA [25] (O(1 + Km /(K − 2Km ))2 ) and Krum [3] (O(1 + Km /(K − 2Km ))). The results for RFA and Krum are taken from [38]. Since the coefficient κ is bounded by a constant, we incorporate these results into our discussion without losing generality. Theorem 3 (Propagation Error Bound). Let Assumption 1 to Assumption 3 hold, and let the local learning rate satisfying
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
6
ηl ≤ 1/(2E). Let W T denote the global model trained over all nodes in the T -th round, and W T,∗ denote the clean global model trained only on benign node set KB . Under backdoor attacks, the propagation error of FedDAB is bounded by: 2 ¯ + 1/τ )T − 1), W T − W T,∗ ≤ 4τ ηg (κ + 2ξ)((1
classified into their true labels. An effective defense method should achieve high TA and RR, while maintaining a low ASR. 3) Attack Settings: The attacker is assumed to control a proportion v (default 30%) of all nodes as malicious nodes. These malicious nodes poison their local training data by embedding a “plus” trigger into clean images. Typical backdoor where κ is given in Theorem 2, and τ := 1/(1 + 3ηg2 L21 ). attacks include BadNet [31] and Distributed Backdoor Attack Remark 3. The error bound reflects the deviation between the (DBA) [32]. To improve attack stealthiness, more advanced clean model W T,∗ and the actual model W T . When the global attacks such as Neurotoxin attacks [24] and Projected Gradient learning rate ηg is fixed, this error bound increases as the Descent (PGD)-based attacks [23] have been further proposed. number of rounds T grows. To mitigate the propagation error, 1) BadNet Attacks. Malicious nodes add uniform “plus” the global learning rate is scheduled to decay over rounds, triggers to clean samples to construct poisoned data. i.e., ηgt gradually decreases as t increases. Thus, FedDAB can 2) DBA Attacks. The “plus” trigger is decomposed into four be integrated with decaying global learning rate schedule [41] sub-triggers, and each malicious node uses only one of to achieve stable convergence. these sub-triggers. 3) Neurotoxin Attacks. Malicious nodes project their local Theorem 4 (Convergence Rate). Suppose Assumptions 1 and updates onto the coordinate set defined by the top 75% 2 hold. Let W ⋆ denote the global optimum. After T rounds of largest-magnitude parameters of the previous global FedDAB, the following bound holds: model, and sets the update in other coordinates to zero. PT 2|Kt |L1 H1 4|Kt | 1 t 2 , 4) PGD Attacks. The malicious local update is projected t=1 E ∥∇F (W )∥2 ≤ T ηg Φ + 4H2 − 4H1 + T ηg 2 onto a sphere to form the tampered update, where the where H1 = (ηl EZ|Kt |)2 , H2 = 2Z 2 12 + |Kt | + 4Z 2 |Kt |2 + 4H1 , sphere’s radius is set to the ℓ2 -norm of the previous0 ⋆ Φ = F (W ) − F (W ), and ηl is the local learning rate. round global model. Remark 4. The bound indicates that the gradient norm of 4) Evaluated Defense Methods: We compare our FedDAB FedDAB decreases as the number of rounds T increases. If all with the non-robust FedAvg [45] and eight defense methods: other parameters are fixed with respect to T , the convergence MKrum [3], Foolsgold [7], RLR [26], RFA [25], MMetric [4], rate of FedDAB is O(1/T ). RoseAgg [9], AlignIns [37], and EndPCA [46]. V. E XPERIMENTAL D ETAILS A. Experimental Settings 1) Hyper-parameter Configuration: Similar to existing FL works [6] [9] [42], we evaluate the defense methods on FMNIST [43], CIFAR10 [44], and CIFAR100 [44]. We simulate a FL system with 20 edge nodes, where all nodes participate in each training round. Notably, we further evaluate the scalability of FedDAB under larger-scale node settings in Section V-B4. Unless otherwise specified, we use the following default settings. The thresholds λDSS and λSAS are set to 1.0. The trade-off parameter µ is set to 0.5, and the buffer length H for both Sl and Sg is set to 3. The selection ratio r is set as 30%, meaning that the Top-30% of model parameters are used for the parameter-level checking. The global learning rate ηg is set to 1, the local learning rate ηl to 0.05, the number of local iterations E to 2, and the temperature parameters q1 and q2 to 1. The number of rounds T is set to 50 for FMNIST, 100 for CIFAR10, and 120 for CIFAR100. For the model architectures, we use ResNet9 for FMNIST and CIFAR10, and VGG9 for CIFAR100. To simulate the Non-IID setting, we partition each dataset among nodes using a Dirichlet distribution Dir(Ψ), where a larger concentration parameter Ψ indicates a more uniform class allocation across nodes. 2) Evaluation Metrics: We use the following metrics to evaluate the defense methods. Test Accuracy (TA) is the proportion of clean samples correctly classified by the global model. Attack Success Rate (ASR) is the proportion of triggered samples misclassified into the target label. Robustness Rate (RR) is the proportion of triggered samples correctly
B. Experimental Results 1) Comparison with Existing Methods: Table I reports the performance of FedDAB and existing defense methods against backdoor attacks under the Dir(1.0) setting. The ASR and RR values are recorded at the round with the highest Top-1 test accuracy. Overall, FedDAB demonstrates superior defense performance under four backdoor attacks. Specifically, on FMNIST, Foolsgold achieves the highest RR (67.36%), but its ASR remains higher than that of FedDAB. This is because Foolsgold cannot filter out malicious updates and can only reduce their weights in global aggregation. Consequently, malicious updates are incorporated into the aggregation process, resulting in a relatively high ASR. In addition, RoseAgg exhibits the highest ASR. This is because RoseAgg evaluates local updates based on their directions, making its assessment susceptible to deviations among benign update directions induced by Non-IID data, preventing it from assigning low aggregation weights to malicious updates. The defense effectiveness of the other methods is also impaired by statistical heterogeneity, making them less robust than FedDAB against these backdoor attacks. More specifically, on CIFAR10, FedDAB reduces the ASR by 1.23 % and improves the RR by 1.38 % over the second-best method. Similarly, on CIFAR100, FedDAB outperforms all existing methods in terms of both ASR and RR. The performance advantages of FedDAB can be attributed to two key designs. (i) The model-contrastive term improves the consistency of benign updates in both direction and magnitude, enabling the alignment checking to distinguish benign updates from malicious
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
7
TABLE I T HE ASR AND RR RESULTS OF BASELINES AND F ED DAB ON THE Dir(1.0) SETTING OF FMNIST, CIFAR10, AND CIFAR100. T HE BEST AND SECOND - BEST RESULTS ARE SHOWN IN BOLD AND UNDERLINED , RESPECTIVELY. BadNet
CIFAR100 (VGG)
CIFAR10 (ResNet9)
FMNIST (ResNet9)
Dataset Methods (Model)
DBA
ASR(%)↓
RR(%)↑
Neurotoxin
ASR(%)↓
RR(%)↑
ASR(%)↓
PGD RR(%)↑
ASR(%)↓
RR(%)↑
v=0.30 v=0.40 v=0.50 v=0.30 v=0.40 v=0.50 v=0.30 v=0.40 v=0.50 v=0.30 v=0.40 v=0.50 v=0.30 v=0.40 v=0.50 v=0.30 v=0.40 v=0.50 v=0.30 v=0.40 v=0.50 v=0.30 v=0.40 v=0.50
Avg. Avg. ASR↓ RR↑
FedAvg
92.38
95.28
98.32
6.84
4.26
1.56
74.14
22.52
61.06
21.74
52.67
29.46
77.07
83.01
88.30
20.22
15.39
10.71
92.38
95.28
98.32
6.84
4.26
1.56
81.50 14.62
MKrum Foolsgold RLR RFA MMetric RoseAgg AlignIns EndPCA FedDAB
2.82 2.90 64.91 36.41 48.13 91.43 2.68 2.03 2.91
5.46 3.18 51.76 41.07 82.20 95.47 4.03 5.64 2.56
10.50 2.96 30.50 45.60 84.91 96.31 8.44 8.24 4.17
71.49 65.79 22.56 49.77 40.67 7.62 68.11 65.42 71.01
56.20 67.86 21.36 47.53 14.77 4.00 68.50 62.40 69.02
45.95 65.49 14.80 40.63 13.90 2.14 50.84 57.63 63.68
2.24 6.44 54.13 5.31 19.62 69.73 2.83 3.88 3.44
3.49 8.25 60.67 14.83 11.40 39.86 3.79 5.43 2.91
12.54 8.42 62.96 34.51 91.44 50.49 6.73 7.75 3.22
68.22 63.32 20.06 60.19 50.34 25.01 67.21 63.67 66.53
59.38 61.81 18.53 52.51 45.43 40.48 68.12 61.09 66.79
40.90 60.41 15.82 37.87 7.67 29.14 47.69 58.33 64.13
2.82 2.12 0.73 3.73 91.44 74.61 2.78 2.16 3.58
2.78 3.52 21.54 3.13 71.99 81.44 3.53 4.12 2.86
12.21 6.54 99.87 9.25 84.13 89.20 7.49 9.87 3.44
71.49 68.46 72.99 64.36 7.67 22.30 66.83 66.49 66.22
57.04 62.35 10.41 71.74 24.19 16.63 68.50 63.34 68.58
46.51 76.42 0.11 59.51 14.33 7.06 53.18 57.51 68.20
2.80 4.53 64.91 36.41 48.13 41.80 3.23 3.92 2.88
5.46 4.63 45.67 41.07 82.20 54.64 4.52 6.89 2.46
13.08 6.26 30.50 48.51 84.91 59.54 7.98 7.79 3.62
71.53 66.34 22.56 49.77 40.67 56.94 66.11 65.18 66.88
56.20 72.63 21.36 47.53 14.77 42.67 67.50 62.93 62.43
37.90 77.47 14.80 37.49 13.90 36.50 54.04 59.89 58.83
6.35 56.89 4.97 67.36 49.01 21.28 26.65 51.57 66.70 24.02 70.37 24.20 4.83 62.21 5.64 61.98 3.17 66.02
FedAvg
99.96
99.94
100
0.02
0.02
0.00
91.58
78.99
98.31
7.18
19.52
1.62
99.82
99.67
99.96
0.17
0.27
0.04
99.96
99.94
100
0.02
0.02
0.00
97.34 2.40
MKrum Foolsgold RLR RFA MMetric RoseAgg AlignIns EndPCA FedDAB
99.77 3.70 80.64 73.73 93.70 97.31 2.88 2.73 3.22
99.98 3.99 85.17 100 93.49 99.36 3.20 4.21 2.97
100 5.78 87.65 100 99.63 100 5.71 6.26 3.02
0.21 76.98 17.63 7.51 5.22 0.07 79.99 78.96 78.91
0.01 75.62 13.20 0.00 4.84 0.01 80.13 76.20 78.93
0.00 71.36 10.18 0.00 0.32 0.00 72.19 73.54 78.26
85.73 3.66 11.54 84.22 53.90 92.54 3.17 3.76 2.99
97.87 5.96 15.61 100 23.61 94.27 3.07 3.31 3.09
100 63.21 30.51 100 67.23 98.65 6.97 5.43 3.87
10.53 77.06 29.98 6.27 35.79 6.51 79.67 77.78 79.59
1.44 75.07 22.58 0.00 53.99 4.36 79.74 77.16 78.83
0.00 71.21 14.04 0.00 27.59 0.98 75.64 75.90 77.08
98.73 3.15 8.54 93.60 99.37 72.64 3.11 4.64 2.82
99.99 4.25 13.00 100 99.96 78.40 2.99 5.31 3.02
100 6.25 14.15 100 99.87 85.83 6.02 7.19 3.73
0.89 77.27 30.60 1.39 0.36 25.71 79.94 76.82 79.67
0.01 74.83 24.94 0.00 0.04 20.96 79.87 75.03 78.97
0.00 71.22 16.49 0.00 0.13 13.25 76.57 74.74 78.08
99.84 3.47 76.35 78.41 91.73 84.69 3.32 3.85 3.10
99.98 4.90 88.80 100 89.74 90.10 4.12 5.47 2.93
100 6.61 95.34 100 98.42 91.63 8.69 8.64 3.66
0.17 78.87 22.30 5.09 7.52 14.02 77.64 78.02 79.07
0.00 73.65 9.35 0.00 9.53 8.53 76.17 76.86 79.19
0.00 70.13 2.09 0.00 0.57 8.02 70.49 75.15 77.90
98.49 1.10 9.57 74.43 50.60 17.78 94.16 1.68 84.22 12.15 90.45 8.53 4.43 77.33 5.06 76.34 3.20 78.71
FedAvg
81.94
96.64
98.14
6.51
2.47
0.54
64.63
69.23
71.32
16.90
15.27
14.80
62.75
72.73
75.61
16.83
10.55
8.18
79.05
96.76
98.15
7.56
2.29
0.37
80.57 8.52
MKrum Foolsgold RLR RFA MMetric RoseAgg AlignIns EndPCA FedDAB
66.48 3.75 63.37 73.73 92.14 95.48 1.45 1.74 0.67
94.47 1.96 100 85.50 99.98 99.64 2.98 3.15 0.26
98.74 4.25 100 87.64 100 100 100 9.53 0.42
14.64 45.65 0.49 7.51 5.22 0.67 45.45 45.10 47.69
3.48 43.41 0.00 5.36 0.00 0.30 44.03 42.48 46.33
0.27 40.62 0.00 5.17 0.00 0.00 0.00 39.29 44.61
54.66 5.25 11.54 67.31 99.63 92.61 1.35 1.95 0.71
93.65 2.46 32.67 84.22 99.83 95.37 1.40 1.82 0.32
97.40 8.15 40.74 90.43 100 99.70 1.87 7.27 0.41
21.33 45.03 32.98 6.27 0.32 1.32 47.44 45.56 47.91
4.29 43.87 11.58 2.61 0.25 0.77 44.97 44.70 46.24
0.57 35.97 4.04 2.29 0.00 0.37 43.20 41.19 44.64
86.46 0.72 8.54 75.31 24.53 74.66 1.13 0.82 0.70
94.47 0.94 60.50 84.42 43.80 86.16 1.28 2.86 0.29
96.16 1.08 81.10 87.06 46.80 99.54 1.59 8.59 0.43
4.79 46.11 30.60 1.63 16.44 2.74 45.26 46.42 47.91
3.48 43.61 14.94 1.27 9.63 1.60 44.29 43.78 46.38
2.90 42.90 6.49 0.95 1.54 0.13 42.61 38.50 44.63
82.62 35.00 21.95 52.81 64.99 79.73 0.81 2.77 0.73
95.00 1.15 29.63 89.92 70.64 85.80 1.25 3.69 0.23
98.37 10.83 36.00 93.15 72.97 96.74 1.41 6.08 0.42
9.66 9.53 22.61 5.09 10.33 2.87 45.80 44.01 48.15
4.07 44.61 10.35 0.62 5.47 1.65 44.66 43.85 46.33
1.80 30.66 5.09 0.31 3.99 0.96 42.30 40.67 44.61
88.20 5.93 6.29 39.33 48.83 11.59 80.95 3.25 76.27 4.43 92.11 1.12 9.71 40.83 4.18 42.96 0.46 46.28
TABLE II T HE ASR (%) RESULTS OF F ED DAB ON VARIOUS N ON -IID SETTINGS OF FMNIST, CIFAR10, AND CIFAR100.
TABLE III P ERFORMANCE OF F ED DAB WITH EXTREME N ON -IID S ETTINGS . Dataset
BadNet DBA Neurotoxin PGD Avg. Dataset Dir(Ψ) v=0.30v=0.40v=0.50v=0.30v=0.40v=0.50v=0.30v=0.40v=0.50v=0.30v=0.40v=0.50 ASR↓ (Model) Ψ = 0.8 3.91 6.97 3.79 4.77 4.23 3.81 3.59 3.57 4.41 2.72 4.03 4.22 4.17 FMNIST Ψ = 0.5 8.70 4.63 4.52 3.68 3.86 3.12 6.21 4.79 4.67 5.10 4.46 3.96 4.81 (ResNet9) Ψ = 0.3 10.14 3.08 4.49 8.15 7.24 3.16 4.03 2.14 3.24 4.64 1.76 8.03 5.01 Ψ = 0.8 3.82 CIFAR10 Ψ = 0.5 3.98 (ResNet9) Ψ = 0.3 2.56
3.07
3.94
3.91
3.77
3.44
3.91
4.52
3.59
3.67
3.09
3.50
3.69
4.17
6.23
4.10
4.01
6.30
4.48
3.89
6.14
4.12
4.24
6.27
4.83
3.16
6.06
2.51
3.17
5.29
3.90
4.04
5.74
2.59
3.81
6.60
4.12
Ψ = 0.8 0.58
1.18
0.84
0.98
1.17
0.92
0.42
1.40
0.96
0.53
1.30
0.82
0.93
CIFAR100 Ψ = 0.5 1.05 (VGG) Ψ = 0.3 0.15
0.76
0.69
0.89
0.82
0.90
0.98
0.82
0.85
0.97
0.75
0.93
0.87
0.49
0.87
0.19
0.90
0.86
0.14
0.56
0.76
0.04
0.49
0.94
0.53
CIFAR10 CIFAR100
v
BadNet Dir(0. 5) Dir(0. 1) ASR↓ RR↑ ASR↓ RR↑
DBA Dir(0. 5) Dir(0. 1) ASR↓ RR↑ ASR↓ RR↑
20% 30% 40% 20% 30% 40%
3.64 4.58 4.23 0.43 0.15 0.29
2.65 3.32 2.83 0.54 0.58 0.71
2) Effectiveness of FedDAB in Varying Non-IID Settings: To evaluate the performance of FedDAB under different data distributions, we conduct experiments under three Non-IID settings: Dir(0.8), Dir(0.5), and Dir(0.3). Table II reports the ASR achieved by FedDAB. Across the four backdoor attacks, the ASR exhibits some fluctuations as statistical heterogeneity increases. Nevertheless, FedDAB maintains its effectiveness across different Non-IID settings. Furthermore, FedDAB achieves lower ASR on CIFAR-10/100 than on FMNIST. This can be attributed to the more complex sample characteristics of CIFAR-10/100, which make it harder for malicious nodes to manipulate the global model toward their backdoor objectives. Moreover, on CIFAR-10/100, the ASR changes marginally when the proportion of malicious nodes increases from 30% to 50%, indicating that FedDAB is robust to variations in
2.17 4.01 5.84 0.19 0.86 1.10
70.71 68.19 68.92 35.88 33.42 32.40
75.84 73.11 70.66 43.46 43.87 41.27
3.58 5.64 3.45 0.59 0.97 0.93
68.19 67.42 65.40 38.48 36.68 33.45
TABLE IV I MPACT OF EACH COMPONENT IN F ED DAB UNDER CIFAR100.
Configuration
ones more accurately. (ii) FedDAB employs both overalldirection checking and parameter-level checking to evaluate local updates from complementary perspectives, making it more difficult for malicious updates to evade detection.
74.15 72.99 71.01 42.83 41.91 40.62
Attack Success Rate (%) ↓ Avg. BadNet DBA Neurotoxin PGD Dir(1.0)Dir(0.5)Dir(1.0)Dir(0.5)Dir(1.0)Dir(0.5)Dir(1.0)Dir(0.5) ASR↓
DSS + MCT
2.24 1.95
20.95 12.30
2.61 1.77
31.60 21.84
2.70 2.09
25.78 17.63
2.48 2.15
33.35 15.21 13.77 9.18
SAS(Top-30%) + MCT
1.65 1.21
12.65 9.18
1.48 1.83
12.31 8.90
1.26 1.55
11.97 10.24
1.01 1.58
12.92 12.37
6.90 5.85
DSS + SAS(Top-100%) + MCT
1.77 0.82
5.70 2.18
1.63 0.96
6.40 1.91
2.04 0.89
5.35 2.24
1.75 0.84
8.50 2.37
4.14 1.52
DSS + SAS(Top-50%) + MCT
1.40 0.83
1.44 1.08
1.38 0.65
1.29 0.95
1.07 0.72
1.13 0.84
0.92 0.68
1.10 0.98
1.21 0.84
FedDAB (H = 0) FedDAB (H = 3)
1.24 0.67
1.30 1.05
1.08 0.71
1.14 0.89
1.15 0.70
1.06 0.98
0.84 0.73
1.25 0.94
1.13 0.83
the proportion of malicious nodes. This conclusion is further supported by the convergence curves in Fig. 3. To evaluate the performance of FedDAB under extreme Non-IID settings, we conduct experiments under the Dir(0.1) setting, using the relatively moderate Dir(0.5) setting as a reference. The results are reported in Table III. We observe that, despite the higher degree of statistical heterogeneity under Dir(0.1), the ASR of FedDAB remains comparable to that under Dir(0.5), without any noticeable increase. These results demonstrate that FedDAB can effectively mitigate backdoor attacks even under extreme Non-IID conditions.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
K