1
Robust Decentralized Personalized Federated Learning via Prediction-Constrained Neighborhood Collaboration Xiao Maa , Hong Shenb , Hui Tianc , Wenqi Lyua , Wei Kea a
Faculty of Applied Sciences, Macao Polytechnic University, Macao SAR, China School of Engineering and Technology, Central Queensland University, Australia c School of Information and Communication Technology, Griffith University, Australia
arXiv:2609.07312v1 [cs.LG] 7 Sep 2026
b
Abstract—Decentralized personalized federated learning enables collaborative model training without centralized coordination while allowing clients to learn client-specific models under heterogeneous data distributions. However, robustness in such systems becomes fundamentally challenging under Byzantine attacks, since honest personalized clients may naturally produce highly diverse model updates that are difficult to be distinguished from malicious perturbations. Existing robust decentralized learning methods typically rely on consensusoriented assumptions or statistical agreement among neighboring updates, which become unreliable in decentralized personalized environments. To address this issue, we propose a robust decentralized personalized federated learning method R-DPFL, that enables clients to reduce the impact of Byzantine attacks via robust neighborhood direction estimation and history-based update trend prediction, rather than purely aggregating client models as in the existing work. In R-DPFL, each client first computes the current-round model update by aggregating the received neighborhood update vectors. It then predicts what this update should be based on its historical values and local model changes. Finally, R-DPFL computes the difference between these two quantities, adaptively clips this difference, and adds it to the local update. We prove convergence of the learning process through rigorous analysis and show that honest clients maintain stable personalized descent dynamics under Byzantine neighbor perturbations without requiring consensus among neighboring models. Extensive experiments on CIFAR-10 demonstrate that RDPFL consistently outperforms state-of-the-art decentralized and personalized federated learning baselines under heterogeneous and adversarial settings. Index Terms—Decentralized Federated Learning, Robustness, Personalized
I. I NTRODUCTION EDERATED Learning (FL)[1] enables distributed clients to collaboratively train models without sharing raw data. Most existing FL systems[2], [1], [3] rely on a centralized server to coordinate client updates, which may introduce a single point of failure, trust concerns, and communication bottlenecks. To overcome these limitations, decentralized federated learning (DFL)[4], [5], [6] removes the central coordinator and allows clients to communicate directly with their neighbors over a peer-to-peer graph. However, decentralization also makes learning more vulnerable to unreliable or malicious neighbors, since each client can only observe local
F
peer information[7], [8], [9]. Meanwhile, real-world federated systems[10], [11], [12] are often highly heterogeneous, where clients may differ in data distributions, usage patterns, and learning objectives. In such cases, learning one shared global model is often suboptimal, motivating decentralized personalized federated learning (DPFL)[13], where each client learns its own personalized model while still benefiting from neighborhood collaboration. Existing DPFL methods[13], [14] have mainly focused on improving personalization, communication efficiency, or resource adaptability in benign environments. For example, prior studies have explored sparse decentralized training[13], knowledge distillation[15], [16], and directed collaboration[14] to address data and system heterogeneity. However, these methods typically do not consider adversarial neighbor behavior during peer-to-peer collaboration. This limitation becomes critical when personalization and Byzantine attacks coexist[17], [18]. In DPFL, even honest clients may naturally produce diverse model updates because their data distributions, optimization trajectories, and local objectives differ. As a result, malicious perturbations can be camouflaged by benign personalized diversity, making them difficult to distinguish from benign personalized differences[7], [19]. This observation reveals a fundamental conflict between robustness and personalization in decentralized learning. Many Byzantine-robust decentralized methods are built upon the statistical regularity that benign updates should remain sufficiently aligned, so that abnormal messages can be detected or suppressed[19], [7]. In contrast, decentralized personalized learning naturally allows clients to follow different optimization trajectories toward client-specific solutions[14], [13], which increases benign update diversity and weakens the reliability of similarity-based anomaly detection. Consequently, robust methods that rely on consensus may become overly conservative in personalized settings: they may suppress useful but heterogeneous neighbor information, causing collaboration to degenerate toward isolated local training. Conversely, overly permissive mechanisms may allow Byzantine perturbations to propagate through peer-to-peer communication, degrading the stability and utility of personalized models[9]. Therefore, the central challenge is not merely to make decentralized learning robust, but to achieve robustness without compromising
2
personalization. This challenge further suggests that the conventional consensus-based view of decentralized convergence becomes insufficient for DPFL. In traditional decentralized optimization, convergence is typically characterized by all honest clients asymptotically approaching a common model or a shared stationary solution through repeated neighborhood averaging and information mixing[20]. Under this view, collaborative learning is largely based on consensus: neighboring updates are expected to remain statistically compatible, and robustness mechanisms are often designed to identify and suppress updates that significantly deviate from the majority trend[7], [19]. However, such assumptions become fundamentally weaker in decentralized personalized learning. Due to heterogeneous local data distributions, device characteristics, user preferences, and application objectives, different honest clients may naturally evolve toward different local optima and follow substantially different optimization trajectories over time[13], [14]. Consequently, disagreement among neighboring models is no longer a reliable indicator of adversarial behavior, but may instead reflect legitimate personalization. In this setting, forcing strong consensus may undesirably suppress useful personalized information exchange, reduce model diversity, and eventually collapse collaborative learning into overly conservative or even isolated local optimization[21]. At the same time, completely removing collaboration is also undesirable[21], since neighborhood interaction remains essential for improving generalization, accelerating convergence, and mitigating local overfitting under limited local data. Therefore, the goal of robust DPFL should not be to recover global consensus under Byzantine attacks, but rather to maintain stable and beneficial personalized optimization in the presence of adversarial neighbor perturbations. Under this perspective, neighborhood collaboration should not aim to make neighboring clients converge to the same model. Instead, it should provide a controlled and reliable correction to each client’s local update, so that useful neighbor information can improve personalized training without dominating the client’s own objective. To address this challenge, we propose R-DPFL, a robust decentralized personalized federated learning method that preserves personalized local optimization while controlling the influence of unreliable neighbors. At each communication round, each client first performs local SGD on its private data and uses the model change after local SGD to construct a local descent direction. The client then exchanges this direction with its neighbors and receives possibly corrupted neighbor directions. Instead of directly averaging these messages or forcing neighboring models toward agreement, R-DPFL first applies robust aggregation to obtain an initial estimate of the neighborhood update. It then predicts the expected neighborhood update from historical changes in the client’s model and historical aggregated neighborhood updates. Finally, the deviation between the current aggregated update and the predicted update is adaptively clipped before being added to the local descent direction. In this way, neighborhood information is used as a bounded correction to personalized optimization:
useful heterogeneous neighbor information can still contribute to learning, while abrupt or malicious deviations are prevented from dominating the update. The main contributions of this work are summarized as follows: We propose a novel method for robust decentralized personalized federated learning in malicious environments that ensures both preservation of client personalization and robustness against Byzantine attacks by computing controlled client model update based on both currentround neighborhood updates and history-predicted update, rather than forcing model consensus. • We design a personalization-preserved neighborhood direction estimation mechanism deploying robust aggregation, historical prediction, and adaptive deviation clipping, that allows each client to suppress the effect of outliers in the received neighbor update directions while preserving personalized optimization without enforcing model consensus. • We provide a theoretical analysis showing that the proposed update rule achieves bounded personalized stationarity. Specifically, under standard smoothness and bounded-error assumptions, each honest client can maintain stable descent on its own local objective, and the influence of Byzantine neighbors appears only as a bounded residual term in the final convergence bound. • We conduct experiments under heterogeneous decentralized settings with multiple Byzantine attacks. The results show that R-DPFL preserves personalized performance in benign settings, improves robustness under adversarial neighbor perturbations, and that both the prediction and clipping components contribute to the final performance. •
II. R ELATED W ORK A. Personalized Federated Learning Personalized federated learning (PFL) aims to address statistical heterogeneity across clients by allowing different participants to learn client-specific models instead of enforcing a single shared global model. Existing PFL methods mainly differ in how they balance global collaboration and local personalization. One line of work adopts partial model personalization, where only a subset of model parameters are shared across clients. Representative examples include FedPer[22] and FedBABU[23], which separate shared feature representations from client-specific classifier layers. Another line of work formulates federated learning as a multi-task optimization problem. Methods such as MOCHA[24] explicitly model task relations among clients, while pFedMe[25] introduces regularized local optimization to balance global consistency and personalized adaptation. Knowledge distillation has also been explored to improve personalization under heterogeneous model architectures. FedMD[26] and FedDF[27] enable collaborative learning through predictionlevel knowledge transfer instead of direct parameter sharing. In addition, clustering-based methods such as FedCluster[28] attempt to group clients with similar data distributions to reduce
3
optimization inconsistency. Although these approaches improve personalization under heterogeneous data distributions, most existing PFL methods are developed under centralized federated learning architectures and assume benign training environments. Consequently, they rely on globally coordinated aggregation or trusted server-side optimization, making them difficult to apply directly in adversarial decentralized settings.
jointly in a decentralized setting. Existing personalized methods usually assume benign peer collaboration, while existing Byzantine-robust decentralized methods mainly aim to protect consensus-oriented learning. In contrast, R-DPFL does not require neighboring clients to learn the same model. Each client keeps its own personalized model, constructs a local update direction from its private data, and uses neighbor information only as a controlled correction to its local update.
B. Decentralized Federated Learning Decentralized federated learning (DFL) removes the central server and allows clients to exchange information directly through peer-to-peer communication. Compared with centralized FL, DFL improves scalability and eliminates the dependency on a trusted coordinator, but also introduces substantial challenges in optimization stability and robustness. Several recent works have explored personalization in decentralized environments. DFedAvgM[6] improves communication efficiency through local momentum and model quantization. DisPFL[13] accelerates decentralized personalized learning by introducing personalized sparse masks and customized local models. KD-PDFL[5], [16] incorporates knowledge distillation into decentralized collaboration to capture statistical relations among clients. From a theoretical perspective, ARDM[29] studies communication and computation complexity in decentralized personalized optimization. Despite these advances, existing decentralized personalized learning methods mainly focus on communication efficiency, model heterogeneity, or optimization performance under benign environments. Most methods implicitly assume that neighboring clients exchange reliable information and do not explicitly consider adversarial peer behavior during decentralized collaboration. C. Byzantine-Robust Decentralized Learning Byzantine-robust decentralized learning has recently attracted increasing attention due to the vulnerability of peer-topeer communication to malicious participants. Existing robust decentralized methods typically attempt to suppress abnormal neighbor updates through robust aggregation, clipping, or consensus-preserving optimization. Representative approaches include Byzantine-resilient decentralized optimization methods based on clipping and robust filtering[19], as well as robust decentralized federated learning methods that leverage neighborhood similarity or robust aggregation to mitigate adversarial influence[7]. These methods generally assume that honest clients remain sufficiently aligned so that malicious deviations can be identified as statistical outliers. However, such assumptions become substantially weaker in decentralized personalized learning. In personalized settings, honest clients may naturally follow different optimization trajectories due to heterogeneous local objectives and client-specific preferences. Consequently, disagreement among neighboring updates may no longer reliably indicate adversarial behavior, which fundamentally challenges consensus-oriented robustness mechanisms. Different from existing personalized federated learning methods and Byzantine-robust decentralized optimization methods, this work studies robustness and personalization
III. P ROBLEM F ORMULATION A. Decentralized Personalized Learning System We consider a decentralized federated learning system consisting of n clients, indexed by the set V = {1, 2, . . . , n}. The clients communicate over a fixed peer-to-peer graph G = (V, E), where an edge (i, j) ∈ E indicates that client i can exchange information with client j. For each client i, let Ni = {j ∈ V : (i, j) ∈ E} denote its neighbor set. Unlike centralized federated learning, there is no server to coordinate training or aggregate model updates. Each client performs local optimization using its private data and communicates only with its immediate neighbors. Each client i maintains a personalized local model wi ∈ Rd and owns a private local dataset Di . The local objective of client i is defined as fi (wi ) = Eξ∼Di [ℓ(wi ; ξ)],
(1)
where ξ ∼ Di denotes that the data sample ξ is (randomly) selected from the local dataset Di , and ℓ(wi ; ξ) denotes the loss of model wi on ξ. The expectation Eξ∼Di means that we take the average loss over the local samples of client i. Therefore, fi (wi ) represents the average local loss of model wi . Due to statistical heterogeneity, the local distributions {Di }i∈V may differ significantly across clients. Therefore, different clients generally do not share the same optimal model. Formally, for two clients i ̸= j, we generally have arg min fi (w) ̸= arg min fj (wj ). w
w
Thus, the goal of decentralized personalized learning is not to force all clients to converge to a single shared model, but to allow each client to learn a model adapted to its own local objective while still benefiting from neighborhood collaboration. A common way to describe such soft collaboration is through the following ideal personalized formulation: X X min fi (wi ) + λ ρ(wi − wj ), (2) {wi }i∈V
i∈V
(i,j)∈E
where λ > 0 controls the strength of neighborhood collaboration, and ρ(·) measures the discrepancy between neighboring models. The first term encourages each client to fit its own local data, while the second term introduces a soft neighborhood regularization effect. This Eq.2 is used only to motivate soft neighborhood collaboration. Our method does not directly optimize the pairwise discrepancy term; instead, it realizes collaboration through robust direction-level correction. However, this formulation should not be interpreted as enforcing hard consensus. In decentralized personalized learning,
4
the discrepancy term is only intended to encourage useful information exchange among related clients. The personalized models are still allowed to remain different due to heterogeneous data distributions and client-specific objectives. B. Robust Personalized Collaboration under Byzantine Neighbors The above formulation becomes more challenging when some neighbors are unreliable or Byzantine. In conventional decentralized optimization, collaboration is often consensusoriented: neighboring clients are expected to maintain statistically compatible updates, and large deviations from the neighborhood trend are usually treated as suspicious. Such a principle is reasonable when honest clients are expected to optimize a shared objective or approach a common solution. In decentralized personalized federated learning, however, this assumption becomes weaker. Since honest clients may have different local objectives and personalized optima, their model updates can naturally diverge even in the absence of attacks. Therefore, a large discrepancy between neighboring clients should not be automatically regarded as malicious. It may instead reflect legitimate personalization caused by data heterogeneity, different optimization trajectories, or clientspecific preferences. This creates a key difficulty: overly conservative robust mechanisms may suppress useful heterogeneous collaboration, while overly permissive mechanisms may allow Byzantine perturbations to spread through peer-to-peer communication. To capture this setting, we view neighborhood interaction as a controlled collaborative correction rather than a consensus constraint. At each communication round t, client i updates its personalized model using two components: a local descent direction uti , computed from its own data, and a neighborhood collaboration direction dti , estimated from the messages received from its neighbors. The general update form is wit = wit−1 − γt uti + λdti , where γt > 0 is the learning rate and λ > 0 controls the strength of neighborhood collaboration. The local direction uti drives client i toward its personalized objective, while dti provides a robust collaborative signal extracted from potentially corrupted neighbor information. We consider a Byzantine adversarial setting in which a subset of clients may send arbitrary messages to their neighbors. Let utj denote the update-related message generated by client j at round t. When client i receives information from neighbor j, the received message may be corrupted as t ũtj→i = utj + δj→i , t where δj→i denotes an arbitrary adversarial perturbation. The perturbation may depend on the current round, local states, or the adversary’s attack strategy. For each honest client i, let Bi ⊆ Ni denote the set of Byzantine neighbors. We assume that the Byzantine fraction in each honest client’s neighborhood is bounded:
1 |Bi | ≤β< . |Ni | 2
Under this threat model, our objective is not to identify every malicious neighbor or exactly recover every clean neighbor update. Instead, we aim to construct a neighborhoodlevel collaboration direction dti whose influence remains useful under benign heterogeneity and bounded under Byzantine perturbations. Accordingly, convergence should not be defined as all honest clients reaching asymptotic consensus. Since clients may pursue different personalized optima, the desired learning goal is that each honest client maintains stable descent toward its own local objective while adversarial neighbor influence only introduces bounded distortion. From the theoretical perspective, we aim to establish a bounded personalized stationarity guarantee. That is, for each honest client i, we want the average expected squared local gradient norm over T iterations of learning to be bounded by a constant C: T −1 1 X Eξit ∥∇fi (wit )∥22 ≤ C. T t=0 This quantity measures the first-order stationarity of the personalized local model. Since a stationary point of a differentiable objective satisfies ∇fi (wi ) = 0, a smaller ∥∇fi (wit )∥22 indicates that the local model is closer to a stationary point. Here, ξ ξit,k denotes the mini-batch sampled by client i at the k-th local SGD step of round t, and ξ ti = (ξit,0 , . . . , ξit,K−1 ) collects all mini-batches used by client i in round t. The expectation Eξ is taken over the mini-batch sampling randomness along the training trajectory, which determines the random iterate wit . IV. M ETHODOLOGY A. Framework Overview Our robust decentralized personalized federated learning method has the following framework. At communication round t, each client i maintains a local personalized model wit−1 . Starting from wit−1 , the client first performs multiple local stochastic gradient updates on its private dataset and constructs a local descent direction uti , which captures the dominant optimization tendency induced by its own local objective. After local optimization, client i exchanges update-related messages with its neighbors. Since some neighboring clients may be Byzantine, the received messages may be arbitrarily corrupted and cannot be directly trusted. To address this issue, each client estimates a robust neighborhood collaboration direction dti from the received neighbor messages together with historical neighborhood dynamics. The estimated direction is designed to preserve useful collaborative trends while suppressing abnormal perturbations caused by malicious neighbors. The personalized model of client i is then updated according to wit = wit−1 − γt uti + λdti , where γt > 0 denotes the learning rate and λ > 0 controls the strength of neighborhood collaboration. The overall procedure of R-DPFL consists of four main steps at each communication round:
5
1) Local optimization: each client performs multiple local SGD updates on its private data; 2) Local direction construction: the cumulative local optimization effect is converted into a local descent direction uti ; 3) Robust neighborhood direction estimation: the client estimates a robust collaborative direction dti using robust aggregation, historical prediction, and adaptive deviation clipping; 4) Personalized model update: the personalized model is updated using both the local descent direction and the robust neighborhood collaboration direction. Algorithm 1 summarizes the complete training procedure of R-DPFL. Algorithm 1 R-DPFL Training Procedure. Require: Graph G = (V, E), local step number K, learning rates ηt , γt , collaboration weight λ Initialize personalized models {wi0 }i∈V for t = 1, 2, . . . , T do // Each client i ∈ V performs in parallel Set wit,0 = wit−1 Perform K steps of local SGD and construct local descent direction K−1
uti =
1 X gi (wit,k ; ξit,k ) K k=0
Exchange local directions with neighboring clients and receive neighbor message set Ueit Compute robust neighborhood collaboration direction dti = ROBUST D IRECTION E STIMATION(Ueit ) Update personalized model: wit = wit−1 − γt uti + λdti
end for Output: Personalized models {wiT }i∈V
B. Local Descent Direction Construction At communication round t, each client i starts from its current personalized model wit−1 and performs multiple steps of local stochastic optimization on its private dataset. Specifically, let wit,0 = wit−1 , for each local step k = 0, 1, . . . , K −1, client i updates its model according to (3) wit,k+1 = wit,k − ηt gi wit,k ; ξit,k , where ηt > 0 is the local learning rate, ξit,k denotes a mini-batch sampled from the local dataset of client i, and gi (wit,k ; ξit,k ) is a stochastic gradient estimator of ∇fi (wit,k ). After K local steps, client i obtains an intermediate locally optimized model w̃it = wit,K . (4) The intermediate model w̃it is only used to construct the local descent direction, while the actual round-level model is updated by Eq.16.
Based on the cumulative effect of local optimization, we define the local descent direction of client i at round t as uti =
wit−1 − w̃it . ηt K
(5)
Substituting (3) into (5), we obtain K−1
uti =
1 X t,k t,k gi wi ; ξi . K
(6)
k=0
Therefore, uti can be interpreted as the average local stochastic descent direction accumulated over K optimization steps.
C. Robust Neighborhood Direction Estimation In decentralized personalized federated learning, neighborhood communication is the only channel through which clients can exchange collaborative information. However, in the presence of Byzantine adversaries, the received neighbor messages may be arbitrarily corrupted and therefore cannot be directly incorporated into model updates. Moreover, under personalized data heterogeneity, even benign neighbor directions may differ substantially from the local direction of a client. Consequently, the key challenge is not to force agreement among neighbors, but to extract a stable and informative neighborhood-level collaboration signal while suppressing abnormal perturbations. To this end, we construct a robust neighborhood collaboration direction for each client by combining robust aggregation, temporal prediction, and adaptive deviation clipping, as shown in Algorithm 2. The resulting direction is designed to preserve benign collaborative trends while limiting the influence of malicious neighbors to a bounded range. Algorithm 2 Robust Neighborhood Direction Estimation. Require: Neighbor message set Ueit , historical model trajectory {wit−1 , wit−2 }, historical neighborhood estimates {d¯t−1 , d¯t−2 }, previous threshold τit−1 i i Compute robust neighborhood aggregation: d¯ti = RobustAgg(Ueit ) if t ≥ 2 then Compute historical variations: st−1 = wit−1 − wit−2 , i t−1 t−1 t−2 y = d¯ − d¯ i
i
i
⟨st−1 ,y t−1 ⟩
Compute prediction coefficient: βit = ∥sit−1 ∥i2 +ϵ 2 i Predict neighborhood direction: dˆti = d¯t−1 + βit (wit−1 − i wit−2 ) else Set dˆti = d¯ti end if Compute deviation: ∆ti = d¯ti − dˆti Update clipping threshold: τit = ατit−1 + (1 − α)c∥∆ti ∥2 Construct robust neighborhood direction: dti = dˆti + clip(∆ti ; τit ) return dti
6
1) Neighbor Message Collection: At communication round t, after constructing its local descent direction uti , each client i transmits this direction to its neighbors and receives updaterelated messages from its neighbor set Ni . Due to Byzantine attacks, the message sent from neighbor j to client i may be corrupted. Therefore, the received message is modeled as t ũtj→i = utj + δj→i ,
(7)
t where δj→i denotes an arbitrary adversarial perturbation. Let Ueit = ũtj→i | j ∈ Ni (8)
denote the set of all received neighbor directions of client i at round t. Our objective is not to identify which neighbor is malicious, nor to exactly recover every clean neighbor direction. Instead, we aim to estimate a neighborhood-level collaboration direction that captures the dominant benign trend in Ueit while remaining robust to outliers and adversarial distortions. 2) Robust Neighborhood Aggregation: As the first step, client i applies a robust aggregation operator to the received set Ueit and obtains a coarse estimate of the current neighborhood collaboration trend: d¯ti = RobustAgg Ueit , (9) where RobustAgg(·) can be instantiated by standard robust aggregation rules such as coordinate-wise median, trimmed mean, or geometric median. The purpose of this step is to suppress the effect of large instantaneous outliers in the received neighbor directions. Compared with direct averaging, robust aggregation provides a more stable estimate of the dominant neighborhood trend when a bounded fraction of received messages are corrupted. However, robust aggregation alone is insufficient in decentralized personalized environments. Since benign neighbor directions may also vary substantially across clients and over time, a purely static aggregation rule may still overreact to naturally occurring heterogeneity or may be misled by temporally coordinated attacks. 3) Historical Prediction of Neighborhood Dynamics: To distinguish benign neighborhood evolution from abrupt abnormal perturbations, client i first predicts the expected neighborhood collaboration direction at round t using only historical information available before receiving the current-round aggregated signal. Let st−1 = wit−1 − wit−2 i
(10)
denote the previous-round change in the local model, and let yit−1 = d¯t−1 − d¯t−2 i i
(11)
denote the previous-round change in the aggregated neighborhood direction. To capture the local relation between the model trajectory and the neighborhood evolution, we introduce a scalar prediction coefficient βit based on a Barzilai--Borwein-style[30] approximation: st−1 , y t−1 , (12) βit = it−1 2i ∥si ∥2 + ϵ
where ϵ > 0 is a small constant for numerical stability. Using this coefficient, the predicted neighborhood direction at round t is defined as dˆti = d¯t−1 + βit wit−1 − wit−2 . (13) i The intuition behind (13) is that, under benign optimization dynamics, the neighborhood collaboration trend should evolve smoothly with the local model trajectory. Therefore, dˆti provides a temporally consistent estimate of the expected neighborhood direction before the current-round neighborhood observation is incorporated. In contrast, adversarial perturbations often induce abrupt deviations that are difficult to explain by historical evolution alone. The coefficient estimates the local sensitivity between the client model trajectory and the evolution of the neighborhood collaboration direction, thereby enabling a first-order approximation of benign neighborhood dynamics. 4) Adaptive Deviation Clipping: After receiving the neighbor messages at round t and computing the robustly aggregated neighborhood direction d¯ti , client i compares the current observation with the historical prediction. Specifically, we define the deviation as ∆ti = d¯ti − dˆti . (14) This deviation quantifies how much the currently observed neighborhood trend differs from the expected benign evolution predicted from history. Since benign deviations may still arise from data heterogeneity and non-stationary local optimization, the deviation should not be treated using a fixed threshold. Instead, we adopt an adaptive clipping threshold that evolves with historical deviation statistics. Specifically, the threshold is updated using an exponential moving average: τit = ατit−1 + (1 − α)c∥∆t−1 ∥2 , i
(15)
where α ∈ [0, 1) is a momentum coefficient and c > 1 is a tolerance factor. The prediction neighborhood direction is first constructed as d˜ti = dˆti + Clip ∆ti ; τit , where the clipping operator is defined by τ . Clip(v; τ ) = v · min 1, ∥v∥2 This operation preserves deviations consistent with recent historical behavior, while clipping large deviations to a bounded limits. To further control the maximum influence of neighborhood collaboration on the personalized update, we then apply: dti = Clip d˜ti ; Gd , where Gd > 0 is the maximum allowed norm of the final neighborhood collaboration direction. During the warm-up stage, each client sets dˆti = d¯t−1 before sufficient historical i information is available. The threshold τit is initialized as τi0 and ∆0i is set to zero.
7
D. Personalized Descent with Robust Neighborhood Collaboration After constructing the local descent direction uti and the robust neighborhood collaboration direction dti , client i updates its personalized model by combining these two components in a unified descent step. Specifically, the update rule at communication round t is defined as wit = wit−1 − γt uti + λdti ,
(16)
where γt > 0 denotes the learning rate and λ > 0 controls the strength of neighborhood collaboration. Unlike consensus-based decentralized optimization methods, the proposed update mechanism does not directly average neighboring models or enforce asymptotic agreement among clients. In traditional decentralized learning schemes, the next iterate is typically obtained by mixing the current local model with neighbor models or their weighted averages, which implicitly treats neighborhood consensus as the optimization target. However, such consensus-oriented updates become less suitable in decentralized personalized learning, where honest clients may naturally follow heterogeneous optimization trajectories toward different local optima. In the proposed method, the local descent direction uti serves as the dominant optimization component. It is constructed solely from the private data of client i and reflects the principal descent tendency of the local personalized objective. Therefore, uti drives the model toward a client-specific desirable solution. In contrast, the neighborhood collaboration direction dti is not intended to replace local optimization or enforce agreement with neighboring models. Instead, it acts as a bounded collaborative correction extracted from neighborhood interactions, whose role is to introduce useful collaborative information while suppressing abnormal perturbations caused by unreliable or Byzantine neighbors. The collaboration weight λ balances personalization and neighborhood interaction. When λ = 0, the proposed method reduces to fully independent local training, where each client optimizes only its own objective without any collaboration. As λ increases, neighborhood information exerts a larger influence on the update process. Nevertheless, even under larger collaboration weights, the proposed method still does not require neighboring models to converge toward a common solution. Instead, collaboration is incorporated in a controlled and reliability-aware manner through robust aggregation, temporal prediction, and adaptive deviation clipping. From an optimization perspective, (16) can be interpreted as a personalized descent process with prediction-constrained robust neighborhood collaboration. The local direction uti determines the dominant personalized optimization trajectory, while the neighborhood term dti introduces only a bounded collaborative correction whose magnitude is explicitly controlled through the robust estimation and clipping procedure in Section IV-C. Consequently, Byzantine perturbations can influence the optimization dynamics only through a bounded neighborhood contribution, while useful heterogeneous collaborative information can still be preserved.
V. T HEORETICAL A NALYSIS In this section, we analyze the stability and convergence behavior of the proposed robust decentralized personalized federated learning method. In our R-DPFL, each honest client performs personalized local descent while incorporating a bounded neighborhood collaboration term extracted from potentially corrupted neighbor messages. Therefore, rather than characterizing convergence through global consensus, we focus on proving a bounded personalized stationarity guarantee: for each honest client, the proposed update rule achieves stable descent on its local objective, while the effect of Byzantine neighbor perturbations enters only through a bounded collaboration error term. A. Assumptions Following the literation for convergence analysis of FL[31], [32], [33] ,we assume that each honest client’s local objective satisfies the following smoothness condition: Definition 1. [34]L-smoothness A differentiable function fi : Rd → R is said to be Lsmooth if its gradient is Lipschitz continuous, i.e., for any w, v ∈ Rd , ∥∇fi (w) − ∇fi (v)∥2 ≤ L∥w − v∥2 .
(17)
Equivalently, for any w, v ∈ Rd , L ∥v − w∥22 . (18) 2 Assumption 1. [35], [1], [36]Unbiased gradients with bounded norm.: For each honest client i, the stochastic gradient computed over mini-batch sample ξ drawn from i’s local dataset, gi (w; ξ), is an unbiased estimator of i’s local gradient: Eξ [gi (w; ξ)] = ∇fi (w). fi (v) ≤ fi (w) + ⟨∇fi (w), v − w⟩ +
Moreover, there exists a constant Gg > 0 such that 2
Eξ ∥gi (w; ξ)∥2 ≤ G2g . This condition also implies bounded stochastic gradient variance, since 2
Eξ ∥gi (w; ξ) − ∇fi (w)∥2 ≤ G2g . B. Proof Sketch For an honest client i, we view one training round as consisting of two components, a personalized local descent component uti and a corrected neighborhood collaboration component dti . The proof first analyzes these two components separately and then combines them in the one-step progress of the local objective fi . 1) We first show that the local descent direction uti remains aligned with the personalized objective of client i. Under standard stochastic gradient assumptions, uti can be decomposed into the true local gradient ∇fi (wit−1 ), a bounded local drift bias, and a bounded stochastic noise term.
8
2) We then show that the corrected neighborhood collaboration direction dti has bounded influence. Since Byzantine messages may be arbitrary, we do not bound each malicious message directly. Instead, the proposed robust aggregation, historical prediction, and adaptive clipping mechanism transforms unreliable neighborhood information into a bounded residual. 3) Finally, we show that the personalized descent effect dominates this bounded collaboration residual in the one-step progress of fi . After accumulating the descent inequalities over communication rounds, Byzantine neighbor perturbations appear only as a bounded residual term in the final bound, yielding a bounded personalized guarantee for honest client i.
C. Bounded Local Descent Direction
Therefore, K−1 h i 1 X Eξti uti | wit−1 = Eξti ∇fi (wit,k ) | wit−1 . K k=0
Define the local drift bias as bti = Eξti uti | wit−1 − ∇fi (wit−1 ). Since ∇fi (wit−1 ) is fixed when conditioning on wit−1 , the bias can be written as bti = Eξti [uti | wit−1 ] − ∇fi (wit−1 )
At the end of round t, after receiving neighborhood information and performing K-step local updates with mini-batch sampling ξ ti , client i obtains the updated model wit , which is then used as the initialization for the next round t + 1. We first show that the local descent direction is a controlled approximation of the true local gradient, with bounded local drift and stochastic variance. Lemma 2. (Bounded local descent direction) Let ξ ti = (ξit,0 , . . . , ξit,K−1 ) denote all mini-batches sampled by client i during K local SGD steps in round t. Conditional on the beginning round model wit−1 , the expectation is taken over the local mini-batch samples ξ ti . Suppose that Assumption 1 holds and that fi is L-smooth. For each honestPclient i and round K−1 t,k t,k 1 t, the local descent direction uti = K k=0 gi (wi ; ξi ) satisfies Eξti uti | wit−1 = ∇fi (wit−1 ) + bti ,
Notice that wit,k depends on wit−1 and the previous minibatches ξit,0 , . . . , ξit,k−1 . Using Assumption 1, we obtain h i Eξti gi (wit,k ; ξit,k ) | wit−1 i h h i = Eξti Eξt,k gi (wit,k ; ξit,k ) | wit,k wit−1 i h i = Eξti ∇fi (wit,k ) | wit−1 .
∥bti ∥2 ≤ Bηt K,
K−1
1 X Eξti [∇fi (wit,k ) | wit−1 ] − ∇fi (wit−1 ) K
=
1 K
=
Eξti
uti − Eξti [uti | wit−1 ]
2 2
wit−1
≤ σu2 ,
t where B > h0 and σu2 ≥ 0 are constants, i and bi = PK−1 t,k t−1 t−1 1 ) | wi is the local k=0 Eξti ∇fi (wi ) − ∇fi (wi K
drift bias induced by the K local SGD steps. Here, wit,k denotes the local model of client i after k local SGD steps in round t, starting from the beginning-round model wit−1 , i.e., Pk−1 wit,0 = wit−1 , wit,k = wit−1 − η r=0 gi (wit,r ; ξit,r ), k = 1, . . . , K. Proof: By the of the local descent direction, Pdefinition K−1 t,k t,k 1 we have uti = K g (w i ; ξi ).Taking the conditional k=0 i expectation over all mini-batches ξ ti sampled in round t, conditioned on the model wit−1 , gives K−1 h i 1 X Eξti uti | wit−1 = Eξti gi (wit,k ; ξit,k ) | wit−1 . K k=0
k=0 K−1 X
1 K
=
Eξti [∇fi (wit,k ) | wit−1 ] − ∇fi (wit−1 )
h i Eξti ∇fi (wit,k ) − ∇fi (wit−1 ) | wit−1 .
k=0
Consequently, Eξti uti | wit−1 = ∇fi (wit−1 ) + bti . Here, bti represents the local drift bias between the expected K-step local descent direction and the gradient at the model wit−1 , while ∥bti ∥2 quantifies the size of this drift. By Jensen’s inequality, the triangle inequality, and the L-smoothness of fi , we have K−1
∥bti ∥2 =
and
k=0 K−1 X
h i 1 X Eξti ∇fi (wit,k ) − ∇fi (wit−1 ) | wit−1 K k=0 K−1
2
h i 1 X ≤ Eξti ∇fi (wit,k ) − ∇fi (wit−1 ) | wit−1 K 2 ≤ ≤
1 K L K
k=0 K−1 X k=0 K−1 X
h Eξti
∇fi (wit,k ) − ∇fi (wit−1 )
2
i h Eξti ∥wit,k − wit−1 ∥2 | wit−1 .
k=0
From the local SGD, wit,k = wit−1 − η
k−1 X
gi (wit,r ; ξit,r ),
r=0
and hence h i Eξti ∥wit,k − wit−1 ∥2 | wit−1 ≤ ηt
k−1 X r=0
Eξti ∥gi (wit,r ; ξit,r )∥2 | wit−1 .
| wit−1
i
9
By the Cauchy--Schwarz inequality, Eξti ∥gi (wit,r ; ξit,r )∥2 | wit−1 1/2 ≤ Gg , ≤ Eξti ∥gi (wit,r ; ξit,r )∥22 | wit−1
Proof: By the definition of the final neighborhood collaboration direction, we have dti = Clip d˜ti ; Gd .
where the last inequality follows from Assumption 1. It follows that h i Eξti ∥wit,k − wit−1 ∥2 | wit−1 ≤ ηkGg .
From the definition of the clipping operator, for any vector v and radius τ > 0, τ . ∥Clip(v; τ )∥2 = v · min 1, ∥v∥2 2
Substituting this bound into the bias estimate gives
Therefore,
τ ∥Clip(v; τ )∥2 = ∥v∥2 min 1, ∥v∥2
K−1
LηGg X ∥bti ∥2 ≤ k K k=0
= min {∥v∥2 , τ } ≤ τ.
Lηt Gg (K − 1) = ≤ BηK, 2 where one may choose
Taking v = d˜ti and τ = Gd , we obtain ≤ Gd . ∥dti ∥2 = Clip d˜ti ; Gd
LGg . 2 It remains to bound fluctuation of uti . tthe stochastic t−1 t Let µi = Eξti ui | wi , using the conditional variance identity, we obtain Eξti ∥uti − µti ∥22 | wit−1 = Eξti ∥uti ∥22 | wit−1 − ∥µti ∥22 ≤ Eξti ∥uti ∥22 | wit−1 .
This completes the proof.
2
B=
By Jensen’s inequality, Eξti ∥uti ∥22 | wit−1 2 K−1 X 1 wit−1 gi (wit,k ; ξit,k ) = Eξti K k=0
≤
1 K
K−1 X
2
h
Eξti ∥gi (wit,k ; ξit,k )∥22 | wit−1
i
Theorem 4. (Stable personalized descent) For an honest client i with initial local model wi0 , if wi is updated by wit = wit−1 − γ (uti + λdti ) according to our R-DPFL Training Procedure Algorithm 1 during the t-th round of learning, where γ > 0 is 1 a constant step size satisfying γ ≤ 8L , and finally stabilized after T rounds of learning. Then, under Assumption 1 and the bounded neighborhood direction condition ∥dti ∥2 ≤ Gd , the average personalized descent during these T rounds is bounded to a constant, i.e., the following inequality holds T
1X 2 ∇fi (wit ) 2 T t=1
Therefore, Eξti
We now establish that the proposed update rule achieves a bounded personalized stationarity guarantee for each honest client. The key idea is to treat the corrected neighborhood collaboration direction as a bounded residual term in the personalized descent process.
k=0
≤ G2g .
E. Stable Personalized Descent
uti − Eξti [uti | wit−1 ]
2 2
wit−1
≤ σu2 ,
where one may take σu2 = G2g . This completes the proof. D. Bounded Neighborhood Collaboration Direction We then show that historical prediction and adaptive clipping transform the corrected neighborhood collaboration direction into a bounded residual term in the personalized update. Lemma 3. (Bounded neighborhood collaboration direction) Let the prediction neighborhood direction be constructed as d˜ti = dˆti + Clip d¯ti − dˆti ; τit , and the final neighborhood collaboration direction be dti = Clip d˜ti ; Gd , where the n o τ clipping operator is defined as Clip(v; τ ) = v·min 1, ∥v∥ . 2 Then the final neighborhood collaboration direction satisfies ∥dti ∥2 ≤ Gd .
≤
4 fi (wi0 ) − fi⋆ + γT
(19)
1 9 5B 2 η 2 K 2 + σu2 + λ2 G2d 2 2 where, fi⋆ = fi (wiT ), B, η, K and σu2 are constants defined in Lemma 2. Proof: We decompose the one-step descent update into the local direction, the neighborhood collaboration direction, and the quadratic term. By bounding these terms, we obtain a descent inequality, which is then averaged over past samples and telescoped over T rounds. Let git−1 = ∇fi (wit−1 ), and Ft be the history available before the model update of round t, including all samples generated before round t. In round t, client i samples minibatches ξ ti = (ξit,0 , . . . , ξit,K−1 ), receives neighborhood information, and computes uti and dti . Conditioned on Ft , the past sampled mini-batches are fixed, while the current mini-batches ξ ti are freshly sampled from Di and remain random. The updated model is then given by wit = wit−1 − γ uti + λdti .
10
Conditioned on Ft , both wit−1 and git−1 are fixed quantities, while uti , dti , and wit depend on the samples generated during round t[31], [32], [33]. Since fi is L-smooth, we have fi (wit ) ≤ fi (wit−1 ) + git−1 , wit − wit−1 +
L 2 wit − wit−1 2 . 2
Using wit = wit−1 − γ uti + λdti . we obtain Lγ 2 t 2 ui + λdti 2 . 2 Taking expectation conditional on Ft in both sides gives E fi (wit ) | Ft ≤ E fi (wit−1 ) | Ft t−1 t − γE gi , ui | Ft − γλE git−1 , dti | Ft (20) i Lγ 2 h t 2 E ui + λdti 2 | Ft + 2 t−1 = E fi (wi ) | Ft + A + B + C, fi (wit ) ≤ fi (wit−1 ) − γ git−1 , uti + λdti +
For A, we first bound the local linear term −E git−1 , uti | Ft . Since git−1 is fixed conditioned on Ft , we have − E git−1 , uti | Ft = − git−1 , E uti | Ft . By Lemma 2, the local descent direction satisfies E uti | Ft = git−1 + bti , ∥bti ∥2 ≤ BηK. Therefore, −E
git−1 , uti | Ft
= − git−1 , git−1 + bti = −∥git−1 ∥22 − git−1 , bti .
By Lemma 3, ∥dti ∥22 ≤ G2d . Therefore, 1 git−1 , dti | Ft ≤ ∥git−1 ∥22 + λ2 G2d . (22) 4 h i 2 For C, we bound the quadratic term E ∥uti + λdti ∥2 | Ft . Using ∥a + b∥22 ≤ 2∥a∥22 + 2∥b∥22 , we have h i 2 E uti + λdti 2 | Ft ≤ 2E ∥uti ∥22 | Ft + 2λ2 E ∥dti ∥22 | Ft . −λE
By the conditional variance decomposition and Lemma 2, E ∥uti ∥22 | Ft h i 2 2 = E uti | Ft 2 + E uti − E uti | Ft 2 | Ft ≤ ∥git−1 + bti ∥22 + σu2 . Using ∥a + b∥22 ≤ 2∥a∥22 + 2∥b∥22 , we further obtain E ∥uti ∥22 | Ft ≤ 2∥git−1 ∥22 + 2∥bti ∥22 + σu2 . By Lemma 3, ∥dti ∥22 ≤ G2d . Therefore, E ∥dti ∥22 | Ft ≤ G2d . Combining the above bounds yields h i 2 E uti + λdti 2 | Ft ≤ 4∥git−1 ∥22 + 4∥bti ∥22 + 2σu2 + 2λ2 G2d .
1 t−1 2 ∥g ∥2 + ∥bti ∥22 . 4 i
Hence, 3 git−1 , uti | Ft ≤ − ∥git−1 ∥22 + ∥bti ∥22 . (21) 4 For B, we bound the neighborhood linear term −γλE git−1 , dti | Ft . Since git−1 is fixed conditioned on Ft , by Young’s inequality, for each realization of dti , we have −E
− λ git−1 , dti ≤ λ git−1 , dti ≤ λ∥git−1 ∥2 ∥dti ∥2 1 ≤ ∥git−1 ∥22 + λ2 ∥dti ∥22 4 Taking conditional expectation conditioned on Ft gives, − λE git−1 , dti | Ft 1 ≤ ∥git−1 ∥22 + λ2 E ∥dti ∥22 | Ft . 4
(23)
Substituting A, B and C in (20) with (21), (22) and (23) yields E fi (wit ) | Ft γ − 2Lγ 2 ∥git−1 ∥22 + γ + 2Lγ 2 ∥bti ∥22 ≤ E fi (wit−1 ) | Ft − 2 + γ + Lγ 2 λ2 G2d + Lγ 2 σu2 . 1 , we have Since γ ≤ 8L
γ γ − 2Lγ 2 ≥ , 2 4
Using Young’s inequality, − git−1 , bti ≤
and γ + Lγ 2 ≤
9γ , 8
γ + 2Lγ 2 ≤ Lγ 2 ≤
5γ , 4
γ . 8
Because ∥bti ∥2 ≤ BηK by Lemma 2, combining the above we have γ E fi (wit ) | Ft ≤ E fi (wit−1 ) | Ft − ∥git−1 ∥22 4 5γ 2 2 2 9γ 2 2 γ 2 + B η K + λ Gd + σu 4 8 8 Rearranging terms yields γ t−1 2 ∥g ∥2 ≤E fi (wit−1 ) | Ft − E fi (wit ) | Ft (24) 4 i 5γ 2 2 2 9γ 2 2 γ 2 + B η K + λ Gd + σu 4 8 8 Since wit−1 is determined by the information of Ft−1 in round t−1, it is Ft−1 -measurable and hencefi (wit−1 ) is Ft−1 measurable because fi is deterministic. Because the added information in Ft in round t over Ft−1 has no impact on wit−1 generated in the preceding round, wit−1 and fi (wit−1 )
11
are also Ft -measurable. Hence, by the measurability property of conditional probability in stochastic optimization, we have: E fi (wit−1 ) | Ft = fi (wit−1 ) E fi (wit ) | Ft = fi (wit ) Replacing the expectation terms of (24) with the above and computing the average of both sides from t = 1 to T yields: ! T T γ X t−1 2 X t−1 t ∥g ∥2 ≤ fi (wi ) − fi (wi ) + 4 t=1 i t=1 5γ 2 2 2 9γ 2 2 γ 2 B η K + λ Gd + σu T 4 8 8 0 T =f (wi ) − f (wi )+ 5γ 2 2 2 9γ 2 2 γ 2 T B η K + λ Gd + σu 4 8 8 =fi (wi0 ) − fi⋆ + 5γ 2 2 2 9γ 2 2 γ 2 B η K + λ Gd + σu T 4 8 8 The inequality of the theorem follows directly by dividing t−1 both sides by γT with ∇fi (wit−1 ). 4 and replacing gi VI. E XPERIMENTS A. Experimental Objectives The experiments are designed to evaluate the proposed method from three aspects. 1) we examine whether the proposed defense can preserve competitive personalized learning performance in the absence of attacks. 2) we investigate whether the proposed method can effectively improve robustness against Byzantine neighbors under different attack strengths. 3) we study whether the performance gains of the proposed method indeed come from its key design components. B. Datasets and Models We conduct experiments on widely used image classification benchmarks, CIFAR-10, to evaluate the robustness of the proposed method under decentralized federated learning with Byzantine neighbors.The datasets consist of 32 × 32 RGB natural images. CIFAR-10 contains 10 object categories and serves as a standard benchmark for robust federated learning. For CIFAR-10, we employ a lightweight convolutional neural network as the local model. The adopted architecture follows a standard convolutional design, consisting of multiple convolutional layers for hierarchical feature extraction and fully connected layers for classification. C. Experimental Setup We simulate a decentralized personalized federated learning environment with 10 clients. All clients participate in every communication round, and communication is performed over a fully connected topology, where each client exchanges updaterelated messages with all other clients. To model statistical heterogeneity, local datasets are partitioned using a Dirichlet
distribution with concentration parameter α = 0.3. Each experiment is run for 500 communication rounds. For local optimization, each client performs 1 local epoch of stochastic gradient descent per round with batch size 64, local learning rate 0.01, momentum 0, and weight decay 0. For the proposed R-DPFL method, the global update coefficient is set to 1.0. The neighborhood collaboration weight λ is varied according to the purpose of each experiment. In clean and sensitivity studies, we examine λ ∈ {0, 0.005, 0.01, 0.05}. In the main robustness and ablation experiments, we use λ = 0.01 as the default setting. The warm-up stage lasts for 5 communication rounds. The exponential moving average coefficient for adaptive threshold estimation is set to 0.9, the tolerance factor is set to 1.5, and the initial threshold is set to 1.0. For numerical stability, we use ϵ = 10−8 in both the Barzilai--Borwein-style prediction coefficient and the clipping operation, and clamp the prediction coefficient to the range [−10, 10]. D. Attack Settings and Evaluation Metrics Attack Models.: To evaluate robustness under adversarial environments, we consider several representative Byzantine attack models commonly used in federated and decentralized learning. The Gaussian noise attack perturbs transmitted directions with random noise and serves as a basic non-strategic baseline[2]. The sign-flipping attack reverses the direction of model updates to maximally oppose the optimization objective and is widely used to simulate strong adversarial behavior[3]. We also consider a scaling attack, where malicious clients amplify their transmitted directions by a large factor in order to dominate neighborhood aggregation[37]. In addition, we include two omniscient attack variants, namely the min-max attack and the min-sum attack, in which Byzantine clients craft adversarial messages using the honest updates observed in the same round[38]. To further assess defense-aware adversaries, we additionally consider an adaptive attack. Instead of sending an obviously abnormal direction, each Byzantine client crafts a poisoned message around the predicted neighborhood trend and then applies a small perturbation in the opposite direction of the honest neighborhood mean. This makes the corrupted message appear more plausible while still biasing the aggregated neighborhood signal. All attacks are implemented in a decentralized message-poisoning manner, where Byzantine clients independently corrupt the directions sent to their neighbors. Unless otherwise specified, we set 3 out of 10 clients as Byzantine attackers, corresponding to a Byzantine ratio of 30%. Evaluation protocol and metrics.: Our primary objective is to evaluate the robustness of the personalized models held by honest participants. Since Byzantine attackers may maintain arbitrary or corrupted model parameters, including them in the evaluation would obscure the true performance of the system. Therefore, we report the Average Test Accuracy of Honest Clients. Let H denote the set of honest clients. The metric is defined as: 1 X Acc(wit ; Dtest ), (25) Accthonest = |H| i∈H
12
where Acc(wit ; Dtest ) represents the classification accuracy of honest client i’s model on the held-out test set. This metric rigorously measures whether the collaborative defense mechanism successfully protects the utility of honest clients against adversarial influence. E. Main Results TABLE I C OMPARISON OF AVERAGE HONEST PERSONALIZED ACCURACY UNDER DIFFERENT VALUES OF λ IN THE CLEAN SETTING . Aggregation Mean Median
λ=0 0.8185 0.8185
λ = 0.005 0.8167 0.8218
λ = 0.01 0.8206 0.8202
λ = 0.05 0.8245 0.8231
1) Clean Performance: We first evaluate the proposed method under the clean setting, where no malicious clients are present, in order to examine whether the introduced neighborhood correction mechanism harms normal personalized training. Fig.1 shows the clean training curves under meanbased and median-based aggregation with different values of λ. In our method, mean-based aggregation and median-based aggregation are not treated as two separate methods. Instead, they are two alternative implementations of the neighborhood aggregation component used to obtain the aggregated neighborhood direction. All settings exhibit stable convergence throughout the training process, indicating that the proposed method remains well-behaved under clean conditions. The average honest personalized accuracy achieved by different λ values is highly comparable under both aggregation rules. In particular, the final performance gap among λ = 0, λ = 0.005, λ = 0.01, and λ = 0.05 is relatively small, which suggests that introducing the neighborhood correction term does not noticeably degrade the quality of local personalized models. A moderate neighborhood regularization strength, such as λ = 0.05, shows a slight advantage in the middle and later stages of training, although the gain remains limited overall. 2) Robustness under Byzantine Attacks: We next evaluate the proposed method under multiple Byzantine attack settings in order to examine whether it can maintain stable personalized performance when neighborhood information is corrupted by malicious clients. Specifically, we consider three standard perturbation-based attacks, namely sign-flip, noise, and scaling, two stronger optimization-based attacks, namely minmax and min-sum, and one moderate defense-aware adaptive attack. Table II reports the final personalized performance under different attack settings, where we present both the average accuracy over honest clients and the average accuracy over all clients. Several observations can be made. First, the proposed method maintains consistently high personalized accuracy under all considered attacks, indicating strong robustness against a range of Byzantine perturbations. Second, the gap between the honest-client accuracy and the overall average accuracy remains small across all attack settings, suggesting that the learned personalized models of honest clients are not severely degraded by the presence of malicious neighbors. Third,
among the considered attacks, the adaptive, sign-flip, and minmax attacks lead to relatively larger performance drops, while the degradation remains limited overall. By contrast, the results under noise and min-sum are very close to, or slightly above, the clean reference. We attribute these small differences to the stochasticity of decentralized training rather than a genuine performance gain caused by attacks. Fig.2 further shows the training curves of average honest personalized accuracy under different Byzantine attacks. It can be observed that all attack settings still exhibit stable convergence throughout the communication rounds, and no attack causes catastrophic collapse or severe training instability. To provide a more direct comparison, Fig. 3 presents the final average honest personalized accuracy under different attack settings, with the clean result shown as a dashed reference line. We observe that the final performance under all attacks remains close to the clean reference, which further confirms the robustness of the proposed method. Although adaptive attack produces one of the largest performance drops, its impact is still moderate, indicating that the proposed defense remains effective even against a defense-aware adversary. F. Ablation and Sensitivity Analysis To further understand the contribution of the key design components in the proposed method, we conduct an ablation study by comparing the full method with two reduced variants, namely w/o Prediction and w/o Clipping. The former removes the neighborhood direction prediction module, while the latter removes the residual clipping mechanism after predictionbased correction. Through this study, we aim to examine whether these two components are both beneficial to robustness under Byzantine attacks. Table III summarizes the final results of different ablated variants under representative Byzantine attacks, and Fig.4 further presents the corresponding final honest personalized accuracy. Several observations can be made. First, the full method achieves the most favorable overall performance across the considered attacks, indicating that the complete prediction-and-correction method provides the strongest robustness among the compared variants. Second, the advantage of the full method is particularly clear under the min-sum attack, where removing either prediction or clipping leads to a more noticeable performance drop. This suggests that both components contribute to suppressing harmful neighborhood deviations under stronger optimization-based attacks. Third, under the adaptive and min-max attacks, the two reduced variants perform similarly and remain slightly below the full method. Fig. 5 shows the training curves of average honest personalized accuracy for the three variants under min-sum, adaptive, and min-max attacks. We observe that all variants remain stable during training, but the full method generally maintains a more favorable convergence trajectory, especially under min-sum. This result is consistent with the final-performance comparison and further suggests that the complete method can better preserve the quality of honest clients’ personalized models throughout training.
13
Fig. 1. Comparison of average honest personalized performance under different values of λ in the clean setting TABLE II C OMPARISON OF FINAL HONEST PERSONALIZED PERFORMANCE UNDER DIFFERENT B YZANTINE ATTACK SETTINGS . Datasets
Defense Method
Cifar10
Mean Median Trimmed-Mean DisPFL Our
sign_flip Honest All 0.8172 0.8195 0.8172 0.8195 0.8172 0.8195 0.0861 0.1001 0.8172 0.8195
noise Honest All 0.8215 0.8215 0.8215 0.8215 0.8215 0.8215 0.1184 0.0972 0.8215 0.8215
scaling Honest All 0.8192 0.8204 0.8192 0.8204 0.8192 0.8204 0.0861 0.1001 0.8192 0.8204
Fig. 2. Training curves of average honest personalized accuracy under different Byzantine attacks.
min_max Honest All 0.8178 0.8198 0.8178 0.8198 0.8178 0.8198 0.0861 0.1001 0.8178 0.8198
min_sum Honest All 0.8236 0.8231 0.8236 0.8231 0.8236 0.8231 0.0861 0.1001 0.8236 0.8231
adaptive Honest All 0.8167 0.8194 0.8167 0.8194 0.8167 0.8194 0.0861 0.1001 0.8167 0.8194
TABLE III A BLATION RESULTS OF KEY DESIGN COMPONENTS UNDER DIFFERENT B YZANTINE ATTACKS . attack_type adaptive
min_max
min_sum
variant Full Method w/o Clipping w/o Prediction Full Method w/o Clipping w/o Prediction Full Method w/o Clipping w/o Prediction
Honest 0.8167 0.8196 0.8196 0.8178 0.8162 0.8162 0.8236 0.8153 0.8153
All 0.8194 0.8199 0.8199 0.8198 0.8179 0.8179 0.8231 0.818 0.818
Fig. 4. Comparison of final honest personalized accuracy for different ablated variants under representative Byzantine attacks.
Fig. 3. Comparison of final honest personalized accuracy under different Byzantine attacks, with the clean reference shown as a dashed line..
We further investigate the sensitivity of the proposed method to the neighborhood collaboration strength controlled by λ. This parameter determines the influence of the corrected neighborhood signal on the final personalized update, and
therefore plays an important role in balancing local training and neighborhood collaboration under Byzantine attacks. Table IV and Fig. IV show the final performance of the proposed method under different values of λ for two representative attack settings, namely sign-flip and min-sum. Several observations can be made. First, when λ = 0, the
14
Fig. 5. Training curves of average honest personalized accuracy for different ablated variants under representative Byzantine attacks.
TABLE IV S ENSITIVITY OF THE PROPOSED METHOD TO DIFFERENT VALUES OF λ UNDER REPRESENTATIVE B YZANTINE ATTACKS . attack_type min_sum
sign_flip
λ 0 0.005 0.01 0.05 0 0.005 0.01 0.05
Honest 0.8144 0.8165 0.8236 0.8229 0.8144 0.8172 0.8172 0.8145
All 0.8185 0.8179 0.8231 0.824 0.8185 0.8193 0.8195 0.8166
Fig. 6. Sensitivity of the proposed method to neighborhood collaboration strength under representative Byzantine attacks.
method reduces to a purely local update without neighborhood correction, and the resulting robustness is relatively limited. As λ increases from 0 to a small or moderate range, the final average honest personalized accuracy improves, indicating that incorporating neighborhood information in a controlled manner is beneficial to robustness. Second, the best performance is achieved at a moderate value of λ, rather than at the largest tested value. In particular, for both sign-flip and min-sum attacks, the performance improves when λ increases from 0 to 0.01, while the gain becomes marginal or slightly decreases when λ is further increased to 0.05. Overall, these results indicate that the proposed method is not overly sensitive to λ within a moderate range, while an appropriate neighborhood collaboration strength is important for achieving the best robustness. In our experiments, λ = 0.01 provides a favorable trade-off between utilizing neighborhood information and avoiding excessive correction.
VII. C ONCLUSIONS In this paper, we proposed R-DPFL, a robust decentralized personalized federated learning method for Byzantine environments. The key motivation is that personalization and robustness can conflict in decentralized learning: honest clients may naturally generate different updates due to heterogeneous data and local objectives, making malicious perturbations harder to distinguish from normal personalized differences. Therefore, instead of forcing neighboring clients to learn the same model, our R-DPFL keeps each client model personalized and uses neighbor information only as a controlled correction to the local update. Specifically, each client first constructs a local descent direction from its private data and exchanges this direction with its neighbors. The received neighbor directions are robustly aggregated into an initial neighborhood update. R-DPFL then predicts what this aggregated neighborhood update should be based on historical local model changes and historical aggregated neighborhood updates. Finally, the difference between the current aggregated update and the predicted update is adaptively clipped before being added to the local descent direction. Theoretically, we established a bounded personalized stationarity guarantee, showing that honest clients can maintain stable descent on their own local objectives under Byzantine neighbor perturbations without enforcing consensus among neighboring models. Empirically, experiments under heterogeneous decentralized settings and multiple Byzantine attacks demonstrated that R-DPFL achieves strong robustness and stable personalized performance. ACKNOWLEDGMENT This work is supported by the Science and Technology Development Fund of Macao (FDCT) (Project #0015/2023/RIA1), Queensland State Department of Environment and Science under the Quantum Challenges 2032 Program (Project #Q2032001). The corresponding author is Hong Shen. R EFERENCES [1] B. McMahan et al., “Communication-efficient learning of deep networks from decentralized data,” AISTATS, 2017. [2] P. Blanchard et al., “Machine learning with adversaries: Byzantine tolerant gradient descent,” in NeurIPS, 2017. [3] D. Yin et al., “Byzantine-robust distributed learning,” ICML, 2018.
15
[4] N. Nguyen et al., “Balance: Byzantine-robust decentralized learning,” IEEE Transactions on Information Theory, 2023. [5] E. Jeong and M. Kountouris, “Personalized decentralized federated learning with knowledge distillation,” arXiv preprint arXiv:2302.12156, 2023. [6] T. Sun, D. Li, and B. Wang, “Decentralized federated averaging,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022. [7] M. Fang, Z. Zhang, Hairi, P. Khanduri, J. Liu, S. Lu, Y. Liu, and N. Gong, “Byzantine-robust decentralized federated learning,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024, pp. 2874–2888. [8] M. Raynal, D. Pasquini, and C. Troncoso, “Can decentralized learning be more robust than federated learning?” arXiv preprint arXiv:2303.03829, 2023. [9] D. Pasquini, M. Raynal, and C. Troncoso, “On the (in) security of peerto-peer decentralized machine learning,” in 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2023, pp. 418–436. [10] A. Imteaj, U. Thakker, S. Wang, J. Li, and M. H. Amini, “A survey on federated learning for resource-constrained iot devices,” IEEE Internet of Things Journal, vol. 9, no. 1, pp. 1–24, 2021. [11] D. C. Nguyen, Q.-V. Pham, P. N. Pathirana, M. Ding, A. Seneviratne, Z. Lin, O. Dobre, and W.-J. Hwang, “Federated learning for smart healthcare: A survey,” ACM Computing Surveys (Csur), vol. 55, no. 3, pp. 1–37, 2022. [12] D. Javeed, M. S. Saeed, P. Kumar, A. Jolfaei, S. Islam, and A. K. M. N. Islam, “Federated learning-based personalized recommendation systems: An overview on security and privacy challenges,” IEEE Transactions on Consumer Electronics, vol. 70, no. 1, pp. 2618–2627, 2024. [13] R. Dai, L. Shen, F. He, X. Tian, and D. Tao, “Dispfl: Towards communication-efficient personalized federated learning via decentralized sparse training,” in International Conference on Machine Learning (ICML), 2022, pp. 4587–4604. [14] Y. Liu, Y. Shi, Q. Li, B. Wu, X. Wang, and L. Shen, “Decentralized directed collaboration for personalized federated learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 23 168–23 178. [15] C. Li, G. Li, and P. K. Varshney, “Decentralized federated learning via mutual knowledge transfer,” IEEE Internet of Things Journal, vol. 9, no. 2, pp. 1136–1147, 2021. [16] E. Jeong and M. Kountouris, “Personalized decentralized federated learning with knowledge distillation,” in ICC 2023-IEEE International Conference on Communications. IEEE, 2023, pp. 1982–1987. [17] A. Zhang, P. Zhao, W. Lu, and G. Zhang, “Personalized decentralized federated learning: A privacy-enhanced and byzantine-resilient approach,” IEEE Transactions on Computational Social Systems, 2025. [18] J. Verbraeken, M. de Vos, and J. Pouwelse, “Bristle: Decentralized federated learning in byzantine, non-iid environments,” arXiv preprint arXiv:2110.11006, 2021. [19] C. Yang and J. Ghaderi, “Byzantine-robust decentralized learning via remove-then-clip aggregation,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 19, 2024, pp. 21 735–21 743. [20] E. Gabrielli, G. Pica, and G. Tolomei, “A survey on decentralized federated learning,” arXiv preprint arXiv:2308.04604, 2023. [21] O. Marfoq, G. Neglia, A. Bellet, L. Kameni, and R. Vidal, “Federated multi-task learning under a mixture of distributions,” Advances in neural information processing systems, vol. 34, pp. 15 434–15 447, 2021. [22] M. Arivazhagan, V. Aggarwal, A. K. Singh, and S. Choudhary, “Federated learning with personalization layers,” arXiv preprint arXiv:1912.00818, 2019. [Online]. Available: https://arxiv.org/abs/1912. 00818 [23] J. Oh, S. Kim et al., “Fedbabu: Towards enhanced representation for federated image classification,” in ICLR Workshop on New Frontiers in Federated Learning, 2022. [Online]. Available: https: //arxiv.org/abs/2106.06042 [24] V. Smith, C.-K. Chiang, M. Sanjabi, and A. Talwalkar, “Federated multi-task learning,” arXiv preprint arXiv:1705.10467, 2017. [Online]. Available: https://arxiv.org/abs/1705.10467 [25] C. T. Dinh, N. H. Tran, and T. D. Nguyen, “Personalized federated learning with moreau envelopes,” NeurIPS, 2020. [26] D. Li and J. Wang, “Fedmd: Heterogenous federated learning via model distillation,” arXiv preprint arXiv:1910.03581, 2019. [Online]. Available: https://arxiv.org/abs/1910.03581 [27] T. Lin et al., “Ensemble distillation for robust model fusion in federated learning (feddf),” in NeurIPS, 2020. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/ 18df51b97ccd68128e994804f3eccc87-Paper.pdf
[28] C. Chen, Z. Chen, Y. Zhou, and B. Kailkhura, “Fedcluster: Boosting the convergence of federated learning via cluster-cycling,” in 2020 IEEE International Conference on Big Data (Big Data), 2020, pp. 5017–5026. [29] A. Sadiev, E. Borodich, A. Beznosikov, D. Dvinskikh, S. Chezhegov, R. Tappenden, M. Takáč, and A. Gasnikov, “Decentralized personalized federated learning: Lower bounds and optimal algorithm for all personalization modes,” EURO Journal on Computational Optimization, vol. 10, p. 100041, 2022. [30] J. Barzilai and J. M. Borwein, “Two-point step size gradient methods,” IMA journal of numerical analysis, vol. 8, no. 1, pp. 141–148, 1988. [31] X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the convergence of fedavg on non-iid data,” in International Conference on Learning Representations, 2020. [32] A. Koloskova, N. Loizou, S. Boreiri, M. Jaggi, and S. U. Stich, “A unified theory of decentralized sgd with changing topology and local updates,” in International Conference on Machine Learning, 2020. [33] L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization methods for largescale machine learning,” SIAM Review, vol. 60, no. 2, pp. 223–311, 2018. [34] M. Gormley, “Lecture 6: Optimality conditions,” https://www.cs. cmu.edu/~mgormley/courses/10425/slides/lecture6-optimality.pdf, 2023, cMU Course 10-425, Accessed April 29, 2025. [35] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” in Proceedings of Machine Learning and Systems, vol. 2, 2020, pp. 429–450. [36] S. U. Stich, “Local sgd converges fast and communicates little,” in International Conference on Learning Representations, 2019. [37] G. Baruch, M. Baruch, and Y. Goldberg, “A little is enough: Circumventing defenses for distributed learning,” in NeurIPS, 2019. [38] M. Fang, X. Cao, J. Jia, and N. Z. Gong, “Local model poisoning attacks to byzantine-robust federated learning,” in USENIX Security, 2020.