Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

cryptography, security, privacy, cybersecurity

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Mingyuan Fan

Cen Chen

East China Normal University Shanghai, China [email protected]

arXiv:2606.22782v1 [cs.LG] 22 Jun 2026

Abstract The proliferation of IoT devices has fueled distributed edge systems to collect vast amounts of sensitive data, creating fertile ground for on-device machine learning applications. While federated learning (FL) mitigates privacy concerns by exchanging model parameters instead of raw data, we identify a critical blind spot in current research. We examine the most commonly used personalized federated learning (PFL) methods, which allow clients to maintain private, personalized models to address data heterogeneity across clients. Through systematic analysis, we reveal that PFL methods exhibit heightened vulnerability to transfer-based adversarial attacks compared to centralized learning paradigms. Wherein, malicious clients can exploit local model knowledge to craft adversarial examples that can compromise peer clients’ personalized models. We establish this vulnerability through both theoretical analysis and empirical evaluation across multiple benchmark datasets, demonstrating significant accuracy drops across various PFL methods. To address this challenge, we propose a defense framework combining stochastic input noise, input-scaled trace regularization, and parameter sensitivity maximization to improve FL’s robustness. Our findings establish the first systematic study of adversarial threats in PFL systems, providing both diagnostic tools and practical countermeasures. CCS Concepts: • Security and privacy; • Computing methodologies; Keywords: Internet of Things, Robustness, Neurual Network, Federated Learning, Adversarial Attack

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. Conference acronym ’XX, Woodstock, NY © 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-XXXX-X/2018/06 https://doi.org/XXXXXXX.XXXXXXX

ACM Reference Format: Mingyuan Fan and Cen Chen. 2018. Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense CoDesign. In Proceedings of Make sure to enter the correct conference title from your rights confirmation email (Conference acronym ’XX). ACM, New York, NY, USA, 17 pages. https://doi.org/XXXXXXX.XXXXXXX

Introduction

The explosive expansion of Internet of Things (IoT) has spawned distributed networks of edge devices that amass vast volumes of sensitive data [11, 15, 17], yet such data remains largely confined to local nodes due to growing privacy regulations [31]. While such data holds immense potential for on-device machine learning applications like health monitoring and intrusion detection, traditional centralized training paradigms requiring raw data centralization causes severe privacy compromises [15, 38]. This fundamental tension has driven the adoption of federated learning (FL) in edge computing systems [4, 42], which enables collaborative model training through parameter exchange while maintaining data localization across distributed devices. In canonical FL, clients (edge devices) jointly train a shared model through FedAvg’s synchronized client-server protocol [25]: clients download the global model, refine it using local datasets, and upload parameter updates for server-side aggregation. This one-model-fits-all approach, however, suffers significant performance degradation when confronted with the intrinsic data heterogeneity of real-world edge environments [22, 31]. To address this, recent efforts have shifted toward personalized federated learning (PFL) [5, 18, 20], which empowers edge devices to cultivate private, personalized models that align with their distinctive data characteristics, while preserving the collaborative benefits of federated knowledge exchange. Beyond personalization efforts, a critical orthogonal research direction is to recognize FL’s intrinsic security challenges. In detail, the distributed nature of FL brings broad attack surfaces, as current systems lack mutual integrity verification mechanisms between servers and clients or among clients themselves, enabling potential adversarial behavior from any participant. Existing studies have identified multiple attack vectors, including model poisoning [2], backdoor injection [30, 35], and privacy breaches [43]. These threats have spurred countermeasures like secure aggregation [8, 40] and differential privacy [26] to fortify FL systems. Systematic

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

exploration of novel attack surfaces remains crucial for both diagnosing FL’s vulnerabilities and designing trust-aware frameworks that balance performance with robustness. Our first contribution is the unveiling of a critical yet unexplored attack surface: the vulnerability of PFL against adversarial attacks. To the best of our knowledge, this constitutes the first investigation at the intersection of FL and adversarial attacks. Adversarial attacks [12, 24] craft imperceptibly perturbed inputs, known as adversarial examples, to mislead target models into making incorrect classifications. We study adversarial attacks in PFL and demonstrate that malicious PFL clients can weaponize local model knowledge to generate transferable adversarial examples capable of compromising peer clients’ personalized models. Our attack is similar to traditional transfer-based attacks [6, 10] where an attacker trains a proxy model to produce adversarial examples against a black-box target model. However, traditional transfer-based attacks primarily assume a pure black-box scenario, characterized by distinct model architectures and parameters between the proxy and target models. In contrast, our focus is on a distributed PFL scenario where clients’ personalized models share identical model architectures but maintain different parameters, i.e., a gray-box scenario. Our second contribution is a comprehensive empirical analysis. Through extensive experimentation, we reveal significant adversarial vulnerability in PFL implementations and provide useful insights for practitioners, particularly contrasting the robustness profiles of PFL models against centrally trained counterparts. We identifiy several fundamental divergences, suggesting distinct vulnerability patterns that warrant further investigation. The third contribution is a principled theoretical framework. We uncover five determinants of attack transferability in PFL systems, including model sensitivity, loss landscape geometry, distribution alignment, fitness difference, and parameter sensitivity. Empirical validation confirms that PFL models exhibit worse values across these compared to centralized models, explaining their heightened susceptibility. The fourth contribution is the proposal of a robustnessenhanced PFL framework. Leveraging our theoretical insight, we present three novel yet effective techniques, namely stochastic input noise augmentation, input-scaled trace regularization and parameter sensitivity minimization, to respectively to mitigate model sensitivity, loss landscape geometry, and parameter sensitivity. These techniques are plugand-play components, making our framework applicable to any PFL method. Extensive experiments demonstrate our framework consistently enhances robustness against transfer-based attacks across diverse PFL implementations.

Background

We provide the necessary background here. Table 1 summarizes the mathematical notation for the reader’s convenience.

Trovato et al.

Table 1. Notation table. Symbol

Definition

𝑥, 𝑦 𝑧 𝑝𝑖 L (𝑖 ) 𝐷𝑖 , 𝑃𝑋𝑌 𝜃𝑔 𝜃𝑖 𝜂 𝛿 𝜖 U, 𝑆𝑡 𝑇 𝜏 𝛼 𝐾 𝑊

Sample and its ground-truth label Feature embedding from feature extraction layers Prediction probability of the 𝑖-th class Loss function Client 𝑖’s local dataset and data distribution Parameters of the global shared model Personalized model parameters of client 𝑖 Learning rate for client models Adversarial perturbation Upper bound on the size of adversarial perturbation Client set and selected clients in round 𝑡 Total number of communication rounds Local epochs Step size for updating adversarial perturbation Number of attack iterations Weights of the classification head

2.1

Personalized Federated Learning

Federated learning. Consider a FL system with 𝑚 clients and a central server. Let U = {1, 2, · · · , 𝑚} denote the client set, where each client holds a private dataset 𝐷𝑖 drawn from 𝑖 , with X as the input space its local data distribution 𝑃𝑋𝑌 and Y as the 𝐾-class label space. Given a loss function L : Y × Y → R, the canonical FL objective [25] aims to learn a global model 𝐹 (𝑥; 𝜃𝑔 ) with parameters 𝜃𝑔 , formulated as: ∑︁ min E (𝑥,𝑦)∼𝐷𝑖 L (𝐹 (𝑥; 𝜃𝑔 ), 𝑦) , (1) 𝜃𝑔

𝑖∈U

where each term corresponds to a client’s local empirical risk. FedAvg [25] optimizes Equation 1 through alternating iterations of: • Local training. During communication round 𝑡, the server selects a subset 𝑆𝑡 ∈ U of clients to participate. Each selected client 𝑖 ∈ 𝑆𝑡 synchronizes its local model 𝜃𝑔𝑖 with the global model, i.e., 𝜃𝑔𝑖 ← 𝜃𝑔 , then performs 𝜏 epochs of SGD updates on 𝐷𝑖 : 𝜃𝑔𝑖 = 𝜃𝑔𝑖 − 𝜂∇𝜃𝑔𝑖 L (𝐹 (𝑥; 𝜃𝑔𝑖 ), 𝑦), (𝑥, 𝑦) ∼ 𝐷𝑖 ,

(2)

where 𝜂 is the learning rate. • Aggregation. The server averages the locallyÍ trained models to form the new global model: 𝜃𝑔 = |𝑆1𝑡 | 𝑖 ∈𝑆𝑡 𝜃𝑔𝑖 . FedAvg, however, suffers from performance degradation un𝑖 der data heterogeneity across clients [20, 22, 31], i.e., 𝑃𝑋𝑌 ≠ 𝑗 𝑃𝑋𝑌 for 𝑖 ≠ 𝑗. As shown in Figure 1, to mitigate this, PFL allows each client to maintain a private personalized model locally [5, 19]. For clarity, the local model refers to the global model downloaded for temporary local training (i.e., 𝜃𝑔𝑖 ), distinct from personalized models. Let 𝜃 𝑖 parameterize client 𝑖’s personalized model.

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Figure 1. Overview. In the training phase, ➀ the server broadcasts the global model to clients. ➁ each client trains the global model using their local dataset. ➂ clients train their personalized models, which are regularized to ensure similarity to the global model through either soft or hard constraints. ➃ the updated global models are then uploaded to the server for aggregation to form a new global model. Notice that in some PFL methods, ➁ may be omitted, or the order of ➁ and ➂ may be swapped. Moreover, in canonical FL, personalized models (➂) do not exist. In the inference phase, we implement adversarial attacks to compromise clients’ personalized models. There are two types of attacks: server-initiated and client-initiated. Both types follow a two-step process: ➀ the given data is fed into the model to generate adversarial perturbations, and ➁ these perturbations are added to the original data to produce adversarial samples to fool the target client’s personalized model. The key difference between the two types of attacks lies in the model used to generate the perturbations: server-initiated attacks use the global model, while client-initiated attacks use the personalized model. Full model-sharing methods. Full model-sharing methods permit all elements of 𝜃 𝑖 to differ from 𝜃𝑔𝑖 , regularized by a coupling term R: ∑︁ min E (𝑥,𝑦)∼𝐷𝑖 [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) + R (𝜃 𝑖 , 𝜃𝑔𝑖 )]. (3) 𝜃𝑖 ,𝑖 ∈ U

𝑖∈U

In Equation 3, L promotes local adaptation, while R preserves global knowledge through model alignment. For example, FedProx [19] implements R as 𝐿2 -norm penalty ||𝜃 𝑖 −𝜃𝑔𝑖 || 22 to constrain 𝜃 𝑖 to remain close to 𝜃𝑔𝑖 . Ditto [18] first trains 𝜃𝑔𝑖 on 𝐷𝑖 , followed by the training of 𝜃 𝑖 in the same manner as FedProx. SCAFFOLD [14] introduces control variables to calibrate updates to 𝜃 𝑖 . Partial model-sharing methods. Partial model-sharing methods instead enforce hard parameter sharing for a subset of model parameters, while allowing others to specialize: 𝑚 ∑︁ min E (𝑥,𝑦)∼𝐷𝑖 [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)], 𝜃𝑖 ,𝑖 ∈ U, (4) 𝑖=1 𝑠.𝑡 ., 𝜃𝑔𝑖 [𝑘] = 𝜃 𝑖 [𝑘], 𝑘 ∈ Λ, 𝑖 ∈ U. where Λ indexes the shared parameters. Partial model-sharing methods generally yield better performance by leveraging prior knowledge to identify which parameters are more likely to capture transferable representations for client sharing, while allowing others to vary freely for capturing clientspecific patterns. FedBN [20] retains batch normalization layers locally while aggregating other layers through FedAvg. Many later approaches adopt classification heads as personalization parameters. For instance, FedRep [5] and

FedBABU [27] synchronize feature extractors but keep classification heads client-specific, with FedPAC [37] further improving these heads through weighted ensemble learning. Moreover, FedAS [39] aligns feature representations between local and personalized models before head training, while GPFL [41] explicitly disentangles feature representations into distinct global and local representations further. FedCAC [34] dynamically assigns Λ via parameter sensitivity analysis, privatizing parameters that exhibit strong dataset-specific characteristics. 2.2

Adversarial Attack

Modern edge computing systems increasingly deploy neural networks to enable intelligent functionalities across mobile applications [11, 22, 28]. However, these systems remain vulnerable to adversarial attacks, in which human-imperceptible perturbations are crafted to deceive models into making erroneous predictions. Formally, given an input 𝑥 with groundtruth label 𝑦, the adversarial perturbation 𝛿 for 𝑥 can be obtained by solving: 𝛿 = arg max L (𝐹 (𝑥 + 𝛿; 𝜃 ), 𝑦), 𝑠.𝑡 ., ||𝛿 || ∞ ≤ 𝜖. 𝛿

A higher loss value indicates stronger effectiveness in misleading the model, and 𝜖 bounds the magnitude of 𝛿 to ensure imperceptibility. Early adversarial attacks focus on white-box scenarios assuming full model access. PGD [24], considered a benchmark method, solves Equation 5 through iterative projected

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Trovato et al.

gradient ascent with random initialization over 𝐾 iterations: 𝛿 = Π [ −𝜖,𝜖 ] {𝛿 + 𝛼 · sign (∇𝛿 L (𝐹 (𝑥 + 𝛿; 𝜃 ), 𝑦)}) ,

(5)

where Π𝜖 projects 𝛿 to the feasible 𝑙 ∞ -norm ball and 𝛼 is the step size. Notably, PGD generalizes two fundamental attacks: basic iterative method [16] with deterministic zero initialization, and fast gradient sign method [12] with 𝐾 = 1. These methods require white-box access to the target model to derive input gradients, which is often impractical. Transfer-based attacks. To relax the white-box assumption, transfer-based attacks leverage adversarial examples generated from a proxy model to compromise black-box target models. The attack effectiveness depends on the transferability of adversarial examples across distinct models. The vanilla transfer-based attack applies PGD on the proxy model to produce adversarial examples, but their transferability is often limited, prompting the development of various transferability-enhancing techniques. These techniques commonly involve modifications to PGD and can be categorized into three main types: optimization-based methods, input regularization methods, and model-based methods. Optimization-based methods employ better optimization methods to escape suboptimal local maxima. Examples include integrating momentum terms [6] or calibrating gradient direction [32] to stabilize update directions. Input regularization methods introduce stochastic transformations to the input during perturbation generation (e.g., random resizing [36], scaling [23], translation [7], and mask operation [9]). Model-based methods alter gradient propagation rules [33] or design proxy architectures that better approximate potential target models [10, 21]. Takeaway for Section 3: 1. PFL methods are highly susceptible to adversarial attacks, with significant accuracy drops observed across various datasets and models. The vulnerability is particularly pronounced in higher-performing models, highlighting an inherent accuracy-vulnerability trade-off. 2. Partial-sharing PFL methods demonstrate better adversarial robustness compared to full-sharing methods at similar accuracy levels. 3. In PFL, the transferability of adversarial examples is independent of the proxy model’s accuracy, contrasting with centralized learning, where higher proxy accuracy correlates with better attack transferability. 4. Server-initiated attacks are less effective against certain partial-sharing PFL methods because clients do not upload their classification heads. 5. Adversarial vulnerability in PFL increases with training rounds, perturbation budgets, and attack iterations. Advanced transfer-based attacks show only marginal benefits in PFL compared to centralized training.

Adversarial Attack in PFL

3.1

Attack Scenario

We investigate a scenario in which multiple clients employ PFL methods to train their personalized models. Our attack occurs during inference after PFL training, operating under the assumption that the attacker can directly interact with the target victim model (e.g., via black-box API queries or physical device access) but cannot access its internal parameters or training data. For instance, consider a distributed IoT surveillance network where edge devices (security cameras) train personalized intrusion detection models using local visual data. An attacker could exploit one device’s model to craft adversarial examples to undermine models deployed on other devices. Moreover, we highlight that both the server and clients faithfully follow PFL training protocols, i.e., without malicious training-time behavior. As shown in Figure 1, in this post-training attack regime, we identify two distinct attacks: • Client-initiated attacks: Malicious clients craft adversarial examples using their local personalized models to produce adversarial examples against other clients’ personalized models. • Server-initiated attacks: The server generates adversarial examples through the global model to target clients’ personalized models. Evaluation criteria. Let 𝛿 = A (𝑥, 𝑦, 𝜃 ) denote the perturbation generated by attack method A (e.g., PGD) for input (𝑥, 𝑦) and model parameters 𝜃 . We define accuracy drop𝑖→𝑗 (AD𝑖→𝑗 ) to quantify the attack effectiveness from client 𝑖 to client 𝑗: Í (𝑥𝑘 ,𝑦𝑘 ) ∈𝐷 𝑗 [I(𝐹 (𝑥𝑘 ; 𝜃 𝑗 ) = 𝑦𝑘 ) − I(𝐹 (𝑥𝑘 + 𝛿𝑘 ; 𝜃 𝑗 ) = 𝑦𝑘 )] , |𝐷 𝑗 | where 𝛿𝑘 = A (𝑥𝑘 , 𝑦𝑘 , 𝜃 𝑖 ) and I represents indicator function. This metric similarly applies to server-initiated attacks by substituting 𝜃 𝑖 with 𝜃𝑔 . This metric is evaluated in the victim’s test set 𝐷 𝑗 to avoid artificial inflation from attackerchosen inputs that might lie outside the model’s operational domain. The system vulnerability of a PFL method is quantified through mean cross-client AD: ∑︁ 1 𝐴𝐷𝑖→𝑗 . (|U| − 1) 2 𝑖,𝑗 ∈ U,𝑖≠𝑗

We explicitly exclude 𝑖 = 𝑗 cases (white-box attacks) based on the practical constraint that attackers cannot access their target models’ parameters. 3.2

Setup

FL configurations. We utilize ResNet-10 and three benchmark datasets, including CIFAR-10, CIFAR-100, and GTSRB, for evaluation. The evaluation results for other models can be found in Appendix B. We simulate real-world statistical heterogeneity across clients using Dirichlet distribution

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

FedAS FedBABU FedBN FedCAC FedProx FedRep GPFL SCAFFOLD

Accuracy Drop

60 50 40 30

120

PFL Method

FedAS FedBABU FedBN FedCAC FedProx FedRep GPFL SCAFFOLD

40 35 30 25 20

PFL Method

100

Accuracy Drop

PFL Method

Accuracy Drop

FedAS FedBABU FedBN FedCAC FedProx FedRep GPFL SCAFFOLD

80 60

Accuracy of Target Model

(a) CIFAR-10

Accuracy of Target Model

100

Accuracy of Target Model

(b) CIFAR-100

Figure 2. Correlation between target model accuracy (%) and AD (%) in three datasets. 35

30 20

FedAS FedBABU FedBN FedCAC FedProx FedRep GPFL SCAFFOLD 30 40

80 FedAS FedBABU FedBN FedCAC FedProx FedRep GPFL SCAFFOLD 15 20 25

20 15

Accuracy of Proxy Model

PFL Method

Accuracy Drop

PFL Method

Accuracy Drop

60 50 40 30

Accuracy of Proxy Model

(a) CIFAR-10

PFL Method

FedAS FedBABU FedBN FedCAC FedProx FedRep GPFL SCAFFOLD 80 85

Accuracy of Proxy Model

(b) CIFAR-100

Figure 3. Correlation between proxy model accuracy (%) and AD (%) in three datasets. Table 2. The effectiveness (average AD) of server-initiated attacks in three datasets. Dataset

FedProx

SCAFFOLD

FedBN

FedRep

FedBABU

GPFL

FedAS

FedCAC

CIFAR-10 CI-FAR100 GTSRB

63.44 33.54 81.87

54.76 33.33 78.01

67.71 30.09 83.74

3.90 3.22 4.81

4.51 1.87 2.40

3.84 3.98 2.34

50.49 30.40 63.75

38.48 25.79 74.89

Accuracy Drop

20 20 10

15 10 5 0

0 40

Accuracy of Proxy Model

Accuracy of Target Model

Figure 4. Centralized learning comparison. Left: AD versus proxy model accuracy with fixed target model. Right: AD versus target model accuracy with fixed proxy model.

𝐷𝑖𝑟 (𝛽) with 𝛽 = 0.5 by default, where each client 𝑖 receives 𝑞𝑐,𝑖 ∼ 𝐷𝑖𝑟 (𝛽) proportion of class 𝑐 samples [41]. This induces both label distribution skew and quantity imbalance across clients, reflecting realistic non-IID scenarios. Our FL system consists of 10 clients with 20% participation rate (|𝑆𝑡 |/|U| = 0.2) over 𝑇 = 50 communication rounds (empirically verified for convergence). All clients employ SGD with a batch size of 32, a learning rate 𝜂 of 0.1, and 𝜏 = 5 local epochs for both local and personalized models. PFL methods. We evaluate eight state-of-the-art PFL methods, including FedProx [19], SCAFFOLD [14], FedBN [20], FedRep [5], FedBABU [27], GPFL [41], FedAS [39], and FedCAC [34]. The first two ones are full model sharing, while the remaining ones are partial model sharing. Attacks. We harness PGD to generate adversarial exam8 ples on the global and personalized models, with 𝜖 = 255

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Accuracy Drop Accuracy Drop

FedProx

Trovato et al.

MaskBlock

PCIFGSM

SCAFFOLD

FedBN

PGD

FedRep

50 25

FedBABU

GPFL

FedAS

FedCAC

50 25 0

Training Round

Accuracy Drop Accuracy Drop

Figure 5. The average AD with different transfer-based attacks across varying training rounds. The black dotted lines indicate the model’s average accuracy (shared on the y-axis).

FedProx

SCAFFOLD

FedBABU

GPFL

FedBN

FedRep

FedAS

FedCAC

50 25

50 25 0

Training Round

Accuracy Drop Accuracy Drop

Figure 6. Impact of perturbation budget 𝜖 on average AD across training rounds using PGD.

FedProx

SCAFFOLD

FedBABU

GPFL

FedBN

FedRep

FedAS

FedCAC

50 25

50 25 0

Training Round

Figure 7. Impact of attack iterations on average AD across training rounds using PGD. and 𝐾 = 10 unless otherwise specified [24]. We will subsequently integrate state-of-the-art transferability-enhancing techniques, including MI [6], PCIFGSM [29], and MaskBlock [9], to study their impact on attack effectiveness. MI stabilizes gradient updates via momentum term to escape local optima, while PCIFGSM adapts gradient computation rules. MaskBlock applies dynamic input masking to regularize the attack process. These methods represent three distinct categories of transferability-enhancing techniques:

optimization-based (MI), model-based (PCIFGSM), and input regularization (MaskBlock). All experiments execute 5 independent trials with different random seeds to ensure statistical significance. 3.3

Evaluation Result

Target model accuracy and attack effectiveness. We first investigate the correlation between the performance of client-specific personalized models and their vulnerability to adversarial attacks. Figure 2 illustrates the per-client model

20 15

PGD MI PCIFGSM MaskBlock

10 5 0

Training Round

(a) Attack Method

100

20 15

4 8 12 16

10 5 0

Training Round

(b) Perturbation Budget

100

Accuracy Drop

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

20 15

5 10 15 20

10 5 0

Training Round

100

Figure 8. Centralized training attack dynamics over different attack methods, perturbation budgets, and attack iterations. X-axis converted via iteration-to-round mapping (iterations / local iterations per round). accuracy plotted against these models’ mean AD (computed by averaging AD𝑖→𝑗 over 𝑖) under client-initiated attacks. Each PFL method has 50 data points, corresponding to five independent experimental trials. We also incorporate linear regression fits to quantify statistical trends. There are two key observations: • Considerable adversarial vulnerability of PFL methods. All personalized models exhibit substantial susceptibility to adversarial attacks, with ≥20%, ≥ 10% and ≥40% AD on CIFAR-10, CIFAR-100, and GTSRB, respectively. Notably, the regression lines reveal a clear positive trend between target model accuracy and adversarial vulnerability, suggesting higher-performing models are more attack-prone, i.e., an accuracy-vulnerability trade-off in PFL. Exceptions arise in specific cases, such as FedAS in CIFAR-10 and all ones in GTSRB, where limited accuracy variance among client models likely obscures measurable trends. • Partial-sharing methods enjoy better accuracyvulnerability trade-off. Partial-sharing PFL methods (FedRep, FedAS, GPFL, FedCAC) demonstrate better adversarial robustness over full-sharing PFL methods at comparable accuracy levels. For instance, in CIFAR10, FedRep achieves 70% ∼ 80% accuracy with AD around 50%, outperforming full-sharing baselines like FedProx (40%∼80% accuracy, 40%∼70% AD). Proxy model accuracy and attack effectiveness. Figure 3 examines the relationship between proxy model accuracy and attack transferability, i.e., the average AD𝑖→𝑗 over target clients 𝑗 versus proxy model accuracy (evaluated on the target clients’ test datasets). We observe a counterintuitive phenomenon: adversarial transferability in PFL is largely independent of proxy model accuracy. That is, AD remains consistent across proxy models with substantially different accuracy. This phenomenon also suggests that any

client can exploit its local personalized model against arbitrary target clients’ personalized models, regardless of how well its own model performs. Actually, one might intuitively expect that higher proxy accuracy correlates with better feature alignment between proxy and target models, thereby enhancing attack transferability. To contextualize this phenomenon, we conduct a centralized learning1 comparison where proxy and target models share identical architectures (ResNet-10) and training configurations (CIFAR-10, SGD with 𝜂=0.1), differing only in initialization: • Fixed target model: We attack a trained target model (with 75% accuracy) using proxy models from different training iterations. • Fixed proxy model: We use a trained proxy model to generate adversarial examples and evaluate their effectiveness in the target model across different training iterations. Results in Figure 4 (over 5 trials) reveal a strong positive correlation between AD and both proxy/target model accuracy in centralized learning. The stark contrast between centralized training and PFL stresses that proxy-accuracyindependent transferability constitutes a distinct vulnerability inherent to PFL. Client-initiated attacks versus server-initiated attacks. Table 2 reports the attack effectiveness of serverinitiated attacks. We see that server-initiated attacks can only achieve very low AD against certain parameter-sharing methods, specifically FedRep, FedBABU, and GPFL. This is because, in these three PFL methods, clients do not upload their classification heads, rendering server’s model ineffective. For other methods, the performance difference between client-initiated attacks and server-initiated attacks is relatively small. Given that server-initiated attacks can be effectively mitigated through partial model-sharing methods (by 1 All client data is pooled together to train a model.

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Trovato et al.

not uploading classification head) and then less threatening overall, subsequent analyses focus on client-initiated attacks.

Assumption 4.1. Client 𝑖’s loss function L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) is 𝐾-Lipschitz continuous over X × Y for some constant 𝐾 ≥ 02 .

Training dynamics and adversarial robustness. Figures 5∼7 analyze how adversarial robustness evolves across training rounds in PFL under various attack configurations, including different attack methods, perturbation budgets, and attack iterations. For comparative analysis, we benchmark against centralized training scenarios (Figure 8), where two ResNet-10 models are trained independently, and the transferability of adversarial examples generated from one model to the other is assessed. There are two observations.

Assumption 4.2. Client 𝑗’s personalized model has converged (𝑗) on 𝑃𝑋𝑌 , i.e., E𝑃 ( 𝑗 ) [∇𝜃 𝑗 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)] = 0.

• Monotonic increase in vulnerability. Accuracy drop grows with training rounds, perturbation budgets, and attack iterations. This trend holds consistently for both PFL and centralized training. • Diminished returns from transfer-based attack methods against PFL methods. While advanced transfer-based attacks, e.g., MaskBlock, improve crossmodel transferability in centralized training by about 8%, their benefits are less pronounced in PFL (≤ 2%).

What makes adversarial attack so effective in PFL?

Theoretical Analysis of Attack Transferability in PFL. Our empirical findings demonstrate that PFL models exhibit heighted vulnerability to transfer-based attacks compared to centralized models. This intriguing phenomenon motivates our investigation into the intrinsic properties of PFL that causes this heighted vulnerability. Consider two clients 𝑖 and (𝑖 ) (𝑗) 𝑗, with data distributions 𝑃𝑋𝑌 , 𝑃𝑋𝑌 , along with personalized model parameters 𝜃 𝑖 and 𝜃 𝑗 . We present a theoretical analysis of adversarial transferability between personalized models, where client 𝑖’s personalized model serves as the proxy and client 𝑗’s personalized model as the target. For (𝑥, 𝑦) with perturbation 𝛿, we expand the loss function via first-order Taylor approximation: L (𝐹 (𝑥 + 𝛿, 𝜃 𝑖 ), 𝑦) = L (𝐹 (𝑥, 𝜃 𝑖 ), 𝑦) + ∇𝑥 L (𝐹 (𝑥, 𝜃 𝑖 ), 𝑦)𝑇 𝛿. The perturbation maximizing loss is given by 𝛼∇𝑥 L (𝐹 (𝑥, 𝜃 𝑖 ), 𝑦), where 𝛼 is a scaling constant ensuring validity of the linear approximation. The adversarial effect of 𝛿 on client 𝑗’s model is then: L (𝐹 (𝑥 + 𝛿, 𝜃 𝑗 ), 𝑦) = L (𝐹 (𝑥, 𝜃 𝑗 ), 𝑦) + 𝛿𝑇 ∇𝑥 L (𝐹 (𝑥, 𝜃 𝑖 ), 𝑦 = L (𝐹 (𝑥, 𝜃 𝑗 ), 𝑦) + 𝛼∇𝑥 L (𝐹 (𝑥, 𝜃 𝑗 ), 𝑦)𝑇 ∇𝑥 L (𝐹 (𝑥, 𝜃 𝑖 ), 𝑦). The above equation reveals that higher gradient inner product implies stronger attack effectiveness. Notably, this conclusion can generalize to arbitrary 𝛿 magnitudes if models maintain gradient direction consistency in any point.

𝑋𝑌

Assumption 4.3. Client 𝑗’s loss function L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) is strongly convex in model parameters 𝜃 𝑗 . Theorem 4.1. (See Appendix A for Proof.) Let 𝜆min be the minimal eigenvalue of ∇𝜃2 𝑗 L (𝐹 (𝑥, 𝜃 𝑗 ), 𝑦). Use Δ to denote 𝜃 𝑖 − 𝜃 𝑗 . The gradient inner product is bounded as follows: ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) ⊤ ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) 1 ≥ ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 22 + ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22 2 2 − 𝐶 ||∇𝑥,𝜃 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22 + O (||Δ|| 2 ), 𝑗 (𝑖 ) (𝑗) 1 where 𝐶 = 𝜆min {𝐾 · 𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ) − E𝑃 (𝑖 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] + 𝑋𝑌 E𝑃 ( 𝑗 ) L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) }. 𝑋𝑌

We employ Wasserstein distance to quantify the distribu(𝑖 ) (𝑗) (𝑖 ) (𝑗) tional divergence between 𝑃𝑋𝑌 and 𝑃𝑋𝑌 , i.e., 𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ). Under Assumptions 4.1∼4.3, the above theorem establishes a lower bound for gradient inner product. Before analyzing Theorem 4.1, we discuss its applicability in PFL context. Assumption 4.1 is generally valid in practice since unbounded Lipschitz constants would imply highly irregular or pathological loss landscapes. Assumption 4.2 is obvious. Assumption 4.3 holds true for linear models with cross-entropy loss. In many PFL methods, gradient differences mainly originate from classification heads, and thus Assumption 4.3 is also likely to hold, at least for those methods that personalize only the classification heads. Furthermore, when the model converges, the loss landscape is often locally convex near the optimum. In Theorem 4.1, the term O (||Δ|| 2 ) when the parameter differences between personalized models are small. Since PFL methods typically introduce regularization terms on parameter differences, either soft (Equation 3) or hard (Equation 4), the parameter differences between personalized models are usually small. Theorem 4.1 reveals several critical factors: • Model sensitivity ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 22 +||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22 : Higher input gradient norms indicate that the model is more sensitive to input perturbations, amplifying the potential for adversarial transferability. • Loss landscape geometry 𝜆min : Intuitively, 𝜆min characterizes the curvature of the loss landscape, specifically along the direction of least resistance. A substantial 𝜆min implies that even the flattest direction exhibits significant curvature, forcing the loss to vary substantially along all directions. This geometric property enhances the target model’s sensitivity to input 2 Formally, we say that a function 𝑓 is 𝐾-Lipschitz continuous if | 𝑓 (𝑥 ) − 1

𝑓 (𝑥 2 ) | ≤ 𝐾 | |𝑥 1 − 𝑥 2 | | 2 for all 𝑥 1 , 𝑥 2 in its domain.

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

2.60

r = 0.503, p = 0.0000

Accuracy Drop

2.55

Loss

2.50 2.45 2.40

40 30

2.35

2.30 0.0010 0.0015 0.0020 0.0025 0.0030

Gradient Magnitude

FedAS FedBABU FedBN FedCAC FedProx FedRep GPFL SCAFFOLD

0.1

0.3

0.5

0.7

0.9

Figure 9. Left: The sum of the input gradient norms on the proxy and target models, along with the loss values of the generated adversarial examples on the target model. Right: The performance of transfer-based attacks under varying degrees of data heterogeneity. Table 3. Comparison of metrics between centralized training and PFL methods. Training Paradigm

Model Sensitivity

Average Eigenvalue

Parameter Sensitivity

Centralized Training PFL methods

0.0078 0.0142

2.984×10−6 9.712×10−6

0.426 0.545

perturbations: minor adversarial perturbations induce pronounced loss changes when 𝜆min is large, thereby enhancing adversarial transferability. • Distribution alignment and model fitness differ ence −E𝑃 (𝑖 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] +E𝑃 ( 𝑗 ) L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) + 𝑋𝑌

𝑋𝑌

(𝑖 ) (𝑗) 𝐾𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ): When the Wasserstein distance between their respective data distributions is small, the corresponding underlying feature distributions are more likely to overlap. Furthermore, if the models achieve comparable performance on their respective datasets (as reflected by their expected loss values), the features learned by the models awill exhibit greater similarity. Together, these factors lead to higher transferability. 2 L (𝐹 (𝑥; 𝜃 ), 𝑦)|| 2 : Smaller • Parameter sensitivity ||∇𝑥,𝜃 𝑗 2 𝑗 values indicate input gradient invariance to parameter changes. Negligible parameter sensitivity (approaching zero) would decouple input gradient behavior from model parameterization. This implies that the regions of high input sensitivity (e.g., adversarial vulnerabilities) remain consistent across different parameter instantiations of the target model. Consequently, adversarial examples crafted on a proxy model by exploiting its input-sensitive regions (high gradient norms) will likely transfer effectively to the target model, as its sensitivity profile remains stable despite parameter changes.

Empirical validations. We empirically validate the above terms. For model sensitivity, we assess the gradient norms of original samples on both the proxy and target models, followed by evaluating the effectiveness (loss magnitude) of adversarial examples on the target model. Figure 9 demonstrates a strong positive correlation between gradient norms and attack effectiveness across diverse PFL client models. Next, we investigate the geometry of the loss landscape by evaluating the average eigenvalue of the Hessian matrix associated with the classification head. The correlation coefficient between the average eigenvalue and AD is -0.176. For distribution alignment, we vary 𝛽 from 0.1 to 0.9, where a larger 𝛽 corresponds to reduced data heterogeneity (a smaller Wasserstein distance). As shown in Figure 9, increasing 𝛽 leads to a more pronounced degradation in the target model’s accuracy, aligning with our theoretical analysis. Furthermore, we evaluate the correlation coefficient between the performance gap of proxy and target models and AD. The resulting correlation coefficient is -0.218. This inverse relationship indicates that smaller fitness gaps between proxy and target models are associated with heightened vulnerability. Finally, we analyze the relationship between parameter sensitivity and AD. Our measurements reveal a moderate correlation coefficient of 0.109, corroborating our theoretical analysis that greater parameter sensitivity makes higher AD. PFL versus centralized training. Table 3 presents a comparison of these metrics associated with models trained with centralized training and PFL methods. Overall, the PFL paradigm exhibits higher model sensitivity, larger average Hessian eigenvalues, and increased parameter sensitivity compared to centralized training. These quantitative differences shed light on the empirically observed vulnerability gap between the two paradigms. These motivate us to propose a robustness-enhanced PFL framework aimed at mitigating these issues. Notice that we do not consider distribution alignment and fitness difference here, as these factors are inherently fixed and cannot be altered within the given scenario. Instead, our efforts concentrate on addressing the modifiable aspects of model behavior to improve robustness effectively.

How to mitigate adversarial attack in PFL?

5.1

Robustness-enhanced PFL Framework

Overview. Building upon our theoretical foundations, we introduce a robustness-enhanced PFL framework, which involves three key techniques, namely stochastic input noise augmentation, input-scaled trace regularization, and parameter sensitivity minimization. These techniques are designed to regularize input gradients, 𝜆min , and parameter sensitivity, respectively. Moreover, these techniques are plug-and-play and thus are compatible with arbitrary PFL methods, making

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Trovato et al.

our framework a universal robustness enhancement solution for both existing and emergent PFL methods. Taking client 𝑖 as a representative example, its original personalized model optimization objective is formulated as: min E (𝑥,𝑦)∼D𝑖 [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] , 𝜃𝑖

classification head, such that the model output is given by 𝐹 (𝑥; 𝜃 𝑖 ) = 𝑊 𝑇 𝑧. For cross-entropy loss, we have: L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) = −

(6)

where we intentionally omit the soft constraints from Equation 3 and hard constraints from Equation 4 for narrative clarity, focusing instead on the core robustness mechanisms. Stochastic input noise augmentation. This technique aims to reduce the model’s sensitivity to input perturbations by regularizing ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 22 . Since direct optimization of ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 22 involves second-order gradients that are computationally prohibitive in high-dimensional spaces [3], we develop a practical approximation solution as follows. Considering Gaussian noise injection 𝜐 ∼ N (0, 𝜎 2 𝐼 ) to input samples, we analyze the first-order Taylor expansion of the perturbed loss function:

𝐾 ∑︁

exp(𝑤𝑇 𝑧 )

𝑘 where 𝑝𝑘 = Í𝐾 exp(𝑤 𝑇 𝑧 ) and 𝑤 𝑘 is 𝑘-th column of 𝑊 . Then, 𝑗 =1

𝑗

the Hessian 𝐻 with respect to 𝑊 is given by: 𝐻 (𝑚, 𝑛) =

𝜕L = 𝑧𝑧𝑇 𝑝𝑚 (I(𝑚 = 𝑛) − 𝑝𝑛 ). 𝜕𝑤𝑚 𝜕𝑤𝑛

Although explicit matrix construction is feasible, spectral decomposition becomes computationally intensive for large 𝑑 (e.g., when 𝑑 = 1024, 𝐾 = 10, the matrix size is about 108 with 10240 eigenvalues). Í Thus, we exploit a basic algebraic relationship 𝑡𝑟 (𝐻 ) = 𝑖=1 𝜆𝑖 , where {𝜆𝑖 } are Hessian eigenvalues. For classification head, this trace reduces to:

L (𝐹 (𝑥 + 𝜐; 𝜃 𝑖 ), 𝑦) = L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) + 𝜐𝑇 ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦), where higher-order terms are neglected under small 𝜎. Rearranging the above equation and multiplying both sides by 𝜐: 𝜐𝜐𝑇 ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) = 𝜐 [L (𝐹 (𝑥 +𝜐; 𝜃 𝑖 ), 𝑦) − L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)]. Taking the expectation over 𝜐 and leveraging the identity E𝜐 [𝜐𝜐𝑇 ] = 𝜎 2 𝐼 , we have: 𝜎 2 ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) = E𝜐 [𝜐𝜐𝑇 ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] =E𝜐 {𝜐 [L (𝐹 (𝑥 + 𝜐; 𝜃 𝑖 ), 𝑦) − L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)]}. For practical implementation, we employ a single-sample Monte Carlo approximation: ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) =

L (𝐹 (𝑥 + 𝜐; 𝜃 𝑖 ), 𝑦) − L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) 𝜐. 𝜎2

Then, taking the norm on both sides yields the computationally tractable regularization term: ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 2 =

|L (𝐹 (𝑥 + 𝜐; 𝜃 𝑖 ), 𝑦) − L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)| ||𝜐 || 2 . 𝜎2

Input-scaled trace regularization. Input-scaled trace regularization aims to increase 𝜆min . Here we consider 𝜆min of classification head, since feature extraction layers are commonly shared across clients in many PFL methods. Another consideration is that the Hessian matrix of classification head admits a closed-form solution (detailed below), facilitating more efficient computation compared to the intractable complexity required for full-network Hessian evaluations. Let 𝑧 ∈ R denote the feature embedding from feature extraction layers and let 𝑊 ∈ R𝐾 ×𝑑 represent the weights of

I(𝑦 = 𝑘) log 𝑝𝑘 .

𝑘=1

𝑡𝑟 (𝐻 ) =

||𝑧|| 22 |{z} feature scaling

𝐾 ∑︁

𝑝𝑖 (1 − 𝑝𝑖 ) .

(7)

𝑖=1

}

prediction uncertainty

As can be seen, the prediction uncertainty term encourages confident predictions by penalizing ambiguous distributions, while the feature scaling term regulates the model’s Lipschitz constant by constraining the magnitude of feature embeddings3 . Together, Equation 7 indeed promotes confident and accurate predictions without significantly increasing the model’s Lipschitz constant. Parameter sensitivity maximization. This technique aims to enhance the sensitivity of model parameters to input data. We focus on the parameter sensitivity of classification 2 L of classification head. The second-order derivative ∇𝑧,𝑤 head enjoys a closed-form expression as follows: 2 ∇𝑧,𝑤 L = (𝑝 − 𝑦) ⊗ 𝐼 + 𝑧 ⊗ [𝑊 (diag(𝑝) − 𝑝𝑝𝑇 )],

where 𝑝 = (𝑝𝑖 ), ⊗ denotes outer product, 𝐼 is identity matrix, and, diag(·) converts the input vector into a diagonal matrix. To maximize parameter sensitivity, we incorporate 2 L into the loss function. the negative 𝐿2 norm of ∇𝑧,𝑤

3 The Lipschitz constant of a function measures the maximum rate at which

the function can change with respect to small changes in its input (see the footnote of Assumption 4.1 for its formal definition). A smaller Lipschitz constant implies that the function is less sensitive to input perturbations, thereby enhancing the model’s robustness. By constraining the magnitude of feature embeddings, we can effectively limit the Lipschitz constant, making the model more stable and less susceptible to adversarial attacks.

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Table 4. The performance of the proposed robustness-enhanced framework in CIFAR-10. PFL Method

FedAS

FedBABU

FedBN

FedCAC

FedProx

FedRep

GPFL

SCAFFOLD

Metric

Acc

Vanilla Ours Ours+AT Ours+AT+RS

79.05 79.32 69.36 51.35

47.61 40.57 8.46 3.57

66.72 78.94 70.49 54.78

57.79 30.72 8.42 3.51

67.39 77.85 66.90 49.46

62.94 53.00 17.12 6.69

62.00 72.62 62.47 47.78

36.19 28.16 11.03 5.94

57.46 68.34 53.27 31.86

55.14 46.01 17.02 4.91

76.83 78.19 68.19 48.66

50.38 42.84 11.50 4.36

51.08 60.97 48.26 34.62

23.52 18.37 14.19 3.40

65.51 74.79 59.41 38.42

60.34 50.21 18.11 5.62

Table 5. The performance of the proposed robustness-enhanced framework in CIFAR-100. PFL Method

FedAS

FedBABU

FedBN

FedCAC

FedProx

FedRep

GPFL

SCAFFOLD

Metric

Acc

Vanilla Ours Ours+AT Ours+AT+RS

48.43 49.02 37.73 16.90

26.98 11.55 6.87 1.79

31.84 44.13 38.04 17.37

30.88 12.12 6.57 1.69

34.43 41.45 33.78 13.05

32.86 39.10 15.38 3.22

36.66 36.63 31.30 20.52

22.83 18.29 8.94 4.35

31.15 35.96 28.64 10.02

29.90 26.38 14.36 2.58

45.51 45.64 37.95 14.87

28.72 21.25 9.90 1.74

30.09 29.98 21.51 7.24

23.89 20.57 8.94 1.04

31.73 36.29 29.64 9.15

30.62 25.32 13.09 1.71

Fianl formulation. Incorporating these techniques, client 𝑖’s complete optimization objective can be formulated:

min E (𝑥,𝑦)∼D𝑖 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) + 𝛽 1 · ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 2 + 𝜃𝑖 2 𝛽 2 · 𝑡𝑟 (𝐻 ) − 𝛽 3 · ||∇𝑧,𝑤 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 2 𝛾 𝛽 1 ||𝜐 || 2 = E (𝑥,𝑦)∼D𝑖 (1 − )L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) 𝜎2 𝐾 ∑︁ 𝛾 𝛽 1 ||𝜐 || 2 2 + L (𝐹 (𝑥 + 𝜐; 𝜃 ), 𝑦) + 𝛽 ||𝑧|| · 𝑝𝑖 (1 − 𝑝𝑖 ) 𝑖 2 2 𝜎2 𝑖=1

− 𝛽 3 ||(𝑝 − 𝑦) ⊗ 𝐼 + 𝑧 ⊗ [𝑊 (diag(𝑝) − 𝑝𝑝𝑇 )|| 2 }, (8)

where 𝛽 1, 𝛽 2, 𝛽 3 are hyperparameters controlling the strengths of the respective regularization terms. Moreover, 𝛾 is a binary indicator that equals 1 if L (𝐹 (𝑥 + 𝜐; 𝜃 𝑖 ), 𝑦) ≥ L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) and −1 otherwise. 5.2

Evaluation

Setup. The experimental configurations follow the specifications outlined in Section 3.2. For our framework, we set 𝛽 1 = 𝛽 2 = 𝛽 3 = 0.01, 𝜎 2 = 1. To comprehensively evaluate the performance of PFL methods, we include an additional metric: clean accuracy (Acc), which is the accuracy of clients’ personalized models on their respective unperturbed datasets, reflecting their generalization performance under normal conditions. Moreover, we increase 𝐾 to 20 to thoroughly validate the robustness of our framework. See Appendix B for ablation study (the proposed three techniques).

The effectiveness of the proposed framework. We first benchmark the proposed framework against vanilla implementations of various PFL methods, with the results summarized in Tables 4∼6. As can be seen, our method consistently reduces AD across all PFL methods compared to their vanilla counterparts. For instance, in CIFAR-10, FedBABU’s AD decreases from 57.70% to 30.72%, representing a reduction of approximately 30%. Similar improvements are observed in other PFL methods, such as FedBN (9.94%) and FedCAC (8.03%) in CIFAR-10. These results confirm our framework’s effectiveness in countering transfer-based attacks. Moreover, surprisingly, our framework also leads to consistent improvements in Acc for most PFL methods. Particularly, in FedBABU, our framework boosts Acc from 66.72% to 78.94% (+12.2%), a notable gain. This simultaneous improvement in both robustness and Acc is noteworthy because there is typically a trade-off between performance and robustness. Generally, enhancing robustness often comes at the cost of reduced Acc (see below). In contrast, our method achieves a win-win scenario where both metrics improve concurrently. Integration with traditional adversarial defenses. To explore complementary robustness gains, we integrate our framework with two widely-used traditional adversarial defenses: adversarial training (AT) [1, 24] and random smoothing (RS) [13]. Both defenses can bolster the robustness of models but also introduce trade-offs in terms of Acc. Below, we analyze the results in detail. AT involves augmenting clients’ local training dataset with adversarial examples to explicitly teach the model to resist adversarial attacks. We implement AT by replacing 𝑥 in Equation 8 with 𝑥 + 𝛿 where 𝛿 is generated via 20-step 8 PGD attacks (𝜖 = 255 ). As Tables 4∼6 illustrate, when combined with AT, our framework yields superior AD reduction

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Trovato et al.

Table 6. The performance of the proposed robustness-enhanced framework in GTSRB. PFL Method

FedAS

FedBABU

FedBN

FedCAC

FedProx

FedRep

GPFL

SCAFFOLD

Metric

Acc

Vanilla Ours Ours+AT Ours+AT+RS

99.53 99.36 97.06 53.85

61.80 22.54 13.55 4.82

98.32 98.42 97.47 54.68

79.00 21.74 13.86 5.64

98.85 99.24 96.83 44.61

84.75 67.69 22.34 7.59

97.81 97.07 91.25 54.30

72.99 55.13 23.29 8.82

98.17 98.64 95.77 39.35

85.59 69.54 24.73 6.22

99.46 99.34 96.79 47.41

68.29 62.24 16.15 4.57

98.23 97.54 90.31 48.12

51.42 42.94 24.16 5.87

97.87 98.67 97.14 40.33

79.28 62.36 22.01 6.82

Table 7. The performance of our framework against state-of-the-art transfer-based attacks in CIFAR-10. Method

Attack

FedAS

FedBABU

FedBN

FedCAC

FedProx

FedRep

GPFL

SCAFFOLD

Ours+AT

PGD MI PCIFGSM MaskBlock

8.46 10.38 10.41 11.25

8.42 10.42 10.46 11.22

17.12 19.73 19.80 20.90

11.03 12.19 12.23 13.02

17.02 20.53 20.58 21.59

11.50 13.89 13.94 14.90

14.19 21.10 21.12 22.84

18.11 21.78 21.87 23.13

Ours+AT+RS

PGD MI PCIFGSM MaskBlock

3.57 4.25 4.28 4.74

3.51 4.35 4.40 4.85

6.69 7.62 7.61 8.52

5.94 6.54 6.58 7.17

4.91 5.45 5.46 6.00

4.36 5.40 5.44 6.03

3.40 5.75 5.74 6.74

5.62 7.59 7.64 8.56

Table 8. The time (s) required to execute a training round on the CIFAR-10. Method

FedAS

FedBABU

FedBN

FedCAC

FedProx

FedRep

GPFL

SCAFFOLD

Vanilla Ours Ours+AT

253.27 255.99 514.16

13.14 18.69 244.476

13.68 16.85 257.478

14.06 19.52 253.36

15.43 17.50 235.656

14.40 17.72 157.696

23.87 27.98 273.914

15.74 17.31 262.042

across all PFL methods. In CIFAR-10, FedBABU’s AD further decreases from 30.72% to 8.42%, albeit with a 8% Acc reduction. This aligns with expectations that explicit adversarial exposure enhances robustness at generalization costs. RS involves adding random noise to the input data during inference to mitigate the impact of adversarial perturbations. We implement RS through Gaussian noise injection (with a variance of 1) followed by majority voting over 50 repeated inferences. The results in tables 4∼6, while RS integration provides moderate AD reduction, its impact is less pronounced than AT’s. This trade-off highlights that while both AT and RS can greatly enhance robustness, they often sacrifices generalization performance on clean data. Nonetheless, the combination of our framework with AT and RS represents a powerful approach for scenarios where robustness is prioritized over accuracy. The robustness against state-of-the-art transfer-based attacks. Table 7 reports the evaluation results of our framework’s robustness against state-of-the-art transfer-based attacks. We observe that these attacks achieve higher AD, but the increase is limited to within 5%. In comparison, as shown in Figure 5, the AD for vanilla implementations generally hovers around 50%. Thus, our framework indeed significantly

enhances the resistance of existing PFL methods against transfer-based attacks. Overhead analysis. Here, we analyze the training and inference overhead. We use a single NVIDIA GTX 4090 GPU. As can be seen in Table 8, our framework introduces a small extra time overhead compared to the vanilla PFL methods, requiring approximately 3∼5 seconds per training round. Actually, the primary time overhead in both training and inference is dominated by the forward and backward passes, and our framework does not require any extra forward and backward passes. This stands in sharp contrast to AT which incurs substantial training overhead by up to (𝐾 + 1)× compared to standard training. This increase occurs because PGD involves 𝐾 forward and backward passes to generate adversarial examples, in addition to a forward and backward pass to compute the model parameter gradients. Regarding inference, the models for Vanilla, Ours, and AT are identical, so their inference costs are almost the same. RS requires processing multiple noise-perturbed samples to conduct majority voting, potentially prolonging inference overhead by the noise sampling factor. Empirically, the inference time for a batch size of 128 is approximately 0.001 seconds. For RS, the inference time is around 0.05 seconds. Practitioners

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

can choose the solution that best fits their needs based on robustness, performance, and time costs.

Conclusion

This work presented a systematic exploration of adversarial vulnerability in PFL systems, revealing critical security gaps in distributed edge systems. The proposed theoretical framework identifies five key determinants of attack transferability—model sensitivity, loss landscape geometry, distribution alignment, fitness difference, and parameter sensitivity, providing interpretable metrics for assessing PFL robustness. To mitigate these risks, we introduced a lightweight yet effective defense framework incorporating stochastic input noise augmentation, input-scaled trace regularization, and parameter sensitivity minimization. Evaluations show our framework can considerably improve the robustness of PFL methods without compromising their performance. The source code will be made publicly available upon the acceptance of this paper to facilitate reproducibility.

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

References [1] Tao Bai, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. 2021. Recent advances in adversarial training for adversarial robustness. arXiv preprint arXiv:2102.01356 (2021). [2] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. 2017. Machine learning with adversaries: Byzantine tolerant gradient descent. Advances in neural information processing systems 30 (2017). [3] S. Boyd and L. Vandenberghe. 2004. Convex Optimization. Convex Optimization. [4] Suo Chen, Yang Xu, Hongli Xu, Zhida Jiang, and Chunming Qiao. 2022. Decentralized federated learning with intermediate results in mobile edge computing. IEEE Transactions on Mobile Computing 23, 1 (2022), 341–358. [5] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai. 2021. Exploiting shared representations for personalized federated learning. In International conference on machine learning. PMLR, 2089– 2099. [6] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. 2018. Boosting Adversarial Attacks with Momentum. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 9185–9193. [7] Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. 2019. Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 4307–4316. [8] Nathan Dowlin, Ran Gilad-Bachrach, Kim Laine, Kristin E. Lauter, Michael Naehrig, and John Robert Wernsing. 2016. CryptoNets: applying neural networks to encrypted data with high throughput and accuracy. In International Conference on Machine Learning. [9] Mingyuan Fan, Cen Chen, Ximeng Liu, and Wenzhong Guo. 2025. Maskblock: Transferable adversarial examples with bayes approach. AsiaCCS. [10] Mingyuan Fan, Wenzhong Guo, Zuobin Ying, and Ximeng Liu. 2023. Enhance transferability of adversarial examples with model architecture. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1–5. [11] Mingyuan Fan, Yang Liu, Cen Chen, Shengxing Yu, Wenzhong Guo, Li Wang, and Ximeng Liu. 2021. Toward Evaluating the Reliability of Deep-Neural-Network-Based IoT Devices. IEEE Internet of Things Journal 9, 18 (2021), 17002–17013. [12] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6572 [13] Jinyuan Jia, Xiaoyu Cao, Binghui Wang, and Neil Zhenqiang Gong. 2020. Certified Robustness for Top-k Predictions against Adversarial Perturbations via Randomized Smoothing. (2020). https: //api.semanticscholar.org/CorpusID:59842968 [14] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning. PMLR, 5132–5143. [15] Latif U Khan, Walid Saad, Zhu Han, Ekram Hossain, and Choong Seon Hong. 2021. Federated learning for internet of things: Recent advances, taxonomy, and open challenges. IEEE Communications Surveys & Tutorials 23, 3 (2021), 1759–1799. [16] A. Kurakin, I. Goodfellow, and S. Bengio. 2016. Adversarial examples in the physical world. (2016). [17] Asif Ali Laghari, Kaishan Wu, Rashid Ali Laghari, Mureed Ali, and Abdullah Ayub Khan. 2021. A review and state of art of Internet of Things (IoT). Archives of Computational Methods in Engineering (2021), 1–19.

Trovato et al. [18] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. 2021. Ditto: Fair and robust federated learning through personalization. In International conference on machine learning. PMLR, 6357–6368. [19] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems 2 (2020), 429–450. [20] Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. 2021. FedBN: Federated Learning on Non-IID Features via Local Batch Normalization. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https://openreview.net/forum?id=6YEQUn0QICG [21] Yingwei Li, Song Bai, Yuyin Zhou, Cihang Xie, Zhishuai Zhang, and Alan Yuille. 2020. Learning transferable adversarial examples via ghost networks. In Proceedings of the AAAI conference on artificial intelligence, Vol. 34. 11458–11465. [22] Yunming Liao, Yang Xu, Hongli Xu, Zhiwei Yao, Liusheng Huang, and Chunming Qiao. 2024. Parallelsfl: A novel split federated learning framework tackling heterogeneity issues. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 845–860. [23] Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E. Hopcroft. 2020. Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks. arXiv: Learning (2020). [24] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations. [25] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2017. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA (Proceedings of Machine Learning Research, Vol. 54), Aarti Singh and Xiaojin (Jerry) Zhu (Eds.). PMLR, 1273–1282. http://proceedings.mlr.press/v54/mcmahan17a. html [26] Chaoyue Niu, Fan Wu, Shaojie Tang, Lifeng Hua, Rongfei Jia, Chengfei Lv, Zhihua Wu, and Guihai Chen. 2020. Billion-scale federated learning on mobile clients: A submodel design with tunable privacy. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1–14. [27] Jaehoon Oh, SangMook Kim, and Se-Young Yun. 2022. FedBABU: Toward Enhanced Representation for Federated Image Classification. In International Conference on Learning Representations. [28] Cong Shi, Tianfang Zhang, Zhuohang Li, Huy Phan, Tianming Zhao, Yan Wang, Jian Liu, Bo Yuan, and Yingying Chen. 2022. Audio-domain position-independent backdoor attack via unnoticeable triggers. In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking. 583–595. [29] Chen Wan and Fangjun Huang. 2023. Adversarial attack based on prediction-correction. arXiv preprint arXiv:2306.01809 (2023). [30] Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. 2020. Attack of the tails: Yes, you really can backdoor federated learning. Advances in Neural Information Processing Systems 33 (2020), 16070–16084. [31] Lun Wang, Yang Xu, Hongli Xu, Min Chen, and Liusheng Huang. 2022. Accelerating decentralized federated learning in heterogeneous edge computing. IEEE Transactions on Mobile Computing 22, 9 (2022), 5001–5016. [32] Xiaosen Wang and Kun He. 2021. Enhancing the Transferability of Adversarial Attacks through Variance Tuning. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 1924–1933.

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY [33] Dongxian Wu, Yisen Wang, Shutao Xia, James Bailey, and Xingjun Ma. 2020. Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNets. ArXiv abs/2002.05990 (2020). [34] Xinghao Wu, Xuefeng Liu, Jianwei Niu, Guogang Zhu, and Shaojie Tang. 2023. Bold but cautious: Unlocking the potential of personalized federated learning through cautiously aggressive collaboration. In Proceedings of the IEEE/CVF international conference on computer vision. 19375–19384. [35] Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li. 2020. DBA: Distributed Backdoor Attacks against Federated Learning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https://openreview.net/ forum?id=rkgyS0VFvr [36] Cihang Xie, Zhishuai Zhang, Jianyu Wang, Yuyin Zhou, Zhou Ren, and Alan Loddon Yuille. 2019. Improving Transferability of Adversarial Examples With Input Diversity. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 2725–2734. [37] Jian Xu, Xinyi Tong, and Shao-Lun Huang. 2023. Personalized Federated Learning with Feature Alignment and Classifier Collaboration. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. https://openreview.net/forum?id=SXZr8aDKia [38] Yang Xu, Yunming Liao, Hongli Xu, Zhenguo Ma, Lun Wang, and Jianchun Liu. 2022. Adaptive control of local updating and model compression for efficient federated learning. IEEE Transactions on Mobile Computing 22, 10 (2022), 5675–5689. [39] Xiyuan Yang, Wenke Huang, and Mang Ye. 2024. Fedas: Bridging inconsistency in personalized federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11986–11995. [40] Chengliang Zhang, Suyi Li, Junzhe Xia, Wei Wang, Feng Yan, and Yang Liu. 2020. BatchCrypt: Efficient Homomorphic Encryption for CrossSilo Federated Learning. In USENIX Annual Technical Conference. [41] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, Jian Cao, and Haibing Guan. 2023. Gpfl: Simultaneously learning global and personalized feature information for personalized federated learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5041–5051. [42] Wuyang Zhang, Zhezhi He, Luyang Liu, Zhenhua Jia, Yunxin Liu, Marco Gruteser, Dipankar Raychaudhuri, and Yanyong Zhang. 2021. Elf: accelerate high-resolution mobile deep vision with content-aware parallel offloading. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. 201–214. [43] Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. Advances in neural information processing systems 32 (2019).

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Trovato et al.

Proof of Theorem 4.1

Proof. Step 1: Gradient Difference Analysis. Let Δ = 𝜃 𝑖 − 𝜃 𝑗 . For gradient alignment, consider the following input gradient difference:

Table 9. Ablation study on the impact of key components on adversarial robustness, measured by AD. Lower AD indicates better robustness. AD is averaged over eight PFL methods. Component

w.o. Stochastic input noise augmentation w.o. Input-scaled trace regularization w.o. Parameter sensitivity maximization All

44.04 41.65 39.92 38.73

∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) − ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) =∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 + Δ), 𝑦) − ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) 2 =∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) + ∇𝑥,𝜃 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)𝑇 Δ 𝑗

−∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) + O (||Δ|| 2 ) 2 =∇𝑥,𝜃 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)𝑇 Δ + O (||Δ|| 2 ). 𝑗

Squaring both sides and applying Cauchy-Schwarz:

Step 3: Combining Step 1 and Step 2. Substituting Equation 9 into Equation 10 through ||Δ|| 22 yields:

||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 22 + ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22

O (||Δ||) + ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 22 + ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22

− 2∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) ⊤ ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) + O (||Δ|| 2 ) −

2 ≤ ||∇𝑥,𝜃 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22 ||Δ|| 22 . 𝑗

𝜆min

(−E𝑃 (𝑖 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] 𝑋𝑌

(𝑗) (𝑖 ) )) + E𝑃 ( 𝑗 ) L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) + 𝐾 · 𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌

Rearranging the above equation yields: ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)|| 22 + ||∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22 2 − ||∇𝑥,𝜃 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)|| 22 ||Δ|| 22 + O (||Δ|| 2 ) 𝑗 ≤ 2∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) ⊤ ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦).

2 L (𝐹 (𝑥; 𝜃 ), 𝑦)|| 2 ||∇𝑥,𝜃 𝑗 2 𝑗

𝑋𝑌

≤ 2∇𝑥 L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦) ⊤ ∇𝑥 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦). (9)

(11) This completes the proof of the gradient alignment bound. □

We next derive the bound of ||Δ|| 22 . B Supplementary Experiment Step 2: Bounding Δ. Using the Kantorovich-Rubinstein The robustness of different models. Table 10 evaluates duality, the Wasserstein distance between distributions ad(𝑖 ) (𝑗) mits the dual form: 𝐾 ·𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ) = sup | | 𝑓 | |𝐿 ≤𝐾 E𝑃 (𝑖 ) [𝑓 (𝑥, 𝑦)]− the influence of model architecture on the robustness across 𝑋𝑌 various PFL methods. Notably, deeper architectures (e.g., E𝑃 ( 𝑗 ) [𝑓 (𝑥, 𝑦)], where ||𝑓 ||𝐿 ≤ 𝐾 denotes K-Lipschitz contiResNet34) generally exhibit higher AD compared to shal𝑋𝑌 nuity. For the model-specific loss function, this implies: 𝐾 · lower ones (e.g., ResNet10), suggesting that increased model (𝑖 ) (𝑗) 𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ) ≤ E𝑃 (𝑖 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)]−E𝑃 ( 𝑗 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)]. capacity may lead to greater vulnerability to adversarial at𝑋𝑌 𝑋𝑌 tacks. We expand the loss difference through Taylor approximation: (𝑖 ) (𝑗) 𝐾 · 𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ) − E𝑃 (𝑖 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] 𝑋𝑌 ≤ −E𝑃 ( 𝑗 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] = −E𝑃 ( 𝑗 ) L (𝐹 (𝑥; 𝜃 𝑗 + Δ), 𝑦) 𝑋𝑌 𝑋𝑌 = −E𝑃 ( 𝑗 ) L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) + ∇𝜃 𝑗 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) ⊤ Δ 𝑋𝑌 ⊤ 2 2 + Δ ∇𝜃 𝑗 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)Δ + O (||Δ|| 2 ) . (𝑗) Using local optimality of 𝜃 𝑗 on 𝑃𝑋𝑌 gives: − E𝑃 (𝑖 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] + E𝑃 ( 𝑗 ) L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) 𝑋𝑌

𝑋𝑌

(𝑖 ) (𝑗) + 𝐾 · 𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ) + O (||Δ|| 22 ) ≤ −E𝑃 ( 𝑗 ) Δ⊤ ∇𝜃2 𝑗 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)Δ .

Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

𝑋𝑌

Under strong convexity, the quadratic term satisfies −Δ⊤ ∇𝜃2 𝑗 L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦)Δ ≤ −𝜆min ||Δ|| 22 . Thus, we have: − E𝑃 (𝑖 ) [L (𝐹 (𝑥; 𝜃 𝑖 ), 𝑦)] + E𝑃 ( 𝑗 ) L (𝐹 (𝑥; 𝜃 𝑗 ), 𝑦) 𝑋𝑌

Ablation study on key components. Table 9 presents an ablation study analyzing the impact of removing individual components from our framework. Each row corresponds to a specific component, and AD quantifies the robustness when that component is excluded. The results indicate that all components contribute to reducing AD, with the full components achieving the lowest of 38.73. This suggests that each component plays a complementary role in improving robustness. Moreover, parameter sensitivity maximization term has the smallest impact, and stochastic input noise augmentation has the largest impact.

𝑋𝑌

(𝑖 ) (𝑗) + 𝐾𝑊 (𝑃𝑋𝑌 , 𝑃𝑋𝑌 ) + +O (||Δ|| 22 ) ≤ −𝜆min ||Δ|| 22 .

(10)

Towards Robust Personalized Federated Learning: Vulnerability Assessment and Defense Co-Design Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Table 10. Impact of model architecture on robustness across various PFL methods. Model

FedAS

FedBABU

FedBN

FedCAC

FedProx

FedRep

GPFL

SCAFFOLD

ResNet10 ResNet18 ResNet34

47.61 55.22 58.85

57.79 66.57 70.59

62.94 70.55 72.41

36.19 42.42 45.80

55.14 62.33 66.06

50.38 61.69 65.22

23.52 34.60 38.19

60.34 73.90 77.29

Record · ID 299833 · SHA-256 2b16d8d8e00d7c67
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.