FedLNS: Leverage LayerNorm Signature Modeling to Mitigate Adversarial Manipulation in Federated LLMs

XXXX, 2026.

1

FedLNS: Leverage LayerNorm Signature Modeling to Mitigate Adversarial Manipulation in Federated LLMs

arXiv:2608.18736v1 [cs.LG] 19 Aug 2026

Kai Li, Jong-Ik Park, Carlee Joe-Wong, Wei Ni, and Falko Dressler

Abstract—Federated training enables language models to learn from distributed private text, but the server cannot directly verify the local supervision or optimization process that produces each client update. A malicious client can therefore train on corrupted targets, introduce incorrect context–token associations, and degrade the global model through repeated aggregation. Such degradation can also increase the risk of unreliable or hallucinatory generation. We propose Federated Learning with Normalization Signatures (F ED LNS), a server-side framework for lightweight maliciousupdate screening. F ED LNS represents each client update through changes in trainable normalization-layer parameters and screens suspicious updates against a robust, history-aware cross-client reference. Because the signatures are extracted at the server from the returned local models, F ED LNS requires no additional client-to-server parameter or metadata exchange compared to standard federated learning (FL) methods. After screening, the retained full-model updates can be aggregated using standard FL or another compatible aggregation rule. Experiments on GPT-style, BERT-style, and LLaMA-style models trained from scratch with 200 clients show that, under 40% population-level target manipulation, F ED LNS achieves lower test perplexity than the strongest of six baselines for all three architectures under both IID (independently and identically distributed) and non-IID data partitions. Index Terms—Federated Learning, Federated Language Models, Malicious-Update Screening, Robust Aggregation, Normalization Layers.

I. I NTRODUCTION Large language models (LLMs) increasingly support search, question answering, software development, decision assistance, and human–computer interaction. Their effectiveness in specialized applications often depends on domain-specific text distributed across users, organizations, and devices [1], [2]. Centralizing such data may be infeasible because of privacy, ownership, confidentiality, or regulatory constraints [3]–[5]. Federated learning (FL) addresses this limitation by coordinating a shared model without collecting raw client data: in each communication round, the server broadcasts the current global model, selected K. Li is with the Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg (E-mail: kaili@ieee.org). J. Park and C. Joe-Wong are with the Department of Electrical and Computer Engineering, Carnegie Mellon University, Pennsylvania, United States (E-mail: {jongikp, cjoewong}@andrew.cmu.edu). W. Ni is with the School of Engineering, Edith Cowan University, Perth, WA 6027, Australia (E-mail: wei.ni@ieee.org). F. Dressler is with the School of Electrical Engineering and Computer Science, TU Berlin, Germany. K. Li and J. Park contributed equally to this work.

clients optimize it locally, and the server aggregates their returned updates [6]–[8] to begin another training round. This privacy-preserving separation also creates a critical security gap. The server observes the returned updates but cannot directly verify the local inputs, targets, and optimization procedures that produced them [7], [9], [10]. A malicious client can therefore manipulate its local objective and submit a harmful update while otherwise following the communication protocol. Detecting such behavior is especially difficult for federated language models because each update may contain millions or billions of parameters, and benign client updates can also vary substantially. This work considers malicious target manipulation, as illustrated in Figure 1. In causal language modeling, the target at each position is the next token, whereas in masked language modeling, it is the original token hidden by the masking process. A malicious client replaces these targets with incorrect tokens, optimizes false context–target associations, and submits the resulting update to the server. If such updates are accepted and repeatedly aggregated at the server, they can propagate corrupted training signals into the global model, reducing test set performance and distorting the model’s predictive distribution over token continuations [11]. These effects may consequently increase the risk of semantically inconsistent or hallucinatory generation [12], [13]. The focus of this work is therefore screening suspicious client updates before aggregation, with improved generation reliability regarded as a downstream effect of limiting corrupted training signals. Existing Byzantine-robust FL methods suppress harmful updates through distance-based selection, coordinate-wise robust statistics, norm control, or clustering-based filtering [14]–[17]. Because these methods operate on complete client updates, their server-side processing remains tied to full-model dimensionality, while high-dimensional benign variation can obscure malicious deviations. Recent studies reduce the representation used for malicious-update screening at a FL server through attacksensitive dimension selection or supervised analysis of parameter-efficient updates [18], [19]. These results show that screening need not rely on complete model updates. However, an important question remains: can suspicious updates be screened using a compact, architecture-aware representation without labeled attack samples, trusted server data, or a separately trained detector? To address this question, we propose Federated Learn-

XXXX, 2026.

2

Fig. 1: Threat model for malicious target manipulation in federated language-model training. A malicious client changes its local supervision and optimizes incorrect context–target associations. If the resulting update is accepted, repeated aggregation can degrade the global model and increase the downstream risk of semantically unreliable generation.

ing with Normalization Signatures (F ED LNS), a serverside framework for lightweight malicious-update screening. Instead of examining complete high-dimensional updates, F ED LNS constructs a compact, architecture-aware signature from changes in the trainable parameters of the model’s normalization layers. These parameters control the scaling and, when supported by the architecture, shifting of hidden representations throughout transformer blocks [20]–[23]. Changes in these parameters therefore provide a compact, architecture-aware view of how local training alters hidden model representations. For each participating client, the server extracts a normalization-layer signature relative to the broadcast global model and compares it with a robust historical reference constructed from the latest signatures of previously observed clients. These are compared to coordinate-wise median and median absolute deviation (MAD) statistics to the influence of anomalous bank entries [15], [24]. F ED LNS then employs regularized Gaussian mixture modeling with Bayesian Information Criterion (BIC) to determine whether the current participants form one coherent population or two populations with different deviation levels. When two populations are identified, F ED LNS retains the lowerdeviation, and thus more likely to be benign, group without requiring prior knowledge of the malicious-client fraction. Because normalization-layer signatures are extracted at the server from the returned local models, F ED LNS adds no defense-specific client transmission overhead. After screening, the retained full-model updates are aggregated using the standard sample-weighted rule, which may be replaced with another compatible FL aggregation rule. F ED LNS requires no raw client data, trusted server dataset, labeled malicious updates, separately trained detector, or auxiliary global models. By limiting the incorporation of target-corrupted updates, F ED LNS aims to improve global-

model robustness and reduce predictive uncertainty under malicious target manipulation. The contributions of this work are as follows: Compact model update representation. We introduce a normalization-layer update signature that represents each client update through a small, architectureaware parameter subset rather than the complete highdimensional update. Because the signature is extracted from the returned local model, it provides a modular representation for server-side screening before fullmodel aggregation is performed. • Unsupervised history-aware screening. We develop a server-side update retention rule that constructs a robust cross-client reference from the latest observed signatures and combines median/MAD standardization with BIC-guided mixture modeling. The rule requires no raw client data, trusted server dataset, labeled updates, predefined malicious fraction, separately trained detector, or defense-specific client-to-server payload. • Cross-architecture evaluation. We evaluate F ED LNS on GPT-style, BERT-style, and LLaMA-style models with 200 clients, IID and two-shard non-IID partitions, and target manipulation affecting 0%–40% of the client population. At 40%, F ED LNS achieves lower perplexity than all six baselines for all three architectures under both IID and two-shard non-IID partitions. The maximum reductions are 18.57% in perplexity and 6.50% in token-level semantic entropy. •

The remainder of this paper is organized as follows. Section II reviews the related work. Section III presents the F ED LNS framework. Section IV provides the theoretical analysis. Section V describes the experimental evaluation and performance analysis. Section VI concludes the paper.

XXXX, 2026.

3

II. R ELATED W ORK Prior work relevant to F ED LNS spans two directions: robust aggregation and malicious-update screening in FL and parameter-efficient or model-internal representations. A. Robust Aggregation and Malicious-Update Screening Byzantine-robust aggregation suppresses harmful FL updates using geometric, coordinate-wise, or magnitude-based properties. Multi-Krum selects updates that remain close to a majority neighborhood under pairwise distance [14]. Coordinate-wise Median and Trimmed Mean apply robust statistics independently across parameter dimensions [15], while norm-bounded aggregation limits client influence through update clipping or rescaling [16]. FLAME combines clustering, clipping, and noise injection to reduce the contribution of suspicious clients [17]. Because these methods operate on complete model updates, their serverside processing remains tied to full-model dimensionality. Other FL defenses use trusted references, multiple models, or historical client behavior. FLTrust constructs a reference update from trusted server data and scores client updates according to their agreement with this reference [25]. FLCert partitions clients into groups, trains multiple global models, and combines their predictions to provide certified robustness [26]. FLDetector predicts client-update trajectories from previous communication rounds and detects persistent deviations from expected behavior [24]. Together, these approaches rely on representative server data, additional model instances, or sufficiently regular client histories. Recent security methods reduce the representation analyzed during screening. Dim-Krum selects dimensions with stronger backdoor-related changes and applies Krum-based detection in the restricted space [18]. Safe-FedLLM extracts features from LoRA updates and trains an offline supervised probe using labeled benign and malicious samples [19]. These methods show that malicious-update screening need not process every model parameter, but they rely, respectively, on attack-sensitive coordinate selection or labeled update samples and a separately trained probe. B. Parameter-Efficient Adaptation and Model-Internal Signals Parameter-efficient adaptation shows that task-relevant model changes can be expressed through restricted trainable subsets. Adapter tuning introduces small trainable modules [27]; BitFit updates only bias parameters [28]; and IA3 learns compact vectors that scale intermediate activations [29]. LN-Tuning adapts language models through normalization gain and bias parameters [30]. Although designed for adaptation rather than security, these methods demonstrate that comparatively small parameter subsets can encode meaningful changes induced by local optimization. Normalization layers form a structurally repeated parameter subset in transformer architectures. They regulate the scaling and, depending on the formulation, shifting of

normalized hidden representations [20]–[22]. Prior work further shows that their placement and behavior affect optimization, gradient propagation, and transformer expressivity [23], [31]. Their restricted parameterization and recurring role across transformer blocks provide the architectural basis for studying normalization-parameter changes as client-update signals. Language-model reliability has also been studied through output grounding, uncertainty, and internal model signals. Grounding-based methods compare generations with source evidence, knowledge representations, or semantic relations [32]–[35], while uncertainty- and consistency-based approaches use semantic entropy, confidence, or response agreement [36]–[39]. Other work examines token-level behavior, multimodal consistency, hidden representations, or activation anomalies [40]–[45], and system-level objectives have incorporated hallucination risk directly [46]. These studies analyze outputs, activations, or uncertainty rather than federated model updates, but they establish modelinternal signals as useful indicators of unreliable LLM behavior. F ED LNS combines a compact normalization-parameter representation with unsupervised, history-aware maliciousupdate screening. Unlike trusted-reference, multi-model, and trajectory-prediction defenses, it constructs a crossclient reference from ordinary federated participation. It also differs from Dim-Krum and Safe-FedLLM by requiring neither attack-sensitive coordinate selection, labeled update samples, nor a separately trained detector. III. T HE P ROPOSED F ED LNS D EFENSE F RAMEWORK This section presents F ED LNS, which integrates a historical normalization-signature bank, layer-wise median/MAD statistical modeling, and BIC-guided Gaussian mixture clustering. Algorithm 1 presents the overall procedure, which includes local client training (Algorithm 2), serverside signature screening and aggregation (Algorithm 3), and the BIC-guided one-vs-two-component GMM rule for retaining client updates (Algorithm 4). We consider an honest server with persistent client identifiers and server-controlled client sampling. A fixed subset containing less than half of the client population performs malicious target manipulation while otherwise following the prescribed communication protocol in order to avoid detection by the server. Thus, our threat model excludes Sybil identities, client-ID resets, and adversaries that explicitly optimize their normalization-parameter changes to mimic benign signatures, all of which may be interesting directions for future work. A. Federated Training and Client Updates Algorithm 1 presents the federated training procedure with K clients indexed by C = {1, . . . , K}. At communication round t, the server samples a participating-client set St ⊆ C and broadcasts the current

XXXX, 2026.

4

Algorithm 1 F ED LNS communication-round driver

Algorithm 2 ClientLocalTrain(wt , Di , E)

Require: Initial global model parameters w0 , client set C, total rounds T , local epochs E, bank activation threshold Mact Ensure: Final global model parameters wT 1: Initialize server-side history bank B−1 = ∅ and client-ID set H−1 = ∅ 2: for t = 0, 1, . . . , T − 1 do 3: Sample participating clients St ⊆ C 4: Broadcast wt to every client i ∈ St 5: for each client i ∈ St in parallel do 6: (wt,i , nt,i ) ← ClientLocalTrain(wt , Di , E) 7: end for 8: (wt+1 , Rt , Bt , Ht ) ← ServerFedLNS (wt , {(wt,i , nt,i )}i∈St , Bt−1 , Ht−1 , Mact ) 9: end for 10: return wT

Require: Global model parameters wt , local dataset Di , local epochs E Ensure: Local model parameters wt,i , processed-example count nt,i 1: Initialize local model parameters wt,i ← wt 2: Set nt,i ← 0 3: for local epoch e = 1, . . . , E do 4: for mini-batch M ∈ MiniBatchLoader(Di ) do 5: Set LM ← 0 6: for each training instance ξ ∈ M do 7: LM ← LM + ℓ(wt,i ; ξ) 8: nt,i ← nt,i + 1 9: end for 10: LM ← LM /|M| 11: wt,i ← OptimizerStep (wt,i , ∇w LM (wt,i )) 12: end for 13: end for 14: return (wt,i , nt,i )

global model parameters wt ∈ RP to the selected clients (Algorithm 1, lines 3–4), where Nt := |St |. Client i trains on its local dataset Di to minimize the empirical objective 1 X Fi (w) := ℓ(w; ξ), (1) |Di | ξ∈Di

where ℓ(·) denotes the language-modeling loss function, e.g., the next-token prediction loss for causal language modeling or the masked-token prediction loss for masked language modeling [47]. Here, ξ represents an individual training instance sampled from Di , such as a text sequence together with its associated supervision signal. Client i initializes its local model from wt (Algorithm 2, line 1), performs E local epochs using mini-batch optimization (lines 3–13), and returns the locally trained parameters and processed-example count (line 14): (wt,i , nt,i ) = ClientLocalTrain (wt , Di , E) .

(2)

Upon completing local training, client i returns the updated model parameters wt,i together with nt,i , the total number of training-example occurrences processed across the E local epochs. The corresponding client update is ∆wt,i := wt,i − wt .

(3)

Because normalization-layer signatures are extracted at the server from the returned local models, clients follow the standard FL training and upload procedure without transmitting defense-specific parameters or metadata. All signature extraction, bank maintenance, and GMM-based screening are performed by the server. After the participating clients return their locally trained models (Algorithm 1, line 6), the server invokes ServerFedLNS (line 8), described in Algorithm 3, to screen the returned updates before global aggregation. B. Server-Side Normalization-Signature Extraction Algorithm 3 computes each complete client update and extracts its layer-wise normalization signature (lines 2–3). Let L denote the ordered set of normalization layers whose trainable parameters are included in the F ED LNS signature,

and let dℓ denote the number of included parameters associated with layer ℓ ∈ L. We define pℓ (w) ∈ Rdℓ as the vector formed by concatenating all trainable normalization parameters of layer ℓ under model parameters w. For a standard LayerNorm layer, pℓ (w) consists of the trainable scaling and bias parameters. For a bias-free normalization mechanism such as RMSNorm, the vector contains only the trainable scaling parameters. For selected client i ∈ St , the layer-wise normalization signature is defined as st,i,ℓ := pℓ (wt,i ) − pℓ (wt ) ∈ Rdℓ .

(4)

Thus, st,i,ℓ measures how local training at client i changes the normalization parameters of layer ℓ relative to the current global model parameters wt . The complete signature of client i is the ordered collection st,i := (st,i,ℓ )ℓ∈L . (5) The signature is computed entirely at the server from the uploaded local models and the current global model. Normalization parameters regulate the scaling and, depending on the architecture, shifting of hidden representations throughout transformer blocks. F ED LNS therefore uses changes in these parameters as a compact, architectureaware screening representation, while the complete returned model updates remain available for aggregation. C. Server-Side History Bank F ED LNS maintains a server-side history bank that stores the latest normalization-layer signature associated with each client observed during training. The resulting collection provides a cross-client reference for constructing robust statistics across communication rounds. Prior to processing round t, the history bank is defined as Bt−1 = {(j, bt−1,j ) : j ∈ Ht−1 } ,

(6)

XXXX, 2026.

5

Algorithm 3 ServerFedLNS: server-side signature screening and aggregation Require: Global model parameters wt , returned client models and counts {(wt,i , nt,i )}i∈St , previous bank Bt−1 , previous ID set Ht−1 , bank activation threshold Mact Ensure: Updated global model parameters wt+1 , retained client set Rt , refreshed bank Bt , refreshed ID set Ht 1: for each client i ∈ St do 2: Compute ∆wt,i = wt,i − wt 3: Extract st,i,ℓ = pℓ (wt,i ) − pℓ (wt ) for all ℓ ∈ L 4: end for 5: Refresh Bt and Ht using (8)–(10) 6: Compute Ntbank = |Ht | 7: if Ntbank < Mact then 8: Set wt+1 = wt and Rt = ∅ 9: return (wt+1 , Rt , Bt , Ht ) 10: end if 11: for each normalization layer ℓ ∈ L do 12: Compute mt,ℓ and dt,ℓ using (14)–(16) 13: end for 14: for each client i ∈ St do 15: Compute zt,i using (17)–(18) 16: Compute δt,i using (20) 17: end for 18: Rt ← BICGMMRetain ({zt,i , δt,i }i∈St ) 19: Aggregate retained updates using (34) to obtain wt+1 20: return (wt+1 , Rt , Bt , Ht )

where Ht−1 denotes the set of client identifiers currently maintained in the bank. For each client j, the stored entry consists of an ordered collection of layer-wise normalization-signature vectors: bt−1,j = (bt−1,j,ℓ )ℓ∈L ,

(7)

where bt−1,j,ℓ represents the latest recorded signature corresponding to normalization layer ℓ for client j. After extracting the current signatures, the server replaces the bank entries of participating clients and retains the latest entries of nonparticipating clients (Algorithm 3, line 5). The updated client-ID set is Ht = Ht−1 ∪ St .

(8)

The refreshed bank is Bt = {(j, bt,j ) : j ∈ Ht } ,

(9)

where ( bt,j =

st,j ,

if j ∈ St ,

bt−1,j ,

if j ∈ Ht−1 and j ∈ / St .

(10)

If a client participates in the current communication round, its previous bank entry is replaced by the newly extracted normalization-layer signature. If the client does not participate, its latest stored signature remains unchanged. As shown in Algorithm 3, the bank is refreshed before current-round screening. Thus, all current signatures contribute to the refreshed bank, including signatures associated with updates that may subsequently be excluded from aggregation. The history bank is maintained entirely by the server and is not stored, accessed, or received by participating clients.

After refreshing the bank, the server computes its current coverage and applies the activation rule (Algorithm 3, lines 6–9). Screening and global aggregation begin when Ntbank ≥ Mact ,

(11)

where Ntbank := |Ht | is the number of distinct client identities represented in the history bank. Before this condition is satisfied, the current round is used to collect normalization-layer signatures, and the global model remains unchanged, i.e., all participating clients in St contribute to the global model: wt+1 = wt .

(12)

The activation threshold does not cap the bank size. After screening begins, the bank continues to incorporate newly observed client identities and ideally approaches full population coverage. The bank-coverage ablation shows that broader coverage at activation generally improves the early active-training performance, including monotonic perplexity improvements under the strongest IID attack setting; see Appendix E, particularly Tables LII and LIII. Waiting for every client to enter the bank can, however, be impractically slow when participation is intermittent or nonuniform. The threshold Mact therefore provides a practical starting point for screening with a partially populated bank while allowing the bank to continue growing thereafter. Under the uniform random participation model used in the experiments, Appendix A bounds the probability that such a partial bank contains a malicious majority and analyzes the time required to reach a chosen activation coverage. D. Layer-Wise Robust Bank Reference Once the activation condition in (11) is satisfied, the server computes the layer-wise median/MAD reference (Algorithm 3, lines 11–13) and then calculates the standardized representation and scalar deviation score of each current client (lines 15–16). These quantities form the input to the GMM screening rule. 1) Robust reference statistics: For vectors u(1) , . . . , u(m) ∈ Rd , the coordinate-wise median is defined as

(m) med u(1) , . . . , u(m) [r] := median u(1) , r , . . . , ur (13) where r = 1, . . . , d denotes the coordinate index. Compared with the arithmetic mean, the coordinate-wise median reduces the influence of extreme values and provides a robust estimate of the population center [48]. For each normalization layer ℓ ∈ L, the server gathers the corresponding normalization-layer signatures from all clients currently stored in the history bank, {bt,j,ℓ }j∈Ht , and computes their coordinate-wise median:   mt,ℓ := med {bt,j,ℓ }j∈Ht ∈ Rdℓ . (14)

XXXX, 2026.

6

The vector mt,ℓ serves as a robust estimate of the central tendency of the normalization-layer signatures at layer ℓ. To quantify the dispersion of the client population around the center, F ED LNS further computes the coordinate-wise MAD:   dℓ e t,ℓ := med {|bt,j,ℓ − mt,ℓ |} (15) d j∈Ht ∈ R ,

substantial portion of the standardized feature space. The subsequent GMM stage identifies the population with lower deviation from the bank reference under the assumption that benign signatures form the dominant reference population.

where the absolute value is applied element-wise. Compared with variance-based measures, MAD provides a robust estimate of scale in the presence of outliers and adversarially manipulated updates [49]. To prevent numerical instability caused by near-zero dispersion values, each coordinate of the MAD vector is lower-bounded by a small positive constant ϵ > 0, yielding n o e t,ℓ [r], ϵ , dt,ℓ [r] := max d r = 1, . . . , dℓ . (16)

Algorithm 4 fits one- and two-component diagonal GMMs (Gaussian mixture models) to clients’ standardized signatures and selects the model order using BIC (lines 2–4). If two components are selected, the algorithm assigns component labels and retains the component with the smaller median deviation from the bank reference (lines 8–13). If the client population is represented by one component, all participating clients are assumed to be benign and retained. Otherwise, the clients are partitioned into two groups, and the group with the smaller overall deviation from the historical normalization-signature reference is selected for aggregation. This rule avoids forcing a two-component partition when a one-component model is selected (e.g., if no malicious clients are present in this training round) and requires neither a predefined maliciousclient fraction nor a manually specified anomaly threshold. In Algorithm 4, the current-round feature matrix is   Nt ×D Zt := z⊤ . (21) t,i i∈S ∈ R

This regularization avoids division-by-zero issues and prevents excessively large standardized deviations arising from extremely small scale estimates. The resulting pair (mt,ℓ , dt,ℓ ) constitutes the robust center and scale reference for normalization layer ℓ. 2) Client deviation scores: After constructing the robust layer-wise reference statistics, F ED LNS projects each current client signature into a bank-standardized feature space, as shown in Algorithm 3. This transformation quantifies the deviation of a client’s normalization-layer signature from the historical reference while placing coordinates from different normalization layers on comparable robust scales within the current screening round. For client i ∈ St and normalization layer ℓ ∈ L, the standardized signature is defined as zt,i,ℓ :=

st,i,ℓ − mt,ℓ ∈ Rdℓ , dt,ℓ

(17)

(18)

where the dimensionality of the resulting feature vector is X D := dℓ . (19) ℓ∈L

To obtain an overall measure of deviation from the historical population, F ED LNS computes a scalar deviation score for each participating client: δt,i := median (|zt,i [1]|, . . . , |zt,i [D]|) .

t

For each candidate number of mixture components c ∈ {1, 2}, F ED LNS fits a diagonal-covariance GMM with covariance regularization 10−4 : fc (z; Θc ) =

c X

πc,k N z; µc,k , Diag σ 2c,k

,

(22)

k=1

where subtraction and division are performed element-wise. Here, st,i,ℓ denotes the normalization-layer signature of client i, while mt,ℓ and dt,ℓ are the robust median and MAD reference vectors derived from the history bank. The complete standardized representation of client i is obtained by concatenating the standardized layer-wise signatures according to a fixed layer ordering: zt,i := concatℓ∈L (zt,i,ℓ ) ∈ RD ,

E. BIC-Guided One-vs-Two GMM Screening

(20)

The score δt,i is the median absolute standardized deviation across the coordinates of the client’s feature vector. A smaller value indicates that the client’s normalization-layer signature is more consistent with the historical reference, whereas a larger value indicates atypical behavior across a

where  c Θc = πc,k , µc,k , σ 2c,k k=1 ,

πc,k > 0,

c X

πc,k = 1.

k=1

(23)

The fitted parameter set is b c := arg max Θ Θc

X

log fc (zt,i ; Θc ).

(24)

i∈St

The BIC score for each candidate model order c is X b c ) + pc log Nt , BIC(c) = −2 log fc (zt,i ; Θ

(25)

i∈St

where the number of free parameters in a diagonal ccomponent GMM is pc = (c − 1) + 2cD.

(26)

F ED LNS first selects the number of mixture components b ct as b ct := arg min BIC(c). (27) c∈{1,2}

If b ct = 1, all participating clients are retained: Rt = St .

(28)

XXXX, 2026.

7

Algorithm 4 BICGMMRetain: adaptive one-vs-two GMM retain rule Require: Standardized features {zt,i }i∈St , deviation scores {δt,i }i∈St Ensure: Retained client set Rt 1: Form feature matrix Zt = [z⊤ t,i ]i∈St 2: Fit one- and two-component diagonal GMMs to Zt 3: Compute BIC(1) and BIC(2) using (25) 4: Set b ct = arg minc∈{1,2} BIC(c) 5: if b ct = 1 then 6: return Rt = St 7: else 8: Assign component labels yt,i using (29) 9: for k ∈ {1, 2} do 10: Compute ρt,k using (30) 11: end for 12: Set retained component ktret = arg mink∈{1,2} ρt,k 13: return Rt = {i ∈ St : yt,i = ktret } 14: end if

A. Influence of Normalization-Parameter Changes on Token Margins For an input sequence x = (x1 , . . . , xm ), a language model with parameters w produces a logit vector z(x; w) ∈ R|V| ,

(35)

where V is the token vocabulary. The probability assigned to token y ∈ V is πw (y | x) = P

exp(zy (x; w)) . v∈V exp(zv (x; w))

(36)

For two candidate tokens u, v ∈ V, define the pairwise token margin mu,v (x; w) := zu (x; w) − zv (x; w).

(37)

The margin measures the model’s relative preference for token u over token v and satisfies If b ct = 2, each client i is assigned to a component:    b 2,k , Diag σ b 22,k . yt,i := arg max π b2,k N zt,i ; µ

mu,v (x; w) = log

πw (u | x) . πw (v | x)

(38)

The median deviation score of component k is

Thus, a client update that changes the pairwise margin also changes the probability ratio assigned to the two tokens. Partition the model parameters as

ρt,k := median ({δt,i : i ∈ St , yt,i = k}) ,

w = (p, q),

k∈{1,2}

(29) k ∈ {1, 2}. (30)

The retained component is ktret := arg min ρt,k . k∈{1,2}

The retained client set is  Rt = i ∈ St : yt,i = ktret .

(31)

(32)

After the retained set Rt is determined, the server aggregates the complete retained updates (Algorithm 3, line 19). For client i ∈ Rt , the sample-weighted aggregation coefficient is αt,i := P

nt,i

.

(33)

αt,i ∆wt,i .

(34)

j∈Rt nt,j

The next global model is wt+1 = wt +

X i∈Rt

F ED LNS therefore acts as a server-side screening stage before full-model aggregation. The retained updates can be passed to the sample-weighted rule above or to another compatible FL aggregation rule.

(39)

where p contains the trainable normalization-layer parameters included in the F ED LNS signature and q contains the remaining model parameters. For client i at communication round t, write the local update as ∆wt,i = (∆pt,i , ∆qt,i ). (40) Here, ∆pt,i is the concatenated normalization-parameter component corresponding to the normalization-layer signature used by F ED LNS. Because a client update affects the model across inputs, we define its prompt-averaged pairwise-margin change as ∆m̄t,i (u, v) := Ex∼X [mu,v (x; wt + ∆wt,i ) − mu,v (x; wt )] . (41) Here, mu,v (x; w) denotes the margin for an individual prompt, whereas ∆m̄t,i (u, v) denotes the change in that margin averaged over the prompt distribution X . The following proposition identifies the direct first-order contribution of the normalization-parameter component to the expected token-margin change.

IV. T HEORETICAL A NALYSIS

Proposition 1 (Normalization-parameter contribution to token-margin change). Suppose mu,v (x; w) is twice continuously differentiable in a neighborhood of wt , and its Hessian is uniformly bounded in operator norm by Lm . Then

This section analyzes two aspects of F ED LNS. First, we examine how changes in trainable normalization-layer parameters contribute to changes in token preferences. Second, we study how screening suspicious clients affects optimization of the benign global objective.

⊤ ⊤ ∆m̄t,i (u, v) = gp,u,v ∆pt,i + gq,u,v ∆qt,i + Rt,i (u, v), (42) where gp,u,v := Ex∼X [∇p mu,v (x; wt )] , (43) gq,u,v := Ex∼X [∇q mu,v (x; wt )] ,

XXXX, 2026.

8

and the prompt-averaged remainder satisfies Lm 2 (44) |Rt,i (u, v)| ≤ ∥∆wt,i ∥2 . 2 Proposition 1 shows that the normalization-parameter component contributes directly to the first-order change in token margins through ⊤ gp,u,v ∆pt,i .

(45)

Consequently, normalization-parameter changes contribute directly to first-order changes in relative token preferences. B. Effect of Screening on the Aggregated Update We next analyze the effect of excluding suspicious clients before aggregation. Let Cben denote the set of benign clients. The benign global objective is X X F (w) := λi Fi (w), λi ≥ 0, λi = 1, i∈Cben

i∈Cben

(46) where Fi is the local objective of benign client i. At round t, F ED LNS returns a retained client set Rt ⊆ St . For retained client i, define the normalized aggregation coefficient nt,i , i ∈ Rt . (47) αt,i = P j∈Rt nt,j The screened update is X

wt+1 = wt +

αt,i ∆wt,i .

(48)

we have PT −1

  ηt E ∥∇F (wt )∥22 2 (F (w0 ) − Finf ) ≤ ST ST
PT −1 P −1 2 2 t=0 ηt + LF ηt2 E[βt2 ] LF Tt=0 ηt E[νt2 ] + + . ST ST (54)

t=0

Theorem 1 bounds the step-size-weighted average squared gradient norm of the benign objective. The righthand side contains three terms: the initial optimization gap, the contribution of the screening-error upper bound βt (i.e., the discrepancy between the expected retained update direction and the gradient of the benign global objective), and the stochastic update-noise contribution governed by νt2 . A smaller βt indicates that the expected direction formed from the retained clients more closely approximates the benign global gradient. The theorem characterizes optimization under such an approximation, but it does not independently guarantee that the F ED LNS screening rule makes βt small in every round. The following corollary gives conditions under which the weighted average gradient norm vanishes. Corollary 1 (Convergence to first-order stationarity). Suppose PT −1 2 T −1 X t=0 ηt → 0. (55) ST = ηt → ∞, ST t=0 Assume that the update variance is uniformly bounded: E[νt2 ] ≤ ν 2

i∈Rt

For analysis, write the same update as wt+1 = wt − ηt gt ,

(49)

where ηt > 0 is an effective step size and gt is the effective retained update direction. Define the screening error et := E [gt | Ft ] − ∇F (wt ),

Theorem 1 (Finite-time bound under screened aggregation). Assume that F is LF -smooth and lower bounded by Finf . Suppose that, for every round t, there exist nonnegative quantities βt and νt2 such that ∥et ∥2 ≤ βt , and

βt ≥ 0,

h i 2 E ∥gt − E[gt | Ft ]∥2 Ft ≤ νt2 .

(51) (52)

If 0 ≤ ηt ≤ 1/(4LF ), then for any T ≥ 1 with ST :=

T −1 X t=0

ηt > 0,

for some finite constant ν 2 . Also assume that the weighted average screening error vanishes: PT −1 2 t=0 ηt E[βt ] → 0. (57) ST Then

(53)

PT −1 t=0

(50)

where Ft contains the information available before the round-t screening, i.e., the vector et measures the difference between the expected retained update direction and the gradient of the benign global objective.

(56)

  ηt E ∥∇F (wt )∥22 → 0. ST

(58)

Equivalently, if a random round τ is sampled from {0, . . . , T − 1} with probability ηt , (59) Pr(τ = t) = ST then   E ∥∇F (wτ )∥22 → 0.

(60)

Corollary 1 shows that the expected gradient norm of a step-size-weighted random iterate vanishes when the step sizes provide sufficient cumulative progress, the stochastic update variance remains bounded, and the average screening error becomes negligible. The final condition requires the retained update direction to approach the benign gradient direction. The following proposition decomposes the screening error into quantities associated with retained malicious updates, benign-client selection, and local-training drift.

XXXX, 2026.

9

Proposition 2 (Interpretation of screening error). Let X µt := αt,a (61) a∈Rt ∩Cmal

denote the total aggregation weight assigned to retained malicious clients. When µt < 1, define the renormalized gradient of the retained benign clients as X αt,i ∇Fi (wt ). (62) ḡtB,ret := 1 − µt i∈Rt ∩Cben

The retained-benign representativeness error is χt := ḡtB,ret − ∇F (wt )

.

(63)

2

Thus, χt measures how closely the retained benign clients, after their weights are renormalized, represent the full benign-client gradient. Let ϱt denote the retained benign local-training drift, as formally defined in Appendix B-C. Under the conditions stated there, the screening error satisfies ∥et ∥2 ≤ (1 − µt )χt + 2Gµt + ϱt .

(64)

Consequently, a valid choice of the screening-error bound in Theorem 1 is βt := (1 − µt )χt + 2Gµt + ϱt .

(65)

Proposition 2 identifies three sources of screening error. The term µt is small when malicious clients receive little retained aggregation weight. The term χt is small when the retained benign clients remain representative of the benign population. The term ϱt is small when local training remains close to the global model. Together, these quantities determine the screening-error contribution in Theorem 1. For a constant effective step size, the finite-time result yields a stationary-neighborhood bound: Corollary 2 (Stationary neighborhood with constant step size). Suppose that ηt = η ≤ 1/(4LF ), ∀t. Define 2

β T :=

T −1

1 X E[βt2 ], T t=0

T −1

ν 2T :=

1 X E[νt2 ]. T t=0

(66)

Then T −1  2 (F (w0 ) − Finf ) 1 X  E ∥∇F (wt )∥22 ≤ T t=0 ηT 2

+ 2 (1 + LF η) β T + LF ην 2T . (67) Corollary 2 shows that, with a constant effective step size, the average gradient norm approaches a neighborhood whose size is controlled by the average screening error and the stochastic update variance. Complete assumptions, proofs, the screening-error decomposition, and the local-training drift bound are provided in Appendix B.

V. E XPERIMENTAL E VALUATION In this section, we evaluate F ED LNS in federated transformer-based language-model training under targetcorruption attacks. The evaluation examines robustness across model architectures, IID and non-IID data partitions, and malicious-client fractions ranging from 0% to 40%. A. Experimental Setup We evaluate F ED LNS on three transformer model families with different language-modeling objectives: • A GPT-style causal transformer [50] trained from scratch on WikiText using autoregressive next-token prediction. • A BERT-style masked-language transformer [51] trained from scratch on Tiny Shakespeare using bidirectional masked-token prediction. • A LLaMA-style decoder-only causal transformer [52] trained from scratch on StackOverflow text. Settings. Unless otherwise specified, each experiment uses K = 200 clients, a 10% participation rate, 20 clients sampled uniformly at random per communication round, and 200 communication rounds [53], [54]. Each configuration is repeated using three random seeds, {100, 200, 300}, under IID and two-shard non-IID data partitions [55]. For the K = 200 clients considered in our experiments, we use the default activation coverage Mact = 100, so screening begins after signatures from half of the client identities have been observed. Malicious clients. We consider a target-corruption attack in which a fixed subset of the client population is designated as malicious at the beginning of each training run and remains malicious throughout that run. For every malicious client selected in a communication round, the input sequence remains unchanged, while the target token at each supervised prediction position is independently replaced, with probability pcorr , by a randomly sampled valid vocabulary token; padding and EOS tokens are excluded from the replacement candidates. We set pcorr = 1.0, so every eligible target position used for local training is replaced. The malicious-client fraction is defined over the complete client population. In each communication round, 20 clients are sampled uniformly without replacement from the K = 200 clients; therefore, the number of malicious clients participating in an individual round varies according to the sampled client set. Performance metrics. We report perplexity and tokenlevel semantic entropy. For the causal language models, perplexity is the exponential of the mean next-token crossentropy. For the masked-language model, masked-token perplexity is the exponential of the mean cross-entropy over masked positions. Token-level semantic entropy is the mean predictive entropy over the evaluated token positions and measures the concentration of the corresponding predictive distributions. For each random seed, the checkpoint with the lowest test loss is selected, and the test loss, perplexity, and tokenlevel semantic entropy are reported from that same model

XXXX, 2026.

10

1600

2800

375 2600 2400

2000 1800

Test PPL

325

2200

Test PPL

Test PPL

350

1400

1200

1000

300 275 250

1600

800

225

1400 1200

200

600

E m vg an ed nd NS ru di AM m dA ou dL -K FL im -B Fe Me Fe lti Tr m u r M No

(a) GPT/WikiText, IID partitions

Fe

vg dA

u

o -B m or

nd

N

E m vg an ed nd NS ru di AM m dA ou dL -K FL im -B Fe Me Fe lti Tr m u r M No

an

E m ed NS ru di AM m dL -K FL im Me Fe lti Tr u M

(b) BERT/Shakespeare, IID partitions

(c) LLaMA/StackOverflow, IID partitions 375

3250

350

1600

3000

2250 2000 1750

Test PPL

Test PPL

2500

Test PPL

325

1400

2750

1200

250 225

1500 600

vg dA

Fe

-B

rm

No

nd ou

Me

an di

im Tr

ed

m

Mu

lti

-

um Kr

FL

E AM

S LN

d Fe

(d) GPT/WikiText, non-IID partitions

275

1000 800

1250

300

Fe

200 vg dA No

o -B rm

d un

a di Me

n

E m ed NS ru AM m dL -K FL im Fe lti Tr Mu

(e) BERT/Shakespeare, non-IID partitions

E m vg an ed nd NS ru di AM m dA ou dL -K FL im -B Fe Me Fe lti Tr m u r M No

(f) LLaMA/StackOverflow, non-IID partitions

Fig. 2: Perplexity (lower is better) under target manipulation affecting 40% of the client population. Bars show the mean over three random seeds, and error bars show one sample standard deviation. F ED LNS consistently shows the lowest perplexity, across both IID and non-IID data partitions.

state. Results are reported as the mean and sample standard deviation across seeds {100, 200, 300} for each model, data partition, aggregation method, and malicious-client fraction. Baselines. We compare F ED LNS with six representative aggregation methods covering conventional averaging, norm-constrained aggregation, coordinate-wise robust statistics, distance-based selection, and clustering-based filtering. FedAvg performs sample-weighted averaging and serves as the non-robust baseline [56]. Norm-Bounded FedAvg clips client updates before aggregation to limit the effect of large update magnitudes [16]. Coordinate-wise Median and Trimmed Mean apply robust statistics independently to each model coordinate [15]. Multi-Krum selects updates with mutually consistent distance profiles [14]. FLAME combines clustering, norm clipping, and Gaussian noise injection [17]. Trimmed Mean and Multi-Krum are configured using the true population-level malicious-client fraction, whereas F ED LNS does not require this fraction as an input. Appendix C provides the complete dataset, tokenization, partition, architecture, optimization, attack, baseline, and F ED LNS configurations; Appendix D reports the full malicious-client-fraction results; and Appendices E and F provide the bank-coverage-at-activation and GMMselection ablations, respectively.

B. Perplexity and Token-Level Semantic Entropy Figure 2 compares the perplexity of F ED LNS and the six baselines when 40% of the client population is malicious. Figures 2(a)–2(c) report the IID results, whereas Figures 2(d)–2(f) report the non-IID results. Error bars show one sample standard deviation across seeds {100, 200, 300}. Under IID data partitions, F ED LNS achieves the lowest perplexity for all three evaluated model families. Relative to the strongest competing baseline, it reduces perplexity by 11.59% for GPT/WikiText (from 1449.72 to 1281.63), 3.88% for BERT/Tiny Shakespeare (from 725.76 to 697.62), and 4.98% for LLaMA/StackOverflow (from 228.52 to 217.14). These results show that the global models trained with F ED LNS exhibit less attack-induced degradation across all three architectures. Under the two-shard non-IID partitions, F ED LNS reduces perplexity by 18.57% for GPT/WikiText (from 1579.81 to 1286.50), 10.05% for BERT/Tiny Shakespeare (from 731.73 to 658.21), and 5.67% for LLaMA/StackOverflow (from 233.42 to 220.18) relative to the strongest non-F ED LNS baseline. The improvement across all three non-IID settings shows that the downstream benefit of the screening procedure persists when benign clients have heterogeneous local data distributions.

11

9.0

9.5

8.5

9.0 8.5 8.0 7.5

7.5

Semantic Entropy

10.0

Semantic Entropy

Semantic Entropy

XXXX, 2026.

8.0 7.5 7.0

7.0

6.5

6.0

6.5

7.0

5.5 6.0

6.5

E m vg an ed nd NS ru di AM m dA ou dL -K FL im -B Fe Me Fe lti Tr m u r M No

E m vg an ed nd NS ru di AM m dA ou dL -K FL im -B Fe Me Fe lti Tr m u r M No

E m vg an ed nd NS ru di AM m dA ou dL -K FL im -B Fe Me Fe lti Tr m u r M No

(a) GPT/WikiText, IID partitions

(b) BERT/Shakespeare, IID partitions

(c) LLaMA/StackOverflow, IID partitions 7.00

10.0

Semantic Entropy

8.5

9.0 8.5 8.0 7.5 7.0 6.5

Semantic Entropy

Semantic Entropy

9.5

8.0 7.5 7.0 6.5

6.75 6.50 6.25 6.00 5.75 5.50

6.0

E vg an ed nd NS um di AM m dA ou dL Kr FL im -B Fe Me tiFe Tr ul rm M o N

E vg an ed nd NS um di AM m dA ou dL Kr FL im -B Fe Me tiFe Tr ul rm M o N

(d) GPT/WikiText, non-IID partitions

(e) BERT/Shakespeare, non-IID partitions

E m vg an ed nd NS ru di AM m dA ou dL -K FL im -B Fe Me Fe lti Tr m u r M No

(f) LLaMA/StackOverflow, non-IID partitions

Fig. 3: Token-level semantic entropy (lower is better) under target manipulation affecting 40% of the client population. Bars show the mean over three random seeds, and error bars show one sample standard deviation. F ED LNS consistently shows the lowest semantic entropy, across both IID and non-IID data partitions.

Because lower entropy does not necessarily imply greater correctness, we interpret the entropy results jointly with perplexity. Figure 3 reports token-level semantic entropy for the three model families under 40% population-level target manipulation. Under IID data partitions, Figures 3(a)–3(c) show that F ED LNS achieves the lowest entropy in all three settings. Relative to the strongest non-F ED LNS baseline, it reduces entropy by 2.30% for GPT/WikiText (from 6.96 to 6.80), 3.41% for BERT/Tiny Shakespeare (from 6.46 to 6.24), and 1.60% for LLaMA/StackOverflow (from 5.61 to 5.52). Under non-IID data partitions, Figures 3(d)–3(f) show reductions of 2.95% for GPT/WikiText (from 7.13 to 6.92), 6.50% for BERT/Tiny Shakespeare (from 6.62 to 6.19), and 1.95% for LLaMA/StackOverflow (from 5.63 to 5.52). Together with the corresponding perplexity reductions, these results show that the global models obtained after F ED LNS screening achieve lower test loss and lower uncertainty over competing token predictions under the evaluated attack.

ing, the population-level malicious-client fraction increases from 0% to 40%. For perplexity, F ED LNS exhibits limited degradation as the malicious-client fraction increases. Its perplexity rises by 2.82%, from 1246.57 at 0% to 1281.63 at 40%. Over the same range, perplexity increases by 27.97% for FedAvg, 48.32% for Norm-Bounded FedAvg, 17.95% for Median, 36.05% for Trimmed Mean, 16.72% for MultiKrum, and 15.15% for FLAME. F ED LNS achieves the lowest perplexity at every nonzero attack fraction from 10% to 40%. Token-level semantic entropy exhibits a similar attackedsetting trend. Although Norm-Bounded FedAvg has the lowest entropy in the attack-free setting, its entropy increases by 47.94% between 0% and 40% malicious clients. In contrast, the entropy of F ED LNS increases by 1.04%, from 6.73 to 6.80, and is the lowest among the evaluated methods for every nonzero attack fraction. The simultaneous stability of perplexity and entropy indicates that F ED LNS limits both predictive degradation and uncertainty as the target-corruption rate increases.

C. Impact of Malicious Clients Appendix D reports the complete numerical results across malicious-client fractions for all model families and data partitions. For GPT/WikiText under IID partition-

D. Impact of Bank Coverage at Activation As defined by Eq. (11), F ED LNS begins screening and global aggregation after the bank reaches the activation

XXXX, 2026.

12

coverage Mact . We evaluate Mact ∈ {0, 50, 100, 150}, corresponding to immediate, 25%, 50%, and 75% population coverage. Setting Mact = 0 activates screening immediately using the currently available bank; it does not remove the bank. In all settings, the bank continues growing after activation as additional client identities are observed. The ablation reports performance over the first ten active aggregation rounds under IID partitioning with 40% malicious clients. Relative to immediate activation with Mact = 0, activation at Mact = 150 reduces perplexity by 10.62% for GPT/WikiText, 31.89% for BERT/Tiny Shakespeare, and 74.68% for LLaMA/StackOverflow. Tokenlevel semantic entropy decreases by 2.53%, 20.20%, and 25.33%, respectively. Thus, the main ablation exhibits a monotonic perplexity improvement as activation coverage increases. These results support maximizing bank coverage whenever the resulting activation delay is acceptable. A broader bank provides a more representative cross-client reference, and the bank should continue growing toward full population coverage after screening begins. The activation threshold is therefore a practical starting criterion rather than a desired final bank size. Smaller activation coverages remain usable when waiting for broader coverage would delay training excessively, but they should not be interpreted as equivalent to a fully populated bank. The largest evaluated activation coverage is 75%; the experiments therefore support the observed benefit of increasing coverage within this range rather than establishing an empirical optimum at 100% coverage. The GMM-selection ablation shows that BICselected one-vs-two-component screening and forced two-component screening produce nearly identical selected-checkpoint results in the evaluated settings. BIC retains the option to keep all participating clients when the one-component model is preferred, but the endpoint metrics exhibit limited sensitivity to this model-selection rule. Complete results are provided in Appendix F. VI. C ONCLUSION This paper introduced F ED LNS, a lightweight serverside framework that develops a normalization-layer update signature to screen malicious updates before standard fullmodel aggregation. Moreover, a server-side retain rule was designed, which constructs a robust cross-client reference from the latest observed signatures and combines median/MAD standardization with BIC-guided mixture modeling. For GPT-style, BERT-style, and LLaMA-style models under both IID and non-IID partitions, F ED LNS achieved lower perplexity than all six baselines when 40% of the client population was malicious. The maximum reductions were 18.57% in perplexity and 6.50% in token-level semantic entropy. ACKNOWLEDGEMENTS This work was in part supported by the Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science

and Technology) through the Carnegie Mellon Portugal Program. R EFERENCES [1] H. Cai, H. Dong, H. Wang, K. Li, and O. B. Akan, “Graph representation-based model poisoning on federated LLMs in CyberEdge networks,” arXiv preprint arXiv:2507.01694, 2025. [2] H. Li, H. Madhukumar, N. Methley et al., “Future factories with 6G: Agentic AI and cyber–physical digital twins,” IEEE Internet of Things Journal, vol. 13, no. 9, pp. 17 990–18 006, 2026. [3] X. An, L. Shen, Y. Luo, H. Hu, and D. Tao, “Adaptive batch size time evolving stochastic gradient descent for federated learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 2, pp. 1158–1170, 2026. [4] Y. Cheng, W. Zhang, Z. Zhang et al., “Toward federated large language models: Motivations, methods, and future directions,” IEEE Communications Surveys and Tutorials, vol. 27, no. 4, pp. 2733– 2764, 2025. [5] K. Li, Z. Zhang, A. Pourkabirian, W. Ni, F. Dressler, and O. B. Akan, “Towards resilient federated learning in CyberEdge networks: Recent advances and future trends,” arXiv preprint arXiv:2504.01240, 2025. [6] J. Hu, D. Wang, Z. Wang et al., “Federated large language model: Solutions, challenges and future directions,” IEEE Wireless Communications, 2024. [7] Y. Yao, J. Zhang, J. Wu et al., “Federated large language models: Current progress and future directions,” arXiv preprint arXiv:2409.15723, 2024. [8] Z. Wang, Y. Zhou, Y. Shi, and K. B. Letaief, “Federated fine-tuning for pre-trained foundation models over wireless networks,” IEEE Transactions on Wireless Communications, 2025. [9] T. Webb, S. S. Mondal, and I. Momennejad, “A brain-inspired agentic architecture to improve planning with LLMs,” Nature Communications, vol. 16, no. 1, p. 8633, 2025. [10] H. Yang, J. Chen, M. Siew, T. Lorido-Botran, and C. Joe-Wong, “LLM-powered decentralized generative agents with adaptive hierarchical knowledge graph for cooperative planning,” arXiv preprint arXiv:2502.05453, 2025. [11] P. Wu, T. Imbiriba, and P. Closas, “A Bayesian framework for clustered federated learning,” IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 48, no. 03, pp. 3471–3481, 2026. [12] H. Liu, W. Xue, Y. Chen et al., “A survey on hallucination in large vision-language models,” arXiv preprint arXiv:2402.00253, 2024. [13] Z. Bai, P. Wang, T. Xiao, T. He, Z. Han, Z. Zhang, and M. Z. Shou, “Hallucination of multimodal large language models: A survey,” arXiv preprint arXiv:2404.18930, 2024. [14] P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” Advances in Neural Information Processing Systems, vol. 30, 2017. [15] D. Yin, Y. Chen, R. Kannan, and P. Bartlett, “Byzantine-robust distributed learning: Towards optimal statistical rates,” in Proceedings of the 35th International Conference on Machine Learning. PMLR, 2018, pp. 5650–5659. [16] Z. Sun, P. Kairouz, A. T. Suresh, and H. B. McMahan, “Can you really backdoor federated learning?” arXiv preprint arXiv:1911.07963, 2019. [17] T. D. Nguyen, P. Rieger, H. Chen et al., “FLAME: Taming backdoors in federated learning,” in 31st USENIX Security Symposium (USENIX Security 22), 2022, pp. 1415–1432. [18] Z. Zhang, Q. Su, and X. Sun, “Dim-krum: Backdoor-resistant federated learning for nlp with dimension-wise krum-based aggregation,” in Findings of the Association for Computational Linguistics: EMNLP 2022, 2022, pp. 339–354. [19] M. Tao, Y. Tian, W. Tu, Y. Yang, X. Yang, and X. Tang, “Safefedllm: Delving into the safety of federated large language models,” in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026, pp. 24 405–24 420. [20] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016. [21] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, 2017. [22] B. Zhang and R. Sennrich, “Root mean square layer normalization,” in Advances in Neural Information Processing Systems, vol. 32, 2019.

XXXX, 2026.

[23] R. Xiong, Y. Yang, D. He, K. Zheng, S. Zheng, C. Xing, H. Zhang, Y. Lan, L. Wang, and T.-Y. Liu, “On layer normalization in the transformer architecture,” in Proceedings of the 37th International Conference on Machine Learning. PMLR, 2020, pp. 10 524–10 533. [24] Z. Zhang, X. Cao, J. Jia, and N. Z. Gong, “FLDetector: Defending federated learning against model poisoning attacks via detecting malicious clients,” in Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 2545– 2555. [25] X. Cao, M. Fang, J. Liu, and N. Z. Gong, “FLTrust: Byzantinerobust federated learning via trust bootstrapping,” in Network and Distributed System Security Symposium (NDSS), 2021. [26] X. Cao, Z. Zhang, J. Jia, and N. Z. Gong, “FLCert: Provably secure federated learning against poisoning attacks,” IEEE Transactions on Information Forensics and Security, vol. 17, pp. 3691–3705, 2022. [27] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. de Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for NLP,” in Proceedings of the 36th International Conference on Machine Learning. PMLR, 2019, pp. 2790–2799. [28] E. Ben Zaken, Y. Goldberg, and S. Ravfogel, “BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language models,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2022, pp. 1–9. [29] H. Liu, D. Tam, M. Muqeeth, J. Mohta, T. Huang, M. Bansal, and C. A. Raffel, “Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 1950–1965. [30] W. Qi, Y.-P. Ruan, Y. Zuo, and T. Li, “Parameter-efficient tuning on layer normalization for pre-trained language models,” arXiv preprint arXiv:2211.08682, 2022. [31] S. Brody, U. Alon, and E. Yahav, “On the expressivity role of LayerNorm in transformers’ attention,” in Findings of the Association for Computational Linguistics: ACL 2023, 2023, pp. 14 211–14 221. [32] X. Fang, Z. Huang, Z. Tian, M. Fang, Z. Pan, Q. Fang, Z. Wen, H. Pan, and D. Li, “Zero-resource hallucination detection for text generation via graph-based contextual knowledge triples modeling,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 22, 2025, pp. 23 868–23 877. [33] N. Nonkes, S. Agaronian, E. Kanoulas, and R. Petcu, “Leveraging graph structures to detect hallucinations in large language models,” arXiv preprint arXiv:2407.04485, 2024. [34] X. Guan, Y. Liu, H. Lin, Y. Lu, B. He, X. Han, and L. Sun, “Mitigating large language model hallucinations via autonomous knowledge graph-based retrofitting,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 16, 2024, pp. 18 126–18 134. [35] K. Furumai, Y. Wang, M. Shinohara, K. Ikeda, Y. Yu, and T. Kato, “Detecting dialogue hallucination using graph neural networks,” in 2023 International Conference on Machine Learning and Applications (ICMLA). IEEE, 2023, pp. 871–877. [36] S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal, “Detecting hallucinations in large language models using semantic entropy,” Nature, vol. 630, no. 8017, pp. 625–630, 2024. [37] Y. Chen, Q. Fu, Y. Yuan et al., “Hallucination detection: Robustly discerning reliable answers in large language models,” in Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, 2023, pp. 245–255. [38] G. Sriramanan, S. Bharti, V. S. Sadasivan, S. Saha, P. Kattakinda, and S. Feizi, “LLM-Check: Investigating detection of hallucinations in large language models,” Advances in Neural Information Processing Systems, vol. 37, pp. 34 188–34 216, 2024. [39] X. Zhou, M. Zhang, Z. Lee, W. Ye, and S. Zhang, “HADEMIF: Hallucination detection and mitigation in large language models,” in The Thirteenth International Conference on Learning Representations, 2025. [40] L. Guo, Y. Fang, F. Chen, P. Liu, and S. Xu, “Large language models with adaptive token fusion: A novel approach to reducing hallucinations and improving inference efficiency,” Authorea, 2024. [41] X. Chen, C. Wang, Y. Xue, N. Zhang, X. Yang, Q. Li, Y. Shen, L. Liang, J. Gu, and H. Chen, “Unified hallucination detection for multimodal large language models,” arXiv preprint arXiv:2402.03190, 2024. [42] A. Gunjal, J. Yin, and E. Bas, “Detecting and preventing hallucinations in large vision-language models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 16, 2024, pp. 18 135–18 143.

13

[43] W. Su, C. Wang, Q. Ai, Y. Hu, Z. Wu, Y. Zhou, and Y. Liu, “Unsupervised real-time hallucination detection based on the internal states of large language models,” arXiv preprint arXiv:2403.06448, 2024. [44] C. Chen, K. Liu, Z. Chen, Y. Gu, Y. Wu, M. Tao, Z. Fu, and J. Ye, “INSIDE: LLMs’ internal states retain the power of hallucination detection,” arXiv preprint arXiv:2402.03744, 2024. [45] X. Du, C. Xiao, and S. Li, “HaloScope: Harnessing unlabeled LLM generations for hallucination detection,” Advances in Neural Information Processing Systems, vol. 37, pp. 102 948–102 972, 2024. [46] Y. Liu, G. Liu, R. Zhang, D. Niyato, Z. Xiong, D. I. Kim, K. Huang, and H. Du, “Hallucination-aware optimization for large language model-empowered communications,” arXiv preprint arXiv:2412.06007, 2024. [47] H. Liu, X. Geng, L. Lee, I. Mordatch, S. Levine, S. Narang, and P. Abbeel, “Towards better few-shot and finetuning performance with forgetful causal language models,” arXiv preprint arXiv:2210.13432, 2022. [48] K. N. Kumar, C. K. Mohan, and L. R. Cenkeramaddi, “The impact of adversarial attacks on federated learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 5, pp. 2672–2691, 2023. [49] S. Xu, Y. Qian, and R. Q. Hu, “Data-driven edge intelligence for robust network anomaly detection,” IEEE Transactions on Network Science and Engineering, vol. 7, no. 3, pp. 1481–1492, 2019. [50] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019. [51] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pretraining of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171–4186. [52] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023. [53] Y. Sun, L. Shen, H. Sun, L. Ding, and D. Tao, “Efficient federated learning via local adaptive amended optimizer with linear speedup,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, pp. 14 453–14 464, 2023. [54] Y. Kang and B. Li, “Polaris: Accelerating asynchronous federated learning with client selection,” IEEE Transactions on Cloud Computing, vol. 12, no. 2, pp. 446–458, 2024. [55] J. Dong, H. Li, Y. Cong, G. Sun, Y. Zhang, and L. Van Gool, “No one left behind: Real-world federated class-incremental learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 4, pp. 2054–2070, 2023. [56] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. PMLR, 2017, pp. 1273–1282.

XXXX, 2026.

14

A PPENDIX C ONTENTS Appendix A: Probabilistic Analysis of Partial Bank Coverage at Activation A-A A Hypergeometric Model of Bank Contamination . . . . . . . . . . . . . . . . . . . . . . . . . . . A-B Concentration of the Malicious Fraction in the Bank . . . . . . . . . . . . . . . . . . . . . . . . . A-C Time to Reach the Activation Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

15 15 15 17

Appendix B: Proofs for Theoretical Analysis B-A Proof of the Normalization-Parameter Margin Decomposition . . . . . . . . . . . . . . . . . . . . B-B Proof of the Screened-Aggregation Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-C Screening-Error Decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-D Local-Training Drift Bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

18 18 20 23 24

Appendix C: Detailed Experimental Settings C-A Federated Learning Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-B Datasets and Tokenization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-C Model Architectures and Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-D Attack and Aggregation-Baseline Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-E FedLNS Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

26 26 26 27 28 29

Appendix D: Complete Results Across Malicious-Client Fractions D-A GPT-Style Causal Language Modeling on WikiText . . . . . . . . . . . . . . . . . . . . . . . . . . D-B BERT-Style Masked Language Modeling on Tiny Shakespeare . . . . . . . . . . . . . . . . . . . . D-C LLaMA-Style Causal Language Modeling on StackOverflow . . . . . . . . . . . . . . . . . . . . .

30 30 32 34

Appendix E: Bank Coverage at Activation E-A Reporting Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-B Activation Rounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-C Results Under IID Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-D Results Under Non-IID Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-E Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

36 36 36 38 41 41

Appendix F: GMM-Selection Ablation F-A Results Under IID Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-B Results Under Non-IID Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-C Interpretation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

45 45 48 48

XXXX, 2026.

15

A PPENDIX A P ROBABILISTIC A NALYSIS OF PARTIAL BANK C OVERAGE AT ACTIVATION F ED LNS ideally benefits from the broadest available client coverage, and the history bank continues expanding after screening begins. Waiting for full population coverage before beginning aggregation may nevertheless be impractically slow. We therefore use Mact to denote a partial bank coverage at which screening is permitted to start. This appendix analyzes, under uniform random participation, the probability that malicious clients form a majority of a partially populated bank and the number of rounds required to reach a chosen activation coverage. The analysis relies on the 50% breakdown point of the coordinate-wise median: the median/MAD reference remains in its benignmajority regime when malicious identities constitute fewer than half of the bank. The resulting bound provides a reliability condition for early activation; it does not imply that a partial bank has the same statistical quality as a fully populated bank. A. A Hypergeometric Model of Bank Contamination Let Ntbank := |Ht |

(68)

denote the number of distinct client identities represented in the refreshed history bank at round t. Let Cmal be the set of malicious clients, with 1 (69) |Cmal | = αK, 0≤α< , 2 where α is the global malicious-client fraction (if αK is not an integer, the same discussion applies with ⌊αK⌋ or ⌈αK⌉ malicious clients). Let Mt := |Ht ∩ Cmal | denote the number of malicious client identities represented in the bank, so the bank has a benign majority when Mt < Ntbank /2. The undesirable event is   N bank Et := Mt ≥ t . (70) 2 F ED LNS uses random client participation, where the server samples participating clients, in each communication round, uniformly at random from the population of K clients. Because the bank stores distinct client IDs, once a client has appeared, the bank holds that client’s most recent signature. Conditioned on the bank’s membership size Ntbank = b, the exchangeability of the sampling process implies that the bank can be treated as a uniformly random subset of size b drawn from K clients. Particularly, the server’s sampling rule does not prefer any client ID; any two subsets of the same size are equally likely after conditioning on the number of distinct observed clients. Since the malicious set Cmal is fixed at the start of the run, the question becomes how many of a random size-b subset, drawn without replacement from K clients, belong to a fixed subset of size αK, which is the hypergeometric setting. Hence, conditioned on Ntbank = b, we have Mt | Ntbank = b ∼ Hypergeometric(K, αK, b), (71) i.e., for feasible m, Pr(Mt = m | Ntbank = b) =

αK m

(1−α)K
b−m
K b

.

(72)

The conditional expectation is E[Mt | Ntbank = b] = αb, so when α < 1/2 the expected malicious fraction in the bank stays below the median’s 50% breakdown point. B. Concentration of the Malicious Fraction in the Bank For a hypergeometric random variable, a Chernoff-type upper-tail bound gives, for any threshold γ > α,

Mt Pr ≥ γ Ntbank ≤ exp −Ntbank D(γ∥α) , bank Nt

(73)

where D(p∥q) is the Bernoulli KL divergence D(p∥q) = p log

1−p p + (1 − p) log . q 1−q

Setting γ = 1/2 and using α < 1/2 gives      Ntbank 1 bank bank Pr Mt ≥ Nt ≤ exp −Nt D α , 2 2

(74)

(75)

XXXX, 2026.

16

with

 D

1 α 2



1 1/2 1 1/2 log + log 2 α 2 1−α

=

(76)

or, equivalently,  D

1 α 2

 =

1 1 log . 2 4α(1 − α)

(77)

This quantity is positive for α < 1/2, so the upper bound on the probability of a malicious-majority bank decays exponentially with Ntbank . This provides the probabilistic motivation for delaying screening until the history bank reaches a sufficiently broad client coverage under random participation. Proposition 3 (Bound on a malicious-majority history bank). Suppose that the fraction of malicious clients satisfies α < 1/2, and let Mt denote the number of malicious signatures in a history bank of size Ntbank . Then the probability that malicious signatures constitute at least half of the bank is bounded by   2 !  1 Ntbank bank bank Nt ≤ exp −2Nt −α . (78) Pr Mt ≥ 2 2 Proof: From the KL-based concentration bound in (75), we have      1 N bank Ntbank ≤ exp −Ntbank D α . Pr Mt ≥ t 2 2

(79)

To obtain a simpler sufficient bound, let

1 −α 2 be the margin between the median breakdown point and the attack fraction. Then    1 1 1 α(1 − α) = −g + g = − g2 . 2 2 4 g :=

(80)

(81)

Therefore, 4α(1 − α) = 1 − 4g 2 .

(82)

Using (77), we obtain  D

1 α 2

 =

1 1 log . 2 1 − 4g 2

1 Since log 1−x ≥ x for 0 ≤ x < 1, it follows that   2  1 1 1 2 2 α ≥ · 4g = 2g = 2 −α . D 2 2 2

Substituting (84) into (75) yields    2 ! Ntbank 1 Pr Mt ≥ Ntbank ≤ exp −2Ntbank −α , 2 2

(83)

(84)

(85)

which proves (78). The bound in Proposition 3 makes the dependence on attack strength explicit. As α → 1/2, the margin g = 1/2 − α shrinks, the exponent weakens, and a larger history bank is required to achieve the same confidence level. In contrast, when α ≪ 1/2, the same bank size already provides a strong guarantee against a malicious-majority bank. Suppose screening must begin before full population coverage is available and the desired probability of a maliciousmajority activation bank is at most η. From (75), it suffices to choose Mact such that exp(−Mact D(1/2∥α)) ≤ η, Mact ≥

log(1/η)
. D 12 α

(86)

Using the lower bound (84), a more conservative sufficient condition is Mact ≥

log(1/η)
2 . 2 12 − α

(87)

When the total number of clients K is fixed, the activation coverage can be expressed as a population fraction Mact = ρK, in which case (75) becomes      Ntbank 1 bank Pr Mt ≥ Nt ≥ ρK ≤ exp −ρKD α . (88) 2 2

XXXX, 2026.

17

TABLE I: Expected round at which the bank reaches each coverage threshold under random participation with K = 200 and participation rate q = 0.1. Bank coverage ρ 25% 50% 75%

Mact = ρK 50 100 150

Expected activation round 3 7 14

C. Time to Reach the Activation Coverage We now estimate how quickly the bank reaches a chosen activation coverage Mact under uniform random participation. Let q be the per-round participation rate, so that in each round approximately qK clients are selected uniformly at random. For a fixed client, the probability of not being selected in one round is 1 − q, so the probability of not appearing after r rounds is (1 − q)r , and the probability of having appeared at least once by round r is 1 − (1 − q)r . The expected number of distinct clients in the bank after r rounds is therefore E[Br ] = K (1 − (1 − q)r ) .

(89)

For a target coverage fraction ρ, the expected bank size reaches ρK once K(1 − (1 − q)r ) ≥ ρK, i.e., (1 − q)r ≤ 1 − ρ, which gives log(1 − ρ) . (90) r≥ log(1 − q) In our experiments, where q = 0.1 and K = 200, Table I presents the expected round at which the bank reaches each coverage threshold. This expectation can be complemented with a tail bound on the probability that the bank has not yet reached the threshold. Let µr := E[Br ] = K(1 − (1 − q)r ). Because the coverage indicators are exchangeable and negatively associated under uniform random sampling, a Chernoff-style lower-tail bound gives Pr(Br < Mact ) ≤ exp −

Mact µr δr2
, δr := 1 − , 2 µr

(91)

whenever µr > Mact . For example, with K = 200, q = 0.1, and Mact = 100, after r = 12 rounds µ12 = 200(1−0.912 ) ≈ 143.5, so δ12 = 1 − 100/143.5 ≈ 0.303 and 143.5 · 0.3032
≈ 1.4 × 10−3 . (92) 2 Under the participation rate used in our experiments, the bank therefore reaches the half-population threshold early with high probability. The activation-time and bank-composition analyses above assume uniform random client participation, matching the sampling procedure used in the experiments. In deployments with nonuniform or intermittent participation, some client identities may appear rarely or never within the training horizon, making full bank coverage impractically slow. The hypergeometric and activation-time bounds do not directly apply to such participation processes. In that setting, Mact should be interpreted as a practical early-activation coverage, while the bank should continue to expand whenever previously unseen clients participate. Pr(B12 < 100) ≤ exp −

XXXX, 2026.

18

A PPENDIX B P ROOFS FOR T HEORETICAL A NALYSIS This appendix provides the assumptions and proofs supporting Section IV. Appendix B-A proves the normalizationparameter margin decomposition, Appendix B-B proves the screened-aggregation and stationarity bounds, Appendix B-C decomposes the screening error, and Appendix B-D bounds local-training drift. A. Proof of the Normalization-Parameter Margin Decomposition This proof establishes the first-order contribution of a client’s normalization-parameter update to output-token margins. We proceed in three steps. First, we analyze the margin change for one fixed prompt x. Second, we split the client update into two parts: the normalization-parameter part and the remaining-parameter part. Third, we average over prompts x ∼ X , because the expected margin change in (41) is defined over a prompt distribution. We use two remainder terms. The term Rt,i (x, u, v) (93) is the pointwise Taylor remainder. It is the Taylor approximation error for one fixed prompt x and token pair (u, v). The term Rt,i (u, v) (94) is the prompt-averaged remainder, defined by Rt,i (u, v) := Ex∼X [Rt,i (x, u, v)] .

(95)

Thus, Rt,i (x, u, v) depends on a specific prompt, while Rt,i (u, v) averages this error over prompts. Assumption 1 (Smooth prompt-dependent margins). Fix a token pair (u, v) ∈ V × V. For every prompt x in the support of X , the mapping w 7→ mu,v (x; w)

(96)

is twice continuously differentiable in a neighborhood of wt . Moreover, there exists Lm > 0 such that ∇2w mu,v (x; w) op ≤ Lm

(97)

for all w in that neighborhood and all relevant prompts x. Conditioned on round t and client i, the update ∆wt,i = (∆pt,i , ∆qt,i )

(98)

is treated as fixed with respect to the prompt draw x ∼ X . Thus, the expectation over x averages prompt-dependent gradients and remainders, while the update vectors ∆pt,i and ∆qt,i do not depend on the sampled prompt. Proof of Proposition 1: Fix a prompt x and token pair (u, v). Define ϕx (s) := mu,v (x; wt + s∆wt,i ) ,

s ∈ [0, 1].

(99)

This scalar function traces the token margin along the straight path from the current global model wt to the locally updated model wt + ∆wt,i . (100) By the fundamental theorem of calculus, Z 1 mu,v (x; wt + ∆wt,i ) − mu,v (x; wt ) =

ϕ′x (s) ds.

(101)

0

Using the chain rule, ⊤

ϕ′x (s) = ∇w mu,v (x; wt + s∆wt,i ) ∆wt,i . Therefore,

Z 1 mu,v (x; wt + ∆wt,i ) − mu,v (x; wt ) =

(102)

⊤

∇w mu,v (x; wt + s∆wt,i ) ∆wt,i ds. 0

We now isolate the first-order effect at the current global model. Add and subtract ∇w mu,v (x; wt )

(103)

XXXX, 2026.

19

inside the integral: mu,v (x; wt + ∆wt,i ) − mu,v (x; wt ) = ∇w mu,v (x; wt )⊤ ∆wt,i + Rt,i (x, u, v),

(104)

where the pointwise Taylor remainder is Z 1h i⊤ Rt,i (x, u, v) := ∇w mu,v (x; wt + s∆wt,i ) − ∇w mu,v (x; wt ) ∆wt,i ds.

(105)

0

This term contains the part of the margin change not captured by the linear approximation at wt . Equivalently, Taylor’s theorem with integral remainder gives Z 1 ⊤ Rt,i (x, u, v) = (1 − s)∆wt,i ∇2w mu,v (x; wt + s∆wt,i ) ∆wt,i ds.

(106)

0

Using Assumption 1, Z 1 |Rt,i (x, u, v)| ≤ 0

2

(1 − s) ∇2w mu,v (x; wt + s∆wt,i ) op ∥∆wt,i ∥2 ds

Z 1 ≤ 0

2

(1 − s)Lm ∥∆wt,i ∥2 ds

Lm 2 = ∥∆wt,i ∥2 . 2 Next, split the first-order term into the normalization-parameter part and the remaining-parameter part. Since w = (p, q),

∆wt,i = (∆pt,i , ∆qt,i ),

(107)

(108)

we have ∇w mu,v (x; wt )⊤ ∆wt,i = ∇p mu,v (x; wt )⊤ ∆pt,i + ∇q mu,v (x; wt )⊤ ∆qt,i .

(109)

Substituting (109) into (104) gives the pointwise decomposition mu,v (x; wt + ∆wt,i ) − mu,v (x; wt ) = ∇p mu,v (x; wt )⊤ ∆pt,i + ∇q mu,v (x; wt )⊤ ∆qt,i + Rt,i (x, u, v).

(110)

We now take expectation over x ∼ X . This step is necessary because ∆m̄t,i (u, v) in (41) is the average margin change over prompts. Taking expectation on both sides of (110) gives     ∆m̄t,i (u, v) = Ex∼X ∇p mu,v (x; wt )⊤ ∆pt,i + Ex∼X ∇q mu,v (x; wt )⊤ ∆qt,i + Ex∼X [Rt,i (x, u, v)] . (111) Because ∆pt,i and ∆qt,i are fixed with respect to the prompt draw x, they can be moved outside the expectation. For the normalization-parameter part, if ∆pt,i ∈ Rdp , then   dp X   ∂mu,v (x; wt ) Ex∼X ∇p mu,v (x; wt )⊤ ∆pt,i = Ex∼X  ∆pt,i [r] ∂p[r] r=1 =

dp X r=1

 Ex∼X

 ∂mu,v (x; wt ) ∆pt,i [r] ∂p[r] ⊤

= Ex∼X [∇p mu,v (x; wt )] ∆pt,i .

(112)

The same argument gives   ⊤ Ex∼X ∇q mu,v (x; wt )⊤ ∆qt,i = Ex∼X [∇q mu,v (x; wt )] ∆qt,i .

(113)

gp,u,v := Ex∼X [∇p mu,v (x; wt )] ,

(114)

gq,u,v := Ex∼X [∇q mu,v (x; wt )] ,

(115)

Rt,i (u, v) := Ex∼X [Rt,i (x, u, v)] .

(116)

⊤ ⊤ ∆m̄t,i (u, v) = gp,u,v ∆pt,i + gq,u,v ∆qt,i + Rt,i (u, v).

(117)

Define

and Using these definitions in (111) yields

XXXX, 2026.

20

Finally, using (107), |Rt,i (u, v)| = |Ex∼X [Rt,i (x, u, v)]| ≤ Ex∼X [|Rt,i (x, u, v)|] Lm 2 ∥∆wt,i ∥2 . ≤ 2

(118)

This proves Proposition 1. The following corollary bounds the difference between two clients’ first-order margin changes using their normalizationparameter and remaining-parameter update differences, together with the second-order remainders. Corollary 3 (Stability of margin changes). Under Assumption 1, for any two clients i and j,  Lm  2 2 ∥∆wt,i ∥2 + ∥∆wt,j ∥2 . |∆m̄t,i (u, v) − ∆m̄t,j (u, v)| ≤ ∥gp,u,v ∥2 ∥∆pt,i − ∆pt,j ∥2 +∥gq,u,v ∥2 ∥∆qt,i − ∆qt,j ∥2 + 2 (119) Proof: Subtract (42) for clients i and j: ⊤ ⊤ ∆m̄t,i (u, v) − ∆m̄t,j (u, v) = gp,u,v (∆pt,i − ∆pt,j ) + gq,u,v (∆qt,i − ∆qt,j ) + Rt,i (u, v) − Rt,j (u, v).

(120)

Taking absolute values and applying Cauchy–Schwarz gives |∆m̄t,i (u, v) − ∆m̄t,j (u, v)| ≤ ∥gp,u,v ∥2 ∥∆pt,i − ∆pt,j ∥2 + ∥gq,u,v ∥2 ∥∆qt,i − ∆qt,j ∥2 + |Rt,i (u, v)| + |Rt,j (u, v)|. (121) Using (44) for both remainders proves the claim. B. Proof of the Screened-Aggregation Bound We now prove Theorem 1. The proof shows how the screening error et enters the descent inequality. If the retained update direction is close to the benign gradient direction, then ∥et ∥2 is small, and the update behaves similarly to benign training. Assumption 2 (Benign objective). The target objective is X X F (w) = λi Fi (w), λi ≥ 0, λi = 1. (122) i∈Cben

i∈Cben

The function F is differentiable, LF -smooth, and lower bounded: F (w) ≥ Finf > −∞.

(123)

The LF -smoothness condition means that, for all w and w′ , F (w′ ) ≤ F (w) + ∇F (w)⊤ (w′ − w) +

LF 2 ∥w′ − w∥2 . 2

(124)

Assumption 3 (Screened update model). At round t, the screened update can be written as wt+1 = wt − ηt gt ,

(125)

where ηt ≥ 0 is the effective step size and gt is the retained update direction. Let Ft contain the information available before the update noise at round t is realized. There exist βt ≥ 0 and νt ≥ 0 such that

and

∥E[gt | Ft ] − ∇F (wt )∥2 ≤ βt ,

(126)

h i 2 E ∥gt − E[gt | Ft ]∥2 | Ft ≤ νt2 .

(127)

Proof of Theorem 1: By smoothness (124) and the update rule (125), F (wt+1 ) ≤ F (wt ) − ηt ∇F (wt )⊤ gt +

LF ηt2 ∥gt ∥22 . 2

(128)

et := ḡt − at .

(129)

Take conditional expectation given Ft . Define at := ∇F (wt ),

ḡt := E[gt | Ft ],

Then ḡt = at + et ,

∥et ∥2 ≤ βt .

(130)

XXXX, 2026.

21

The first-order term satisfies ⊤ −ηt a⊤ t ḡt = −ηt at (at + et )

= −ηt ∥at ∥22 − ηt a⊤ t et .

(131)

1 ∥at ∥22 + ∥et ∥22 . 4

(132)

3ηt ∥at ∥22 + ηt ∥et ∥22 . 4

(133)

By Young’s inequality, |a⊤ t et | ≤ Therefore, −ηt a⊤ t ḡt ≤ − Next,

    E ∥gt ∥22 | Ft = ∥ḡt ∥22 + E ∥gt − ḡt ∥22 | Ft ≤ ∥at + et ∥22 + νt2 ≤ 2∥at ∥22 + 2∥et ∥22 + νt2 .

(134)

Substituting (133) and (134) into (128) gives  E [F (wt+1 ) | Ft ] ≤ F (wt ) −

 LF ηt2 2 3ηt − LF ηt2 ∥at ∥22 + (ηt + LF ηt2 )∥et ∥22 + νt . 4 2

(135)

3ηt ηt − LF ηt2 ≥ . 4 2

(136)

If 0 ≤ ηt ≤ 1/(4LF ), then

Using ∥et ∥22 ≤ βt2 , we obtain ηt LF ηt2 2 ∥∇F (wt )∥22 + (ηt + LF ηt2 )βt2 + νt . 2 2 Taking total expectation and summing from t = 0 to T − 1 yields E [F (wt+1 ) | Ft ] ≤ F (wt ) −

T −1 T −1 T −1 X   1X LF X 2 ηt E ∥∇F (wt )∥22 ≤ F (w0 ) − E[F (wT )] + ηt E[νt2 ]. (ηt + LF ηt2 )E[βt2 ] + 2 t=0 2 t=0 t=0

(137)

(138)

Since F (wT ) ≥ Finf , T −1   1X ηt E ∥∇F (wt )∥22 ≤ F (w0 ) − Finf 2 t=0

+

T −1 X

(ηt + LF ηt2 )E[βt2 ]

t=0

+

T −1 LF X 2 η E[νt2 ]. 2 t=0 t

Dividing both sides by 12 ST proves (54). Proof of Corollary 1: From Theorem 1,   PT −1 PT −1 PT −1 2 2 (F (w0 ) − Finf ) 2 t=0 (ηt + LF ηt2 )E[βt2 ] LF t=0 ηt2 E[νt2 ] t=0 ηt E ∥∇F (wt )∥2 ≤ + + . ST ST ST ST We show that each term on the right-hand side goes to zero. First, since ST → ∞ and F (w0 ) − Finf is finite, 2 (F (w0 ) − Finf ) → 0. ST Second, because ηt ≤ 1/(4LF ), LF ηt2 ≤

ηt + LF ηt2 ≤

(140)

(141)

1 ηt , 4

and therefore

(139)

(142)

5 ηt . 4

(143)

Thus, PT −1

2 2 t=0 (ηt + LF ηt )E[βt ]

ST By (57),

5 ≤ 4

PT −1 t=0

ηt E[βt2 ] . ST

(144)

PT −1

2 2 t=0 (ηt + LF ηt )E[βt ]

ST

→ 0.

(145)

XXXX, 2026.

22

Third, since E[νt2 ] ≤ ν 2 , LF

PT −1

ηt2 E[νt2 ]

t=0

ST

≤ LF ν

2

PT −1 t=0

ηt2

ST

.

(146)

By (55), LF ν

2

PT −1 t=0

ST

ηt2

→ 0.

(147)

Combining (141), (145), and (147) in (140) gives PT −1 t=0

  ηt E ∥∇F (wt )∥22 → 0. ST

(148)

Now define a random round τ by Pr(τ = t) =

ηt , ST

t = 0, . . . , T − 1.

(149)

Then, by the law of total expectation, −1     TX Pr(τ = t)E ∥∇F (wt )∥22 E ∥∇F (wτ )∥22 = t=0

PT −1 =

t=0

  ηt E ∥∇F (wt )∥22 . ST

(150)

Therefore,   E ∥∇F (wτ )∥22 → 0.

(151)

This proves the corollary. Proof of Corollary 2: Set ηt = η in Theorem 1. Then ST =

T −1 X

η = ηT.

(152)

  T −1  ηE ∥∇F (wt )∥22 1 X  E ∥∇F (wt )∥22 . = ηT T t=0

(153)

t=0

The left-hand side of (54) becomes PT −1 t=0

The optimization term becomes 2 (F (w0 ) − Finf ) . ηT

(154)

The screening-error term becomes 2

T −1

PT −1

2 2 t=0 (η + LF η )E[βt ]

ηT

= 2(1 + LF η)

1 X E[βt2 ] T t=0 2

= 2(1 + LF η)β T .

(155)

The variance term becomes LF

PT −1 t=0

η 2 E[νt2 ]

ηT

T −1

= LF η

1 X E[νt2 ] T t=0

= LF ην 2T . Substituting these three expressions into (54) proves (67).

(156)

XXXX, 2026.

23

C. Screening-Error Decomposition We now connect βt to the retained set returned by F ED LNS. This decomposition relates the screening error to retained malicious weight, benign-client representativeness, and local-training drift. Let Cmal denote the malicious client set and Cben denote the benign client set. At round t, F ED LNS retains Rt . For each retained client i ∈ Rt , define nt,i . (157) αt,i := P j∈Rt nt,j The total retained malicious weight is µt :=

X

αt,a .

(158)

a∈Rt ∩Cmal

For a retained benign client i, define its conditional expected local direction as B ḡt,i := E [gt,i | Ft ] ,

i ∈ Cben .

(159)

We decompose it as B ḡt,i = ∇Fi (wt ) + rt,i ,

(160)

where rt,i is the local-training drift vector. For a retained malicious client a, define ht,a := E [gt,a | Ft ] , The vector ht,a can be arbitrary. The conditional mean retained direction is X E[gt | Ft ] =

a ∈ Cmal .

(161)

X

αt,i (∇Fi (wt ) + rt,i ) +

i∈Rt ∩Cben

αt,a ht,a .

(162)

a∈Rt ∩Cmal

Assume µt < 1. Define the retained benign gradient after renormalizing the benign retained weights: X αt,i ḡtB,ret := ∇Fi (wt ). 1 − µt

(163)

i∈Rt ∩Cben

The retained benign representativeness error is χt := ḡtB,ret − ∇F (wt )

.

(164)

.

(165)

2

The retained benign local-drift magnitude is X

ϱt :=

αt,i rt,i

i∈Rt ∩Cben

Assumption 4 (Bounded retained directions). There exists G > 0 such that ∥ht,a ∥2 ≤ G,

2

∥∇F (wt )∥2 ≤ G

(166)

for all retained malicious clients a and all rounds considered. Proposition 4 (Screening-error bound). Under the definitions above and Assumption 4, ∥E[gt | Ft ] − ∇F (wt )∥2 ≤ (1 − µt )χt + 2Gµt + ϱt .

(167)

βt = (1 − µt )χt + 2Gµt + ϱt .

(168)

Therefore, one may choose Proof: Using (162), E[gt | Ft ] − ∇F (wt ) =

X i∈Rt ∩Cben

αt,i ∇Fi (wt ) − ∇F (wt ) +

X

αt,a ht,a +

a∈Rt ∩Cmal

X

αt,i rt,i .

(169)

i∈Rt ∩Cben

The benign-gradient part satisfies X αt,i ∇Fi (wt ) − ∇F (wt ) = (1 − µt )ḡtB,ret − ∇F (wt ) i∈Rt ∩Cben

  = (1 − µt ) ḡtB,ret − ∇F (wt ) − µt ∇F (wt ).

(170)

XXXX, 2026.

24

Thus, X

≤ (1 − µt )χt + µt ∥∇F (wt )∥2 .

αt,i ∇Fi (wt ) − ∇F (wt )

i∈Rt ∩Cben

(171)

2

The retained malicious part satisfies X a∈Rt ∩Cmal

X

≤

αt,a ht,a

αt,a ∥ht,a ∥2

a∈Rt ∩Cmal

2

≤ Gµt .

(172)

The local-drift term is ϱt by definition. Combining the three bounds and using ∥∇F (wt )∥2 ≤ G gives ∥E[gt | Ft ] − ∇F (wt )∥2 ≤ (1 − µt )χt + 2Gµt + ϱt . This proves the result. Lemma 1 (Sufficient condition for vanishing screening error). Suppose βt ≤ (1 − µt )χt + 2Gµt + ϱt . If

PT −1

PT −1 t=0

ηt E[χ2t ] → 0, ST

t=0

then

ηt E[µ2t ] → 0, ST

(173)

(174) PT −1

ηt E[ϱ2t ] → 0, ST

t=0

(175)

PT −1 t=0

ηt E[βt2 ] → 0. ST

(176)

Proof: Using (a + b + c)2 ≤ 3a2 + 3b2 + 3c2 , we obtain βt2 ≤ 3(1 − µt )2 χ2t + 12G2 µ2t + 3ϱ2t .

(177)

βt2 ≤ 3χ2t + 12G2 µ2t + 3ϱ2t .

(178)

2

Since (1 − µt ) ≤ 1, this gives Multiplying by ηt , summing from t = 0 to T − 1, taking expectation, and dividing by ST yields PT −1 PT −1 PT −1 PT −1 2 ηt E[χ2t ] ηt E[µ2t ] ηt E[ϱ2t ] t=0 ηt E[βt ] ≤ 3 t=0 + 12G2 t=0 + 3 t=0 . ST ST ST ST Each term on the right-hand side converges to zero by assumption. Therefore, PT −1 2 t=0 ηt E[βt ] → 0. ST

(179)

(180)

D. Local-Training Drift Bound For completeness, we provide a bound on ϱt under local gradient descent. Assumption 5 (Bounded benign gradients during local training). For every benign client i, local step e, and round t, (e)

∇Fi (wt,i )

2

≤ GB .

(181)

Also, each Fi is Li -smooth with Li ≤ Lmax . Suppose benign client i performs E local gradient steps with local learning rate ηloc : (0)

wt,i = wt ,

(e+1)

wt,i

(e)

(e)

= wt,i − ηloc ∇Fi (wt,i ),

e = 0, . . . , E − 1.

(182)

Define the average local gradient direction E−1

B ḡt,i =

1 X (e) ∇Fi (wt,i ). E e=0

(183)

Then B rt,i = ḡt,i − ∇Fi (wt ).

(184)

XXXX, 2026.

25

Lemma 2 (Local-training drift bound). Under Assumption 5, ∥rt,i ∥2 ≤

Lmax ηloc GB (E − 1) . 2

(185)

Consequently, ϱt ≤ (1 − µt )

Lmax ηloc GB (E − 1) . 2

(186)

Proof: By Li -smoothness, (e)

(e)

∇Fi (wt,i ) − ∇Fi (wt )

2

≤ Li wt,i − wt

. 2

(187)

Because each local step has norm at most ηloc GB , (e)

wt,i − wt

2

≤ eηloc GB .

(188)

Thus, ∥rt,i ∥2 =

E−1 i 1 Xh (e) ∇Fi (wt,i ) − ∇Fi (wt ) E e=0

2

E−1 X

≤

1 (e) Li wt,i − wt E e=0 2

≤

1 X Lmax eηloc GB E e=0

=

Lmax ηloc GB (E − 1) . 2

E−1

(189)

Finally, ϱt =

X

αt,i rt,i

i∈Rt ∩Cben

≤

X

2

αt,i ∥rt,i ∥2

i∈Rt ∩Cben

≤ (1 − µt ) This proves the lemma.

Lmax ηloc GB (E − 1) . 2

(190)

XXXX, 2026.

26

A PPENDIX C D ETAILED E XPERIMENTAL S ETTINGS This appendix provides the complete experimental configuration used in Section V, including the datasets, tokenization, client partitions, model architectures, optimization settings, attack configuration, baseline parameters, and F ED LNS implementation details. A. Federated Learning Protocol Table II summarizes the simulation protocol shared by all three model families: a population of K = 200 clients participates at rate 0.1 (20 clients per round) over 200 communication rounds, with local AdamW optimization (weight decay 0.01), evaluation after every round, and results averaged over three random seeds under both IID and non-IID partitions. TABLE II: Federated simulation protocol used across model families. Item

Value

Number of clients Participation rate Clients per round Communication rounds Random seeds Data partitions Non-IID construction Local optimizer Weight decay Evaluation frequency

K = 200 0.1 20 200 100, 200, 300 IID and non-IID Two shards per client AdamW 0.01 Every communication round

B. Datasets and Tokenization Table III lists the dataset, tokenizer, and sequence-construction choices for each model family, and Table IV describes how the resulting training units are partitioned across clients. TABLE III: Dataset and tokenization settings. All models are initialized from scratch; tokenizer names specify tokenization only. Model family

Dataset

Tokenizer

Sequence construction

Evaluation split used for selection and reporting

GPT-style LM

WikiText-2 raw

gpt2

Causal LM blocks of length 128

BERT-style MLM

Tiny Shakespeare

LLaMA-style LM

StackOverflow posts

MLM blocks of length 128 with [CLS]/[SEP] TinyLlama/TinyLlama-1.1B-Chat-v1.0 Causal LM blocks of length 128

Official validation split Final 10% of the text Evaluation split defined during dataset construction

bert-base-uncased

TABLE IV: Client data partition construction for IID and non-IID settings. Model family

Training units before partitioning

IID partition

Non-IID partition

GPT-style LM / WikiText-2

Causal LM token blocks of length 128

Shuffle all block indices and assign them round-robin to 200 clients

BERT-style MLM / Tiny Shakespeare

BERT MLM blocks of length 128 from the first 90% of the text Causal LM token blocks of length 128 from up to 50,000 training posts

Shuffle all block indices and assign them round-robin to 200 clients

Split ordered block indices into 400 contiguous shards; assign two shuffled shards per client Split ordered block indices into 400 contiguous shards; assign two shuffled shards per client Split ordered block indices into 400 contiguous shards; assign two shuffled shards per client

LLaMA-style LM / StackOverflow

Shuffle all block indices and assign them round-robin to 200 clients

For each model family, the training corpus is first tokenized and converted into fixed-length training blocks, after which the client partition is applied over the training-block indices. The evaluation split is never partitioned across clients. It is used for global checkpoint selection and for reporting test loss, perplexity, and token-level semantic entropy.

XXXX, 2026.

27

In the IID setting, all training block indices are shuffled using the experiment seed and distributed to the K = 200 clients in round-robin order, so each client receives an approximately equal number of blocks sampled from across the entire training corpus. In the non-IID setting, the ordered list of block indices is first divided into K × 2 = 400 contiguous shards, the shard list is shuffled, and each client is assigned two shards. Clients therefore still have approximately balanced data sizes, but each one observes only a small number of localized regions of the corpus – inducing client-level heterogeneity through topic, style, speaker, or source locality rather than through class-label skew. C. Model Architectures and Optimization Table V lists the scratch transformer architecture used for each model family, and Table VI lists the corresponding local optimization and evaluation settings. TABLE V: Scratch transformer architectures. Hyperparameter

GPT-style LM

BERT-style MLM

LLaMA-style LM

Hidden / embedding dimension Number of layers Attention heads Key-value heads Intermediate dimension Context / block size Max position embeddings Dropout RMSNorm epsilon Initializer range

256 12 16 – – 128 128 0.1 – Default config

256 12 16 – 1024 128 130 Default config – Default config

256 8 16 8 1536 128 256 Default config 10−6 0.02

TABLE VI: Optimization and evaluation settings. Setting

GPT-style LM

BERT-style MLM

LLaMA-style LM

Local epochs Batch size Learning rate Weight decay Train-evaluation batches Evaluation batches MLM masking probability Maximum training examples Maximum evaluation examples

3 8 3 × 10−4 0.01 200 200 – Full split Full split

3 8 3 × 10−4 0.01 200 200 0.15 Full split Full split

2 8 10−5 0.01 50 50 – 50,000 5,000

XXXX, 2026.

28

D. Attack and Aggregation-Baseline Configuration Table VII details the target-corruption attack used throughout the experiments: input tokens are left unchanged, only training targets are corrupted, and each malicious client corrupts its labels with probability pcorr = 1.0. The main text reports results at the malicious-client fraction α = 0.4 (80 of 200 clients); this appendix evaluates the full sweep α ∈ {0, 0.1, 0.2, 0.3, 0.4}. TABLE VII: Target-corruption attack settings. Item

Value

Attack type Input tokens Corrupted quantity Corruption probability Malicious fractions in sweep Strongest main-text attack fraction Malicious clients in the population at 0.4 Special-token handling

Target corruption Kept unchanged Training targets / labels pcorr = 1.0 0, 0.1, 0.2, 0.3, 0.4 0.4 80 out of 200 Padding labels avoided; EOS/padding avoided when sampling corrupt labels

Table VIII summarizes the aggregation methods compared against F ED LNS, and Table IX lists the attack-dependent parameters used for each population-level malicious-client fraction: the trim fraction for Trimmed Mean and the parameters f and m for Multi-Krum, with 20 clients selected per round. These two baselines are therefore configured using the population-level corruption fraction. TABLE VIII: Aggregation methods compared in the experiments. Method

Implementation summary

FedAvg Norm-Bounded FedAvg Coordinate Median Trimmed Mean Multi-Krum FLAME F ED LNS

Sample-weighted average of client updates Each update is clipped by an ℓ2 norm bound before FedAvg Coordinate-wise median of flattened updates Coordinate-wise trimming followed by averaging Distance-based update selection followed by averaging selected updates Two-cluster filtering, majority-cluster retention, norm clipping, and Gaussian noise Normalization-signature bank, median/MAD standardization, and BIC-guided GMM screening

TABLE IX: Robust-aggregation parameters used for each malicious-client fraction. The number of selected clients per round is 20. Malicious fraction

Trim fraction

Multi-Krum f

Multi-Krum m

0.0 0.1 0.2 0.3 0.4

0.0 0.1 0.2 0.3 0.4

0 2 4 6 8

20 18 16 14 12

XXXX, 2026.

29

E. FedLNS Implementation Details Table X lists the F ED LNS implementation settings used in all experiments, covering normalization-parameter extraction, bank update timing, robust standardization, and BIC-guided one-vs-two-component GMM screening. TABLE X: FedLNS implementation settings used in the experiments. Item

Setting

Signature source

Trainable normalization-layer parameters whose names contain ln or layernorm and end with weight or bias One latest per-layer signature dictionary per client Before screening in the current round Latest replacement; EMA coefficient β = 0 Coordinate-wise bank median Coordinate-wise MAD 10−6 Median absolute standardized deviation One diagonal Gaussian vs. two diagonal Gaussians BIC for default F ED LNS Diagonal 10−4 K-means initialization, 5 restarts 200 Keep component with smaller median standardized deviation If no client is retained, retain arg mini∈St δt,i

Bank entry Bank update timing Bank update rule Robust center Robust scale MAD floor Client deviation score GMM candidates GMM model selection GMM covariance GMM regularization GMM initialization Maximum GMM iterations Two-component retain rule Safety fallback

XXXX, 2026.

30

A PPENDIX D C OMPLETE R ESULTS ACROSS M ALICIOUS -C LIENT F RACTIONS This appendix reports the complete numerical results for the malicious-client fraction sweep, evaluated over α ∈ {0, 0.1, 0.2, 0.3, 0.4},

(191)

which for K = 200 clients correspond to 0, 20, 40, 60, and 80 malicious clients, respectively. All tables follow the reporting rule used in the main text: for each seed, the checkpoint with the lowest test loss is selected, and test loss, perplexity, and token-level semantic entropy are reported from that same model state. For each model, partition, aggregation method, and malicious-client fraction, each entry reports the mean and sample standard deviation over seeds {100, 200, 300}, with lower values indicating better performance for all three metrics. The tables are grouped by model family and data partition. For each model and partition, they report test loss, perplexity, and token-level semantic entropy across the full attack-fraction sweep from the clean setting to the strongest evaluated setting of 40%. A. GPT-Style Causal Language Modeling on WikiText Tables XI–XIII report the IID results for GPT/WikiText, and Tables XIV–XVI report the non-IID results. TABLE XI: Test Loss under different malicious-client fractions for GPT / WikiText with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

7.13 ± 0.01 7.50 ± 0.00 7.30 ± 0.01 7.12 ± 0.01 7.12 ± 0.01 7.19 ± 0.00 7.13 ± 0.00

7.17 ± 0.01 7.51 ± 0.01 7.27 ± 0.01 7.19 ± 0.01 7.15 ± 0.01 7.19 ± 0.01 7.13 ± 0.00

7.23 ± 0.00 7.59 ± 0.01 7.28 ± 0.01 7.25 ± 0.01 7.19 ± 0.00 7.20 ± 0.00 7.13 ± 0.00

7.29 ± 0.01 7.69 ± 0.02 7.30 ± 0.01 7.32 ± 0.03 7.21 ± 0.00 7.21 ± 0.00 7.14 ± 0.00

7.37 ± 0.02 7.89 ± 0.01 7.47 ± 0.03 7.43 ± 0.01 7.28 ± 0.02 7.33 ± 0.03 7.16 ± 0.00

TABLE XII: Test PPL under different malicious-client fractions for GPT / WikiText with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

1242.91 ± 6.58 1809.27 ± 8.31 1484.69 ± 11.20 1241.55 ± 10.08 1242.08 ± 8.40 1323.55 ± 4.87 1246.57 ± 2.52

1303.02 ± 9.59 1828.13 ± 10.69 1441.44 ± 9.33 1326.15 ± 17.45 1275.34 ± 7.09 1328.88 ± 7.21 1247.28 ± 5.97

1373.61 ± 1.63 1985.85 ± 25.34 1452.99 ± 11.40 1411.15 ± 15.57 1320.41 ± 5.63 1345.05 ± 2.82 1249.11 ± 1.78

1472.41 ± 15.31 2196.84 ± 33.99 1477.26 ± 17.65 1516.84 ± 43.71 1356.21 ± 6.19 1355.34 ± 3.27 1258.21 ± 4.70

1590.56 ± 35.51 2683.53 ± 34.31 1751.07 ± 52.62 1689.16 ± 18.00 1449.72 ± 22.36 1524.02 ± 40.71 1281.63 ± 1.74

TABLE XIII: Semantic Entropy under different malicious-client fractions for GPT / WikiText with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

6.80 ± 0.06 6.55 ± 0.02 6.89 ± 0.04 6.77 ± 0.03 6.80 ± 0.07 7.11 ± 0.05 6.73 ± 0.03

7.08 ± 0.08 8.04 ± 0.12 7.30 ± 0.15 7.30 ± 0.18 7.14 ± 0.24 7.13 ± 0.10 6.81 ± 0.01

7.37 ± 0.18 8.31 ± 0.10 7.27 ± 0.14 7.29 ± 0.19 7.07 ± 0.09 7.20 ± 0.05 6.75 ± 0.02

7.56 ± 0.16 8.92 ± 0.03 7.61 ± 0.13 7.47 ± 0.13 7.03 ± 0.11 7.11 ± 0.09 6.74 ± 0.04

7.96 ± 0.07 9.69 ± 0.06 7.55 ± 0.19 7.55 ± 0.39 7.13 ± 0.37 6.96 ± 0.10 6.80 ± 0.06

XXXX, 2026.

31

TABLE XIV: Test Loss under different malicious-client fractions for GPT / WikiText with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

7.10 ± 0.01 7.56 ± 0.01 7.21 ± 0.00 7.11 ± 0.01 7.11 ± 0.01 7.21 ± 0.00 7.11 ± 0.01

7.16 ± 0.01 7.54 ± 0.00 7.18 ± 0.00 7.17 ± 0.01 7.18 ± 0.00 7.25 ± 0.01 7.12 ± 0.00

7.23 ± 0.04 7.63 ± 0.01 7.26 ± 0.02 7.25 ± 0.01 7.25 ± 0.01 7.28 ± 0.01 7.13 ± 0.01

7.29 ± 0.06 7.75 ± 0.02 7.33 ± 0.02 7.38 ± 0.03 7.30 ± 0.00 7.29 ± 0.01 7.15 ± 0.02

7.37 ± 0.08 7.94 ± 0.09 7.51 ± 0.03 7.51 ± 0.05 7.36 ± 0.03 7.40 ± 0.04 7.16 ± 0.01

TABLE XV: Test PPL under different malicious-client fractions for GPT / WikiText with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

1211.23 ± 6.70 1916.47 ± 13.00 1350.32 ± 1.34 1225.89 ± 8.21 1225.07 ± 8.06 1359.58 ± 3.82 1225.90 ± 12.06

1282.08 ± 13.22 1886.00 ± 3.63 1309.94 ± 6.40 1295.29 ± 16.41 1311.67 ± 5.60 1415.19 ± 13.32 1238.52 ± 4.97

1378.40 ± 55.98 2060.01 ± 13.63 1419.46 ± 32.27 1406.67 ± 11.55 1408.79 ± 20.45 1449.95 ± 14.72 1251.65 ± 11.37

1471.79 ± 90.09 2314.33 ± 44.03 1529.64 ± 31.03 1599.60 ± 52.85 1474.93 ± 3.80 1467.85 ± 14.75 1277.28 ± 23.52

1595.68 ± 132.68 2811.07 ± 257.29 1820.62 ± 48.31 1819.34 ± 98.93 1579.81 ± 43.94 1637.88 ± 63.90 1286.50 ± 15.34

TABLE XVI: Semantic Entropy under different malicious-client fractions for GPT / WikiText with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

6.79 ± 0.07 6.37 ± 0.02 6.74 ± 0.15 6.75 ± 0.15 6.79 ± 0.17 7.04 ± 0.03 6.80 ± 0.09

7.31 ± 0.17 7.85 ± 0.19 7.31 ± 0.34 7.36 ± 0.07 7.08 ± 0.12 7.08 ± 0.15 6.79 ± 0.05

7.12 ± 0.37 8.44 ± 0.20 7.14 ± 0.32 7.27 ± 0.07 7.07 ± 0.45 7.32 ± 0.20 7.04 ± 0.40

7.61 ± 0.30 9.02 ± 0.09 7.52 ± 0.10 7.39 ± 0.11 7.25 ± 0.47 6.94 ± 0.07 6.90 ± 0.02

7.66 ± 0.26 9.60 ± 0.36 7.45 ± 0.25 7.72 ± 0.28 7.18 ± 0.46 7.13 ± 0.58 6.92 ± 0.24

XXXX, 2026.

32

B. BERT-Style Masked Language Modeling on Tiny Shakespeare Tables XVII–XIX report the IID results for BERT/Tiny Shakespeare, and Tables XX–XXII report the non-IID results. The reported perplexity is masked-token perplexity and should be interpreted within the masked-language-modeling setting. TABLE XVII: Test Loss under different malicious-client fractions for BERT / Shakespeare with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

6.49 ± 0.03 6.80 ± 0.03 6.80 ± 0.03 6.49 ± 0.03 6.49 ± 0.03 6.49 ± 0.03 6.50 ± 0.05

6.54 ± 0.01 6.80 ± 0.01 6.82 ± 0.01 6.66 ± 0.01 6.51 ± 0.01 6.47 ± 0.02 6.49 ± 0.02

6.57 ± 0.01 6.92 ± 0.01 6.82 ± 0.02 6.72 ± 0.02 6.52 ± 0.01 6.49 ± 0.02 6.49 ± 0.02

6.62 ± 0.02 7.11 ± 0.01 6.80 ± 0.03 6.80 ± 0.03 6.54 ± 0.02 6.52 ± 0.00 6.51 ± 0.01

6.68 ± 0.02 7.30 ± 0.05 6.97 ± 0.08 6.97 ± 0.07 6.59 ± 0.02 6.70 ± 0.18 6.55 ± 0.02

TABLE XVIII: Test PPL under different malicious-client fractions for BERT / Shakespeare with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

659.93 ± 21.92 901.04 ± 29.47 894.85 ± 27.58 660.99 ± 20.10 661.00 ± 20.10 658.52 ± 18.95 663.68 ± 29.76

693.01 ± 4.36 897.17 ± 10.13 912.29 ± 9.07 779.94 ± 9.18 672.89 ± 7.97 645.26 ± 15.47 655.97 ± 11.66

710.15 ± 6.19 1008.61 ± 7.00 913.38 ± 19.06 831.19 ± 20.60 676.44 ± 4.57 657.21 ± 10.53 661.38 ± 11.94

750.09 ± 12.08 1227.23 ± 15.36 898.46 ± 22.55 900.65 ± 26.88 695.35 ± 12.21 679.98 ± 3.26 669.37 ± 5.86

794.54 ± 15.01 1477.67 ± 70.84 1066.13 ± 91.56 1062.80 ± 79.49 725.76 ± 17.84 818.57 ± 157.53 697.62 ± 13.62

TABLE XIX: Semantic Entropy under different malicious-client fractions for BERT / Shakespeare with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

6.26 ± 0.18 6.31 ± 0.03 6.60 ± 0.06 6.28 ± 0.16 6.28 ± 0.16 6.26 ± 0.03 6.28 ± 0.09

6.79 ± 0.19 7.27 ± 0.21 6.92 ± 0.22 6.63 ± 0.17 6.40 ± 0.07 6.26 ± 0.11 6.38 ± 0.16

6.95 ± 0.07 7.65 ± 0.07 7.05 ± 0.34 6.97 ± 0.18 6.69 ± 0.38 6.11 ± 0.05 6.19 ± 0.09

7.22 ± 0.36 8.07 ± 0.15 7.41 ± 0.22 7.35 ± 0.54 6.53 ± 0.36 6.15 ± 0.10 6.24 ± 0.10

7.37 ± 0.33 8.54 ± 0.16 7.47 ± 0.78 6.76 ± 0.47 6.61 ± 0.28 6.46 ± 0.16 6.24 ± 0.06

TABLE XX: Test Loss under different malicious-client fractions for BERT / Shakespeare with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

6.36 ± 0.03 6.83 ± 0.01 6.86 ± 0.01 6.54 ± 0.03 6.54 ± 0.03 6.57 ± 0.02 6.47 ± 0.05

6.38 ± 0.04 6.85 ± 0.01 6.89 ± 0.01 6.72 ± 0.01 6.59 ± 0.02 6.56 ± 0.03 6.36 ± 0.06

6.51 ± 0.10 6.95 ± 0.01 6.86 ± 0.03 6.80 ± 0.02 6.60 ± 0.01 6.56 ± 0.01 6.42 ± 0.14

6.56 ± 0.12 7.14 ± 0.02 6.86 ± 0.01 6.84 ± 0.02 6.61 ± 0.02 6.57 ± 0.01 6.38 ± 0.18

6.59 ± 0.11 7.34 ± 0.02 6.95 ± 0.05 6.94 ± 0.02 6.69 ± 0.06 6.86 ± 0.17 6.48 ± 0.13

XXXX, 2026.

33

TABLE XXI: Test PPL under different malicious-client fractions for BERT / Shakespeare with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

576.49 ± 19.76 922.98 ± 7.18 950.49 ± 10.79 691.27 ± 19.12 691.27 ± 19.11 710.27 ± 16.45 643.56 ± 33.46

588.64 ± 24.25 941.70 ± 9.95 985.58 ± 6.47 829.90 ± 7.29 727.43 ± 13.60 705.47 ± 23.82 576.63 ± 36.58

672.55 ± 66.84 1039.58 ± 9.18 949.59 ± 26.06 893.88 ± 19.37 732.73 ± 10.31 704.63 ± 4.08 618.30 ± 91.12

708.50 ± 88.05 1262.16 ± 26.29 953.32 ± 10.76 938.21 ± 17.35 739.83 ± 12.52 712.01 ± 8.24 594.61 ± 108.75

731.73 ± 83.15 1533.59 ± 36.83 1047.36 ± 53.66 1033.35 ± 24.28 804.39 ± 53.06 963.65 ± 157.76 658.21 ± 91.66

TABLE XXII: Semantic Entropy under different malicious-client fractions for BERT / Shakespeare with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

5.87 ± 0.14 6.27 ± 0.01 6.59 ± 0.06 6.24 ± 0.10 6.24 ± 0.10 6.19 ± 0.06 6.12 ± 0.05

6.34 ± 0.41 6.99 ± 0.14 7.28 ± 0.10 6.61 ± 0.15 6.67 ± 0.24 6.27 ± 0.16 6.06 ± 0.18

6.96 ± 0.20 7.63 ± 0.15 7.20 ± 0.36 7.00 ± 0.13 6.43 ± 0.06 6.30 ± 0.12 6.33 ± 0.09

7.19 ± 0.38 7.98 ± 0.19 7.41 ± 0.26 7.23 ± 0.17 6.69 ± 0.13 6.41 ± 0.12 6.00 ± 0.38

6.94 ± 0.14 8.48 ± 0.02 7.20 ± 0.59 6.75 ± 0.27 6.82 ± 0.31 6.62 ± 0.14 6.19 ± 0.13

XXXX, 2026.

34

C. LLaMA-Style Causal Language Modeling on StackOverflow Tables XXIII–XXV report the IID results for LLaMA/StackOverflow, and Tables XXVI–XXVIII report the non-IID results. TABLE XXIII: Test Loss under different malicious-client fractions for LLaMA / StackOverflow with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

5.36 ± 0.03 5.49 ± 0.03 5.39 ± 0.03 5.36 ± 0.03 5.36 ± 0.03 5.48 ± 0.03 5.36 ± 0.03

5.40 ± 0.03 5.56 ± 0.03 5.40 ± 0.03 5.39 ± 0.03 5.37 ± 0.03 5.48 ± 0.03 5.37 ± 0.03

5.46 ± 0.04 5.63 ± 0.04 5.43 ± 0.03 5.43 ± 0.03 5.38 ± 0.03 5.48 ± 0.03 5.37 ± 0.03

5.53 ± 0.02 5.77 ± 0.02 5.46 ± 0.03 5.47 ± 0.03 5.39 ± 0.03 5.49 ± 0.03 5.37 ± 0.03

5.66 ± 0.04 5.86 ± 0.08 5.54 ± 0.03 5.54 ± 0.04 5.43 ± 0.04 5.58 ± 0.05 5.38 ± 0.03

TABLE XXIV: Test PPL under different malicious-client fractions for LLaMA / StackOverflow with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

213.15 ± 6.77 243.23 ± 6.54 218.98 ± 7.04 213.15 ± 6.77 213.15 ± 6.77 240.02 ± 6.40 213.42 ± 6.57

221.94 ± 6.63 258.63 ± 6.54 222.39 ± 7.10 219.60 ± 7.16 215.53 ± 7.04 239.85 ± 6.77 213.93 ± 6.56

234.03 ± 8.53 279.81 ± 11.33 227.13 ± 7.36 227.49 ± 7.80 217.38 ± 6.99 240.36 ± 6.81 214.57 ± 6.75

251.01 ± 5.88 319.10 ± 6.04 235.44 ± 6.60 237.43 ± 7.51 219.76 ± 6.92 242.54 ± 7.33 215.61 ± 6.70

286.73 ± 10.99 349.83 ± 27.13 254.94 ± 8.06 254.52 ± 9.07 228.52 ± 8.72 264.80 ± 13.70 217.14 ± 6.85

TABLE XXV: Semantic Entropy under different malicious-client fractions for LLaMA / StackOverflow with IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

5.52 ± 0.02 5.70 ± 0.01 5.55 ± 0.02 5.52 ± 0.02 5.52 ± 0.02 5.64 ± 0.05 5.52 ± 0.02

5.80 ± 0.02 5.96 ± 0.01 5.73 ± 0.03 5.73 ± 0.04 5.54 ± 0.04 5.61 ± 0.04 5.52 ± 0.04

5.97 ± 0.12 6.43 ± 0.22 5.81 ± 0.08 5.81 ± 0.09 5.55 ± 0.03 5.64 ± 0.05 5.54 ± 0.03

6.12 ± 0.06 6.83 ± 0.07 5.91 ± 0.02 5.89 ± 0.04 5.56 ± 0.03 5.65 ± 0.04 5.52 ± 0.04

6.58 ± 0.34 6.95 ± 0.43 5.97 ± 0.09 5.95 ± 0.08 5.61 ± 0.03 5.74 ± 0.05 5.52 ± 0.02

TABLE XXVI: Test Loss under different malicious-client fractions for LLaMA / StackOverflow with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

5.37 ± 0.03 5.51 ± 0.03 5.40 ± 0.03 5.37 ± 0.03 5.37 ± 0.03 5.49 ± 0.03 5.37 ± 0.03

5.41 ± 0.03 5.57 ± 0.03 5.42 ± 0.03 5.40 ± 0.03 5.38 ± 0.03 5.49 ± 0.03 5.38 ± 0.03

5.47 ± 0.04 5.65 ± 0.04 5.44 ± 0.03 5.44 ± 0.04 5.39 ± 0.03 5.49 ± 0.03 5.38 ± 0.03

5.54 ± 0.03 5.79 ± 0.02 5.48 ± 0.03 5.49 ± 0.03 5.41 ± 0.03 5.50 ± 0.03 5.38 ± 0.03

5.67 ± 0.03 5.83 ± 0.05 5.56 ± 0.03 5.56 ± 0.04 5.45 ± 0.04 5.59 ± 0.04 5.39 ± 0.03

XXXX, 2026.

35

TABLE XXVII: Test PPL under different malicious-client fractions for LLaMA / StackOverflow with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

215.38 ± 6.80 246.30 ± 6.76 221.84 ± 7.19 215.55 ± 6.92 215.55 ± 6.92 243.12 ± 6.82 215.55 ± 6.85

224.27 ± 6.68 261.73 ± 6.66 225.51 ± 7.38 222.49 ± 7.43 218.10 ± 7.52 242.99 ± 6.85 216.13 ± 6.70

237.35 ± 9.12 283.49 ± 10.99 230.73 ± 7.84 230.96 ± 8.21 219.94 ± 7.17 243.25 ± 6.41 216.97 ± 6.91

255.84 ± 7.17 326.42 ± 6.26 239.51 ± 6.99 241.46 ± 8.09 222.74 ± 7.63 245.58 ± 7.14 218.02 ± 7.07

288.74 ± 8.31 340.88 ± 18.09 260.33 ± 7.99 259.56 ± 9.62 233.42 ± 10.15 269.08 ± 11.90 220.18 ± 6.45

TABLE XXVIII: Semantic Entropy under different malicious-client fractions for LLaMA / StackOverflow with Non-IID partition. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. Method FedAvg Norm-Bound Median Trimmed Multi-Krum FLAME FedLNS

0%

10%

20%

30%

40%

5.53 ± 0.03 5.70 ± 0.03 5.56 ± 0.03 5.53 ± 0.03 5.53 ± 0.03 5.66 ± 0.06 5.54 ± 0.03

5.80 ± 0.01 5.98 ± 0.01 5.76 ± 0.02 5.75 ± 0.03 5.55 ± 0.04 5.63 ± 0.06 5.54 ± 0.03

5.94 ± 0.11 6.44 ± 0.21 5.84 ± 0.09 5.84 ± 0.10 5.56 ± 0.03 5.62 ± 0.01 5.53 ± 0.03

6.22 ± 0.11 6.90 ± 0.09 5.91 ± 0.04 5.92 ± 0.06 5.56 ± 0.02 5.66 ± 0.07 5.53 ± 0.02

6.61 ± 0.27 6.56 ± 0.42 5.99 ± 0.12 5.96 ± 0.10 5.63 ± 0.06 5.74 ± 0.06 5.52 ± 0.03

XXXX, 2026.

36

A PPENDIX E BANK C OVERAGE AT ACTIVATION This appendix studies the amount of bank coverage available when F ED LNS first activates screening and global aggregation. Appendix A analyzes the benign-majority probability of a partially populated bank; here, we evaluate how broader activation coverage affects the early active training trajectory. A. Reporting Protocol The bank activation threshold determines when the server begins screening and global aggregation. We therefore focus on the early active period immediately after each threshold is reached, where differences in bank coverage and activation timing are most visible. For each seed and activation coverage, the activation round is the first communication round in which the bank reaches Mact . We then consider the first ten active aggregation rounds. Within this window, the checkpoint with the lowest test loss is selected, and test loss, perplexity, and token-level semantic entropy are reported from the same checkpoint. B. Activation Rounds Tables XXIX–XXXVIII report the activation rounds for all population-level malicious-client fractions and partitions. The observed timing follows the pattern analyzed in Appendix A: Mact = 0 activates immediately, while thresholds covering 25%, 50%, and 75% of the client population activate after approximately 3, 7, and 13–14 rounds, respectively. TABLE XXIX: Estimated activation rounds for the bank-size ablation under IID partition with 0% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 6.7 ± 0.6 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 6.7 ± 0.6 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 6.7 ± 0.6 13.3 ± 0.6

TABLE XXX: Estimated activation rounds for the bank-size ablation under IID partition with 10% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.0 ± 0.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.0 ± 0.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.0 ± 0.0

TABLE XXXI: Estimated activation rounds for the bank-size ablation under IID partition with 20% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.3 ± 0.6

TABLE XXXII: Estimated activation rounds for the bank-size ablation under IID partition with 30% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.3 ± 0.6 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.3 ± 0.6 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.3 ± 0.6 14.0 ± 1.0

XXXX, 2026.

37

TABLE XXXIII: Estimated activation rounds for the bank-size ablation under IID partition with 40% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 14.0 ± 1.0

TABLE XXXIV: Estimated activation rounds for the bank-size ablation under Non-IID partition with 0% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 6.7 ± 0.6 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 6.7 ± 0.6 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 6.7 ± 0.6 13.3 ± 0.6

TABLE XXXV: Estimated activation rounds for the bank-size ablation under Non-IID partition with 10% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.0 ± 0.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.0 ± 0.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.0 ± 0.0

TABLE XXXVI: Estimated activation rounds for the bank-size ablation under Non-IID partition with 20% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.3 ± 0.6

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 13.3 ± 0.6

TABLE XXXVII: Estimated activation rounds for the bank-size ablation under Non-IID partition with 30% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.3 ± 0.6 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.3 ± 0.6 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.3 ± 0.6 14.0 ± 1.0

TABLE XXXVIII: Estimated activation rounds for the bank-size ablation under Non-IID partition with 40% malicious clients. The activation round is the first communication round where the bank reaches Mact . Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 14.0 ± 1.0

1.0 ± 0.0 3.0 ± 0.0 7.0 ± 0.0 14.0 ± 1.0

XXXX, 2026.

38

C. Results Under IID Partitions Tables XXXIX–LIII report the IID bank-threshold results. Across model families and attack fractions, larger thresholds generally improve the early active-window values; these comparisons reflect both broader client coverage in the bank and later activation of global aggregation. At the strongest 40% malicious-client setting, the 75% activation coverage gives the lowest test loss and perplexity for all three model families under IID partitioning: perplexity falls from 1820.21 ± 4.44 to 1626.87 ± 9.50 for GPT/WikiText, from 1532.76 ± 114.78 to 1043.96 ± 22.20 for BERT/Tiny Shakespeare, and from 2408.15 ± 162.25 to 609.87 ± 24.44 for LLaMA/StackOverflow. Token-level semantic entropy also decreases substantially for BERT and LLaMA, indicating that broader bank coverage can stabilize the early active trajectory. TABLE XXXIX: Bank-size threshold ablation under IID partition with 0% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.50 ± 0.00 7.48 ± 0.01 7.44 ± 0.01 7.38 ± 0.01

7.33 ± 0.01 7.20 ± 0.03 7.04 ± 0.04 6.94 ± 0.02

7.72 ± 0.04 7.37 ± 0.04 6.87 ± 0.09 6.42 ± 0.03

TABLE XL: Bank-size threshold ablation under IID partition with 0% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1802.40 ± 2.97 1764.16 ± 11.60 1695.11 ± 18.37 1601.83 ± 17.25

1519.44 ± 14.38 1341.88 ± 42.96 1143.14 ± 48.58 1028.47 ± 24.40

2258.01 ± 88.68 1584.71 ± 60.01 969.72 ± 83.50 616.50 ± 19.83

TABLE XLI: Bank-size threshold ablation under IID partition with 0% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.01 ± 0.03 6.96 ± 0.09 6.90 ± 0.00 6.90 ± 0.06

7.46 ± 0.02 7.26 ± 0.38 6.39 ± 0.16 6.37 ± 0.12

9.95 ± 0.02 9.67 ± 0.03 8.88 ± 0.18 7.49 ± 0.04

TABLE XLII: Bank-size threshold ablation under IID partition with 10% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.50 ± 0.00 7.48 ± 0.01 7.44 ± 0.01 7.39 ± 0.00

7.32 ± 0.01 7.18 ± 0.05 7.06 ± 0.04 6.92 ± 0.02

7.72 ± 0.04 7.37 ± 0.04 6.84 ± 0.03 6.44 ± 0.03

XXXX, 2026.

39

TABLE XLIII: Bank-size threshold ablation under IID partition with 10% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1808.57 ± 3.95 1772.48 ± 13.23 1701.03 ± 11.07 1615.82 ± 5.52

1512.51 ± 12.15 1310.31 ± 71.29 1163.34 ± 42.36 1011.33 ± 20.24

2256.49 ± 90.48 1585.07 ± 62.27 933.03 ± 30.89 625.74 ± 17.90

TABLE XLIV: Bank-size threshold ablation under IID partition with 10% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.01 ± 0.10 6.98 ± 0.06 6.92 ± 0.05 6.91 ± 0.06

7.51 ± 0.04 6.88 ± 0.12 6.45 ± 0.01 6.33 ± 0.03

9.95 ± 0.02 9.67 ± 0.03 8.79 ± 0.06 7.54 ± 0.04

TABLE XLV: Bank-size threshold ablation under IID partition with 20% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.50 ± 0.01 7.48 ± 0.01 7.43 ± 0.01 7.39 ± 0.01

7.32 ± 0.01 7.19 ± 0.05 7.05 ± 0.01 6.93 ± 0.04

7.72 ± 0.04 7.37 ± 0.04 6.84 ± 0.03 6.43 ± 0.02

TABLE XLVI: Bank-size threshold ablation under IID partition with 20% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1810.07 ± 10.64 1774.75 ± 21.70 1693.93 ± 23.24 1612.54 ± 12.60

1505.92 ± 11.61 1328.38 ± 66.28 1157.90 ± 17.29 1027.72 ± 37.62

2256.98 ± 91.12 1585.63 ± 62.41 933.10 ± 31.26 617.43 ± 15.49

TABLE XLVII: Bank-size threshold ablation under IID partition with 20% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.02 ± 0.07 6.92 ± 0.03 6.95 ± 0.04 6.88 ± 0.07

7.52 ± 0.06 6.91 ± 0.13 6.48 ± 0.11 6.29 ± 0.05

9.95 ± 0.02 9.67 ± 0.03 8.79 ± 0.06 7.50 ± 0.07

TABLE XLVIII: Bank-size threshold ablation under IID partition with 30% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.50 ± 0.01 7.48 ± 0.01 7.44 ± 0.01 7.39 ± 0.01

7.32 ± 0.07 7.18 ± 0.04 7.06 ± 0.05 6.92 ± 0.03

7.72 ± 0.04 7.37 ± 0.04 6.81 ± 0.05 6.40 ± 0.03

XXXX, 2026.

40

TABLE XLIX: Bank-size threshold ablation under IID partition with 30% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1813.92 ± 12.17 1778.81 ± 10.18 1694.88 ± 16.38 1619.16 ± 11.42

1508.04 ± 110.34 1312.65 ± 46.41 1169.29 ± 61.52 1011.10 ± 35.22

2257.69 ± 91.15 1585.24 ± 62.65 903.36 ± 43.16 600.06 ± 15.82

TABLE L: Bank-size threshold ablation under IID partition with 30% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.03 ± 0.08 6.96 ± 0.02 6.96 ± 0.04 6.88 ± 0.07

7.51 ± 0.07 6.94 ± 0.18 6.43 ± 0.04 6.40 ± 0.07

9.95 ± 0.02 9.67 ± 0.03 8.71 ± 0.12 7.42 ± 0.08

TABLE LI: Bank-size threshold ablation under IID partition with 40% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.51 ± 0.00 7.49 ± 0.01 7.45 ± 0.00 7.39 ± 0.01

7.33 ± 0.07 7.24 ± 0.04 7.07 ± 0.05 6.95 ± 0.02

7.79 ± 0.07 7.42 ± 0.05 6.87 ± 0.02 6.41 ± 0.04

TABLE LII: Bank-size threshold ablation under IID partition with 40% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1820.21 ± 4.44 1782.72 ± 10.82 1718.82 ± 4.65 1626.87 ± 9.50

1532.76 ± 114.78 1388.21 ± 62.55 1171.92 ± 53.02 1043.96 ± 22.20

2408.15 ± 162.25 1674.97 ± 86.91 964.01 ± 22.51 609.87 ± 24.44

TABLE LIII: Bank-size threshold ablation under IID partition with 40% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.11 ± 0.06 7.02 ± 0.04 6.97 ± 0.07 6.93 ± 0.03

7.92 ± 0.48 7.13 ± 0.22 6.53 ± 0.06 6.32 ± 0.10

9.99 ± 0.04 9.73 ± 0.06 8.88 ± 0.09 7.46 ± 0.14

XXXX, 2026.

41

D. Results Under Non-IID Partitions Tables LIV–LXVIII report the non-IID results. Larger thresholds again generally improve the early active-window values, while also postponing the first global aggregation rounds. At the strongest 40% malicious-client setting, the 75% activation coverage gives the lowest test loss and perplexity for all three model families under Non-IID partitioning: perplexity falls from 1920.35 ± 56.71 to 1698.50 ± 44.56 for GPT/WikiText and from 1514.80 ± 139.13 to 1078.90 ± 33.23 for BERT/Tiny Shakespeare, while for LLaMA/StackOverflow the 0% threshold is highly unstable (5266.58 ± 4615.45) and the 75% threshold reduces this to 759.15 ± 249.94, which shows that delaying screening until the bank has broader coverage can substantially stabilize early active training under severe attack and non-IID heterogeneity. For token-level semantic entropy, the 75% threshold is again best for BERT and LLaMA at the strongest attack setting, while for GPT/non-IID the 50% threshold gives the lowest value with 75% close behind, indicating that the best threshold can vary slightly by metric, though the overall pattern still favors larger bank coverage for stable early training. TABLE LIV: Bank-size threshold ablation under Non-IID partition with 0% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.58 ± 0.01 7.55 ± 0.02 7.51 ± 0.03 7.44 ± 0.02

7.35 ± 0.03 7.25 ± 0.01 7.11 ± 0.06 6.96 ± 0.04

7.73 ± 0.05 7.37 ± 0.05 6.88 ± 0.09 6.43 ± 0.04

TABLE LV: Bank-size threshold ablation under Non-IID partition with 0% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1959.50 ± 22.90 1896.90 ± 42.06 1823.97 ± 46.01 1710.88 ± 32.52

1550.67 ± 47.69 1403.02 ± 11.73 1221.73 ± 76.35 1050.53 ± 37.04

2273.70 ± 102.86 1596.15 ± 71.84 976.65 ± 92.33 622.54 ± 22.03

TABLE LVI: Bank-size threshold ablation under Non-IID partition with 0% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.86 ± 0.10 6.82 ± 0.03 6.72 ± 0.03 6.71 ± 0.03

7.82 ± 0.38 6.95 ± 0.18 6.60 ± 0.15 6.33 ± 0.07

9.95 ± 0.02 9.68 ± 0.04 8.88 ± 0.20 7.48 ± 0.06

TABLE LVII: Bank-size threshold ablation under Non-IID partition with 10% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.55 ± 0.02 7.52 ± 0.02 7.48 ± 0.03 7.42 ± 0.02

7.33 ± 0.01 7.25 ± 0.01 7.10 ± 0.07 6.95 ± 0.05

7.73 ± 0.05 7.37 ± 0.05 6.84 ± 0.04 6.44 ± 0.03

E. Interpretation The study characterizes Mact as the bank coverage at which screening and global aggregation begin, rather than as a desired final bank size. The bank continues to grow after activation. Across the evaluated settings, the dominant trend

XXXX, 2026.

42

TABLE LVIII: Bank-size threshold ablation under Non-IID partition with 10% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1899.89 ± 44.69 1853.08 ± 43.58 1768.44 ± 51.11 1672.14 ± 37.85

1531.88 ± 13.60 1402.99 ± 7.86 1213.39 ± 83.14 1047.93 ± 54.20

2265.83 ± 100.94 1592.05 ± 71.15 937.25 ± 35.56 628.96 ± 18.82

TABLE LIX: Bank-size threshold ablation under Non-IID partition with 10% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.76 ± 0.01 6.71 ± 0.01 6.68 ± 0.05 6.71 ± 0.02

7.63 ± 0.05 6.97 ± 0.15 6.61 ± 0.19 6.42 ± 0.02

9.96 ± 0.02 9.68 ± 0.04 8.80 ± 0.08 7.53 ± 0.05

favors broader activation coverage: perplexity improves consistently as coverage increases, and token-level semantic entropy generally follows the same pattern. This supports maximizing the bank coverage available at activation whenever the resulting delay is acceptable. We use 50% population coverage as the default practical activation point, not because it is statistically preferable to broader coverage, but because it begins aggregation earlier than the largest evaluated threshold of 75%. In deployments with intermittent or nonuniform participation, waiting for all client identities may be impractically slow. A smaller activation bank can therefore provide an operational starting point, while the probability analysis in Appendix A quantifies its benign-majority reliability under uniform random participation. The ablation does not evaluate 100% activation coverage and therefore does not establish full coverage as an empirical optimum.

XXXX, 2026.

43

TABLE LX: Bank-size threshold ablation under Non-IID partition with 20% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.55 ± 0.02 7.52 ± 0.02 7.47 ± 0.04 7.42 ± 0.03

7.27 ± 0.12 7.14 ± 0.17 7.08 ± 0.16 6.95 ± 0.11

7.73 ± 0.05 7.38 ± 0.05 6.85 ± 0.04 6.43 ± 0.02

TABLE LXI: Bank-size threshold ablation under Non-IID partition with 20% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1892.83 ± 45.80 1843.09 ± 43.59 1759.53 ± 65.05 1673.23 ± 42.10

1439.58 ± 163.65 1270.95 ± 202.01 1199.79 ± 182.48 1048.68 ± 111.65

2284.63 ± 122.32 1603.79 ± 84.10 941.83 ± 41.03 622.50 ± 15.12

TABLE LXII: Bank-size threshold ablation under Non-IID partition with 20% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.78 ± 0.08 6.84 ± 0.10 6.75 ± 0.06 6.74 ± 0.02

7.47 ± 0.31 6.84 ± 0.45 6.59 ± 0.16 6.48 ± 0.10

9.96 ± 0.03 9.69 ± 0.05 8.81 ± 0.08 7.50 ± 0.06

TABLE LXIII: Bank-size threshold ablation under Non-IID partition with 30% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.56 ± 0.02 7.52 ± 0.04 7.47 ± 0.03 7.43 ± 0.02

7.30 ± 0.06 7.20 ± 0.09 7.09 ± 0.07 6.97 ± 0.03

7.73 ± 0.05 7.38 ± 0.05 6.81 ± 0.06 6.41 ± 0.03

TABLE LXIV: Bank-size threshold ablation under Non-IID partition with 30% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1914.41 ± 35.81 1849.76 ± 65.54 1760.25 ± 53.04 1677.76 ± 37.72

1484.60 ± 89.71 1339.75 ± 121.30 1206.50 ± 80.07 1063.79 ± 26.84

2279.76 ± 116.94 1600.19 ± 79.79 911.93 ± 52.64 605.22 ± 18.26

TABLE LXV: Bank-size threshold ablation under Non-IID partition with 30% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.74 ± 0.07 6.79 ± 0.01 6.75 ± 0.05 6.78 ± 0.05

7.62 ± 0.11 6.92 ± 0.14 6.53 ± 0.10 6.40 ± 0.04

9.96 ± 0.03 9.68 ± 0.05 8.72 ± 0.15 7.43 ± 0.08

XXXX, 2026.

44

TABLE LXVI: Bank-size threshold ablation under Non-IID partition with 40% malicious clients. Metric: Test Loss. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.56 ± 0.03 7.53 ± 0.03 7.49 ± 0.04 7.44 ± 0.03

7.32 ± 0.09 7.21 ± 0.11 7.13 ± 0.08 6.98 ± 0.03

8.33 ± 0.82 7.96 ± 0.82 7.28 ± 0.61 6.60 ± 0.31

TABLE LXVII: Bank-size threshold ablation under Non-IID partition with 40% malicious clients. Metric: Test PPL. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

0% 25% 50% 75%

1920.35 ± 56.71 1860.94 ± 60.73 1783.26 ± 65.14 1698.50 ± 44.56

1514.80 ± 139.13 1355.84 ± 138.76 1248.37 ± 101.92 1078.90 ± 33.23

5266.58 ± 4615.45 3651.86 ± 3205.90 1649.52 ± 1092.41 759.15 ± 249.94

TABLE LXVIII: Bank-size threshold ablation under Non-IID partition with 40% malicious clients. Metric: Semantic Entropy. For each seed and threshold, we select the checkpoint with the lowest test loss within the first 10 communication rounds after the bank reaches Mact . The best case is highlighted in bold. Mact /K 0% 25% 50% 75%

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.80 ± 0.12 6.81 ± 0.10 6.73 ± 0.04 6.76 ± 0.06

7.85 ± 0.34 6.90 ± 0.19 6.62 ± 0.15 6.29 ± 0.18

10.12 ± 0.17 9.96 ± 0.31 9.36 ± 0.66 8.00 ± 0.97

XXXX, 2026.

45

A PPENDIX F GMM-S ELECTION A BLATION This appendix reports the ablation study for the GMM selection rule in F ED LNS. The default F ED LNS rule fits both a one-component and a two-component diagonal Gaussian mixture model to the current bank-standardized normalizationsignature features, computes the Bayesian Information Criterion (BIC) for each model, and selects the lower-BIC model. This adaptive rule is compared with a forced two-component variant. BIC-selected 1-vs-2 GMM.: This is the default F ED LNS rule. It selects b ct = arg min BIC(c).

(192)

c∈{1,2}

When b ct = 1, all clients are retained. When b ct = 2, F ED LNS retains the component with the smaller median standardized deviation from the bank reference. Forced 2-GMM.: This variant always fits a two-component GMM after bank activation and retains the component with the smaller median absolute standardized deviation from the bank reference. It is more aggressive than the default method because it always tries to split the current round into two groups, even when the current client signatures may be better explained by a single population. All GMM ablation runs use the same model, dataset, partition, attack, optimizer, seeds, and bank threshold as the corresponding main experiment. The only changed variable is the GMM decision rule. For each seed, we select the checkpoint with the lowest test loss and report test loss, perplexity, and token-level semantic entropy from that same checkpoint. Thus, token-level semantic entropy is not independently optimized; it is measured at the checkpoint selected by test loss. All entries are reported as mean ± sample standard deviation over seeds {100, 200, 300}. A. Results Under IID Partitions Tables LXIX–LXXXIII report the IID GMM-selection ablation results across all malicious-client fractions. The BICselected and forced two-component rules produce nearly identical selected evaluation-checkpoint values across the three model families. At the strongest 40% malicious-client setting, both rules obtain the same values up to the reported precision for BERT/Tiny Shakespeare and LLaMA/StackOverflow. For GPT/WikiText, the two rules are also essentially identical: the BIC-selected rule gives test loss 7.16 ± 0.00, perplexity 1281.63 ± 1.74, and token-level semantic entropy 6.80 ± 0.06, while forced 2-GMM gives test loss 7.16 ± 0.00, perplexity 1281.62 ± 1.73, and token-level semantic entropy 6.80 ± 0.06. In the IID results, the default BIC-selected rule and the forced two-component rule have nearly identical endpoint values. The ablation does not establish that the current-round feature distribution is statistically identifiable as two components in every attacked setting. TABLE LXIX: GMM-selection ablation under IID partition with 0% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.13 ± 0.00 7.13 ± 0.00

6.50 ± 0.05 6.50 ± 0.05

5.36 ± 0.03 5.36 ± 0.03

TABLE LXX: GMM-selection ablation under IID partition with 0% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1246.57 ± 2.52 1246.57 ± 2.52

663.68 ± 29.76 663.68 ± 29.76

213.42 ± 6.57 213.42 ± 6.57

XXXX, 2026.

46

TABLE LXXI: GMM-selection ablation under IID partition with 0% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.73 ± 0.03 6.73 ± 0.03

6.28 ± 0.09 6.28 ± 0.09

5.52 ± 0.02 5.52 ± 0.02

TABLE LXXII: GMM-selection ablation under IID partition with 10% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.13 ± 0.00 7.13 ± 0.00

6.49 ± 0.02 6.49 ± 0.02

5.37 ± 0.03 5.37 ± 0.03

TABLE LXXIII: GMM-selection ablation under IID partition with 10% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1247.28 ± 5.97 1247.28 ± 5.97

655.97 ± 11.66 655.97 ± 11.66

213.93 ± 6.56 213.93 ± 6.56

TABLE LXXIV: GMM-selection ablation under IID partition with 10% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.81 ± 0.01 6.81 ± 0.01

6.38 ± 0.16 6.38 ± 0.16

5.52 ± 0.04 5.52 ± 0.04

TABLE LXXV: GMM-selection ablation under IID partition with 20% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.13 ± 0.00 7.13 ± 0.00

6.49 ± 0.02 6.49 ± 0.02

5.37 ± 0.03 5.37 ± 0.03

TABLE LXXVI: GMM-selection ablation under IID partition with 20% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1249.11 ± 1.78 1249.11 ± 1.78

661.38 ± 11.94 661.38 ± 11.94

214.57 ± 6.75 214.57 ± 6.75

TABLE LXXVII: GMM-selection ablation under IID partition with 20% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.75 ± 0.02 6.75 ± 0.02

6.19 ± 0.09 6.19 ± 0.09

5.54 ± 0.03 5.54 ± 0.03

XXXX, 2026.

47

TABLE LXXVIII: GMM-selection ablation under IID partition with 30% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.14 ± 0.00 7.14 ± 0.00

6.51 ± 0.01 6.51 ± 0.01

5.37 ± 0.03 5.37 ± 0.03

TABLE LXXIX: GMM-selection ablation under IID partition with 30% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1258.21 ± 4.70 1258.21 ± 4.70

669.37 ± 5.86 669.37 ± 5.86

215.61 ± 6.70 215.61 ± 6.70

TABLE LXXX: GMM-selection ablation under IID partition with 30% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.74 ± 0.04 6.74 ± 0.04

6.24 ± 0.10 6.24 ± 0.10

5.52 ± 0.04 5.52 ± 0.04

TABLE LXXXI: GMM-selection ablation under IID partition with 40% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.16 ± 0.00 7.16 ± 0.00

6.55 ± 0.02 6.55 ± 0.02

5.38 ± 0.03 5.38 ± 0.03

TABLE LXXXII: GMM-selection ablation under IID partition with 40% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1281.63 ± 1.74 1281.62 ± 1.73

697.62 ± 13.62 697.62 ± 13.62

217.14 ± 6.85 217.14 ± 6.85

TABLE LXXXIII: GMM-selection ablation under IID partition with 40% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.80 ± 0.06 6.80 ± 0.06

6.24 ± 0.06 6.24 ± 0.06

5.52 ± 0.02 5.52 ± 0.02

XXXX, 2026.

48

B. Results Under Non-IID Partitions Tables LXXXIV–XCVIII report the corresponding non-IID results. The non-IID results follow the same pattern: BICselected one-vs-two GMM and forced two-component GMM have nearly indistinguishable endpoint values in most settings. At the strongest 40% malicious-client setting, the two methods produce identical reported values for all three model families. For GPT/WikiText, both rules obtain test loss 7.16 ± 0.01, perplexity 1286.50 ± 15.34, and token-level semantic entropy 6.92 ± 0.24. For BERT/Tiny Shakespeare, both obtain test loss 6.48 ± 0.13, perplexity 658.21 ± 91.66, and token-level semantic entropy 6.19 ± 0.13. For LLaMA/StackOverflow, both obtain test loss 5.39 ± 0.03, perplexity 220.18 ± 6.45, and token-level semantic entropy 5.52 ± 0.03. The main visible difference appears only in a small number of intermediate settings. For example, under GPT/WikiText with non-IID partitioning and 30% malicious clients, BIC-selected GMM gives perplexity 1277.28 ± 23.52, while forced 2-GMM gives 1277.81 ± 22.93. For token-level semantic entropy in the same setting, forced 2-GMM gives 6.87 ± 0.06, while BIC-selected GMM gives 6.90 ± 0.02. These differences are small and do not indicate a systematic advantage of forced splitting. TABLE LXXXIV: GMM-selection ablation under Non-IID partition with 0% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.11 ± 0.01 7.11 ± 0.01

6.47 ± 0.05 6.47 ± 0.05

5.37 ± 0.03 5.37 ± 0.03

TABLE LXXXV: GMM-selection ablation under Non-IID partition with 0% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1225.90 ± 12.06 1225.89 ± 12.06

643.56 ± 33.46 643.56 ± 33.46

215.55 ± 6.85 215.55 ± 6.85

TABLE LXXXVI: GMM-selection ablation under Non-IID partition with 0% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.80 ± 0.09 6.80 ± 0.09

6.12 ± 0.05 6.12 ± 0.05

5.54 ± 0.03 5.54 ± 0.03

TABLE LXXXVII: GMM-selection ablation under Non-IID partition with 10% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.12 ± 0.00 7.12 ± 0.00

6.36 ± 0.06 6.36 ± 0.06

5.38 ± 0.03 5.38 ± 0.03

C. Interpretation The BIC-selected and forced two-component rules yield nearly identical endpoint values in the evaluated runs. The forced rule always partitions the current participants, whereas BIC preserves the option to retain all clients when the one-component model receives the lower criterion value. This ablation therefore characterizes endpoint sensitivity to the model-selection rule; it does not establish statistical identifiability of the fitted mixtures.

XXXX, 2026.

49

TABLE LXXXVIII: GMM-selection ablation under Non-IID partition with 10% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1238.52 ± 4.97 1238.52 ± 4.97

576.63 ± 36.58 576.63 ± 36.58

216.13 ± 6.70 216.13 ± 6.70

TABLE LXXXIX: GMM-selection ablation under Non-IID partition with 10% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.79 ± 0.05 6.79 ± 0.05

6.06 ± 0.18 6.06 ± 0.18

5.54 ± 0.03 5.54 ± 0.03

TABLE XC: GMM-selection ablation under Non-IID partition with 20% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.13 ± 0.01 7.13 ± 0.01

6.42 ± 0.14 6.42 ± 0.14

5.38 ± 0.03 5.38 ± 0.03

TABLE XCI: GMM-selection ablation under Non-IID partition with 20% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1251.65 ± 11.37 1251.65 ± 11.37

618.30 ± 91.12 618.30 ± 91.12

216.97 ± 6.91 216.97 ± 6.91

TABLE XCII: GMM-selection ablation under Non-IID partition with 20% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.04 ± 0.40 7.04 ± 0.40

6.33 ± 0.09 6.33 ± 0.09

5.53 ± 0.03 5.53 ± 0.03

TABLE XCIII: GMM-selection ablation under Non-IID partition with 30% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.15 ± 0.02 7.15 ± 0.02

6.38 ± 0.18 6.38 ± 0.18

5.38 ± 0.03 5.38 ± 0.03

TABLE XCIV: GMM-selection ablation under Non-IID partition with 30% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1277.28 ± 23.52 1277.81 ± 22.93

594.61 ± 108.75 594.63 ± 108.74

218.02 ± 7.07 218.02 ± 7.07

XXXX, 2026.

50

TABLE XCV: GMM-selection ablation under Non-IID partition with 30% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.90 ± 0.02 6.87 ± 0.06

6.00 ± 0.38 6.00 ± 0.38

5.53 ± 0.02 5.53 ± 0.02

TABLE XCVI: GMM-selection ablation under Non-IID partition with 40% malicious clients. Metric: Test Loss. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

7.16 ± 0.01 7.16 ± 0.01

6.48 ± 0.13 6.48 ± 0.13

5.39 ± 0.03 5.39 ± 0.03

TABLE XCVII: GMM-selection ablation under Non-IID partition with 40% malicious clients. Metric: Test PPL. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

BIC 1/2-GMM Forced 2-GMM

1286.50 ± 15.34 1286.50 ± 15.34

658.21 ± 91.66 658.21 ± 91.66

220.18 ± 6.45 220.18 ± 6.45

TABLE XCVIII: GMM-selection ablation under Non-IID partition with 40% malicious clients. Metric: Semantic Entropy. Values are mean ± standard deviation over three seeds. For each seed, all metrics are taken from the checkpoint with the lowest test loss. The best case is highlighted in bold. GMM rule BIC 1/2-GMM Forced 2-GMM

GPT / WikiText

BERT / Shakespeare

LLaMA / StackOverflow

6.92 ± 0.24 6.92 ± 0.24

6.19 ± 0.13 6.19 ± 0.13

5.52 ± 0.03 5.52 ± 0.03

Related documents

• SGDE: Secure Generative Data Exchange for Cross-Silo Federated Learning#385449
• Information-Theoretically Secure Aggregation for Lightweight Federated Learning: Resilient to Dropouts and Adversaries#987757
• FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Models#670363
• FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Models#668298
• What the Eyes See, the LLMs Miss: Exploiting Human Perception for Adversarial Text Attacks#267549