SWB-DM: A Calibrated Sliced-Wasserstein-Barycenter Aggregator with Delayed-Momentum Caching for Byzantine-Robust Federated Learning under Partial Participation
arXiv:2609.16099v1 [cs.LG] 14 Sep 2026
1
Saranraj S. Department of AIML Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology Chennai, India [email protected]
2
Saranya M. S. Assistant Professor, Department of AIML Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology Chennai, India [email protected]
3
Alex David S. Professor, Department of AIML Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology Chennai, India [email protected]
4
Ajay Kumar A. Department of AIML Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology Chennai, India [email protected] Abstract—Most robust aggregation methods for federated learning are designed and benchmarked under an implicit assumption: that the clients sampled in any given round look roughly like the full population. In practice, partial participation breaks this assumption. Even when the overall fraction of Byzantine clients is modest, a small per-round sample can easily end up dominated by adversaries – and that is enough to quietly violate the finite-sample guarantees that methods like coordinate-wise median, Krum, Bulyan, and trimmed mean rely on. We introduce SWB-DM, which combines two ideas. The first, SWB, is a randomized-slicing aggregator that treats each chunk of a client’s update vector as a one-dimensional empirical distribution, computes a trimmed Wasserstein barycenter across clients, and then restores coordinate identity through a medoidbased gauge-fixing step – a heuristic we developed ourselves and make no claim it belongs to standard optimal-transport theory. The second is a DeMoA-style delayed-momentum cache that aggregates over the entire client population each round, not just whoever happened to be sampled. Calibrating SWB’s trim ratio to the assumed corruption level turns out to be essential, not cosmetic: under-trimming leads to collapse at corruption levels a properly calibrated version survives. We evaluate SWB-DM across 448 CIFAR-10 configurations (7 methods, 16 attack–corruption pairings, 2 participation rates, 2 seeds), supplemented by CIFAR100, FEMNIST, and a 500-client scalability experiment. What emerges is not one failure mode but several mechanistically distinct ones: median degrades to a deterministic wrong answer when the sample size is small and even, Krum can silently violate its own n > 2f + 2 precondition and diverge without any warning, and Bulyan’s n ≥ 4f + 3 threshold produces a sharp, reproducible pass/fail boundary. On the attack side, IPM defeats order-statistic
defenses – including SWB – more reliably than ALIE, a finding we corroborate through delta-space error measurements against a semi-formal convergence bound. SWB-DM’s caching does come with a genuine warm-up cost: all 64 CIFAR-10 and all 8 CIFAR100 configurations improved when given more rounds, and the magnitude of this cost tracks participation rate and task difficulty exactly as the caching mechanism would predict. To put this in perspective, we extended every baseline to the same round budget and found that SWB-DM’s gain on CIFAR-10 is disproportionately large – though on CIFAR-100, FLTrust benefits even more from the extra rounds, for an entirely different reason unrelated to caching. Index Terms—federated learning, Byzantine robustness, sliced optimal transport, delayed momentum, robust aggregation, partial participation, adversarial machine learning
Code Availability— The complete source code, experimental grids, and diagnostic scripts are publicly available at https: //github.com/saranrajsnkr/SWB-DM-full-empirical-study. I. I NTRODUCTION The promise of federated learning is that a shared model can be trained across many clients without ever pooling their raw data – but the flip side is that the server must accept gradient updates it has no way to independently verify [2]. The standard response has been to replace FedAvg’s simple weighted average [1] with aggregation rules designed to limit how much a handful of corrupted clients can steer the result: coordinate-wise median or trimmed mean [3], Krum’s nearest-
1
neighbor selection [2], Bulyan’s layered combination of both placed in context by extending all six baselines to the same [4], or FLTrust’s cosine-similarity scoring against a server-held round budget. On CIFAR-10 SWB-DM’s gain turns out reference [5]. Every one of these carries a precondition – an to be disproportionately large; on CIFAR-100, however, assumed upper bound on how many adversaries are present FLTrust benefits even more – for an entirely different, nonand, in some cases, a trim parameter that must exceed the caching reason. true corruption fraction – and those preconditions are almost II. R ELATED W ORK always stated in terms of the full client population, not the FedAvg and robust aggregation. FedAvg [1] simply much smaller group that actually participates in any single averages client updates weighted by sample count, offering round. Under partial participation, this distinction matters im- zero protection against even a single unbounded adversarial mensely. The population-level corruption fraction β tells us contribution. Coordinate-wise median and trimmed mean [3] nothing about how many attackers actually end up in a single limit per-coordinate influence, but they need a certain minimum round’s sample n, which is usually much smaller than the full number of honest samples to do so: when n is small and even, population N . Simple sampling noise can easily spike effective the median convention just picks the lower of two middle values, corruption well past β in any given round—and a strategic which is deterministic and has no meaningful robustness left. attacker who actively skips unfavorable rounds creates an even Krum [2] selects the single update whose summed distance to its n − f − 2 nearest neighbors is smallest, tolerating f bigger problem. In both cases, the aggregator’s safety precondition quietly adversaries only when n > 2f + 2. Bulyan [4] stacks iterative fails, even if you configured the system with an exact population Krum-style selection on top of trimmed mean, tightening the parameter. Otsuka, Takezawa, and Yamada tackle this directly requirement to n ≥ 4f + 3. FLTrust [5] takes a fundamentally with Delayed Momentum Aggregation (DeMoA) [8]: by different approach: it scores each client by cosine similarity caching every client’s latest update and aggregating over the with a small reference update the server computes on its own entire cache each round, the server removes selection luck data, then rescales passing clients’ updates to match the server reference norm before averaging. Unlike the others, it does not from the sample size equation entirely. depend on a minimum sample size to function. Our contributions are as follows: Byzantine attacks. Label-flipping, sign-flipping, and addi1) SWB, a new aggregator that takes a rotated chunk of client tive Gaussian noise corrupt updates without adapting to the updates, treats it as a one-dimensional empirical distribution, defense. ALIE [6] computes the largest perturbation that stays and computes a trimmed Wasserstein barycenter across inside a robust statistic’s acceptance band for a given (n, f ). clients. Coordinate identity – lost during the sorting step IPM [7] negates the honest gradient direction and norm-matches. that makes the 1-D transport tractable – is restored through Despite ALIE’s analytical sophistication, IPM defeats more a Wasserstein-medoid gauge-fixing step that we want to aggregators in our experiments. be upfront about: it is our own heuristic, not something Delayed Momentum Aggregation. DeMoA [8] is the most borrowed from the established sliced optimal-transport directly relevant prior work. It maintains a server-side cache literature (Section IV). of every client’s most recent update; only sampled clients 2) SWB-DM, which pairs SWB with DeMoA-style caching refresh their entries each round. The aggregator then runs on and includes a demonstration that calibrating the trim ratio the full cache of N entries, keeping effective corruption near to the assumed corruption level is not optional – an under- the population-level β regardless of sampling luck. This is the trimmed variant collapses at corruption levels the properly caching mechanism we adopt for SWB-DM. Our contribution calibrated version handles without difficulty. is pairing it with a different inner aggregator and characterizing 3) A systematic 448-configuration CIFAR-10 study (7 methods, the resulting warm-up dynamics. 16 constrained attack–corruption pairings, 2 participation Sliced Wasserstein barycenters. The formal SWB [10] rates, 2 seeds), complemented by CIFAR-100 and FEMNIST computes barycenters over N probability distributions projected generalization experiments and a 500-client scalability onto random 1-D lines where optimal transport has a closed trial. Crucially, we trace per-round diagnostics – accuracy, form. Federated aggregation does not fit this framework — parameter norms, prediction-class histograms – that let us each client contributes a point, not a distribution. Section IV tease apart the failure mechanisms of median, Krum, and describes how we work around this by slicing coordinates Bulyan, rather than burying them all under one aggregate within a chunk. Doing so creates a coordinate-identity problem accuracy number. not addressed in prior OT work; we propose a solution in 4) A semi-formal convergence bound validated against two Section IV. structurally different adaptive attacks (ALIE and IPM), III. T HREAT M ODEL revealing that their error profiles are qualitatively different: ALIE’s is non-monotonic while IPM’s rises strictly, which A server coordinates N clients over T communication rounds. goes a long way toward explaining why IPM breaks more In each round, a subset St of size n = pN is sampled at defenses in our grid. participation rate p. A fixed fraction β of the population is 5) An honest accounting of SWB-DM’s caching warm-up cost, Byzantine and remains so for the entire run, though which
2
SWB-DM Federated Learning Architecture under Partial Participation
Partial Client Selection Global Model Parameters w(t) Broadcast to S(t)
broadcast
N total clients Sample S(t), n = pN
Local Training selected clients
2 local epochs (SGD) Benign trained states
Dirichlet non-IID data
Threat Injection trained states
s_i(t), i in S(t)
Label-flip, sign-flip, Gaussian ALIE or IPM adaptive attack Byzantine fraction beta
sampled client states
Delayed-Momentum Client Cache
Sliced-Wasserstein-Barycenter (SWB) Aggregator
Refresh: c_i(t) = s_i(t), i in S(t) Retain: c_i(t) = c_i(t-1), i not in S(t) Aggregate all N cached states
full cache
4. Gauge Fix
5. Restore
X in R^(N x C)
Y = XQ
Sort each row
Nearest medoid
Apply inverse Q^T
P random views
1. Chunk
Q orthogonal
2. Rotate
3. Sort and Trim
Trimmed barycenter b
Borrow permutation
Average P views
Effective corruption remains near beta calibrated trim
next round
aggregated state
Trim ratio: r = min(0.45, beta + 0.05)
Robust Global State Replacement w(t+1) = SWB({c_i(t)}), i = 1,...,N Aggregate replaces the global state Accuracy and stability diagnostics
Full-population state caching reduces round-wise Byzantine concentration; calibrated SWB limits adversarial influence.
Fig. 1. Architecture of the proposed SWB-DM framework, integrating partial client participation, Byzantine threat injection, delayed-momentum full-population caching, and calibrated sliced-Wasserstein-barycenter aggregation.
of those adversaries happen to fall into St varies from round C projected values within a single client’s chunk are treated to round. For every aggregator that carries a precondition as samples from an empirical distribution. on the number of tolerable adversaries (Krum, Bulyan), we 3) Compute a trimmed mean across clients at each sorted-rank set f = ⌊nβ⌋ – a fixed, population-level estimate computed position, discarding the top and bottom ⌊n · r⌋ values, to without any access to the true per-round attacker count. Giving produce a barycenter row b ∈ RC in sorted-rank space. a defender oracle knowledge of who is actually malicious 4) Medoid gauge fixing. The sorting in step 2 destroys each round would be unrealistic and would unfairly advantage each client’s original mapping from coordinates to ranks. methods whose guarantees depend on knowing the count. To undo this, we identify the medoid client m = Figure 1 illustrates the overall architecture. Each communiarg mini ∥sort(Yi ) − b∥2 – the client whose sorted chunk cation round, the global server selects a subset of clients for is closest to the barycenter – and borrow that client’s rankparticipation. After local training, any Byzantine clients in the to-coordinate permutation to place b back into the correct sample may corrupt their updates using label-flipping, signcoordinate positions before rotating back with Q⊤ . flipping, Gaussian noise, ALIE, or IPM attacks. The delayed- We repeat this procedure over P =2 independent random momentum cache then refreshes entries for participating clients rotations and average the results. The trim ratio is set as while retaining the most recent updates from everyone else. r = min(0.45, β + 0.05), tying it directly to the assumed SWB processes the complete cached population through a corruption level (Section VI). pipeline of chunking, randomized orthogonal rotation, rowStep 4 is specific to our setting; the standard SWB literature wise sorting, calibrated trimming, medoid-based gauge fixing, operates on unordered point clouds where coordinate identity inverse rotation, and multi-view averaging. The resulting robust is not an issue. An alternative approach — restoring identity by aggregate replaces the global model heading into the next round. resorting the barycenter according to original coordinate ranks — did not produce usable aggregates. The medoid approach IV. M ETHOD borrows the rank-to-coordinate permutation from the nearest A. SWB: A Randomized-Slicing, Wasserstein-Inspired Aggrereal client. It performs well empirically but lacks a formal gator guarantee. We refer to SWB throughout as inspired by, rather Given n client vectors {δ1 , . . . , δn } ⊂ RD — in our than an instance of, the formal SWB framework. implementation, these are full locally-trained model states submitted for aggregation, not deltas from the global model B. Delayed-Momentum Caching — we split them into chunks of size C. For each chunk, let (t) Let ci be the server’s cached copy of client i’s latest locallyX ∈ Rn×C be the matrix stacking the n clients’ values. SWB (t) (t) (t) trained model state at round t: ci ← si for i ∈ St , where si then proceeds as follows: (t) (t−1) otherwise. 1) Draw a random orthogonal matrix Q ∈ RC×C (obtained by is the trained state client i submits, and ci = ci QR-decomposing a Gaussian random matrix) and project: The server then replaces the global model with the aggregate (t) (t) Y = XQ. over the full cache, w(t+1) = Agg({c1 , . . . , cN }), for any 2) Sort each client’s row of Y independently. This is the step base aggregator Agg(·). SWB-DM uses SWB as that base that makes one-dimensional optimal transport tractable: the aggregator; we also test a DelayedMomentum baseline using
3
plain median instead, to separate what the caching contributes from what the inner statistic contributes.
VII. R ESULTS A. Main Grid
V. T HEORETICAL A NALYSIS
Table I presents the mean CIFAR-10 accuracy for each method across both participation rates, averaged over all six We adopt the standard robust-mean-estimation bound shape √ attack conditions and random seeds. E(β) ≤ E0 + C β (Eq. 1), where E(β) denotes the L2 At p=0.1, SWB-DM falls behind all other methods within aggregation error relative to the true benign mean, E0 captures 10 rounds. Section VII-E attributes this to a warm-up effect the irreducible finite-sample error when β=0, and C reflects in the caching layer. FLTrust maintains stable accuracy across the spread of the benign update distribution [9]. p corruption levels and participation rates because its trust scoring E(β) ≤ E0 + C β (1) uses a server-held clean dataset rather than statistics derived We fit C empirically to SWB’s measured error. This does not from the participating pool. Krum achieves higher accuracy at p=0.1 than at p=0.5 constitute a tight, SWB-specific derivation; we use it to check (43.10 vs. 34.11 at β=0). This reflects its single-client selection whether the general bound shape holds against two structurally rule: Krum selects one update and discards the rest. At p=0.5 different adaptive attacks (Section VII-C). A tighter analysis with n=10, nine updates are discarded per round. Under nonis left to future work. IID partitions where each client captures only part of the global VI. E XPERIMENTAL S ETUP distribution, the information loss from discarding most updates Datasets and model. We use CIFAR-10 as the primary accumulates across rounds. Table II counts outright collapses — configurations where testbed, with N =20 clients for most experiments and N =500 accuracy fell to ≤ 10.5%, essentially chance-level on CIFARfor the scalability study. For generalization, we also run on 10. CIFAR-100 and FEMNIST (the EMNIST-ByClass variant with FLTrust never collapses across all 64 configurations. Median 62 classes). All experiments use a compact two-convolutionalis the most collapse-prone at 10 out of 64, predominantly under layer CNN, trained for 2 local epochs per round with SGD IPM (Section VII-B). (learning rate 0.01, momentum 0.9). Partitioning. Data is split across clients using a Dirichlet B. Failure-Mode Diagnostics non-IID partition with α=0.5, and the partition seed is held At the grid’s most extreme setting (n = 2: IPM with fixed across all methods and random seeds so that everyone trains on exactly the same data split. At N =500, per-client β = 0.3 at p = 0.1), a 20-round per-round trace (see sample counts range from 13 to 288 (median 92) – a 22× diagnostics.py in our repository, https://github.com/ spread that we verified by direct measurement rather than saranrajsnkr/SWB-DM-full-empirical-study) exposes two distinct failure modes. assumption. a) Median under IPM at n=2: With two participating Baselines. For the main 448-configuration grid, we compare against FedAvg, Median, Krum, Bulyan, and FLTrust, alongside clients, the coordinate-wise median returns the smaller of the our proposed SWB and SWB-DM (7 methods total). For the two per-coordinate values. IPM constructs its update to point 500-client scalability study (Section VII-F), we additionally opposite the honest gradient direction on most coordinates, so include a plain DelayedMomentum baseline (DeMoA with the smaller value is typically the attacker’s. Accuracy holds at coordinate-wise median) to isolate the inner statistic’s contri- exactly 10.00% across all 20 rounds — chance level for ten classes — and all test samples are assigned the same label in bution from the caching mechanism. Attacks. The attack suite includes no attack (clean), label- every round. The parameter norm increases from 9.89 at round 1 to 31.56 flip, sign-flip, Gaussian noise, ALIE [6], and IPM [7]. ALIE’s perturbation magnitude is computed analytically from the at round 20, indicating that gradient updates are applied but in directions that do not reduce the training loss. round’s (n, f ) values. b) Krum at n=2: Krum’s guarantee needs n > 2f +2. At Grid. Corruption levels are β ∈ {0, 0.1, 0.2, 0.3}, but not every combination runs: the clean (no-attack) condition n = 2, no value of f satisfies that. We built a guarded version only appears at β=0, and the five actual attacks run only at that checks the bound before aggregating; it simply refuses to β ∈ {0.1, 0.2, 0.3}, yielding 1 + 5 × 3 = 16 attack–corruption run when n = 2, which is the right call. Our main grid, though, pairings per method rather than the full 6 × 4=24. Participation uses the unguarded variant everywhere for consistency across rates are p ∈ {0.5, 0.1}, seeds are {42, 7}, and each run goes configurations. Left to compute a neighbor-distance score over for 10 rounds. In total, that gives 7 × 16 × 2 × 2 = 448 two clients, it has almost nothing to rank against. By round 7, configurations. For Krum and Bulyan, we set f = ⌊nβ⌋. it picks the corrupted client as the global update. Bulyan needs a larger sample to satisfy its n ≥ 4f + 3 The damage is immediate and total: 545,066 of the model’s requirement, so we draw n=11 clients per round for it (versus 545,098 parameters turn to NaN in that single step, and the n=10 everywhere else) — this is enough to meet the bound other 32 overflow to Inf. Most aggregation rules blend several at β=0.2, but not at β=0.3. We leave that violation in on clients together, so one bad update gets diluted. Krum doesn’t purpose, as a stress test (Section VII-B). blend — it copies. Whichever client it selects becomes the
4
TABLE I M EAN ACCURACY (%) AS A FUNCTION OF CORRUPTION β, SHOWN FOR p=0.5 AND p=0.1, AVERAGED ACROSS ALL SIX ATTACK CONDITIONS AND 2 RANDOM SEEDS . p = 0.5
p = 0.1
β=0
β=0.1
β=0.2
β=0.3
β=0
β=0.1
β=0.2
β=0.3
FedAvg 65.78 Median 61.78 Krum 34.11 Bulyan 65.40 FLTrust 58.54 SWB 60.64 SWB-DM 52.17
62.61 59.92 41.55 62.42 56.94 59.22 49.16
57.87 52.48 34.63 52.37 56.51 52.18 44.21
38.90 32.89 26.77 32.91 54.40 31.67 35.87
53.00 41.22 43.10 53.98 54.36 53.37 22.63
52.83 29.06 45.96 53.55 54.76 53.61 19.76
47.32 25.85 44.78 48.31 53.94 46.96 15.78
39.71 21.13 41.08 39.15 47.74 40.59 14.54
Method
Fig. 2. Empirical aggregation error against corruption level under ALIE (left) and IPM (right), compared with the fitted bound.
d) Krum under IPM within the safe zone: At β=0.2, n=10, f =2, Krum’s precondition n > 2f + 2=6 holds with margin. Under ALIE, Krum reaches 51.15% accuracy. Under IPM with the same (n, f ), it drops to 10.00%. IPM negates the honest gradient direction and matches its norm, producing corrupted updates that minimize the neighbordistance score Krum uses to rank clients. The corrupted update receives a low distance score and is selected as the representative. The precondition n > 2f + 2 is necessary but not sufficient against this class of attack.
TABLE II N UMBER OF CONFIGURATIONS THAT COLLAPSED (≤ 10.5% ACCURACY ) OUT OF 64 TESTED FOR EACH METHOD . Method FedAvg Median Krum Bulyan FLTrust SWB SWB-DM Count
3
10
5
4
0
5
2
global state, so if that client is corrupted, so is the model, permanently. We see the same fragility in the extended CIFAR-100 runs (Section VII-E), though it shows up as inconsistency rather than collapse. One seed under label-flip corruption drops from 13.82% at round 10 to 1.96% by round 25, hovering near the ˜1% chance rate for 100 classes. A second seed, same corruption, same everything else, keeps improving the whole time. Krum’s accuracy on any given run is really a question of which two or three clients happened to get sampled. c) Bulyan at the precondition boundary: Bulyan needs n ≥ 4f + 3. At β=0.2, with n=11 and f =2, that works out to 11 ≥ 11 — satisfied with nothing to spare. And the method holds: 51.70–51.98% accuracy under both ALIE and IPM, right where the guarantee says it should land. Push β to 0.3 and f becomes 3, so the requirement jumps to n ≥ 15. We’re still at n=11. The bound fails, and Bulyan doesn’t degrade gracefully about it — accuracy falls straight to 10.00% under both attacks. One step up in β is the difference between a working defense and pure chance, with nothing in between. We reran the configuration a second time to make sure; the result didn’t move.
C. Bound vs. Empirical Error, Two Attacks Figure 2 plots the delta-space L2 aggregation error for Median and SWB as the corruption count f ranges from 0 to 4 out of 10 sampled clients, under ALIE (left panel) and IPM (right panel), with the bound from Eq. 1 fitted to SWB. The two attacks leave very different fingerprints on the error curve. Under ALIE, error for both methods actually dips below the f =0 baseline at f =1 and f =2, before it starts climbing. SWB, specifically, goes 1.478 → 1.101 → 1.087 before turning upward at f =3 and f =4. That dip isn’t noise — it’s what ALIE is built to do. The attack tunes each poisoned update to sit just inside a robust statistic’s tolerance threshold, so a little corruption can briefly pull the aggregate closer to the true mean. Add more corrupted clients, though, and the cumulative damage eventually overwhelms that effect. IPM doesn’t behave this way at all. SWB’s error climbs at every step — 1.483, 1.529, 1.769, 2.686, 3.680 — with no dip anywhere. That tracks with how IPM works: it pushes updates along the negative gradient direction and simply scales the
5
TABLE III FEMNIST MEAN ACCURACY (%), β=0.2, p=0.5, 2- SEED AVERAGE . Method
None Label-flip Sign-flip Gaussian
FedAvg Median Krum Bulyan FLTrust SWB SWB-DM
84.60 83.62 75.66 84.94 83.08 82.96 80.68
71.24 80.69 71.61 81.68 82.41 79.44 79.15
74.50 75.11 75.93 80.84 82.52 76.72 77.38
TABLE IV M EAN ACCURACY GAIN ( PERCENTAGE POINTS ) FROM ROUND 10 TO ROUND 25, ALL SEVEN METHODS , CIFAR-10 ( FULL GRID ) AND CIFAR-100 ( FOUR - ATTACK SUBSET ).
83.78 81.26 76.09 82.93 82.48 80.20 78.43
Method
CIFAR-10 gain
CIFAR-100 gain
+3.72 +2.43 +5.05 +5.12 +7.91 +4.07 +13.12
+6.05 +8.57 +0.93 +6.69 +14.19 +6.77 +10.39
FedAvg Median Krum Bulyan FLTrust SWB SWB-DM
push with each additional corrupted client. It’s the same story we saw on the accuracy side in Section VII-B and Table II. One thing worth flagging: SWB’s aggregation error is worse than Median’s across the board, every f , both attacks. That’s the price of chunked rotation — it shows up clearly here in delta space, but end-to-end accuracy tends to hide it. Put plainly, SWB wins downstream (Sections VII-A–VII-E) despite, not because of, its per-round aggregation error.
TABLE V 500- CLIENT RESULTS : 10- ROUND MAIN RUN , AND 40- ROUND EXTENSION FOR THE TWO CACHING METHODS . WALL - CLOCK AND PER - ROUND AGGREGATION TIME (WALL , AGG ) REFER TO THE 10- ROUND RUN IN EVERY ROW; THE LAST TWO ROWS ’ ACCURACY COLUMN ADDITIONALLY REPORTS THE 40- ROUND ENDPOINT.
D. Generalization: CIFAR-100 and FEMNIST
Method
Wall(s) Agg(s/rnd) Pool
FedAvg Median Krum Bulyan FLTrust SWB DelayedMomentum SWB-DM
29.6 29.6 30.3 52.6 32.1 91.1 63.3 163.0
0.01 0.02 0.10 2.34 0.28 6.18 3.36 13.35
Acc.%
50 30.34 50 24.39 50 10.10 50 28.53 50 22.87 50 22.79 500 8.61 → 20.61 (r40) 500 8.68 → 17.16 (r40)
Table III presents mean FEMNIST accuracy across 62 classes, at β=0.2, p=0.5, over 10 rounds. On FEMNIST, FedAvg takes a much steeper hit under labelflip and sign-flip attacks (71.24%, 74.50%) than its CIFAR10 numbers at comparable corruption levels would lead you to expect. Bulyan, FLTrust, and Median, on the other hand, all hold up well, landing comfortably in the 76–85% range. the same 25-round budget, covering the full CIFAR-10 grid If anything, this makes an even stronger case for robust and the CIFAR-100 subset. Table IV reports the round-10-toround-25 accuracy gain for every method. aggregation on harder tasks, not a weaker one. Every method benefits from additional rounds. On CIFARCIFAR-100 accuracies are lower across the board, as one would expect with five times as many classes (Krum sits at 10, SWB-DM’s gain is 1.7–5.4× larger than any baseline’s, 13–15%, FedAvg at 19–32%), and SWB-DM again trails at 10 confirming that its warm-up cost exceeds what continued rounds (8.43–14.49%) for the same warm-up reason confirmed training alone accounts for. CIFAR-100 shows a different pattern: FLTrust’s gain (+14.19) exceeds SWB-DM’s (+10.39). in Section VII-E. FLTrust’s server-side reference model requires more training E. SWB-DM’s Warm-Up Cost iterations to mature on a 100-class task, which is independent Because SWB-DM’s cache only gets refreshed for whichever of the caching mechanism. The large warm-up effect observed clients happen to be sampled each round, it naturally takes on CIFAR-10 does not transfer uniformly to other datasets. longer to “fully warm up” when participation is low or the task Krum is the other outlier in Table IV. Its CIFAR-100 gain is inherently harder. We wanted to quantify this directly, so we is negligible (+0.93), and as discussed in Section VII-B, this re-ran all 64 CIFAR-10 configurations out to 25 rounds and near-zero average conceals a divergence in one of its two did the same for 8 CIFAR-100 configurations. The results were seeds. Krum’s poor CIFAR-100 performance reflects the same unambiguous: every single configuration improved. On CIFAR- structural fragility documented above, not a warm-up cost that 10, all 64 out of 64 showed gains, with a mean improvement additional rounds can resolve. of 13.12 percentage points. The improvement was larger at p=0.1 (15.25 points) than at p=0.5 (11.00 points) – exactly F. Scalability at 500 Clients Table V reports results for a larger deployment with N =500 the direction you would expect if this is genuinely a warmup effect. On CIFAR-100, all 8 out of 8 improved, gaining clients, p=0.1 (so 50 are sampled per round), over 10 rounds 10.39 points on average and moving the mean from 11.93% on clean data. The final two rows also show what happens up to 22.32%, an 87% relative improvement. Not a single when we extend the two caching methods out to 40 rounds. configuration got worse, and none stayed flat. The 10-round The aggregation cost column captures each method’s compufigures in Tables I and III should be interpreted alongside these tational overhead, ranging from FedAvg’s lightweight average extended results. (0.01s per round) all the way to SWB-DM’s full 500-client Is the warm-up cost unique to caching? To answer this chunked-rotation pipeline (13.35s). Both caching methods hover fairly, we extended all six baselines – not just SWB-DM – to near chance level at 10 rounds but recover substantially by
6
round 40, confirming that the mechanism still works at this IX. C ONCLUSION larger scale. That said, SWB-DM’s recovery is noisier and We introduced SWB-DM, combining a randomized-slicing less complete than DelayedMomentum’s (17.16% vs. 20.61%, aggregator inspired by sliced-Wasserstein barycenters with with a dip from 18.14% at round 35) – the reverse of what we delayed-momentum caching. Across 448 CIFAR-10 configurasee at every smaller scale in this paper, where SWB’s trimmed tions, CIFAR-100, FEMNIST, and a 500-client scalability study, statistic consistently beat plain median under the same cache. we observe that robust aggregation guarantees depend on finiteWe leave this as an open question: SWB’s chunked rotation sample assumptions that partial participation can violate. The may introduce per-aggregation variance that becomes more resulting failure modes — convergence to a fixed wrong answer, damaging when each cache slot only refreshes on average once numerical divergence, sharp precondition boundaries — are every N/n=10 rounds. Separately, Krum’s near-chance clean- distinct and require per-round diagnostic tracing to differentiate. data result (10.10%) reflects the same single-client-selection SWB-DM’s caching introduces a warm-up cost that on CIFARinefficiency we have seen throughout, made worse by the 10 exceeds all baselines extended to the same round budget. The 22× spread in client data sizes at this scale; extending to 40 500-client recovery gap and the lack of a formal guarantee for rounds brings it to 23.09%, but non-monotonically (a dip to medoid gauge-fixing remain open. Designing adaptive attacks 11.59% at round 20), consistent with occasionally selecting an that exploit the caching mechanism directly is a direction for unrepresentative, data-poor client. future work. VIII. D ISCUSSION AND L IMITATIONS
R EFERENCES
The failure modes documented above — convergence to a fixed wrong answer (Median), numerical divergence (Krum), and sharp precondition boundaries (Bulyan) — are distinct in mechanism, though they can produce similar aggregate accuracy numbers. Distinguishing them required per-round tracking of parameter norms and prediction histograms rather than final accuracy alone. Limitations. We identify two primary limitations of SWBDM. The first is the medoid gauge-fixing step. It held up across every configuration we tested, but we don’t have a proof that it has to. Picture an adversarial case where the medoid client’s own permutation is corrupted — that corruption could ride straight through into the aggregate. Nothing in our runs actually did this. Whether it’s fragile in that specific way is just something we can’t rule out yet. The second is a recovery gap at 500 clients (Section VII-F). Every smaller scale in this paper tells the same story: SWB’s trimmed statistic beats plain median under the same cache, consistently. Then at 500 clients that story flips. DelayedMomentum climbs to 20.61%; SWB-DM gets stuck at 17.16% and even dips non-monotonically around round 35. Our best guess is that chunked rotation is the culprit — each cache slot only refreshes about once every N/n=10 rounds at this scale, and stale entries may end up amplifying noise instead of averaging it away. We don’t have a clean proof of this yet, so we’re leaving it as an open question rather than a claim. A few smaller limitations round these out. At p=0.5 with low corruption, FedAvg, Bulyan, and Median all beat SWBDM (Table I) — the warm-up cost from Section VII-E biting harder at higher participation. Our bound (Eq. 1) borrows its functional form from robust-mean-estimation theory rather than being derived specifically for SWB; a tighter, SWB-specific bound is future work. The adaptive attacks we test assume the attacker knows the benign updates in a round but not the cache state itself — a cache-aware attacker would be a natural next step. Further per-round diagnostics are in the repository (https: //github.com/saranrajsnkr/SWB-DM-full-empirical-study).
[1] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Proc. AISTATS, 2017. [2] P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Machine learning with adversaries: Byzantine tolerant gradient descent,” in Proc. NeurIPS, 2017. [3] D. Yin, Y. Chen, R. Kannan, and P. Bartlett, “Byzantine-robust distributed learning: Towards optimal statistical rates,” in Proc. ICML, 2018. [4] E. M. El Mhamdi, R. Guerraoui, and S. Rouault, “The hidden vulnerability of distributed learning in Byzantium,” in Proc. ICML, 2018. [5] X. Cao, M. Fang, J. Liu, and N. Z. Gong, “FLTrust: Byzantine-robust federated learning via trust bootstrapping,” in Proc. NDSS, 2021. [6] G. Baruch, M. Baruch, and Y. Goldberg, “A little is enough: Circumventing defenses for distributed learning,” in Proc. NeurIPS, 2019. [7] C. Xie, O. Koyejo, and I. Gupta, “Fall of empires: Breaking Byzantinetolerant SGD by inner product manipulation,” in Proc. UAI, 2020. [8] R. Otsuka, K. Takezawa, and M. Yamada, “Delayed momentum aggregation: Byzantine-robust federated learning with partial client participation,” arXiv:2509.02970, 2025. [9] G. Lugosi and S. Mendelson, “Mean estimation and regression under heavy-tailed distributions: A survey,” Foundations of Computational Mathematics, 2019. [10] N. Bonneel, J. Rabin, G. Peyré, and H. Pfister, “Sliced and Radon Wasserstein barycenters of measures,” Journal of Mathematical Imaging and Vision, 2015.
7