Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy Asim Ukaye* 1 , Mubarak Abdu-Aguye1 , Nurbek Tastan1 , and Karthik Nandakumar1,2
arXiv:2604.22562v1 [cs.LG] 24 Apr 2026
1
MBZUAI, UAE
2
Michigan State University, USA
Abstract
global model. Existing aggregation schemes, such as FedAvg [12], implicitly assume honest self-reporting (e.g., sample counts), while other approaches, such as CFFL [8] and FedCE [3], rely on an auxiliary validation set to measure client model performance. Both assumptions can be problematic in practice: validation sets may not exist or may be biased relative to the clients’ distributions, and selfreported metadata can be strategically misrepresented, especially in incentive-based settings. Recent data-free approaches attempt to circumvent these issues by evaluating client updates without direct access to private data. For instance, CGSV [21] scores clients by the cosine similarity between their gradients and the cohort average, yet such similarity-based measures may penalize genuinely informative but dissimilar updates. Authors of ShapFed [17] propose Class-specific Shapley Values (CSSV) that computes class-wise alignment of the final layer of the model update. They show that this approach helps in improving the contribution estimates by accounting for label skew present in real-world data settings. In this work, we propose a novel data-free, non-selfreported metric for estimating client contribution in FL based on the matrix von Neumann entropy or spectral entropy [1], [15] of each client’s model update. Intuitively, the spectral entropy of the final layer’s weight update captures the diversity and richness of a client’s learned representation without requiring access to any data. We show that higher entropy correlates with more informative client updates and thus greater contribution to the global model. While spectral entropy is a strong per-round indicator, the true contribution of a client drifts over rounds due to local learning progress. Per-round measurements are therefore noisy and occasionally volatile. To turn these noisy observations into stable scores, we introduce a Rank-Adaptive Kalman Filter [6] component that models a client’s latent contribution as a hidden state evolving over time and treats spectral entropy as one of its observations. It uses CSSV as another observation and fuses them together to obtain a robust contribution estimate. We propose two variants of
Client contribution estimation in Federated Learning is necessary for identifying clients’ importance and for providing fair rewards. Current methods often rely on serverside validation data or self-reported client information, which can compromise privacy or be susceptible to manipulation. We introduce a data-free signal based on the matrix von Neumann (spectral) entropy of the final-layer updates, which measures the diversity of the information contributed. We instantiate two practical schemes: (i) SpectralFed, which uses normalized entropy as aggregation weights, and (ii) SpectralFuse, which fuses entropy with class-specific alignment via a rank-adaptive Kalman filter for per-round stability. Across CIFAR-10/100 and the naturally partitioned FEMNIST and FedISIC benchmarks, entropy-derived scores show a consistently high correlation with standalone client accuracy under diverse non-IID regimes – without validation data or client metadata. We compare our results with data-free contribution estimation baselines and show that spectral entropy serves as a useful indicator of client contribution. The code is available at: github.com/asimukaye/spectralfuse
1. Introduction Federated Learning (FL) enables the training of machine learning models across a population of clients without directly sharing their private data [12]. This paradigm addresses fundamental concerns about data privacy and regulatory compliance while leveraging distributed computational resources. Despite its promise, FL introduces unique challenges beyond standard centralized training. Chief among these are the statistical heterogeneity of client data, system heterogeneity of client devices, and the need for mechanisms to ensure trustworthy participation. [5]. A central yet underexplored aspect of trustworthiness in FL is the estimation of each client’s contribution to the * Corresponding author: [email protected]
1
Server @ Round
Update of final layer Von Neumann Entropy
SVD(
Rank Adaptive Kalman Filter
)
Eigenvalue Spectrum
Aggregation Weights Model Aggregation
Class-Specific Shapley Values layers Update Alignment
Client 1
Client i
Global Model
Client k
Client N
Figure 1. Illustration of the proposed SpectralFuse approach. We incorporate the von Neumann Entropy and Class-specific Shapley values of the clients final-layer updates to form a robust client contribution estimate which serves as weights for global model aggregation.
our approach: SpectralFed, which uses normalized spectral entropy directly for aggregation weights, and SpectralFuse, which fuses spectral entropy with class-specific Shapley values over time via a Kalman filter update to produce robust contribution estimates. We conduct extensive experiments on CIFAR-10 and CIFAR-100 under multiple partitioning regimes using different model architectures, as well as on the naturally partitioned FEMNIST [2], and FedISIC [13] datasets. For each client, we compute entropy-derived weights (for Spectralfed) and fusion-derived weights (for SpectralFuse) and compare these to standalone client accuracies. Our results show a strong positive correlation between the proposed weights and client standalone performance across most settings, suggesting that the metric reliably reflects client quality. When incorporated into the aggregation process, our weighting scheme yields global models with test accuracies that are on par with or outperform standard baselines. The key contributions of this paper are: • We introduce a data-free, non-self-reported contribution signal using the von Neumann entropy of client updates. • We formulate a Rank Adaptive Kalman Filter that treats entropy and CSSV as noisy observations of a latent contribution-estimation process. • We instantiate these ideas in SpectralFed (direct entropy
weighting) and SpectralFuse (fusion-based weighting), integrating them into standard FL process. • We provide an extensive evaluation across vision benchmarks under non-IID regimes, showing strong correlation with client quality and competitive predictive performance, without the need for auxiliary data or metadata.
2. Related Work Client Contribution Estimation in Federated Learning. A central challenge in FL is to fairly weight or reward clients based on their actual contribution to the global model. Early work such as FedAvg [12] uses the number of local samples as a proxy for client weight, implicitly assuming honest self-reporting. However, when incentives are tied to contribution, this assumption may not hold. More recent approaches rely on an auxiliary validation set to assess each client’s update. For example, CFFL [8] and FedCE [3] evaluate client models on server-side validation data to estimate their goodness. Yet, validation sets may be unavailable or biased relative to client data distributions, limiting the reliability of these metrics. Data-Free and Non-Self-Reported Methods. To avoid privacy risks or reliance on client metadata, several works propose data-free scoring of client updates [17–19, 21]. 2
CGSV [21] computes the cosine similarity between the gradient of each client and the average gradient of the cohort, rewarding updates that are better aligned with the average of the group. ShapFed [17] improves upon the gradient alignment strategy by proposing class-wise gradient alignment for the final layer of the model. This allows for better estimates of client contributions in scenarios with high skew in class labels within each client. Fed-LWR [22] introduces layer-wise relevance weighting of client updates, while FedPCA [20] leverages pairwise correlated agreement to score clients without a test set and uses those scores for aggregation and a strategy-proof incentive mechanism. FedBary [7] computes a Wasserstein barycenter over client distributions and uses barycentric distances as validation-free relevance scores. These methods highlight a growing interest in server-side evaluation of client updates without raw data.
ated learning is to minimize the objective:
\label {eq:opt} f^{\star } = \min _{w \in \mathbb {R}^d} f(w) ,& \quad \text {where} \quad f(w) \coloneqq \frac {1}{n} \sum _{i=1}^n f_i(w) \\ \ f_i(w) &\coloneqq \mathbb {E}_{z_i\sim \mathcal {D}_i}\big [F_i(w, z_i)\big ] (2) where Fi is typically the loss of the prediction over sample set zi drawn from the client distribution Di . In real-world settings, the client data distributions may not be identically distributed, i.e. Di ̸= Dj , which leads to inherent data heterogeneity in the client cohort. In a typical federated training setup, the clients and server communicate their models for a total of T communication rounds. At communication round t, the server broadcasts the global model parameters (t) wg to all participating clients who set this as their initial (t) model wg|t−1 . The clients then update their models using (t)
Despite their promise, similarity-based or projection-based methods can mischaracterize genuinely useful but diverse updates. Moreover, Shapley-based approaches often incur high computational costs and scale poorly with client count.
SGD and send back the updated model wi to the server. The server computes a new global model using an aggre(t) (t+1) = G(wi , c, . . .), where c is an gation scheme , wg aggregation coefficient used to perform optimal model aggregation. Different approaches use different variants of c along with additional parameters to obtain the global model. We focus on the subclass of methods, where c ∈ RN is the weight vector consisting of each client’s weight as a scalar used for the linear weighted aggregation of the client models. Our goal is to compute an online data-free contribution (t) estimate, c(t) = [ci ]T , that reflects the client’s true contribution to the global model. To motivate this better, we first look at the standard Federated Averaging algorithm.
Weight-based Measures. A growing line of work gauges model quality directly from weight-space geometry, requiring no training or test data. An initial empirical analysis shows that several training-free or minimally supervised proxies, grounded in weight spectra, are competitive predictors of generalization [4]. Random-matrix analyses argue that deep networks exhibit implicit self-regularization, where heavy-tailed spectra and scale metrics track generalization [10]. Building on this perspective, WeightWatcher predicts the accuracy trends of state-of-the-art networks using power-law exponents and other spectral properties without data [11]. Similar conclusions are shown in NLP, where training-free generalization metrics computed from parameters correlate with downstream performance across tasks [23]. Further, spectral or information-theoretic measures have been used to characterize model complexity or representation diversity [1]. However, to our knowledge, von Neumann entropy of client weight updates has not yet been explored as a metric for contribution in FL.
Federated averaging. A common approach to solving (1) is FedAvg [12]. This algorithm involves the participants performing several local steps of SGD and communicating with the server over multiple communication rounds. In each communication round, t, the updates from the participants are averaged on the server and sent back to all participants. For a local epoch e and participant i ∈ K, the local iterate is updated according to: w_g^{t+1} = \sum _{i=1}^n \frac {m_i}{M} w_i^t.
In summary, our work differs from prior methods by (i) requiring no auxiliary validation data, (ii) avoiding similarityto-average assumptions, and (iii) offering an inexpensive, information-theoretic proxy for client contribution that correlates with standalone performance across diverse datasets.
(3)
where, Pn mi is the total data size held by a client and M = i=1 mi is the total data size across all clients. Note that mi is a self-reported metric by the client. A variant of the FedAvg algorithm, FedOpt [16], uses a server side optimizer to perform aggregation. Von Neumann Entropy. Given a symmetric positive semidefinite matrix M with eigenvalues {λj }, the von Neumann entropy is defined as:
3. Preliminaries We consider a standard cross-silo federated learning scenario with N clients indexed by i ∈ K, where K = {1, 2, . . . , n}. The optimization problem posed for feder-
S(M) = -\sum _{j}\lambda _j \log \lambda _j. \label {eq:entropy_definition}
3
(4)
Data Size
6000
Kalman Filter. Kalman Filters are a popular choice of statistical filters used in the sensor fusion and control theory domains. They are known to be statistically optimal under Gaussian noise assumptions [6]. The typical Kalman Filter (t) is described as follows. Given xi as the target state to be (t) observed at time t, and yi as the measurements for that state. A standard Kalman Filter update is then given as: Predict:
Standalone accuracies 0.8
4000 2000 0
0
1 Client ID
2
0.4 0.2
2
4 6 Epochs
Entropy
Client 2
8 Client 0 Client 1 Client 2
0.8 0.6 0.4 0.2 0.0 0
Update:
0
Entropy
Client 1
Eigenvalue Magnitude
(5)
Client 0 Client 1 Client 2
0.6
Final layer Eigenvalues
Client 3
\hat {x}_{i|t-1} = \hat {x}_{i}^{(t-1)}, \quad P_{i|t-1}=P_{i}^{(t-1)}+Q, \label {eq:kf_pred} \\
Client Data Distribution Accuracy
Intuitively, this quantity measures the “spectral spread” or information content of M . In other words, matrices with more uniform eigenvalue distributions have higher entropy.
2 4 6 8 Eigenvalue Index
0
2
4 6 Epochs
8
Figure 2. Top left: Three clients with equal amount of data but varying number of labels from the MNIST dataset. Bottom left: Eigenvalue spectrum corresponding to the final-layer updates of each client. Top right: Accuracies obtained by each client on a hold out test set as training progresses. Bottom right: Spectral entropy for each client as the training progresses.
\label {eq:kf_upd_1} K_{i}^{(t)}&=P_{i|t-1}H^{\!\top }\! \Bigl (H\,P_{i|t-1}H^{\!\top }\! + R_{i}^{(t)}\Bigr )^{-1},\\ \hat {x}_{i}^{(t)}&=\hat {x}_{i|t-1}+K_{i}^{(t)} \Bigl (y_{i}^{(t)}-H\hat {x}_{i|t-1}\Bigr ),\label {eq:kf_upd}\\ P_{i}^{(t)}&=(I-K_{i}^{(t)}H)P_{i|t-1}.
(8) where, H is the mapping from the measurement space to (t) (t) state space. Pi is the state covariance, Ki is the Kalman Gain , and Q and R are the process and measurement noise covariances, respectively. We refer the interested reader to [6] for further details on the Kalman Filter.
von Neumann entropy of \rho is given as S(\rho ) \;=\; - \sum _{j=1}^C p_j \log p_j,
4. Methodology
(9)
Under the neural-collapse regime, the last-layer weight vectors align with class prototypes and span an almostorthogonal frame of class directions {uc }C c=1 , [14]. A client’s update decomposes along these directions, and the class–space energy allocated to direction uc is
4.1. Spectral Entropy as a Data-Free Indicator The first component of our method is based on the von Neumann (spectral) entropy. We present a brief sketch of the relation between spectral entropy and its potential in assessing client utility. Specifically, we turn to the phenomenon of Neural Collapse presented by [14]. In one of their propositions they show that the last-layer weights align with the class means and form a simplex ETF (See NC2 and NC3 in [14]). The authors of [24] further prove that Simplex ETF corresponds to the global minima for unconstrained feature models. The authors of [9] utilize Neural Collapse during optimization for faster convergence. In our setting, the model update from client i at round t is (t) given as ∆wi . We denote the weight matrix corresponding (t) (t) to the last layer, L, as the update matrix as Mi = ∆wi,L . Without loss of generality, we drop the superscript (t) and subscript i for further analysis. For the update, M ∈ RC×d , let A := M M ⊤ ∈ SC + be its class-space Gram matrix, where C is the number of classes. Let ρ := A/ Tr(A) be the density matrix P corresponding to A with eigenvalues {pj }C , giving j=1 j pj = 1 . Then the
\alpha _c \;:= \; u_c Au_c^\top = \; \big \| M^\top u_c \big \|_2^2 .
(10)
Since {uc }C ≈ c=1 are approximately orthogonal, A PC ⊤ c=1 αc uc uc and the eigenvalue spectrum of A is approximately {αc }C c=1 . Normalizing αc to sum to one, p_c \;=\; \frac {\alpha _c}{\sum _{k=1}^C \alpha _k}, \qquad S(\rho ) \;=\; -\sum _{c=1}^C p_c \log p_c, \label {eq:entropy_proof}
(11)
Therefore, the von Neumann entropy is approximately the Shannon entropy of the normalized class-space energies. It increases when the update’s energy spreads more evenly across class directions and decreases when it concentrates in a few directions. We demonstrate this phenomenon empirically in Fig. 2 by computing the entropy scores for clients with the same amount of data but a varying number of class labels. The entropy scores correlate positively with the standalone accuracies of the clients. 4
Algorithm 1 SpectralFed
column vectors of the final-layer update matrices, Mi , Mg , for j ∈ {1, . . . C}. The per class contribution (CSSV) Γ̂i,j of participant i is the cosine similarity between m̂i,j and m̂s,j which is aggregated over C classes to obtain a single (t) scalar Γi per client :
wg0 , local learning rate η, no.
Input: global weights: of communication rounds T , no. of local epochs E, momentum factor: µ, set of clients: K = {1, 2 . . . n} 1: Initialize client weights : c0i ← 1/n, ∀i ∈ K 2: for t = 1 . . . T do (t) (t) 3: Broadcast global model: wi ← wg 4: for all clients i ∈ K in parallel do 5: for e = 1 . . . E do 6: Sample zi ∼ Di (t) 7: Compute gradient: ∇Fi wi,e , zi (t)
\hat {\Gamma }_{i,j} = \cos \bigl (\hat {m}_{i,j},\hat {m}_{g,j}\bigr ) \quad ; \quad \Gamma _{i}^{(t)} = \frac {1}{C}\sum _{j=1}^{C}\hat {\Gamma }_{i,j}^{(t)} \label {eq:cssv}
4.3. Rank-Adaptive Kalman Filtering (RAKF) of Contribution Metrics
(t)
wi,e+1 ← SGD (wi,e , ∇Fi , η) end for 10: end for (t) 11: Compute Si using Equations 12, (1) (1) 12: S̃ = S S̃ (t) = µS̃ (t−1) + (1 − µ)S (t) (t) (t) (t) Pn 13: ci = S̃i / i=1 S̃i Pn (t) (t) (t+1) ← i=1 ci wi,E 14: Update global model wg 15: end for 8: 9:
In each communication round t, we obtain for every client i two complementary indicators of contribution quality: (t) • Final-layer gradient entropy Si , which measures the dispersion of the client’s final-layer update distribution. (t) • Class-specific Shapley alignment Γi , computed as the directional alignment between the client’s final-layer update and the global update. Since the reliability of these metrics varies with client heterogeneity, we introduce a Rank-Adaptive Kalman Filter to (t) produce a fused contribution score x̂i that automatically emphasizes the more informative metric at each round. We (t) adopt the filter per client whose latent state xi models the “true” (but unobserved) contribution. Following the standard notations from section 3, The process model is
In implementation, we use the WeightWatcher tool [11], which internally computes entropy as: S_{i}^{(t)} = S\bigl (\tilde A_i^{(t)}\bigr ) = -\sum _{j}\lambda _j \log \lambda _j. \label {eq:entropy}
(t)
where Ãi
(12)
\label {eq:proc_model} x_{i}^{(t)} = x_{i}^{(t-1)} + w_{i}^{(t)}, \qquad w_{i}^{(t)}\sim \mathcal {N}(0,Q),
(t)
=
Ai
(t)
∥Ai ∥F
(13)
; This yields entropy values equiva-
(14)
where Q is the process noise variance. For observations, we (t) first standardize the smoothed entropy estimates (S̃i ) and (t) smoothed CSSV (Γ̃i ) estimates to sum to one:
lent to those in Eqn. 11 up to a scaling factor. Utility implication. In federated aggregation, clients that possess a wider range of classes provide updates cover multiple class directions tend to reduce global loss more broadly and less redundantly. Neural Collapse theory shows that late-stage representations and classifier weights organize into near-orthogonal class directions. Because a client’s last-layer update decomposes along these directions, the von Neumann entropy measures how broadly its update energizes the class directions, providing a data-free proxy for utility in aggregation. We construct our first algorithm, S PECTRAL F ED, first applying a momentum term to the estimates and then normalizing these smoothed entropy estimates to obtain the final (t) client weights ci . The complete S PECTRAL F ED method is described in Algorithm 1.
\label {eq:normalize} s^{(t)}_{i}= \frac {\tilde {S}^{(t)}_{i}}{\sum _{i=1}^n \tilde {S}^{(t)}_{i}}; \quad \gamma ^{(t)}_{i}= \frac {\tilde {\Gamma }^{(t)}_{i}}{ \sum _{i=1}^n \tilde {\Gamma }^{(t)}_{i}}
(15)
At each round, we form the measurement vector \label {eq:meas_model} y_{i}^{(t)} = \begin {bmatrix} s_{i}^{(t)} \\[3pt] \gamma _{i}^{(t)} \end {bmatrix} = Hx_{i}^{(t)} + v_{i}^{(t)}, \quad v_{i}^{(t)}\sim \mathcal {N}(0,R_{i}^{(t)}), (16) where H = [1 1]⊤ . To make the filter rank-adaptive, we first compute the Spearman rank correlation between the predicted client estimates before the measurement update, (t) (t) x̂|t−1 = [x̂i|t−1 ]T , and each of the incoming observations (t)
(t)
s(t) = [si ]T and γ (t) = [γi ]T . We then map the observation noise covariance to inversely vary with these correlations, with ϵ as a small stability factor:
4.2. Complementary Class-specific Shapley Values While spectral entropy captures the spread of the updates well, it is less discriminative under scenarios where the class distribution is balanced but sample quantities differ as we show in section 6. To complement the entropy score we leverage Class-specific Shapley values (CSSV) proposed by the authors of ShapFed [17]. Let m̂i,j , m̂s,j denote the j-th
\label {eq:rupdate} R^{(t)}=\operatorname {diag} \bigl ( (1 - \rho _s^{(t)})+\epsilon ,\; (1 - \rho _\gamma ^{(t)})+\epsilon \bigr ) \forall i \in \mathcal {K} (17) The filter schematic is illustrated in Figure 3 and the algorithm is outlined in Appendix A. Intuitively, we seek to 5
Measurements: CSSV : Entropy :
Rank Correlation
Algorithm 2 SpectralFuse
Standard Kalman Filter Components
Adaptive measurement variance
Input: wg,0 , η, T , E, µ, K = {1, 2 . . . n} 1: Initialize client weights : c0i ← 1/n, ∀i ∈ K 2: Initialize filter ← Rank Adaptive Kalman Filter() 3: for t = 1 . . . T do (t) (t) 4: Broadcast global model: wi ← wg ∀i ∈ K (t) (t) 5: wi,E , Ŝi ← Run Steps 4-12 of SpectralFed 1
Application specific Components
Predict
Update
(t)
(t)
Compute si , γi from Equation 15 (t) (t) (t) 7: x̂i ← filter(γi , si ) (t) (t) (t) Pn 8: Normalize: ci = x̂i / i=1 x̂i Pn (t) (t) (t+1) 9: Update global model wg ← i=1 ci wi,E 10: end for 6:
Aggregation Weights
State vector:
Figure 3. Schematic of the Rank Adaptive Kalman Filter used to fuse the CSSV estimates with the entropy estimates to get the final aggregation weights
tion round. The CIFAR experiments were run for 200 rounds, FEMNIST for 100 rounds, and FedISIC for 300 rounds. Unless otherwise stated, we used stochastic gradient descent without momentum, initial learning rate 0.1 for CIFAR-10/100 and FEMNIST and 0.001 for FedISIC, and cosine annealing scheduler with a final learning rate of 10−6 . The loss function used was cross-entropy loss. We use a batch size of 64 for all settings. For the Kalman Filter, we set the process noise variance Q = 10−4 and the measurement noise stability factor ϵ = 10−3 . An ablation study on the sensitivity of Q and ϵ is shown in Appendix. The momentum factor for smoothing the entropy and CSSV estimates was set to µ = 0.9 similar to [17]. Data Splits. We considered five types of non-IID splits for CIFAR-10 and CIFAR-100 along with the standard IID split: (i) Only Label Skew: imbalanced labels with equal data points, (ii) Step Quantity Skew: progressively increasing data quantity with balanced labels, (iii) Step Label Skew: progressively increasing label variety and data quantity, Dirichlet splits with (iv) α = 0.1, and (v) α = 0.01. The FEMNIST and FedISIC datasets use the naturally provided client split. Entropy and CSSV Computation. In each communication round, we compute the von Neumann entropy Si of the spectral distribution of the final-layer gradient update for each client i. This is done using the WeightWatcher tool [11]. We also compute the class-specific Shapley value vector Γi using the approach of [17]. Baselines. We compare our methods against the following baselines: FedAvg [12], CGSV [21], and ShapFed [17]. For FedAvg, we consider the variant with uniform weighting (FedAvg-uniform) for a fair comparison with data-free approaches and no self-reported metadata. Both CGSV and ShapFed are data-free methods that do not require auxiliary validation data, but provide client rewards commensurate to their contribution estimates. Since we focus mainly on the contribution estimates, we disable the client reward mecha-
penalize observations that frequently change the ordering of the clients’ contribution scores. If, say, the client rankings obtained from the entropy estimates varies more frequently than the CSSV estimates, it is declared less reliable and should contribute less towards the final clients’ contribution estimates.
4.4. Putting It All Together: SpectralFuse Our complete pipeline is illustrated in Fig. 1. Each client update is first processed to compute the spectral entropy (t) (t) Si and the Class-specific Shapley value Γi . The S PEC TRAL F ED variant uses only the entropy to set aggregation weights, which is simple and effective in label-skew scenar(t) (t) ios. The S PECTRAL F USE variant feeds both Si and Γi into the rank-adaptive Kalman Filter to produce a smoothed and reliable contribution estimate. This estimate is then used to weight client updates in the global aggregation step. Our methodology is entirely server-side and data-free. It does not require access to any validation data, nor to selfreported client statistics, and it is robust to heterogeneous client behavior.
5. Implementation Details Datasets and Models. For CIFAR-10, we use a randomly initialized 5-layer CNN model , while for the CIFAR100 experiments we use a randomly initialized ResNet-50 model. For FEMNIST we use a 4-layer MLP and for the FedISIC dataset, we used the ImageNet-1k pretrained ViT-B/16 model. Hyperparameters. We simulated 5 clients for CIFAR10/100 and 6 clients for FedISIC. We subsample 100 clients for the FEMNIST dataset to prevent memory overruns. Each client performed one local epoch per communica6
Table 1. Average Pearson Correlation of client weights ci with standalone final accuracies of methods across datasets and splits. The best results are shown in bold and the second best results are underlined.
Dataset
Split
FedAvg
CGSV
ShapFed
SpectralFed
SpectralFuse
CIFAR-10
Only Label Skew Step Label Skew Step Quantity Dirichlet (α = 0.1 ) Dirichlet (α = 0.01)
-0.01 ± 0.02 -0.01 ± 0.02 0.03 ± 0.04 -0.01 ± 0.02 -0.00 ± 0.02
0.88 ± 0.03 0.95 ± 0.02 0.92 ± 0.06 0.74 ± 0.19 0.88 ± 0.04
0.89 ± 0.02 0.93 ± 0.05 0.91 ± 0.03 0.58 ± 0.53 0.94 ± 0.03
0.95 ± 0.02 0.96 ± 0.01 0.81 ± 0.02 0.72 ± 0.20 0.97 ± 0.03
0.97 ± 0.01 0.97 ± 0.01 0.95 ± 0.01 0.80 ± 0.14 0.97 ± 0.03
CIFAR-100
Only Label Skew Step Label Skew Step Quantity Dirichlet (α = 0.1 ) Dirichlet (α = 0.01)
0.02 ± 0.01 0.02 ± 0.02 -0.01 ± 0.02 0.02 ± 0.03 -0.02 ± 0.06
0.38 ± 0.33 0.56 ± 0.11 0.15 ± 0.17 -0.07 ± 0.24 0.51 ± 0.03
0.81 ± 0.13 0.87 ± 0.08 0.92 ± 0.03 0.08 ± 0.61 0.79 ± 0.22
0.90 ± 0.14 0.90 ± 0.04 0.92 ± 0.02 0.68 ± 0.15 0.94 ± 0.01
0.88 ± 0.15 0.94 ± 0.02 0.95 ± 0.01 0.74 ± 0.05 0.95 ± 0.03
FEMNIST FedISIC
Natural split Natural split
-0.00 ± 0.01 -0.01 ± 0.01
0.12 ± 0.03 0.88 ± 0.01
0.41 ± 0.03 0.92 ± 0.01
0.35 ± 0.02 0.82 ± 0.01
0.40 ± 0.01 0.86 ± 0.01
nisms in CGSV and ShapFed for a fair comparison. Evaluation Protocol. We mainly report the Pearson correlation between the client weights ci and the standalone accuracies of each client. The standalone accuracies are obtained when each client independently trains on its own data and evaluates on a held-out test set. We also report the global model accuracy obtained at the end of training for each method and each split. All reported metrics are reported as mean and standard deviation over 3 seeds for CIFAR100, FEMNIST, and FedISIC and 5 for CIFAR10.
1.0
(a) CIFAR-10, Dirichlet ( =0.01)
(b) CIFAR-100, Step Label Skew
Correlation
0.9 0.8 0.7
CGSV ShapFed SpectralFed SpectralFuse
0.6 0.5
0
50
100
Rounds
150
200
0
50
100
Rounds
150
200
Figure 4. SpectralFed (green) shows consistent Pearson correlation with the client standalone accuracies across rounds. SpectralFuse (red) benefits from fusing both CSSV and Entropy signals and improves correlation by resisting frequent rank changes.
6. Results The results of our experiments are summarized in Tables 1 and 2. Table 1 examines how well each contribution estimation method correlates with standalone client accuracy. CGSV and ShapFed attain strong correlations in several settings but are inconsistent: CGSV performs well on quantityskew settings on CIFAR-10 yet collapses under more heterogeneous CIFAR-100 splits, while ShapFed performs relatively better under label skew settings. In contrast, our entropy-based methods (SpectralFed and SpectralFuse) provide consistently high correlations. On CIFAR-10, SpectralFuse remains above 0.95 across nearly all non-IID splits, with SpectralFed close behind. A similar pattern holds on CIFAR-100, where the entropy-based methods clearly outperform the baselines in the heterogeneous regimes, maintaining substantial positive correlation. Even on the naturally partitioned FEMNIST and FedISIC datasets, SpectralFed and SpectralFuse remain competitive, indicating that the entropy signal transfers to real-world non-IID distributions. Furthermore, RAKF allows SpectralFuse to maintain sustained high correlations across all settings. To illustrate the consistency of the entropy-derived weights, we
show the per-round Pearson correlation for the proposed methods and baselines in Figure 4. Further analysis on the operating principle of the filter is provided in the Appendix. Table 2 compares the predictive performance of the global model under each aggregation scheme. In the IID condition, all methods perform reasonably well. Under non-IID splits, however, the advantage of entropy-based weighting is more evident. On CIFAR-10, SpectralFed and SpectralFuse perform well in the Only Label Skew setting, while maintaining parity in Step Quantity and Step Label Skew. Under the more challenging Dirichlet partitions, entropy weighting remains comparable to or slightly better than the baselines. A similar trend appears on CIFAR-100. On FEMNIST and FedISIC, the benefits are more pronounced. SpectralFed achieves a balanced test accuracy of 55.59%, exceeding FedAvg by over four points and outperforming ShapFed. These findings indicate that the data-free entropy signal not only aligns with client quality but can also trans7
Table 2. Predictive performance of the methods across datasets and splits. * CGSV does not compute the global model, so we report the average client performance. †We report the balanced accuracy for FedISIC to account for the label imbalance in its test set. The best results are shown in bold and the second best results are underlined.
Dataset
Split
FedAvg
CGSV*
ShapFed
SpectralFed
SpectralFuse
CIFAR-10
IID Only Label Skew Step Label Skew Step Quantity Dirichlet (α = 0.1 ) Dirichlet (α = 0.01)
82.45 ± 0.11 67.97 ± 1.71 81.95 ± 0.24 82.66 ± 0.23 76.42 ± 1.26 67.14 ± 2.20
75.48 ± 0.27 36.72 ± 0.81 55.75 ± 1.06 75.16 ± 0.34 36.49 ± 2.16 21.73 ± 0.85
81.99 ± 0.19 67.81 ± 0.79 81.73 ± 0.34 82.12 ± 0.29 75.72 ± 1.17 58.14 ± 3.22
82.27 ± 0.09 69.07 ± 1.42 82.35 ± 0.28 82.81 ± 0.20 75.76 ± 1.13 65.21 ± 2.35
82.25 ± 0.18 69.43 ± 0.84 82.15 ± 0.23 82.75 ± 0.15 76.24 ± 0.89 66.18 ± 2.38
CIFAR-100
IID Only Label Skew Step Label Skew Step Quantity Dirichlet (α = 0.1 ) Dirichlet (α = 0.01)
43.19 ± 1.22 34.67 ± 0.38 43.90 ± 0.27 42.66 ± 1.11 39.01 ± 0.45 33.57 ± 1.73
25.69 ± 0.46 15.66 ± 0.15 21.53 ± 0.48 25.44 ± 0.43 16.04 ± 0.36 11.89 ± 0.90
40.39 ± 0.26 35.78 ± 0.45 41.65 ± 0.55 41.83 ± 1.15 39.42 ± 1.06 36.33 ± 0.16
41.57 ± 1.42 35.62 ± 0.84 43.88 ± 0.13 42.97 ± 0.37 39.20 ± 1.33 35.99 ± 0.59
42.70 ± 2.38 34.28 ± 1.63 43.63 ± 1.03 42.69 ± 3.16 40.45 ± 1.71 34.87 ± 0.46
FEMNIST FedISIC†
Natural split Natural split
60.64 ± 0.26 51.26 ± 0.98
29.98 ± 0.22 37.23 ± 0.22
60.82 ± 0.22 51.58 ± 1.92
61.17 ± 0.27 55.59 ± 0.67
61.06 ± 0.23 55.34 ± 0.48
late into tangible improvements in the aggregated model’s accuracy. However, the absolute gains are often modest, suggesting that while finding good contribution estimates may help with fairness and incentives distribution, it may not necessarily translate to superior global model performance. We discuss this further in the next section. Across CIFAR-10/100, FEMNIST, and FedISIC, entropyderived weights strongly correlate with standalone client performance and yield improvements in global accuracy over uniform weighting. The Kalman Filter fuses the signals from CSSV and entropy to provide a smooth and noise robust estimate. The modular nature of the filter allows its extension to potentially work with multiple input signals. The evidence supports that von Neumann entropy in conjunction with alignment-based methods, can serve as a reliable proxy for client contributions.
On the other hand, similarity/alignment based approaches benefit in scenarios with diverse client quantities, as the best clients produce bigger updates and dominate the aggregate gradient. However, under label-diversity, they may struggle to account for the full range of client contributions. Interpreting the accuracy gains. The accuracy improvements we observe are typically modest relative to FedAvg on CIFAR-10/100, and a bit larger on FEMNIST and FedISIC. We hypothesize that two factors help explain the modest gains: (i) since we supply the full global model to all the clients, all clients eventually train on the same model and any gains obtained from commensurate weight assignment eventually taper off as the training ends. (ii) Our method adjusts aggregation weights but leaves local training unchanged, which may be the primary bottleneck for the global model performance. We suspect that the combination of these factors leads to the observed improvements. Further research on these hypotheses may help validate them.
7. Discussion Why does spectral entropy work? The final layer encodes the task’s class geometry and aligns with the mean class directions under the neural-collapse regime. Clients whose data are informative for decision boundaries tend to induce weight updates with richer spectral profiles. This helps in non-IID regimes where valuable clients may contribute updates that deviate from the cohort average (e.g., minority labels or rare features). Unlike similarity based approaches that reward conformity to the mean gradient, the entropy view can value these informative, diverse updates without access to any data or labels. The strong empirical correlation with standalone accuracy supports this interpretation.
Future directions. A practical benefit of data-free contribution score is that it can drive incentive mechanisms without auxiliary validation sets. Enhanced aggregation schemes that can better utilize the contribution estimates to improve global model performance are a potential avenue for future exploration. In summary, entropy-based contribution estimation provides a simple, privacy-preserving, and effective signal for weighting clients in federated learning. Combined with alignment-based approaches using filtering, it can help ensure that client contributions are accurately reflected in the training process. 8
Acknowledgements
gold, Tanguy Marchand, Othmane Marfoq, Erum Mushtaq, Boris Muzellec, Constantin Philippenko, Santiago Silva, Maria Teleńczuk, Shadi Albarqouni, Salman Avestimehr, Aurélien Bellet, Aymeric Dieuleveut, Martin Jaggi, Sai Praneeth Karimireddy, Marco Lorenzi, Giovanni Neglia, Marc Tommasi, and Mathieu Andreux. FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings. Advances in Neural Information Processing Systems, 35:5315–5334, 2022. 2 [14] Vardan Papyan, X. Y. Han, and David L. Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40):24652–24663, 2020. 4 [15] Asher Peres. Quantum theory: concepts and methods. Springer, 2002. 1 [16] Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and Hugh Brendan McMahan. Adaptive Federated Optimization. In International Conference on Learning Representations, 2021. 3 [17] Nurbek Tastan, Samar Fares, Toluwani Aremu, Samuel Horváth, and Karthik Nandakumar. Redefining Contributions: Shapley-Driven Federated Learning. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI-24, pages 5009–5017. International Joint Conferences on Artificial Intelligence Organization, 2024. Main Track. 1, 2, 3, 5, 6 [18] Nurbek Tastan, Samuel Horváth, and Karthik Nandakumar. Aequa: Fair Model Rewards in Collaborative Learning via Slimmable Networks. In Proceedings of the 42nd International Conference on Machine Learning, pages 59210– 59236. PMLR, 2025. [19] Nurbek Tastan, Samuel Horváth, and Karthik Nandakumar. CYCle: Choosing Your Collaborators Wisely to Enhance Collaborative Fairness in Decentralized Learning. Transactions on Machine Learning Research, 2025. 2 [20] Nannan Wu, Zengqiang Yan, Nong Sang, Li Yu, and Chang Wen Chen. Fedpca: Noise-robust fair federated learning via performance-capacity analysis. arXiv preprint arXiv:2503.10567, 2025. 3 [21] Xinyi Xu, Lingjuan Lyu, Xingjun Ma, Chenglin Miao, Chuan Sheng Foo, and Bryan Kian Hsiang Low. Gradient Driven Rewards to Guarantee Fairness in Collaborative Machine Learning. In Advances in Neural Information Processing Systems, pages 16104–16117. Curran Associates, Inc., 2021. 1, 2, 3, 6 [22] Yunlu Yan, Lei Zhu, Yuexiang Li, Xinxing Xu, Rick Siow Mong Goh, Yong Liu, Salman Khan, and Chun-Mei Feng. A new perspective to boost performance fairness for medical federated learning. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 13–23. Springer, 2024. 3 [23] Yaoqing Yang, Ryan Theisen, Liam Hodgkinson, Joseph E. Gonzalez, Kannan Ramchandran, Charles H. Martin, and Michael W. Mahoney. Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data, 2023. arXiv:2202.02842 [cs]. 3
This material is partly based on work supported by the Office of Naval Research N00014-24-1-2168.
References [1] Michele Benzi, Michele Rinelli, and Igor Simunec. Computation of the von neumann entropy of large matrices via trace estimators and rational krylov methods. Numerische Mathematik, 155(3):377–414, 2023. 1, 3 [2] Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. Leaf: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018. 2 [3] Meirui Jiang, Holger R Roth, Wenqi Li, Dong Yang, Can Zhao, Vishwesh Nath, Daguang Xu, Qi Dou, and Ziyue Xu. Fair federated medical image segmentation via client contribution estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16302–16311, 2023. 1, 2 [4] Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. Fantastic generalization measures and where to find them. In International Conference on Learning Representations, 2020. 3 [5] Peter Kairouz and H Brendan McMahan. Advances and open problems in federated learning. Foundations and trends in machine learning, 14(1-2):1–210, 2021. 1 [6] Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME–Journal of Basic Engineering, 82(Series D):35–45, 1960. 1, 4 [7] Wenqian Li, Shuran Fu, Fengrui Zhang, and Yan Pang. Data valuation and detections in federated learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12027–12036, 2024. 3 [8] Lingjuan Lyu, Xinyi Xu, Qian Wang, and Han Yu. Collaborative fairness in federated learning. In Federated Learning: Privacy and Incentive, pages 189–204. Springer, 2020. 1, 2 [9] Evan Markou, Thalaiyasingam Ajanthan, and Stephen Gould. Guiding neural collapse: Optimising towards the nearest simplex equiangular tight frame. Advances in Neural Information Processing Systems, 37:35544–35573, 2024. 4 [10] Charles H Martin and Michael W Mahoney. Implicit selfregularization in deep neural networks: Evidence from random matrix theory and implications for learning. Journal of Machine Learning Research, 22(165):1–73, 2021. 3 [11] Charles H. Martin, Tongsu, Peng, and Michael W. Mahoney. Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data. Nature Communications, 12(1):4122, 2021. 3, 5, 6 [12] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. CommunicationEfficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, pages 1273–1282. PMLR, 2017. 1, 2, 3, 6 [13] Jean Ogier du Terrail, Samy-Safwan Ayed, Edwige Cyffers, Felix Grimberg, Chaoyang He, Regis Loeb, Paul Man-
9
[24] Zhihui Zhu, Tianyu Ding, Jinxin Zhou, Xiao Li, Chong You, Jeremias Sulam, and Qing Qu. A Geometric Analysis of Neural Collapse with Unconstrained Features. In Advances in Neural Information Processing Systems, pages 29820– 29834. Curran Associates, Inc., 2021. 4
10
Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy Supplementary Material A. Rank Adaptive Kalman Filter Algorithm
1.0
(a) True Pearson Correlation
1.0
Algorithm 3 Rank Adaptive Kalman Filter Initialize: x0i , P 0 , Q, ϵ (t−1) (t) (t) Input: x̂i , si , γi (t) 1: Predict: x̂i|t−1 using Equation 5 = Spearman (x̂|t−1 , s(t) )
(t)
= Spearman (x̂|t−1 , γ (t) )
0.6
(t)
(t)
(t)
(t)
(t)
= (si , γi )⊤ (t) 6: Update Ri from Equation 17 (t) 7: Compute: x̂i using Equation 7 (t) 8: return x̂i
5: yi
0
50
100
1.00 0.75 0.50 0.25 0.00
150
0.6
200
(cssv)
s(entropy)
0
50
100
Rounds
0.8 CSSV Entropy Fused
0.7
(c) Estimated Spearman Correlation
Correlation
4: ργ
CSSV Entropy Fused
150
200
0
50
100
150
200
(d) Estimated Measurement Variance Variance
(t)
0.8 0.7
2: Update: 3: ρs
0.9
Correlation
Correlation
0.9
(b) True Spearman Correlation
1.00 0.75 0.50 0.25 0.00
2(cssv) 2 s (entropy)
0
50
100
Rounds
150
200
Figure 5. The RAKF’s working principle demonstrated for a run on the Dirichlet (α = 0.1) split of CIFAR-10. (a) The RAKF fused weights smoothen and track the true Pearson correlation trend of the perceived reliable signal. (see ‘Fused’ vs. ‘CSSV’) (b) Fused weights resist rank changes and therefore maintain a stable Spearman Rank correlation profile over rounds. (c) The filter estimates the rank correlation between its previous state and each incoming signal. (d) The signal that has a higher estimated rank correlation has lower variance and is considered more trustworthy.
We provide an extended analysis of the RAKF proposed in section 4.3 here. The filter fuses two complementary, data-free signals :(i) the spectral (von Neumann) entropy of the final-layer update and (ii) the class-specific Shapley alignment (CSSV), into a single per-client contribution estimate that is stable over rounds yet responsive to persistent changes. Algorithm 3 complements Figure 3 to illustrate the filter methodology.
B. Filter hyperparameter sensitivity To assess the impact of filter hyperparameters Q and ϵ, on weight correlations with standalone accuracies, we provide a brief analysis of their sensitivity. Process noise covariance Q. In Figure 6, we tested the impact of varying the process noise covariance Q on correlation performance over rounds. In order to interpret the results, we revisit the definition of Q given in Equation 14. Q is the estimate of the covariance of the normally distributed process noise in the state model. In a sense, it is a prior estimate of the fidelity of the process model that the filter is operating under. The better a process is modeled, the lower the value of Q should be. In our case, the process model is (t) (t−1) (t) (t) given as xi = xi + wi , where xi is the estimate of the weights of the client i in round t. This implies that the client weights should remain static. In principle, this is a sound model as the clients’ utility is expected to remain constant under the assumption that no client adds any additional data during the training process. However, given that the filter initializes under a uniform weight for each client, the client state should be allowed to change based on external heuristics. The choice of process noise covariance Q
We demonstrate the filter operation process in Figure 5 by analyzing the internal values of the filter. In Figure 5 (a), the filter initially tracks the correlation trajectory of the entropy and later transitions to CSSV. Figures 5 (b-c) illustrate this: we observe that while the entropy-based rankings are initially consistent, they exhibit greater variation between rounds 50-150. This, in turn, reduces its variance (Figure 5 (d)), and therefore the fused weights are more dependent on the CSSV signal. An key caveat is that the filter does not have access to ground-truth correlations, as the client utility (standalone accuracy) is unknown to the server. Therefore, it may trust a signal as long as it is consistent, even if it may have a lower actual correlation. This is evident in Figure 5 (a), where the signal with lower correlation (CSSV) is more trusted due to its consistency despite having a lower true correlation score. Together, these plots illustrate the intended behavior of the filter, which initially reduces rank volatility but adapts to the signals when a sustained regime change occurs. 1
correlation
0.98
0.92
Q=10 6
0.88 0.86 0 1.00
50
100
Q=10 6
0.97
Q=10 5 Q=10 4 Q=10 3 Q=10 2
0.90
Q=10 5 Q=10 4 Q=10 3 Q=10 2
0.96 150
Split: Quantity Skew
200
0.95
0
1.00
0.98
50
100
150
Split: Dirichlet, = 0.01
0.94 Q=10 6 Q=10 5 Q=10 4 Q=10 3 Q=10 2
0.90 0.88 0.86 0
50
100
Rounds
200
0 1.00
0.95
50
100
Q=10 6 Q=10 5 Q=10 4 Q=10 3 Q=10 2 0
50
0.96 150
Split: Quantity Skew
200
150
=10 5 =10 4 =10 3 =10 2 =10 1
0.90
0
Figure 6. Effect of choice of filter process noise covariance Q on the per-round Pearson correlation with standalone accuracies for different splits of the CIFAR-10 dataset. Lower values of Q (see Q = 10−6 ) yield smoother but stagnant curves as the filter ‘trusts’ its internal process model of persistent client ranks more.
50
100
150
200
100
150
200
Split: Dirichlet, = 0.01
0.98
0.92
200
0
1.00
0.94
0.86 100
0.95
0.99
0.88
Rounds
=10 5 =10 4 =10 3 =10 2 =10 1
0.97
0.96
0.96 200
=10 5 =10 4 =10 3 =10 2 =10 1
0.90
0.98
0.97
150
0.98
0.92
0.86
0.98
0.92
0.94
0.88
0.99
0.96
0.99
0.96
correlation
correlation
0.94
Split: Step Label Skew
1.00
0.98
0.99
0.96
Split: Only Label Skew
1.00
50
100
Rounds
=10 5 =10 4 =10 3 =10 2 =10 1
0.97 0.96 150
200
0.95
0
50
Rounds
Figure 7. Effect of choice of measurement variance floor ϵ on the per-round Pearson correlation with standalone accuracies for different splits of the CIFAR-10 dataset. Higher values of ϵ raise the variance floor for the measurements (entropy and CSSV) and cap the filter’s confidence in the measurement signals. The fused weights do not respond to rapid changes in the signals, resulting in a smoother correlation curve (see ϵ = 10−2 )
governs how fast the state should respond to external signals when they differ from the internal state. We observe this phenomenon in practice in Figure 6. We observe that lower values of Q smooth the correlation curves and avoid the noise in the external signals (entropy and CSSV). Although this is a desirable behavior if the initial correlation is high (as in the Only Label Skew and Quantity Skew cases), it can be problematic when early correlations are poor and improve in later rounds (as in the Step Label Skew case). An intermediate value of Q provides a good balance between these behaviors. Across most splits, a broad middle band of values yields a similar performance, indicating low sensitivity in practice.
Q
10 2 10 3 10 4 10 5 10 6
Only Label Skew 0.94 0.95 0.95 0.96 0.97 0.94 0.94 0.95 0.96 0.96 0.94 0.94 0.94 0.95 0.96 0.94 0.94 0.94 0.94 0.95 0.93 0.94 0.94 0.94 0.95
Q
10 2 10 3 10 4 10 5 10 6
Quantity Skew
Measurement variance floor ϵ. Similarly to the study of the sensitivity of the process noise covariance, we study the impact of the choice of ϵ. To recap, the measurement variance floor provides a lower bound to the measurement noise covariance matrix R as described in Equation 17. The measurement noise covariance acts as a counterweight to the process noise covariance Q. In a typical Kalman Filter design, it provides a fixed prior on the covariance of external measurements. In the case of RAKF, R(t) is time-dependent and varies inversely with the rank correlation of the measurement signals (Equation 17). Intuitively, a higher value of R implies a relatively lower confidence in the external measurements as compared to the process model. Higher ϵ
0.94 0.95 0.95 0.96 0.97 0.94 0.94 0.95 0.96 0.96 0.94 0.94 0.94 0.95 0.96 0.94 0.94 0.94 0.94 0.95 0.94 0.94 0.94 0.94 0.95 10 5 10 4 10 3 10 2 10 1
1.00 0.98 0.96 0.94 0.92 0.90 1.00 0.98 0.96 0.94 0.92 0.90
Step Label Skew 0.98 0.98 0.99 0.99 0.99 0.98 0.98 0.99 0.99 0.99 0.98 0.98 0.98 0.99 0.99 0.98 0.98 0.98 0.99 0.99 0.98 0.98 0.98 0.99 0.99
Dirichlet, = 0.01 1
1
1
1
1
0.99 0.99 0.99 1
1
0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 0.99 10 5 10 4 10 3 10 2 10 1
1.00
Avg. Pearson Correlation
0.98
correlation
Split: Step Label Skew
1.00
0.98 0.96 0.94 0.92 0.90 1.00
Avg. Pearson Correlation
Split: Only Label Skew
1.00
0.98 0.96 0.94 0.92 0.90
Figure 8. Sensitivity of choice of Q and ϵ on the average Pearson correlation of the fused weights and standalone accuracies for different splits on the CIFAR-10 dataset. Average values of correlation are robust with respect to changes in filter hyperparameters.
results in a generally higher value of R. Figure 7 captures the effect of changing ϵ on the true Pearson correlation over rounds for different splits of the CIFAR-10 dataset. The em2
0
0.02
0
0.04
1
0
0
0
0
0.1
0
0.26
0
0
0.1 0.32
0
0
0
0
0
0.08
0
0
0
0
1
0
0
0.02
0
0
0.32
0
0
0
0
1
1
2
3
4
5
1
2
3
4
5
1 0.061 0
0
0
3
0
0.54
0.76
0
0
0
0
0
1
0
0
0
0
0
1
1
2
3
4
5
True Free Rider
C. Free-rider detection
Detected Client
True Free Rider
0
0.24 0.061 0
0
0
0.61 0.082 0
0
0
0
0
0.16
0
0
0
0
0
0.86
1
2
3
4
5
1
0
2
1
0
3
2
0
0
4
0
True Free Rider
0
5
0
1
0.34
4
Rounds 151 - 199
1
5
Rounds 101 - 150
Detected Client
Detected Client
0.8
Detection Rate
0.56
1.0
0.6 0.4 0.2 0.0
1.0 0.8
Detection Rate
0
1
0
2
2
0
3
0
3
0
4
0
4
0
Detected Client
1
5
0
1
0
True Free Rider
Although the choice of Q and ϵ affects the behavior and performance of the RAKF, the variation observed in the correlation values when aggregated over rounds remains negligible. We capture this in Figure 8, which shows the variation in the average correlation values as a function of (Q, ϵ). Correlations remain high over wide regions of the grid for all splits, with very small variations for all splits. Practically, one can pick Q to target desired smoothness and set a modest ϵ to avoid over-trusting rapidly fluctuating signals. We therefore show that the reported results are reasonably robust to the choice of filter hyperparameters.
Rounds 51 - 100
0.22
5
Rounds 1 - 50
pirical results validate this hypothesis, as higher values of ϵ yield a delayed, more sluggish response to entropy/CSSV.
0.6 0.4 0.2 0.0
(a) SpectralFed
0
0
0
0
0.06
0
0
0
0
0.36
0
0
0.02
0
0
0.16
1
2
3
4
5
1
1
0
0
0
0
2
0
1
0
0
0
3
0
1
0
0
0
0
0
0.12
0
0
0
0
0
1
0
0
0
0
0
1
2
3
4
5
0
0 0.061 0
0
0
0
0
0.27
0
0
0
0
0
1
2
3
4
5
0
2
0
1
0
0
0
3
0
0
1
0
0
0
0
0
1
0
0
0
0
0
1
1
2
3
4
5
Detected Client
True Free Rider
True Free Rider
1
0
True Free Rider
0
1
0
4
Rounds 151 - 199
1
5
Rounds 101 - 150
Detected Client
1
Detected Client
1.0 0.8
Detection Rate
0
4
0
1
0
5
0
2
0.04
3
0
3
2
0.12
True Free Rider
0
4
0
Detected Client
1
5
0
1
0
4
Rounds 51 - 100
0
5
Rounds 1 - 50 0.1
0.6 0.4 0.2 0.0
1.0 0.8
Detection Rate
Contribution estimation schemes should not only be effective in identifying client utility under data heterogeneity but also should be able to detect scenarios where there are noncontributing clients or free-riders. A free-rider could be modeled in several ways. In the trivial scenario, it could be a client that does not perform training but participates only to obtain the global model or participation rewards. At the other extreme, a free rider could be malicious and attempt to actively derail the FL process. The first scenario can be easily identified by a simple client gradient norm check. The latter model requires further assumptions about a white-box, grey-box, or black-box knowledge setting and is better suited for treatment under the robust FL paradigm. We simulate a non-adversarial free-rider that participates by duplicating a very small amount of data to match the average cohort data size. Any method that correlates with the amount of data only would not be able to distinguish this free-rider.
0.6 0.4 0.2 0.0
(b) SpectralFuse
In order to detect the free-rider from the contribution estimates, we first transform the estimates to the real space (contribution estimates are compositional as they have a fixed sum, one, and thus lie on a simplex). We then compute the robust Z-score zi = (xi − median(xi )/MAD) where MAD is the Median Absolute Deviation. Free-riders are clients that are flagged as outliers in the Z-score. We run 5 experiments on the Dirichlet-partitioned (α = 1.0) CIFAR10 dataset for both SpectralFed and SpectralFuse. One out of the 5 clients is the free rider in each experiment. We show the fraction of detections for each client in four 50-round intervals in Figure 9. For both SpectralFed (Fig. 9a) and SpectralFuse (Fig. 9b), free-riders are detected with high accuracy towards the middle and later phases of training. We observe lower earlyphase detection with occasional off-diagonal false positives, likely because the weights are not yet well-separated. These results indicate that simple thresholding of our data-free
Figure 9. Free-rider detection accuracy of the proposed methods at four different intervals of training. Each row denotes an independent federated training procedure, with the y-axis indicating which client is the free rider. The values in the grid represent the rate of detecting the client, denoted on the x-axis as the free rider. High values on the diagonal indicate correct detections, while offdiagonal values indicate false positives.
scores is effective for free-rider screening, without the need for auxiliary validation data or self-reported metadata.
D. Choice of final layer for entropy evaluation In this section, we motivate the choice of using the final layer of the gradients for the entropy computation with empirical evidence. Figure 10 shows the average Pearson correlation between entropy and standalone accuracy as a function of layer depth for a 5-layer CNN on CIFAR-10 and 3
Pearson Correlation
CIFAR-10 w. 5-layer CNN
Only Label Skew
1.0
Dirichlet, = 0.1
Quantity Skew
Dirichlet, = 0.01
0.8 0.6 0.4 0.2 0
1
2
3
4
0
1
2
3
4
0
1
2
3
4
0
1
2
3
4
0
1
2
3
4
1.0
Pearson Correlation
CIFAR-100 w. ResNet-50
Step Label Skew
0.8 0.6 0.4 0.2 0.0 0 10 20 30 40 50
Layer Index
0 10 20 30 40 50
Layer Index
0 10 20 30 40 50
Layer Index
0 10 20 30 40 50
Layer Index
0 10 20 30 40 50
Layer Index
Figure 10. Average Pearson correlation of the entropy of the gradient for each layer with the standalone accuracy. The top row displays the average correlation for the 5-layer CNN used on the CIFAR-10 dataset, with each entry corresponding to a specific split. The x-axis denotes the layer index. The bottom row shows a similar analysis for the ResNet-50 model used on the CIFAR-100 dataset. We observe a general trend of increasing correlation as we move deeper in the network, with the final layer usually showing the highest correlation.
and evaluate the entropy from the normalized eigenvalues in O(C) time. Therefore, the total time complexity of entropy computation over all participating clients is
ResNet-50 on CIFAR-100. We observe a generally increasing correlation as we move deeper in the model with the highest correlations observed in the final layer. This provides strong empirical grounds for choosing the final layer only, rather than, e.g., averaging across layers Furthermore, these results align with the theoretical basis for the role of the classifier head (final layer) in encoding the class geometry, as discussed in subsection 4.1. When a client’s data is uniform and diverse in labels, the final layer update exhibits a richer and more balanced spectrum (higher entropy), which better tracks client utility. The choice of using the last layer for entropy computation motivates the use of Class-wise Shapley Values (CSSV) as an auxiliary signal for fusion. As discussed in subsection 4.2, CSSV is also computed on the last layer. Therefore, both the metrics used as inputs for fusion are derived from the same sub-part of the model, albeit with different approaches. Additionally, computing entropy on the last layer is the most efficient, as the Gram matrix A ∈ RC×C (see subsection 4.1) and scales only with the number of classes.
O\!\left (n\,d\,C^2 + n\,C^3\right ).
(18)
Because these computations are performed independently per client, they are embarrassingly parallel across clients. The additional cost of S PECTRAL F USE relative to S PEC TRAL F ED comes from computing CSSV and applying the Kalman filter. We report the per-round server-side wall-clock time for the baselines and our methods in Table 3. The reported values are mean ± standard deviation in milliseconds over 10 rounds with all methods evaluated under identical CPU/GPU settings. Table 3. Per-round server-side wall-clock time (ms) for running contribution estimation and aggregation.
E. Compute Complexity For n participating clients, let the final-layer update of each client be Mi ∈ Rd×C , where d is the feature dimension and C is the number of classes, with typically d > C. For each client, we first form the class-space Gram matrix, Ai = Mi⊤ Mi ∈ RC×C , which costs O(d C 2 ). We then compute the eigenspectrum of Ai using SVD, which costs O(C 3 ),
Dataset
CGSV
ShapFed
SpectralFed
SpectralFuse
CIFAR-100 FedISIC
50 ± 2 172 ± 6
467 ± 31 398 ± 49
207 ± 2 53 ± 10
378 ± 12 102 ± 5
Table 3 shows that S PECTRAL F ED is substantially cheaper than ShapFed in both settings. The entropy-based method are the fastest on FedISIC as they scale only with the number of classes. In contrast, the time complexity of CGSV scales with the full model size resulting in slower runtimes for larger models. 4