FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning Radwan Selo, Majid Kundroo and Taehong Kim∗

arXiv:2608.09221v1 [cs.LG] 10 Aug 2026

School of Information and Communication Engineering, Chungbuk National University, Cheongju, 28644, Republic of Korea

ARTICLE INFO

ABSTRACT

Keywords: Federated Learning Total Variation Distance Independent and Identically Distributed

Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence. To address this, we propose FedTVD, a novel FL algorithm that weights client contributions during aggregation by considering both data quality and quantity. Unlike traditional FL approaches such as FedAvg, which rely solely on dataset size for client weighting, FedTVD integrates Total Variation Distance (TVD) to measure the divergence between each client’s local label distribution and a uniform global distribution. Clients with highly skewed distributions receive lower weights, preventing unbalanced datasets with imbalances from disproportionately influencing the global model. At the same time, dataset size is incorporated to ensure scalability and fairness. This dual-weighting mechanism effectively mitigates the impact of data imbalance, leading to more stable and generalized global models. Experimental results show that FedTVD consistently outperforms state-of-the-art methods across all datasets (FMNIST, CIFAR-10, and CIFAR-100) and all levels of data heterogeneity. Notably, it achieves up to 10.6% improvement over FedAvg on CIFAR-10 under highly skewed data, while maintaining top performance even under moderate and IID settings.

1. Introduction FL [1] has emerged as a revolutionary paradigm in distributed machine learning, enabling collaborative model training across multiple decentralized clients without requiring them to share their raw data. This design inherently enhances data privacy and security, addressing significant concerns in sensitive domains such as healthcare, finance, and personalized devices [2, 3, 4]. In contrast to traditional centralized learning, where data is aggregated in a single location, FL leverages local training on client devices, transmitting only model updates to a central server for aggregation. This decentralized approach aligns with stringent data privacy regulations, such as the General Data Protection Regulation (GDPR), and mitigates the risks associated with transferring large volumes of sensitive data over networks [5, 6]. Moreover, FL significantly reduces communication overhead and computational burden on centralized systems, making it well-suited for large-scale, heterogeneous environments. Despite its advantages, FL faces several challenges that hinder its practical deployment in real-world scenarios. These challenges can be broadly categorized into communication efficiency, system heterogeneity, and data heterogeneity. Communication efficiency is critical as FL systems often operate in resource-constrained environments, such as © 2026. This manuscript version is made available under the CCBY-NC-ND 4.0 license (http://creativecommons.org/licenses/by-nc-nd/4. 0/). This is the accepted manuscript version of an article published in Future Generation Computer Systems. The final published version is available at https://doi.org/10.1016/j.future.2025.108177. ∗ Corresponding author radwan-selo@cbnu.ac.kr (R. Selo); kundroomajid@cbnu.ac.kr (M. Kundroo); taehongkim@cbnu.ac.kr (T. Kim) ORCID (s):

: Preprint submitted to Elsevier

edge devices or Internet of Things (IoT) networks, where bandwidth and latency [7, 8] are limiting factors [9, 10, 11, 12]. System heterogeneity, on the other hand, arises from the diverse computational capacities of client devices, which can lead to unequal participation and delays in model updates [13]. Among these, data heterogeneity, where the data across clients is non-Independent and Identically Distributed (non-IID) exhibit significant variations in size and class distributions, hence. This issue is inherently tied to the decentralized nature of the system. Clients typically generate data based on their local environments and user behaviors [14]. For example, healthcare organizations contributing data to a federated medical imaging model may have patient populations that differ in demographics, disease prevalence, and imaging protocols, while mobile devices participating in FL-based recommendation systems may exhibit distinct usage patterns. This heterogeneity becomes particularly problematic during model aggregation [15, 16], as some clients possess large but skewed datasets, while others contribute smaller yet more balanced datasets. Standard aggregation methods, such as FedAvg [17], tend to prioritize clients with larger datasets, often resulting in global models that overfit dominant distributions [18] and struggle with fairness and generalization. Addressing this imbalance is crucial for developing robust and equitable FL models that generalize well across diverse client populations. A key barrier to effective FL is ensuring that the global model remains both fair and robust despite these challenges. This challenge is amplified by the fact that clients often learn from skewed local data, while the global model must perform well on more balanced data. This mismatch highlights the importance of a client’s data quality, particularly its label distribution. However, considering only quality may Page 1 of 14

FedTVD

undervalue clients with large but imbalanced data. Existing aggregation strategies often lead to models that overfit to dominant data distributions, causing model drift [19, 20], a phenomenon where the model’s performance deteriorates as local data deviates from the global model over time. This drift is particularly prevalent with non-IID dataset, where clients hold data with varying class distributions. Thus, it is essential to consider both the distributional quality and quantity of each client’s data, penalizing highly skewed distributions while still leveraging useful large-scale data, to support stable convergence and better generalization. To address these challenges, this study introduces Federated Learning with Total Variation Distance (FedTVD), a novel algorithm designed to mitigate the impact of heterogeneous label distributions. By leveraging the TVD [21, Chapter 4, p. 47], a robust and interpretable metric for quantifying the divergence between probability distributions, FedTVD employs a dual weighting mechanism that balances label distribution skewness with dataset size, thereby promoting better convergence, improved generalization, and enhanced fairness. By intelligently adjusting each client’s influence based on the nature of its data, our approach aims to ensure that the final model is both inclusive and robust. Ultimately, by addressing the inherent data heterogeneity among clients, we aim to create a system that achieves a balanced integration of diverse data sources, leading to a global model that is more accurate, generalizable, and fair. In light of the challenges posed by data heterogeneity and the limitations of traditional aggregation methods, this work makes several significant contributions to advancing FL. In particular, our work provides a comprehensive algorithm that not only addresses fairness and robustness in global model training but also enhances overall performance through practical, scalable solutions. Our key contributions are as follows: 1. TVD-based Weighting: We introduce a novel weighting mechanism that assigns adaptive weights to clients based on both label distribution skewness and dataset size. This ensures that clients with heavily skewed data have a moderated influence on the global model, while those with substantial and balanced data are appropriately emphasized, thereby promoting better convergence, improved generalization, and enhanced fairness. 2. Empirical Validation: Through extensive empirical evaluations across diverse datasets and application scenarios, we demonstrate that FedTVD consistently outperforms existing methods, particularly in environments with highly heterogeneous data. By dynamically adjusting client contributions based on data quality and quantity, FedTVD ensures stable convergence and enhanced performance, making it particularly suitable for large-scale, non-IID environments. The remainder of this paper is structured as follows: Section 2 reviews related work in FL, exploring various methods used to handle challenges like non-IID data and : Preprint submitted to Elsevier

model drift. Section 3 describes the methodology of this study, outlining the problem formulation, including model drift and data quality/quantity, and detailing the innovative TVD-based weighting mechanism used to improve global model convergence. Section 4 explains the experimental setup, providing information on the environment, dataset distribution strategies, hardware configuration, model architecture and hyperparameters employed in the study. Section 5 presents the results and a comprehensive discussion, highlighting the performance of the FedTVD approach under both IID and non-IID data conditions. Additionally, experiments were conducted to evaluate the effect of increasing the number of clients and the Client Participation Rate (CPR) on model accuracy. Finally, Section 6 concludes the paper, summarizing the key findings and suggesting directions for future research.

2. Related Works FL has been the focus of extensive research, with numerous algorithms proposed to address its inherent challenges. A key area of exploration is mitigating the effects of non-IID data across clients, which significantly impact the efficiency and fairness of collaborative training. This section reviews prominent FL algorithms and their strategies for handling data heterogeneity, highlighting their contributions and limitations in the context of developing more robust aggregation methods, with a summarized overview provided in Table 1.

2.1. Data Distribution Heterogeneity

Balancing the contribution of clients with diverse dataset sizes and data distributions remains a fundamental challenge in FL. Classical methods such as FedAvg aggregate updates using dataset-size-weighted averaging, which is effective for IID data but can result in biased models under nonIID scenarios. FedProx [22] augments this by penalizing local updates that deviate from the global model, leading to greater algorithmic stability. However, neither FedAvg nor FedProx fully mitigates issues arising from non-uniform label or feature distributions. FedNova [23] further adjusts for system heterogeneity by normalizing updates based on each client’s local progress. More recently, FedDkw [27] has introduced KL divergence to weight the similarity between client and server label distributions for aggregation, yet this approach can suffer from instability due to the unbounded nature of KL and neglects the size of each client’s dataset. Overall, while these algorithms address parts of the problem, a unified solution that robustly balances both data quantity and distributional diversity is still lacking.

2.2. Feature Heterogeneity

Distinct feature distributions among clients are especially prominent in domains such as medical imaging and multi-institutional datasets. FedBN [25] provides an effective remedy by maintaining client-specific batch normalization parameters while synchronizing only the remaining

Page 2 of 14

FedTVD Table 1 Summary of Key FL Methods, Addressed Challenges and Limitations Method

Year

Key Idea

Addressed Challenges

Limitations

FedAvg [17]

2017

Weighted average of local models by dataset size

Data quantity imbalance (assumes IID data)

Poor performance with non-IID data bias toward majority distributions

FedProx [22]

2020

Adds proximal term to local objectives for stability

System heterogeneity

Insufficient for distributional (label/feature) heterogeneity

FedNova [23]

2020

Normalizes client updates by training progress

Computational (system) imbalance

Assumes data distribution uniformity poor with skewed data

FedMA [24]

2020

Aligns and averages layer-wise model representations across clients

Feature representation heterogeneity

Does not explicitly account for variations in class distribution skew or disparities in client dataset sizes during aggregation

FedBN [25]

2021

Local batch normalization parameters global weight aggregation

Feature distribution shift

Does not address label skew or data quantity imbalance

Scaffold [26]

2021

Control variates to reduce client drift and update divergence

Client drift and training stability

Requires careful tuning limited handling of label and feature imbalance

FedDkw [27]

2023

Weights updates by KL divergence between client and global data distributions

Data distribution heterogeneity

Ignores dataset size and unbounded KL may destabilize aggregation

BN-Scaffold [28]

2024

Incorporates local batch normalization into Scaffold to address drift and feature shift

Client drift and feature shift

May depend on hyperparameter tuning does not fully resolve label skew or dataset size imbalance

Clustered FL [29, 30, 31]

20212024

Client clustering to train subgroup-specific models

Highly diverse client data distributions

Relies on accurate and stable clustering struggles with dynamic client populations

Personalized FL [32, 33]

2023

Client-specific model adaptation or fine-tuning of global model

Local data heterogeneity and personalization

Adds complexity focuses on local optimization not global solutions

FedOpt [34]

2021

Adaptive optimizers applied to global aggregation

Convergence under data and system heterogeneity

Depend on hyperparameter tuning and does not fully resolve data skew

model weights. This preserves locality in feature statistics, improving global performance where input distributions vary. Another approach, FedMA [24], aligns and averages layer-wise model representations across clients to reduce feature representation mismatch during aggregation. These strategies enhance model robustness under heterogeneous input spaces, but they still largely overlook skew in class distributions and disparities in dataset sizes.

this by incorporating local batch normalization into the Scaffold framework, thereby addressing both update divergence and feature shift in heterogeneous environments. Although these approaches demonstrate improved convergence, they do not fully resolve underlying label skew or dataset size imbalance, and their effectiveness may depend on careful hyperparameter tuning.

2.3. Client Drift

Clustered FL addresses the challenge of highly diverse data distributions by partitioning clients into subgroups with similar local data characteristics, enabling each cluster to train its own specialized model [30, 31, 29, 35]. These clusters are typically formed using gradient-based statistics, distributional similarity, or auxiliary metadata. By aligning models more closely with subgroup distributions, clustered FL improves personalization and convergence in non-IID settings. However, the effectiveness of this approach relies

Local model updates often diverge from the global trajectory in the presence of data heterogeneity, resulting in diminished convergence rates. Scaffold [26] directly addresses this challenge by utilizing control variates, which help align local updates with the direction of global optimization and significantly improve training stability under non-IID data. A recent variant, BN-Scaffold [28], enhances : Preprint submitted to Elsevier

2.4. Clustered Federated Learning

Page 3 of 14

FedTVD

heavily on accurate and stable clustering, which can be compromised in dynamic environments with shifting client populations or evolving data characteristics, where computational complexity requires multi-objective optimization [36, 37].

2.5. Personalized Federated Learning

Personalized FL (PFL) addresses the challenge of local data heterogeneity by allowing each client to fine-tune a shared global model or maintain personalized components [32, 33]. This improves local performance by adapting to individual client distributions. However, it differs from standard FL by focusing on client-specific optimization, which resolves local heterogeneity but does not offer a unified global solution.

2.6. Optimization and Aggregation Enhancements

This line of research aims to improve training efficiency and convergence under heterogeneous conditions by refining global optimization and aggregation strategies. FedOpt and its variants (FedAdam, FedYogi) [34] apply adaptive learning rate techniques based on aggregated gradients, resulting in more stable updates across diverse clients. Additionally, hyperparameter optimization methods such as FedAdap [38, 39] fine-tune training parameters to improve performance in non-IID scenarios. While these techniques enhance optimization dynamics and support convergence, they do not fully resolve deeper issues related to data distribution skew, system variability, or long-term generalization in complex federated settings. In summary, while existing federated learning methods have advanced the handling of non-IID data, they often struggle to effectively balance the influence of data quantity and distribution heterogeneity during aggregation. Many approaches address individual aspects like system heterogeneity, feature shifts, or client drift, yet a unified, principled solution that dynamically accounts for both dataset size and label distribution similarity remains lacking. To address this, we propose FedTVD, a novel aggregation method that leverages robust metrics to fairly and efficiently weight client contributions, leading to more stable, equitable, and generalized global models.

3. Methodology This section introduces the FedTVD algorithm and explains how it addresses the challenges of model aggregation in non-IID FL environments. We first discuss how the imbalance between data quantity and data quality in client updates can negatively affect model convergence and performance. We then present the FedTVD strategy, which corrects for this imbalance by jointly considering both factors during aggregation. Finally, we provide a detailed breakdown of the algorithm, including the client-server workflow and how TVD is incorporated into the aggregation process.

: Preprint submitted to Elsevier

3.1. Overview of FedTVD

FedTVD is an aggregation strategy that considers both the size of each client’s dataset (data quantity) and how well that dataset represents the global data distribution (data quality). This dual perspective enables more balanced and effective model aggregation unlike FedAvg, which relies solely on data quantity. Data quantity refers to the total number of local samples available to each client. In conventional FL frameworks such as FedAvg, client aggregation weights are determined solely based on dataset size, under the assumption that larger datasets produce more reliable updates. However, in non-IID scenarios, larger datasets may be significantly biased toward specific classes and thus may not contribute proportionally to the generalization of the global model. In contrast, data quality measures how closely a client’s local data distribution aligns with the global model distribution, which is typically assumed to be IID and classbalanced. For example, in a classification problem with three classes, the ideal global distribution would allocate one-third of the data to each class. In practical FL settings, clients typically exhibit varying degrees of deviation from this ideal distribution due to user-specific or environmental factors. To quantify this deviation, we employ the TVD, a statistical metric that measures the disparity between a client’s local label distribution and the global balanced distribution. A TVD of zero indicates a perfectly balanced client dataset, while higher TVD values signal more severe distributional skewness. Fig. 1 illustrates the difference between FedAvg and FedTVD across three clients. Each client holds samples from three classes. Client 1 has a balanced dataset with (30, 30, 30) samples per class, yielding a TVD of zero. Client 2 displays moderate imbalance with (20, 40, 40), and Client 3 has a notably skewed dataset of (10, 30, 70). In FedAvg, Client 3 receives the highest aggregation weight, driven solely by its larger dataset size. Conversely, FedTVD mitigates this issue by adjusting weights based on both data quantity and TVD. As a result, Client 1 receives a higher aggregation weight under FedTVD, reflecting its higher data quality, despite having fewer samples. By dynamically balancing data quantity and quality, FedTVD promotes equitable and robust model aggregation, steering updates away from clients with skewed datasets and toward those with representative distributions. This fosters improved convergence and generalization in heterogeneous FL environments. At the system level, FedTVD introduces minor yet impactful modifications to the standard FL workflow. Clients locally compute their label distributions and associated TVD values relative to the global distribution. These metrics, along with model updates and dataset sizes, are communicated to the server. The server then performs a refined aggregation by jointly incorporating data quantity and TVD, replacing the simplistic quantity-only weighting used in FedAvg. The subsequent subsection provides a detailed description of the FedTVD workflow, outlining its clientserver interactions and aggregation mechanism. Page 4 of 14

FedTVD Data Quantity (FedAvg)

Data Quantity and Quality (FedTVD)

Client

dataset

Size of

Client 2

Client 1

Total number of samples across clients

Client 3

Distribution of Balanced Distribution

Client 1

Client 2

Client 3 P=(

,

,

)

P=(

,

,

)

Q=(

,

,

)

Q=(

,

,

)

P=( Q=(

,

, ,

,

)

Client

scaling factor

Client

model

Global model

)

Client 1 contributes more to the global model due to its more balanced local dataset

Data Quantity

where is the number of samples at client

Aggregation

Data Quality

Data Quantity

Aggregation Client 1 receives a lower weighting in the aggregation process because data quality is not considered.

Fig. 1: Comparison of FedAvg and FedTVD aggregation strategies. FedAvg relies only on data quantity (number of samples), while FedTVD incorporates both data quantity and quality, ensuring more balanced aggregation and mitigating model drift in non-IID settings.

3.2. Detailed FedTVD Algorithmic Steps

where |𝐷𝑖𝑘 | denotes the number of local samples of class 𝑖, ∑ 𝑘 and |𝐷𝑘 | = 𝐶 𝑖=1 |𝐷𝑖 | is the total number of local samples for client 𝑘. To evaluate how this local distribution deviates from a balanced target, the client constructs a reference distribution 𝑄, in which all classes are assumed to be equally likely: ( ) 1 1 1 𝑄= , ,…, (2) 𝐶 𝐶 𝐶

3.2.1. Client-Side Operations (1) Local Training: Each client 𝑘 begins with the global parameters 𝑤 and proceeds to train on its private dataset 𝐷𝑘 using an appropriate optimizer, such as stochastic gradient descent (SGD). Multiple local epochs can be executed if network constraints allow. (2) Label Distribution Computation: Following training, the client computes its label distribution 𝑃 by counting how many examples fall into each class. Formally, if the total number of classes is 𝐶, the label distribution 𝑃 is given by: ) ( 𝑘 |𝐷1 | |𝐷2𝑘 | |𝐷𝐶𝑘 | , ,…, 𝑘 𝑃 = (1) |𝐷𝑘 | |𝐷𝑘 | |𝐷 |

This balanced distribution serves as a baseline for assessing the level of skew in the client’s local data. (3) TVD Calculation: The client calculates the TVD to measure the deviation between its local data distribution 𝑃 and the balanced reference distribution 𝑄. The TVD is defined as:

The FedTVD algorithm follows the typical client-server paradigm in FL. In each round 𝑡, the server distributes the current global model 𝑤𝑡 to participating clients. Clients then update these parameters using their local datasets and send back the updated parameters, along with additional statistics. Below we provide a step-by-step outline of our approach, culminating in Algorithm 1.

: Preprint submitted to Elsevier

1 ∑| 𝑃 − 𝑄𝑖 || 2 𝑖=1 | 𝑖 𝐶

TVD =

(3)

TVD quantifies the total difference between the two distributions by summing the absolute differences of their corresponding probabilities. A higher TVD indicates a larger imbalance in the local data distribution, which is useful for identifying clients with more skewed data. This helps Page 5 of 14

FedTVD

the server adjust the model aggregation process, giving less weight to clients with higher TVD values to ensure a more balanced global model. (4) Return of Results: Finally, the client 𝑘 sends its updated model parameters 𝑤𝑘 , the computed TVD𝑘 , and its local sample size |𝐷𝑘 | back to the server.

3.2.2. Server-Side Operations (1) Model Broadcast: The server holds a global model parameter vector 𝑤, which is initialized at the start. At each round 𝑡, the server broadcasts 𝑤𝑡 to all participating clients. (2) Reception of Client Updates: Once clients finish their local training, the server collects the updated model parameters 𝑤𝑘 , TVD𝑘 , and local sample sizes |𝐷𝑘 | from each client 𝑘 in the current round. (3) Softmax-Based TVD Weighting: The server applies the softmax function to the set of TVD values received from all clients to transform them into a normalized weighting distribution: 𝑘

𝑒−TVD 𝑎𝑘 = ∑𝐾 −TVD𝑗 𝑗=1 𝑒

(4)

This transformation adjusts each client’s weight based on the degree of balance in their local label distribution. Clients with more balanced distributions (i.e., lower TVD values) are assigned higher weights, thereby contributing more significantly to the global model update. In contrast, clients with skewed distributions (higher TVD) are down-weighted. The softmax function also ensures a smooth and differentiable weighting scheme, preventing any single client from dominating the aggregation process. (4) Composite Scaling Factors: Following the softmaxbased weighting described above, the server integrates each client’s weight 𝑎𝑘 , which reflects the balance of its local label distribution, with its relative data volume. First, it computes the total number of samples across all clients: |𝐷| =

𝐾 ∑

Algorithm 1: Federated Learning with TVD (FedTVD) 1 Function CLIENT_SIDE_OPERATIONS(𝑤): 2 Train local model 𝑤 on local data 𝐷𝑘 3

// Calculate local label distribution

(

𝑃 = 4

(

1 1 1 , ,…, 𝐶 𝐶 𝐶

(5)

𝑘=1

This allows the computation of the proportion of data held 𝑘| by each client 𝑘, expressed as |𝐷 , which captures the |𝐷| contribution of the client in terms of dataset size. The final scaling factor 𝑠𝑘 for client 𝑘 is then computed by combining the softmax-derived weight 𝑎𝑘 , which reflects the balance of its local label distribution, with its relative data volume. This combination is controlled via a tunable parameter 𝜆 ∈ [0, 1] as follows: 𝑠𝑘 = 𝜆 ⋅ 𝑎𝑘 + (1 − 𝜆) ⋅

|𝐷𝑘 | |𝐷|

(6)

Here, 𝜆 enables flexible control over the influence of data quality versus quantity in the aggregation process. A higher 𝜆 emphasizes data quality (i.e., label balance), while a lower 𝜆 emphasizes dataset size. Notably, setting 𝜆 = 1 results in weighting purely based on data quality, and setting 𝜆 = 0 : Preprint submitted to Elsevier

Eq. (1)

)

Eq. (2)

// Calculate TVD

1 ∑| 𝑃 − 𝑄𝑖 || 2 𝑖=1 | 𝑖 𝐶

TVD =

Eq. (3)

return 𝑤, TVD, and |𝐷𝑘 | 7 End Function 8 Function SERVER_SIDE_OPERATIONS(): 9 Initialize global model parameters 𝑤 10 for each round 𝑡 = 1, 2, … , 𝑇 do 11 Distribute global model 𝑤𝑡 to all participating clients 12 for each client 𝑘 = 1, 2, … , 𝐾 do 13 𝑤𝑘 , TVD𝑘 , and |𝐷𝑘 | ← CLIENT_SIDE_OPERATIONS(𝑤𝑡 ) 14 end 6

15

// Calculate softmax of TVD values 𝑘

𝑒−TVD 𝑎𝑘 = ∑𝐾 −TVD𝑗 𝑗=1 𝑒 16

Eq. (4)

// Calculate total number of samples across clients

|𝐷| =

𝐾 ∑

|𝐷𝑘 |

Eq. (5)

𝑘=1 17

// Compute scaling factor

𝑠𝑘 = 𝜆 ⋅ 𝑎𝑘 + (1 − 𝜆) ⋅ |𝐷𝑘 |

)

// Initialize global balanced distribution

𝑄= 5

|𝐷1𝑘 | |𝐷2𝑘 | |𝐷𝐶𝑘 | , , … , |𝐷𝑘 | |𝐷𝑘 | |𝐷𝑘 |

18

|𝐷𝑘 | Eq. (6) |𝐷|

// Aggregate using

𝑤𝑡+1 =

𝐾 ∑

𝑠𝑘 𝑤𝑘

Eq. (7)

𝑘=1 19 20

end End Function

reduces the algorithm to FedAvg, where only dataset size is considered. This composite scaling factor therefore accounts for both the quality of the client’s data (via 𝑎𝑘 , which penalizes 𝑘| distributional skew) and its quantity (via |𝐷 ). By jointly |𝐷| considering these two dimensions, the algorithm ensures that clients with both large and well-balanced datasets exert greater influence on the global model, while clients with limited or highly skewed data are appropriately down-weighted. Page 6 of 14

FedTVD

This strategy enables a principled trade-off between fairness and statistical representativeness in federated model aggregation. (5) Model Aggregation: Finally, the server aggregates the local models into the new global model 𝑤𝑡+1 using: 𝑤𝑡+1 =

𝐾 ∑

(7)

𝑠𝑘 𝑤𝑘

𝑘=1

This updated model 𝑤𝑡+1 is then broadcast to the clients in the next round.

3.3. Complexity Analysis

To evaluate the scalability of FedTVD, we analyze its computational complexity and compare it to that of FedAvg. We consider the dominant operations per communication round on both client and server sides. Client-Side Complexity: The primary computational cost on the client side is local training. Assuming each client performs 𝐸 epochs of SGD over their local dataset 𝐷𝑘 , the training complexity is: (𝐸 ⋅ |𝐷𝑘 | ⋅ |𝑤|)

• Softmax over 𝐾 TVD values: (𝐾) • Total sample count: (𝐾) • Composite scaling factor computation: (𝐾) • Weighted model aggregation: (𝐾 ⋅ |𝑤|) Although FedTVD adds four extra steps at the server, all but one are lightweight (𝐾) scalar operations. Only the final step—weighted model aggregation—shares the same dominant complexity (𝐾 ⋅ |𝑤|) as FedAvg. Thus, FedTVD preserves the same overall order of server-side complexity as FedAvg, with only minor constant-factor overhead.

3.4. Convergence Analysis

We analyze the convergence behavior of FedTVD under standard assumptions in federated optimization. Our approach builds upon the convergence theory of FedAvg, which we briefly recall here to provide a foundation for comparison. In FedAvg, the global objective function is defined as: 𝐹 (𝑤) =

𝑝𝑖 𝐹𝑖 (𝑤),

𝑖=1

: Preprint submitted to Elsevier

where all gradients are evaluated at the current global model 𝑤𝑡 , and the client drift term is defined as: 1 2 𝛿FedAvg = 𝐾

where

|𝐷 | , 𝑝𝑖 = ∑𝑁 𝑖 𝑗=1 |𝐷𝑗 |

𝐾 ∑ 𝑘=1

‖∇𝐹𝑘 (𝑤𝑡 ) − ∇𝐹 (𝑤𝑡 )‖2 . ‖ ‖

FedTVD modifies only the aggregation step of FedAvg. Instead of relying solely on the sample proportion |𝐷𝑖 |, it incorporates both data quantity and data quality. In particular, each client is assigned a weight 𝑠𝑘 that depends on its dataset size and the TVD of its label distribution. The updated global model is then computed using a normalized weighted average: 𝑤𝑡+1 =

where |𝑤| is the number of model parameters. In addition to training, FedTVD computes the local label distribution and TVD. Label counting has linear complexity in the dataset size and class count, (|𝐷𝑘 | + 𝐶), while TVD computation is (𝐶). Given that 𝐶 is typically much smaller than |𝐷𝑘 |, the total client-side complexity remains dominated by the training phase. Server-Side Complexity: At the server, FedAvg aggregates 𝐾 client models per round, costing (𝐾 ⋅ |𝑤|). FedTVD introduces lightweight additional operations:

𝑁 ∑

and each local objective function 𝐹𝑖 (𝑤) is assumed to be 𝐿-smooth and 𝜇-strongly convex. Under these assumptions, FedAvg achieves the following convergence bound after 𝑇 communication rounds: ( ) 1 2 𝔼[𝐹 (𝑤𝑇 ) − 𝐹 (𝑤∗ )] ≤ 𝑂 + 𝑂(𝛿FedAvg ), 𝑇

𝐾 ∑

𝑠𝑘 𝑤𝑡𝑘 ,

with

𝐾 ∑

𝑠𝑘 = 1,

𝑠𝑘 ≥ 0.

𝑘=1

𝑘=1

This structure remains consistent with the class of algorithms analyzed in FedAvg, and thus FedTVD inherits the same theoretical convergence guarantees under convexity and smoothness. Formally, the expected optimality gap for FedTVD satisfies: ( ) 1 2 ), + 𝑂(𝛿FedTVD 𝔼[𝐹 (𝑤𝑇 ) − 𝐹 (𝑤∗ )] ≤ 𝑂 𝑇 where the modified client drift term is given by: 2 𝛿FedTVD =

𝐾 ∑ 𝑘=1

𝑡 𝑡 ‖2 𝑠𝑘 ‖ ‖∇𝐹𝑘 (𝑤 ) − ∇𝐹 (𝑤 )‖ .

Since TVD captures the degree of label imbalance, which often leads to biased local gradients, FedTVD naturally reduces drift by assigning lower weights to clients with highly skewed distributions. This weighting effect suppresses the contribution of highly biased updates, which 2 2 implies that 𝛿FedTVD ≤ 𝛿FedAvg in typical non-IID scenarios. In conclusion, FedTVD maintains the same convergence rate as FedAvg under standard convexity and smoothness assumptions. By incorporating distributional information into the aggregation process, it reduces the impact of client drift, which is often the dominant source of error in non-IID FL. As a result, FedTVD is theoretically as sound as FedAvg while offering improved robustness in practical scenarios.

3.5. Advantages and Novelty

(A) Mitigating non-IID Effects: FedTVD explicitly moderates clients exhibiting large distributional discrepancies from a balanced reference distribution. By computing the TVD of each client’s local label distribution with respect to Page 7 of 14

FedTVD

an ideal (uniform) distribution, our method identifies those with skewed or biased data. This mechanism prevents the global model from becoming dominated by a few skewed distributions, ensuring a more equitable update that remains resilient in heterogeneous settings. (B) Robustness to Label Skew Outliers: Clients with highly imbalanced label distributions can act as statistical outliers and introduce bias during model aggregation. FedTVD addresses this issue by assigning lower aggregation weights to such clients using an exponential penalty based on their TVD. This reduces their influence on the global model and helps improve robustness against distributional skew. We note that our notion of robustness is limited to mitigating the effect of label imbalance, and does not address other types of outliers such as clients with corrupted data labels or adversarial clients [40]. (C) Balanced Data-Driven Weighting: While addressing skewness is vital, large datasets may hold significant information. FedTVD reconciles these needs by incorporating both the TVD-based weight (𝑎𝑘 ) and the sample-size frac𝑘| ). This hybrid scheme strikes a balance between tion ( |𝐷 |𝐷| distributional alignment and data volume, preventing large but skewed datasets from overwhelming the global model while still leveraging their information-rich contribution. (D) Light Overhead: Compared to traditional FL, the main overhead of FedTVD is the computation and transmission of TVD for each client. However, this overhead remains minimal compared to raw data exchange or large-scale gradient sharing. The primary computational overhead in FedTVD arises from computing the TVD metric, which involves summing over class distributions, This complexity is negligible compared to the overall cost of FL, making FedTVD scalable to large deployments. (E) Extensibility: Though introduced in the context of a standard FL approach, the TVD modification is designed as a lightweight, drop-in mechanism that can be integrated into diverse federated algorithms. Because it involves minimal additional computation and does not require substantial changes to communication or optimization routines, it can be readily adopted in any FL framework to handle non-IID data. (F) Real-World Use Cases: FedTVD is particularly wellsuited to FL environments where client data is skewed due to user-specific, demographic, or operational factors. In healthcare applications, for instance, different hospitals or medical centers may collect patient data that naturally varies in disease prevalence or age distribution, which leads to highly imbalanced local datasets. FedTVD can mitigate the influence of such skewed clients during model aggregation while preserving meaningful contributions. Similarly, in large-scale sensor networks or environmental monitoring, data collected from different regions or sensor types may reflect localized patterns that deviate from the overall distribution. While FedTVD computes TVD using a uniform reference distribution to avoid domination by any particular class, this reference can be adapted in practice to reflect : Preprint submitted to Elsevier

domain-specific priors if a more accurate global distribution is known. This flexibility ensures that FedTVD remains applicable across both standard and application-aware FL scenarios, enhancing fairness and robustness in aggregation. Summary: In conclusion, FedTVD offers a balanced and adaptive aggregation strategy by coupling data quantity with a principled TVD-based penalty. This approach effectively addresses heterogeneous data distributions and outlier risks while imposing minimal overhead. The key operations and line-by-line equations are outlined in Algorithm 1, providing a clear blueprint for practical deployment in a wide range of FL scenarios.

4. Experimental Setup This section outlines the experimental setup, which includes the datasets, data partitioning techniques, computational environment, and configurations used to evaluate the proposed approach. In order to evaluate the model’s effectiveness under various client participation rates and data heterogeneity levels, we provide a comprehensive understanding of its performance in diverse scenarios.

4.1. Implementation Details and System Configuration

All experiments were conducted using Python 3.11, with PyTorch 2.2.2 serving as the core deep learning framework. FL simulations were implemented using the FedEasy framework [41], enabling efficient orchestration of clientserver interactions. To ensure consistency across experimental runs, all computations were performed on a highperformance computing server equipped with an Intel Core i9-10900X CPU running at 3.70 GHz, 192 GB of RAM, and four NVIDIA GeForce RTX 4090 GPUs. This computational setup facilitated large-scale federated training while maintaining reproducibility and efficiency.

4.2. Datasets and Model Architectures

The evaluation was conducted on three standard benchmark datasets: FashionMNIST (FMNIST) [42], CIFAR10 [43], and CIFAR-100, each presenting a different level of complexity. FMNIST, a grayscale image classification dataset, was used with a standard Convolutional Neural Network (CNN) architecture [44]. CIFAR-10 was evaluated using a ResNet-18 model [45], while CIFAR-100 was trained on a deeper ResNet-34 architecture. These datasets provided a diverse range of classification challenges, allowing for an in-depth analysis of the proposed method’s robustness across both simple and complex tasks.

4.3. Data Partitioning and Experimental Variants

To simulate realistic non-IID conditions, data partitioning was performed using a Dirichlet allocation strategy [46], controlling the extent of statistical heterogeneity across clients. The concentration parameter 𝛼 was varied to reflect different levels of non-IID data distributions, with lower values 𝛼 = 0.1 indicating highly skewed distributions and Page 8 of 14

FedTVD Table 2 Mean test accuracy for the last 10 rounds across different datasets (FMNIST, CIFAR-10, and CIFAR-100) under varying Dirichlet non-IID distributions (𝛼 = 0.1, 0.5, 1.0) and IID settings. Dataset

FMNIST

CIFAR10

CIFAR100

Model

CNN

Resnet-18

Resnet-34

Dir (𝛼)

FedAvg

FedProx

FedNova

FedDkw

FedTVD (ours)

0.1

82.64 ± 0.71

80.69 ± 1.68

83.63 ± 0.22

83.99 ± 0.61

85.26 ± 0.65

0.5

86.11 ± 0.56

86.08 ± 0.39

86.65 ± 0.14

87.20 ± 0.19

87.21 ± 0.21

1.0

87.69 ± 0.36

87.62 ± 0.41

88.01 ± 0.10

88.03 ± 0.21

88.39 ± 0.14

IID

88.95 ± 0.16

88.83 ± 0.25

88.89 ± 0.15

88.89 ± 0.25

88.74 ± 0.23

0.1

48.24 ± 3.35

49.44 ± 2.69

53.52 ± 2.20

56.30 ± 1.61

58.82 ± 1.75

0.5

73.33 ± 0.42

72.61 ± 1.00

74.26 ± 0.19

74.51 ± 0.37

74.69 ± 0.29

1.0

76.17 ± 0.32

75.22 ± 0.62

75.37 ± 0.43

75.13 ± 0.41

76.22 ± 0.45

IID

77.56 ± 0.37

77.34 ± 0.45

77.23 ± 0.32

77.12 ± 0.40

78.04 ± 0.25

0.1

35.31 ± 1.12

36.12 ± 0.44

36.26 ± 0.57

36.16 ± 0.87

37.68 ± 0.38

0.5

38.91 ± 0.36

38.36 ± 0.75

38.89 ± 0.44

38.21 ± 1.08

39.58 ± 0.71

1.0

39.30 ± 0.55

39.30 ± 0.92

39.09 ± 0.27

39.06 ± 0.69

39.79 ± 0.51

IID

39.24 ± 0.68

39.33 ± 0.29

39.76 ± 0.59

39.95 ± 0.20

40.51 ± 0.45

higher values 𝛼 = 1 representing more balanced partitions. For reference, a fully IID scenario was also included in the analysis by setting 𝛼 to a sufficiently large value. Two major experimental settings were designed to analyze the impact of data heterogeneity and CPR. The first set of experiments investigated the effect of varying nonIID levels on model convergence and performance across all three datasets. In this setting, different values of 𝛼 = 0.1, 0.5, 𝑎𝑛𝑑1 were considered, allowing an in-depth evaluation of the impact of statistical heterogeneity. Each experiment involved 100 clients, with 10% of them randomly selected in each round to participate in training. The second set of experiments specifically focused on assessing how the number of clients and CPR influence learning dynamics. These trials were conducted using the CIFAR-10 dataset under a highly non-IID setting 𝛼 = 0.1. The number of clients was varied between 100 and 200, while the CPR was systematically adjusted to 10%, 20%, and 50%, enabling an analysis of scalability and participation effects on model convergence.

4.4. Hyperparameters and Training Process

To ensure fair comparisons, all experiments followed a standardized training protocol. Model updates were optimized using SGD with a learning rate of 0.01 and a momentum coefficient of 0.9. Each client performed four local epochs per training round with a batch size of 32 before transmitting model updates to the central server. Training was conducted over 300 communication rounds to thoroughly evaluate model convergence and stability across different settings. For FedTVD, the hyperparameter 𝜆 was fixed at 0.50, reflecting an equal balance between client label distribution quality and dataset size during aggregation. To ensure robustness and reproducibility, each experiment was repeated five times using different random seeds, with data resampling performed independently in each experiment for : Preprint submitted to Elsevier

training only. The testing dataset was fixed on the server and remained unchanged throughout all runs. The reported results represent the mean ± standard deviation over these runs.

5. Results & Discussion In this section, we present the results from comprehensive comparative experiments conducted to verify the effectiveness of the proposed FedTVD algorithm. These experiments benchmark FedTVD against several state-of-theart FL algorithms, including FedAvg, FedProx, FedNova, and FedDkw. The results, presented in the following subsections, demonstrate the superior performance of FedTVD in terms of test accuracy and convergence speed, particularly in extreme non-IID data settings.

5.1. Performance Analysis Across Varying Data Distributions and Datasets

In scenarios with extreme data heterogeneity, where the Dirichlet parameter 𝛼 = 0.1, FedTVD demonstrates a clear advantage over other algorithms. As shown in Fig. 2a, 3a and Fig. 4a, FedTVD achieves faster convergence and higher accuracy compared to FedAvg, FedProx, and FedNova, particularly on the CIFAR-10 dataset. The results in Table 2 also reinforce this finding, where FedTVD reaches 58.82% test accuracy on CIFAR-10, surpassing FedAvg (48.24%) and FedDkw (56.30%). This performance is especially significant in environments where the clients’ data are highly skewed, resulting in significant heterogeneity between client datasets. In such settings, FedTVD’s adaptive aggregation mechanism prioritizes the updates that contribute most effectively to improving the global model, which helps mitigate the negative impact of extreme data imbalances. Specifically, FedTVD is able to focus on the more informative updates from clients with data distributions that are more Page 9 of 14

50

100

FedAvg FedProx FedNova 150 200 Round

FedDkw FedTVD 250

300

90 80 70 60 50 40 30 20 10 0

Accuracy (%)

90 80 70 60 50 40 30 20 10 0

Accuracy (%)

Accuracy (%)

FedTVD

50

(a) 𝛼 = 0.1

100

FedAvg FedProx FedNova 150 200 Round

FedDkw FedTVD 250

300

90 80 70 60 50 40 30 20 10 0

50

(b) 𝛼 = 0.5

100

FedAvg FedProx FedNova 150 200 Round

FedDkw FedTVD 250

300

(c) 𝛼 = 1.0

50

100

FedAvg FedProx FedNova 150 200 Round

FedDkw FedTVD 250

300

80 70 60 50 40 30 20 10 0

Accuracy (%)

80 70 60 50 40 30 20 10 0

Accuracy (%)

Accuracy (%)

Fig. 2: Accuracy trends over rounds for the FMNIST dataset using a CNN model architecture under different levels of non-IID data distribution controlled by the Dirichlet parameter 𝛼.

50

(a) 𝛼 = 0.1

100

FedAvg FedProx FedNova 150 200 Round

FedDkw FedTVD 250

300

80 70 60 50 40 30 20 10 0

50

(b) 𝛼 = 0.5

100

FedAvg FedProx FedNova 150 200 Round

FedDkw FedTVD 250

300

(c) 𝛼 = 1.0

Fig. 3: Accuracy trends over rounds for the CIFAR-10 dataset using a ResNet-18 model architecture under different levels of non-IID data distribution controlled by the Dirichlet parameter 𝛼.

FedAvg, FedProx, and FedNova. On CIFAR-100, FedTVD achieves 39.58% accuracy for 𝛼 = 0.5, slightly surpassing FedAvg (38.91%) and FedDkw (38.21%). For 𝛼 = 1.0, it reaches 39.79%, outperforming FedAvg (39.30%) and FedDkw (39.06%). This suggests that even in less extreme non-IID scenarios, FedTVD effectively balances client contributions and mitigates data imbalance, leading to stable and efficient convergence. Unlike FedAvg and FedNova, which struggle with slower adaptation in moderately heterogeneous settings, FedTVD dynamically adjusts client weights to prioritize more informative updates. This adaptability results in consistent performance improvements across both CIFAR10 and CIFAR-100. Even in IID settings, where client data distributions are uniform, FedTVD maintains competitive accuracy and

45

45

40

40

40

35

35

35

30 25 20 15 10 0

50

100

FedAvg FedProx FedNova 150 200 Round

(a) 𝛼 = 0.1

FedDkw FedTVD 250

300

30 25 20 15 10 0

50

100

FedAvg FedProx FedNova 150 200 Round

(b) 𝛼 = 0.5

FedDkw FedTVD 250

300

Accuracy (%)

45

Accuracy (%)

Accuracy (%)

representative of the global model, thus enabling faster and more stable convergence. The benefit of FedTVD becomes more pronounced as the data heterogeneity increases, demonstrating its ability to adapt to highly non-IID settings and outperform other methods that struggle to converge efficiently under such conditions. In conclusion, FedTVD excels in highly heterogeneous environments, where its adaptive aggregation scheme allows it to outperform other FL algorithms by better handling the skewed data distributions across clients. As data heterogeneity becomes more moderate, represented by 𝛼 = 0.5 and 𝛼 = 1.0, FedTVD maintains its advantage in both accuracy and convergence speed. The trends in Fig. 3b, Fig. 3c, Fig. 4b, and Fig. 4c, along with Table 2, demonstrate that FedTVD consistently outperforms

30 25 20 15 10 0

50

100

FedAvg FedProx FedNova 150 200 Round

FedDkw FedTVD 250

300

(c) 𝛼 = 1.0

Fig. 4: Accuracy trends over rounds for the CIFAR-100 dataset using a ResNet-34 model architecture under different levels of non-IID data distribution controlled by the Dirichlet parameter 𝛼. : Preprint submitted to Elsevier

Page 10 of 14

FedTVD Table 3 Mean test accuracy for the last 10 rounds on the CIFAR-10 dataset under extreme non-IID settings (𝛼 = 0.1) with varying numbers of clients and CPR. #Clients 100

200

CPR

FedAvg

FedProx

FedNova

FedDkw

FedTVD (ours)

0.1

48.24 ± 3.35

49.44 ± 2.69

53.52 ± 2.20

56.30 ± 1.61

58.82 ± 1.75

0.2

56.45 ± 0.82

56.55 ± 1.58

58.37 ± 0.77

59.98 ± 1.14

61.68 ± 1.57

0.5

60.60 ± 0.94

61.18 ± 0.82

62.00 ± 0.65

62.35 ± 0.89

62.93 ± 0.86

0.1

46.71 ± 1.04

45.28 ± 2.42

52.91 ± 1.84

55.11 ± 1.83

57.39 ± 3.30

0.2

52.05 ± 2.49

50.99 ± 2.22

56.24 ± 1.48

57.57 ± 1.61

58.78 ± 1.88

0.5

55.18 ± 2.26

56.41 ± 2.17

58.62 ± 1.49

59.19 ± 1.18

61.12 ± 0.97

stable convergence, as shown in Table 2 and Figs. 2c and 3c. While the performance gap with existing methods like FedAvg and FedDkw is marginal, FedTVD’s adaptive aggregation remains effective, ensuring consistent updates across clients. Its ability to perform well across both IID and non-IID environments highlights its versatility in FL. This minor gain may arise because, despite IID sampling, the Dirichlet-based partitioning with finite 𝛼 can still lead to slight class imbalances, allowing the TVD term to act as a weak regularizer.

5.2. Influence of Total Number of Clients and CPR

This section analyzes the performance according to CPR with 100 clients. For a low CPR of 0.1, where only 10% of clients participate in each communication round, FedTVD achieves an accuracy of 58.82%, outperforming FedDkw (56.30%) and FedAvg (48.24%), as shown in Table 3. As CPR increases, FedTVD’s performance improves, reaching 62.93% at a CPR of 0.5, maintaining its lead over FedDkw (62.35%). This improvement can be attributed to the increased diversity and quantity of updates as more clients contribute to the aggregation process, allowing FedTVD to better capture the global data distribution and enhance test set accuracy. FedTVD benefits from broader data access, further boosting its performance. As the client count increases to 200, the overall performance of all algorithms slightly decreases due to greater data distribution heterogeneity and reduced data volume per client. At a CPR of 0.1, FedTVD achieves 57.39%, slightly outperforming FedDkw (55.11%) and significantly surpassing FedAvg (46.71%). At a higher CPR of 0.5, FedTVD achieves 61.12%, maintaining its lead over FedDkw (59.19%). The reduced accuracy compared to the 100client scenario is expected, as the broader distribution of data increases variability in local updates, and the reduced data per client limits the representativeness of individual models. Despite these challenges, FedTVD’s robust adaptive mechanism minimizes the performance drop by aggregating the most relevant updates, ensuring that the global model quality remains high even with a larger number of clients.

: Preprint submitted to Elsevier

5.3. Impact of Data Quality

In FL, both data quantity and data quality influence model performance. This section explores the impact of quality-aware weighting in FL aggregation by comparing FedAvg (which prioritizes dataset size) and FedTVD (which dynamically adjusts weights based on both label distribution and dataset size) on the CIFAR-10 dataset under a Dirichlet distribution with 𝛼 = 0.1, using 100 clients (CPR of 0.1). Fig. 5 illustrates the difference between traditional quantitybased aggregation (Fig. 5a) and FedTVD’s quality-aware approach (Fig. 5b). In Fig. 5a, larger datasets dominate aggregation, leading to an imbalance where a few clients contribute excessively while others have minimal influence. In contrast, Fig. 5b shows a more even distribution of contributions, as FedTVD dynamically adjusts weights based on both dataset size and label distribution. This ensures that clients with highly skewed data do not disproportionately impact the global model while still leveraging large, highquality datasets. As a result, FedTVD achieves a 10.58% accuracy improvement over FedAvg in extreme non-IID settings (𝛼 = 0.1, Table 2), demonstrating the importance of incorporating data quality into FL aggregation. Table 4 Mean test accuracy over the last 10 rounds on CIFAR10 for different 𝜆 values under two non-IID settings: extreme (𝛼 = 0.1) and moderate (𝛼 = 1.0). Each setting uses 100 clients (CPR of 0.1). 𝜆 0.00 0.25 0.50 0.75 1.00

Accuracy (mean ± std) 𝛼 = 0.1

𝛼 = 1.0

48.24 ± 3.35 55.88 ± 0.71 58.82 ± 1.75 58.16 ± 0.79 60.18 ± 0.96

76.17 ± 0.32 76.25 ± 0.22 76.22 ± 0.45 76.05 ± 0.55 76.03 ± 0.52

To further examine the impact of balancing data quantity and quality, we provide an ablation study on the weighting parameter 𝜆, which controls the trade-off between the two components during aggregation. A value of 𝜆 = 0.00 corresponds to using data quantity only (i.e., FedAvg), while 𝜆 =

Page 11 of 14

FedTVD Client 10

Client 10

Client 9

Client 9

Client 8

Client 8 Client 7

Client 6

Clients

Clients

Client 7 Client 5 Client 4

Client 6 Client 5 Client 4

Client 3

Client 3

Client 2

Client 2

Client 1 1

2

3

4

5 6 Rounds

7

8

9

10

Client 1 1

(a) Data Quantity only

2

3

4

5 6 Rounds

7

8

9

10

(b) Data Quantity and Quality

Fig. 5: Conceptual illustration of different weighting strategies in FL on the CIFAR-10 dataset under a Dirichlet distribution with 𝛼 = 0.1, using 100 clients (CPR of 0.1). The bubble size represents the relative contribution of each client during aggregation. (a) When only data quantity is considered, clients with larger datasets dominate the updates. (b) A combined approach balances both factors, ensuring that high-quality contributions are prioritized alongside larger datasets.

1.00 corresponds to FedTVD’s full reliance on data quality. Intermediate values interpolate between these extremes. As shown in Table 4, under extreme non-IID conditions (𝛼 = 0.1), even small increases in 𝜆 lead to notable accuracy gains—for example, 𝜆 = 0.25 improves performance by over 7% compared to 𝜆 = 0.00. Although 𝜆 = 1.00 achieves the highest accuracy in this extreme setting, it performs slightly worse in the moderate skewed setting (𝛼 = 1.0). Relying solely on data quality, as with 𝜆 = 1.00, may not generalize well across all FL scenarios since, in less skewed environments, data quantity becomes more important. In contrast, 𝜆 = 0.50 delivers consistently strong results across both scenarios—ranking second-best for 𝛼 = 0.1 and matching the top accuracy for 𝛼 = 1.0. This balanced approach, which combines contributions from both data quality and quantity, makes 𝜆 = 0.50 a robust default choice for diverse FL environments. These results highlight the critical role of balancing data quality and quantity in FL aggregation. Unlike traditional methods that overweight large but potentially biased datasets, FedTVD with 𝜆 = 0.50 ensures a fairer contribution from all clients, leading to better generalization across diverse non-IID scenarios and more stable convergence.

6. Conclusion FedTVD advances the state of the art in FL by incorporating a lightweight yet effective metric, the TVD, into the aggregation procedure. By combining TVD-derived weights with sample-based proportions, it captures both distributional alignment and data volume, helping to mitigate the common challenges of non-IID data. This leads to more robust convergence, particularly in heterogeneous settings. FedTVD’s ability to handle outliers and imbalanced data makes it well-suited for real-world applications where client data is naturally diverse. While our approach specifically targets label distribution skew, it does not directly address feature distribution skew, concept drift, or fairness across subgroups defined by non-label features. Tackling these more complex forms of heterogeneity would require significant methodological extensions and richer datasets. We view this as an important future direction, with potential to further improve fairness, robustness, and adaptability in large-scale federated systems.

Funding This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (No. NRF2022R1I1A3072355).

Data Availability The data used in this study are publicly available.

Declaration of Competing Interests The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. : Preprint submitted to Elsevier

Page 12 of 14

FedTVD

CRediT authorship contribution statement Radwan Selo: Conceptualization, Methodology, Visualization, and Writing – original draft. Majid Kundroo: Formal analysis, Validation, and Writing – review and editing. Taehong Kim: Resources, Validation, Supervision and Writing – review and editing.

References [1] Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency, 2017. URL https://arxiv.org/abs/1610.05492. [2] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: Concept and applications. ACM Trans. Intell. Syst. Technol., 10(2), January 2019. ISSN 2157-6904. doi: 10.1145/ 3298981. URL https://doi.org/10.1145/3298981. [3] Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. A review of applications in federated learning. Computers & Industrial Engineering, 149: 106854, 2020. ISSN 0360-8352. doi: https://doi.org/10.1016/j.cie. 2020.106854. URL https://www.sciencedirect.com/science/article/ pii/S0360835220305532. [4] Chen Zhang, Yu Xie, Hang Bai, Bin Yu, Weihong Li, and Yuan Gao. A survey on federated learning. Knowledge-Based Systems, 216:106775, 2021. ISSN 0950-7051. doi: https://doi.org/10.1016/j. knosys.2021.106775. URL https://www.sciencedirect.com/science/ article/pii/S0950705121000381. [5] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. Advances and open problems in federated learning. Foundations and trends® in machine learning, 14(1–2):1–210, 2021. [6] Han Xu, Priyadarsi Nanda, and Jie Liang. Reciprocal federated learning framework: Balancing incentives for model and data owners. Future Generation Computer Systems, 161:146–161, 2024. ISSN 0167739X. doi: https://doi.org/10.1016/j.future.2024.06.055. URL https: //www.sciencedirect.com/science/article/pii/S0167739X24003558. [7] Xuechen Chen, Aixiang Wang, Xiaoheng Deng, and Jinsong Gui. Latency-efficient wireless federated learning with sparsification and quantization for heterogeneous devices. IEEE Internet of Things Journal, 12(1):488–501, 2025. doi: 10.1109/JIOT.2024.3462722. [8] Abdul Wahab Mamond, Majid Kundroo, and Taehong Kim. Autoencoder-based decentralized federated learning for efficient communication. Computer Networks, 272:111676, 2025. ISSN 1389-1286. doi: https://doi.org/10.1016/j.comnet.2025.111676. URL https://www.sciencedirect.com/science/article/pii/ S1389128625006437. [9] Pham Khanh Quan, Majid Kundroo, and Taehong Kim. Experimental evaluation and analysis of federated learning in edge computing environments. IEEE Access, 11:33628–33639, 2023. doi: 10.1109/ ACCESS.2023.3262945. [10] Bernardo Barreto, Carlos Senna, Pedro Rito, and Susana Sargento. Mobfedls: A framework to provide federated learning for mobile nodes in v2x environments. Future Generation Computer Systems, 163:107514, 2025. ISSN 0167-739X. doi: https://doi.org/10.1016/ j.future.2024.107514. URL https://www.sciencedirect.com/science/ article/pii/S0167739X24004783. [11] Hemant Kumar Apat, Rashmiranjan Nayak, and Bibhudatta Sahoo. A comprehensive review on internet of things application placement in fog computing environment. Internet of Things, 23:100866, 2023. ISSN 2542-6605. doi: https://doi.org/10.1016/j.iot.2023. 100866. URL https://www.sciencedirect.com/science/article/pii/ S2542660523001890. [12] Hemant Kumar Apat, Veena Goswami, Bibhudatta Sahoo, Rabindra K. Barik, and Manob Jyoti Saikia. Fog service placement optimization: A survey of state-of-the-art strategies and techniques. Computers, 14(3), 2025. ISSN 2073-431X. doi: 10.3390/

: Preprint submitted to Elsevier

computers14030099. URL https://www.mdpi.com/2073-431X/14/3/99. [13] Fatih Ilhan, Gong Su, Qingyang Wang, and Ling Liu. Scalable federated learning with system heterogeneity. In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS), pages 1037–1040, 2023. doi: 10.1109/ICDCS57875.2023.00113. [14] Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3):50–60, 2020. doi: 10.1109/MSP. 2020.2975749. [15] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. Federated learning with non-iid data. NA, 2018. doi: 10.48550/ARXIV.1806.00582. URL https://arxiv.org/ abs/1806.00582. [16] Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin. Federated learning on non-iid data: A survey. Neurocomputing, 465:371–390, 2021. ISSN 0925-2312. doi: https://doi.org/10.1016/j.neucom.2021. 07.098. URL https://www.sciencedirect.com/science/article/pii/ S0925231221013254. [17] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273–1282. PMLR, 2017. [18] Yi Hu, Hanchi Ren, Chen Hu, Jingjing Deng, and Xianghua Xie. An element-wise weights aggregation method for federated learning. In 2023 IEEE International Conference on Data Mining Workshops (ICDMW), page 188–196. IEEE, December 2023. doi: 10.1109/icdmw60847.2023.00031. URL http://dx.doi.org/10.1109/ ICDMW60847.2023.00031. [19] Ellango Jothimurugesan, Kevin Hsieh, Jianyu Wang, Gauri Joshi, and Phillip B Gibbons. Federated learning under distributed concept drift. In International Conference on Artificial Intelligence and Statistics, pages 5834–5853. PMLR, 2023. [20] Leyla Rahimli, Feras M. Awaysheh, Sawsan Al Zubi, and Sadi Alawadi. Federated learning drift detection : An empirical study on the impact of concept and data drift. In 2024 2nd International Conference on Federated Learning Technologies and Applications, FLTA 2024 :, pages 241–250, 2024. ISBN 9798350354812. doi: 10.1109/FLTA63145.2024.10839814. [21] David A. Levin, Yuval Peres, and Elizabeth L. Wilmer. Markov Chains and Mixing Times. American Mathematical Society, Providence, RI, 2nd edition, 2017. ISBN 978-1-4704-2962-1. [22] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2: 429–450, 2020. [23] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor. Tackling the objective inconsistency problem in heterogeneous federated optimization. Advances in neural information processing systems, 33:7611–7623, 2020. [24] Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni. Federated learning with matched averaging, 2020. URL https://arxiv.org/abs/2002.06440. [25] Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. Fedbn: Federated learning on non-iid features via local batch normalization, 2021. URL https://arxiv.org/abs/2102.07623. [26] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference on machine learning, pages 5132–5143. PMLR, 2020. [27] Boyuan Li, Shengbo Chen, and Keping Yu. Feddkw – federated learning with dynamic kullback–leibler-divergence weight. ACM Trans. Asian Low-Resour. Lang. Inf. Process., April 2023. ISSN 2375-4699. doi: 10.1145/3594779. URL https://doi.org/10.1145/ 3594779. Just Accepted. [28] Gonzalo Iñaki Quintana, Laurence Vancamberg, Vincent Jugnon, Mathilde Mougeot, and Agnès Desolneux. Bn-scaffold: controlling the drift of batch normalization statistics in federated learning, 2024.

Page 13 of 14

FedTVD URL https://arxiv.org/abs/2410.03281. [29] Felix Sattler, Klaus-Robert Müller, and Wojciech Samek. Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8):3710–3722, 2021. doi: 10.1109/TNNLS. 2020.3015958. [30] Minghao Li, Dmitrii Avdiukhin, Rana Shahout, Nikita Ivkin, Vladimir Braverman, and Minlan Yu. Federated learning clients clustering with adaptation to data drifts, 2024. URL https://arxiv. org/abs/2411.01580. [31] Yuxiao Deng, Anqi Wang, Lei Zhang, Ying Lei, Beibei Li, and Yizhou Li. Fedrfc: Federated learning with recursive fuzzy clustering for improved non-iid data training. Future Generation Computer Systems, 160:835–843, 2024. ISSN 0167-739X. doi: https://doi.org/ 10.1016/j.future.2024.06.049. URL https://www.sciencedirect.com/ science/article/pii/S0167739X24003509. [32] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, 34(12):9587–9603, 2023. doi: 10.1109/ TNNLS.2022.3160699. [33] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(9):11237–11244, Jun. 2023. doi: 10. 1609/aaai.v37i9.26330. URL https://ojs.aaai.org/index.php/AAAI/ article/view/26330. [34] Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H. Brendan McMahan. Adaptive federated optimization, 2021. URL https://arxiv.org/ abs/2003.00295. [35] Abdul Wahab Mamond, Majid Kundroo, Seong-eun Yoo, Seonghoon Kim, and Taehong Kim. Fldqn: Cooperative multi-agent federated reinforcement learning for solving travel time minimization problems in dynamic environments using sumo simulation. Sensors, 25(3), 2025. ISSN 1424-8220. doi: 10.3390/s25030911. URL https: //www.mdpi.com/1424-8220/25/3/911. [36] Hemant Kumar Apat and Bibhudatta Sahoo. Jagw: A hybrid metaheuristic algorithm for iot workflow placement in fog computing environment. Simulation Modelling Practice and Theory, 144:103163, 2025. ISSN 1569-190X. doi: https://doi.org/10.1016/j.simpat.2025. 103163. URL https://www.sciencedirect.com/science/article/pii/ S1569190X2500098X. [37] Hemant Kumar Apat, Pranati Sattarapu, Rudra Narayan Dash, Veena Goswami, Sagarika Mohanty, and Rabindra K. Barik. Leveraging towards multi-objective iot application placement in fog computing environment. In 2023 IEEE 3rd International Conference on Smart Technologies for Power, Energy and Control (STPEC), pages 1–6, Dec 2023. doi: 10.1109/STPEC59253.2023.10430628. [38] Majid Kundroo and Taehong Kim. Efficient federated learning with adaptive client-side hyper-parameter optimization. In 2023 IEEE 43rd International Conference on Distributed Computing Systems (ICDCS), pages 973–974, 2023. doi: 10.1109/ICDCS57875.2023. 00103. [39] Majid Kundroo and Taehong Kim. Demystifying impact of key hyper-parameters in federated learning: A case study on cifar-10 and fashionmnist. IEEE Access, 12:120570–120583, 2024. doi: 10.1109/ACCESS.2024.3450894. [40] Kummari Naveen Kumar, Chalavadi Krishna Mohan, and Linga Reddy Cenkeramaddi. The impact of adversarial attacks on federated learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2672–2691, 2024. doi: 10.1109/TPAMI.2023.3322785. [41] Majid Kundroo, Ghani Haider, Nguyen Khoa, Abdul Wahab Mamond, and Taehong Kim. Fedeasy : Federated learning with ease. SoftwareX, 31:102276, 2025. ISSN 2352-7110. doi: https://doi.org/ 10.1016/j.softx.2025.102276. URL https://www.sciencedirect.com/ science/article/pii/S2352711025002432.

: Preprint submitted to Elsevier

[42] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747, 2017. [43] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009. [44] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998. [45] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778, 2016. [46] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification, 2019. URL https://arxiv.org/abs/1909.06335.

Page 14 of 14

Related documents

• FedNIA: Noise-Induced Activation Analysis for Mitigating Data Poisoning in Federated Learning#1061591
• A Resilient Genetic-centric Aggregation for Hierarchical Federated Learning against Scaling Attacks#280393
• Totoro$^+$: An Adaptive and Scalable Edge Federated Learning System#229472
• Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labels#189226
• Towards Modality-imbalanced Federated Graph Learning: A Data Synthesis-based Approach#290587