TILBench: A Systematic Benchmark for Tabular Imbalanced Learning Across Data Regimes Ruizhe Liua , Jiaqi Luoa,∗
arXiv:2605.14915v1 [cs.LG] 14 May 2026
a
School of Mathematical Sciences, Soochow University, No.1 Shizi Street, Suzhou, 215006, Jiangsu Province, China
Abstract Imbalanced learning remains a fundamental challenge in tabular data applications. Despite decades of research and numerous proposed algorithms, a systematic empirical understanding of how different imbalanced learning methods behave across diverse data characteristics is still lacking. In particular, it remains unclear how different method families compare in predictive performance, robustness under varying data characteristics, and computational scalability. In this work, we present Tabular Imbalanced Learning Benchmark (TILBench), a large-scale empirical benchmark for tabular imbalanced learning. TILBench evaluates more than 40 representative algorithms across 57 diverse tabular datasets, resulting in over 200000 controlled experiments across a wide range of data characteristics. Our findings show that no single method consistently dominates across all settings; instead, the effectiveness of imbalanced learning methods depends strongly on dataset characteristics and computational constraints. Based on these findings, we provide practical recommendations for selecting appropriate methods in realworld applications. Keywords: Tabular data, Imbalanced Learning, Benchmark 1. Introduction Imbalanced learning is a fundamental challenge in tabular machine learning [1, 2, 3, 4], where minority cases are rare but often critical. Such settings ∗
Corresponding author Email addresses: [email protected] (Ruizhe Liu), [email protected] (Jiaqi Luo)
Tabular Imbalanced Learning Benchmark
A Systematic Empirical Benchmark for Tabular Imbalanced Learning BENCHMARK SCOPE 40+ Algorithms
Performance Comparison · Individual ranking · Family-level aggregation
PRACTICAL RECOMMENDATION Method Selection Guide Data-level methods:
Data-level
Algorithm-level
Ensemble
57 Datasets · Binary classification tasks · Multi-class classification tasks · Diverse sizes, feature dimensions, imbalance ratios, and missing values
Performance Across Dataset Characteristics · Sample size · Feature dimensionality · Imbalance severity · Missing values Scalability and Computational Efficiency · Training time · Sample size scaling · Feature dimensionality scaling · Class number scaling
· severe imbalance · multi-class task
Algorithm-level methods: · large-scale · high dimensionality · missing values
Ensemble-based methods: · small-scale · binary task
Trade-offs · Predictive performance · Computational cost · Scalability and robustness
Figure 1: Overview of TILBench. The benchmark evaluates more than 40 representative imbalanced learning algorithms across 57 tabular datasets. It provides a structured analysis of overall predictive performance, behavior across dataset characteristics, and computational scalability, leading to regime-aware recommendations for method selection.
are common in real-world applications, including fraud detection [5], medical diagnosis [6], and fault diagnosis [7]. In these scenarios, standard learning algorithms tend to favor the majority class, resulting in poor recognition of minority instances that often carry greater practical importance. To address this issue, a wide range of imbalanced learning techniques have been developed, including data-level methods [8, 9, 10], algorithm-level approaches [11, 12, 13], and ensemble-based strategies [14, 15, 16]. Although these methods can be effective in specific scenarios, their relative behavior across diverse tabular data conditions remains insufficiently understood. Existing empirical studies provide valuable comparisons, but are often limited in scale and scope [17, 18, 19]. Many studies consider a restricted set of algorithms or datasets, focus primarily on average predictive performance, and provide limited analysis across data regimes or computational constraints. As a result, there remains a lack of systematic guidance for 2
selecting appropriate imbalanced learning methods under different tabular data conditions. In this work, we present TILBench, a large-scale empirical benchmark for imbalanced learning on tabular data. TILBench evaluates over 40 representative algorithms across 57 diverse datasets under a unified and reproducible evaluation framework, resulting in more than 200000 controlled experiments. The benchmark covers a broad range of data characteristics, including sample size, feature dimensionality, imbalance severity, missing values, and class number. Unlike existing studies that primarily focus on overall predictive performance, TILBench provides a structured evaluation of imbalanced learning methods from three complementary perspectives: performance comparison, behavior across data regimes, and computational scalability under increasing data complexity. Rather than seeking a single universally best method, our goal is to develop a regime-aware understanding of when different method families are effective and where they face performance or scalability limitations. This analysis further supports guidance for method selection in practical tabular imbalanced learning. The main contributions of this work are summarized as follows: • We establish a unified large-scale benchmark for imbalanced learning on tabular data, covering more than 40 representative methods and 57 datasets under consistent evaluation protocols. • We conduct a comprehensive comparison of different imbalanced learning families, including data-level, algorithm-level, and ensemble-based methods, in terms of both individual algorithm performance and familylevel predictive performance. • We systematically analyze how imbalanced learning methods behave across different data characteristics, including sample size, feature dimensionality, imbalance severity, and missing values. • We investigate the computational scalability and efficiency of different methods under increasing sample size, feature dimensionality, and class number, characterizing how computational cost changes with data scale and complexity. • Based on these findings, we derive practical guidelines for method selection, demonstrating that effective strategies depend on both data 3
properties and system constraints rather than a single universally optimal approach. 2. Preliminaries 2.1. Imbalanced Learning for Tabular Data Existing methods can be broadly categorized into three families based on how imbalance is handled during learning: data-level methods, which modify the training data distribution; algorithm-level methods, which incorporate imbalance awareness into the learning process; and ensemblebased methods, which integrate imbalance-handling strategies within ensemble frameworks. An overview of this taxonomy is illustrated in Fig. 2.
Figure 2: Method Categorization
2.1.1. Data-level Methods Data-level methods, also known as resampling methods, address class imbalance by modifying the training data distribution prior to model learning while leaving the underlying classifier unchanged. As external preprocessing techniques, they are model-agnostic and can be applied across a wide range of learning algorithms. The core idea is to rebalance class distributions by adjusting the composition of the training data, thereby reducing the bias toward majority classes. 4
These methods are typically categorized into undersampling [8, 20, 21, 22], oversampling [9, 10, 23], and hybrid approaches [24, 25]. Oversampling increases the representation of minority classes by duplicating existing samples or generating synthetic instances, which can improve minority recall but may introduce redundancy and increase the risk of overfitting. Undersampling reduces class imbalance by removing majority class samples, but aggressive removal may discard informative data and degrade decision boundaries. Hybrid methods combine both strategies to balance minority enrichment and information preservation, at the cost of increased complexity and parameter sensitivity. 2.1.2. Algorithm-level Methods Algorithm-level methods address class imbalance by modifying the learning process itself rather than altering the input data. These approaches incorporate imbalance awareness directly into model optimization or decision mechanisms, enabling the learner to account for skewed class distributions during training. They can be broadly categorized into cost-sensitive learning [12, 13, 26], learner modification [11, 27, 28], and one-class classification [29]. Costsensitive learning introduces class-dependent weights into the loss function, effectively rebalancing sample contributions during optimization while preserving the original data distribution. Learner modification methods adjust the internal structure or training rules of models, such as decision criteria or regularization mechanisms, to mitigate majority bias, although they are often model-specific. One-class classification focuses on modeling the minority class as the target distribution and treats other samples as outliers, which is suitable for extreme imbalance but may struggle when class distributions overlap. 2.1.3. Ensemble-based Methods Ensemble-based methods address class imbalance by integrating imbalanceaware strategies within ensemble learning frameworks. By combining multiple base learners, these methods exploit both sample diversity and model diversity to improve robustness under skewed distributions. They can be broadly divided into data-level ensembles and algorithm-level ensembles. Data-level ensembles construct multiple balanced subsets via resampling and train separate models on each subset, increasing diversity and
5
improving minority coverage, but at the cost of higher computational overhead [14, 30, 31, 32]. Algorithm-level ensembles incorporate imbalance-aware mechanisms, such as cost-sensitive weighting [15, 16, 33], directly into ensemble training, often achieving strong performance while introducing additional complexity. 2.1.4. Unified Perspective and Key Differences Despite their apparent differences, existing imbalanced learning methods can be understood from a unified perspective based on where and how class imbalance is addressed during model learning. Data-level methods modify the empirical data distribution and indirectly affect learning through resampled training data. Algorithm-level methods incorporate imbalance awareness into the learning process by modifying the objective function, decision rule, or learner structure. Ensemble-based methods address imbalance through model aggregation, improving robustness by combining multiple imbalance-aware base learners. From this viewpoint, the key distinction lies in whether imbalance is handled at the level of data distribution, learning process, or model aggregation. These different mechanisms lead to distinct advantages and limitations, as summarized in Table 1. The table provides a compact comparison of the three method families in terms of their operating mechanism, main operation, primary advantage, and potential limitation. These differences suggest that each method family addresses imbalance through a distinct form of intervention. Data-level methods are flexible and model-agnostic, but may distort the original data distribution. Algorithmlevel methods provide more direct control over learning bias, but can be model-dependent. Ensemble-based methods improve robustness through aggregation, but often incur higher computational cost. Therefore, these families are expected to exhibit different performance in terms of stability, robustness, and computational cost. This mechanism-level distinction provides the conceptual basis for the empirical comparisons in the following sections. 2.2. Existing Surveys and Benchmarks Imbalanced learning has been extensively studied over the past decades, and several comprehensive surveys have summarized the development of this field. Foundational surveys and books such as [1, 2, 3] provide systematic overviews of classical techniques, including resampling, learning process, and 6
Table 1: Key differences among imbalanced learning method families. Aspect
Data-level
Algorithm-level
Ensemble-based
Mechanism Operation Advantage Limitation
Data distribution Resampling Flexibility Distribution distortion
Learning process Algorithm modification Bias control Model dependence
Model aggregation Integrated strategies Robustness High cost
ensemble methods. More specialized surveys focus on particular subcategories. For example, [34] focuses on data-level methods, [35] and [36] study cost-sensitive learning, and [37] provides an in-depth review of SVM-based approaches for imbalanced classification. Beyond surveys, several studies have conducted empirical comparisons of imbalanced learning methods. Some works focus on specific methodological categories. For example, [17] focuses on data augmentation and ensemble strategies, [18] studies oversampling methods, and [11] investigates loss modification approaches. Other studies, such as [38] and [19], provide broader evaluations, but still consider only a subset of method families, primarily cost-sensitive learning, resampling, and ensemble methods, without covering learner modification techniques. In addition, several benchmarks focus on specific application domains, such as business [39], finance [40], and education [41]. However, these studies are not designed as comprehensive benchmarks for imbalanced learning in tabular data. Despite these efforts, existing benchmarks remain limited in several important aspects. First, many studies consider only a restricted set of algorithms or datasets, which limits the generality of their conclusions. Second, most benchmarks focus primarily on predictive performance and do not systematically analyze how methods behave across different data regimes. Third, practical factors such as computational cost and scalability are often overlooked. These limitations motivate the empirical study presented in this paper. 3. Benchmark Design 3.1. Scope and Objectives We conduct a large-scale empirical study to systematically evaluate imbalanced learning methods for tabular data. Our benchmark is designed to address the following key research questions (RQ): 7
• RQ1: How do different families of imbalanced learning methods compare in overall predictive performance? • RQ2: How does the effectiveness of imbalanced learning methods vary across different data characteristics, including sample size, feature dimensionality, imbalance severity, and missing values? • RQ3: How do different imbalanced learning methods compare in computational scalability and efficiency under increasing data scale and complexity? To answer these questions, we design a controlled evaluation pipeline that enables fair comparison across diverse datasets and methods. 3.2. Datasets Our evaluation spans 57 tabular datasets, including 34 binary classification tasks and 23 multi-class classification tasks. Among them, 7 datasets contain natural missing values. The datasets are collected from OpenML [42] and imbalanced-learn1 , covering a wide range of domains and diverse characteristics in terms of sample size, feature dimensionality, and imbalance severity. Detailed dataset statistics are summarized in Table A.12 in Appendix A. 3.3. Baseline Models We evaluate more than 40 representative imbalanced learning methods spanning three major methodological families introduced in Section 2: datalevel methods, algorithm-level methods, and ensemble-based methods. We choose XGBoost [43] as the reference baseline because Gradient Boosting Decision Trees (GBDTs) consistently demonstrate strong performance on tabular data [44, 45, 46]. For fair comparison, XGBoost is also adopted as the base classifier for all data-level resampling methods. Data-level methods. We consider three categories of resampling approaches. Under-sampling methods include TomekLinks [20], EditedNearestNeighbors [21], NeighborhoodCleaningRule [22], InstanceHardnessThreshold [8], ClusterCentroids [47], CondensedNearestNeighbor [48], AllKNN [49], NearMiss [50], and One-SidedSelection [51]. 1
https://imbalanced-learn.org/stable/index.html
8
Over-sampling methods include SMOTE [9], BorderlineSMOTE [10], PolyfitSMOTE [23], SMOTEIPF [52], Lee [53], and SMOBD [54]. Hybrid methods include SMOTEENN [24] and SMOTETomek [25]. Algorithm-level methods. We evaluate both cost-sensitive learning and learner modification approaches. Cost-sensitive methods include XGBoostCost, LogisticRegressionCost, DecisionTreeCost, and RandomForestCost. Learner modification methods are implemented by combining XGBoost with different imbalance-aware loss functions, including Asymmetric Loss (XGBoostASL) [55], Asymmetric Cross-Entropy (XGBoostACE) [11], Asymmetric Weighted Cross-Entropy (XGBoostAWE) [11], Focal Loss (XGBoostFL) [56], Weighted Cross-Entropy (XGBoostWCE) [57], and ClassBalanced Cross-Entropy (XGBoostCBE) [58]. Ensemble-based methods. We consider both data-level and algorithm-level ensemble approaches. Data-level ensemble methods include SelfPacedEnsemble [14], BalanceCascadeEnsemble [30], BalancedRandomForest [31], EasyEnsemble [30], RUSBoost [59], UnderBagging [60], OverBoost [32], SMOTEBoost [32], OverBagging [60], and SMOTEBagging [61]. Algorithm-level ensemble methods include AdaCost [33], AdaUBoost [15], and AsymBoost [16]. Abbreviation. For readability in figures and tables, abbreviations of all methods are summarized in Table A.13 in Appendix A. 3.4. Evaluation Protocol To ensure fair and reproducible evaluation, we adopt a standardized protocol across all datasets and methods. Each dataset is randomly split into 80% training and 20% testing sets using stratified sampling, with 10% of the training data reserved for validation. Hyperparameters are tuned using Optuna [62] with 20 trials per method. Models are then retrained on the full training set and evaluated on the test set. The entire process is repeated five times with different random seeds, and we report the mean and standard deviation of all metrics, including F1-score and G-mean. All resampling techniques are applied only to the training data to avoid data leakage. Experiments are conducted on a workstation with an Intel Core 9
i9-14900HX CPU and 48 GB RAM. Hyperparameter settings and implementation details are provided in Table A.14 and Table A.15 in Appendix A. 4. Results and Analyses 4.1. Overall Performance Comparison We evaluate the overall effectiveness of different imbalanced learning methods from two perspectives: global ranking of individual algorithms and family-level aggregation. 4.1.1. Global Ranking We first analyze the performance of individual methods across all datasets. Table 2 reports the top-performing algorithms in terms of F1-score and Gmean. Table 2: Top 10 performing methods ranked by F1-score and G-mean for binary and multi-class tasks. Rank
Method
F1-score
Rank
Method
G-mean
UnderBagging BalancedRandomForest SMOTEENN BalanceCascadeEnsemble AdaUCost EasyEnsemble XGBoostCost SelfPacedEnsemble SMOTEIPF SMOTETomek
89.32 ± 2.12 88.34 ± 2.38 87.27 ± 3.21 87.20 ± 2.72 86.74 ± 3.43 86.70 ± 2.40 86.22 ± 3.39 85.97 ± 3.14 85.00 ± 3.58 84.92 ± 3.99
XGBoostCost SMOTE SMOTETomek BorderlineSMOTE Lee Tomek SMOTEIPF XGBoostFL XGBoostCBE XGBoostWCE
83.91 ± 1.16 83.63 ± 1.00 83.44 ± 0.94 83.23 ± 1.26 83.08 ± 1.04 82.60 ± 1.29 82.49 ± 1.11 82.15 ± 1.29 82.15 ± 1.19 82.12 ± 0.95
Binary 1 2 3 4 5 6 7 8 9 10
SelfPacedEnsemble XGBoostWCE XGBoostASL XGBoostAWE BalanceCascadeEnsemble XGBoostACE SMOTEBagging XGBoostLee XGBoostFL XGBoostCost
73.88 ± 4.01 72.96 ± 4.91 72.62 ± 5.48 72.54 ± 4.57 72.51 ± 4.55 72.07 ± 6.16 70.93 ± 4.73 70.53 ± 5.97 70.14 ± 5.84 69.90 ± 4.90
1 2 3 4 5 6 7 8 9 10
SMOTE XGBoostCost SMOTETomek BorderlineSMOTE Lee Tomek XGBoostFL XGBoostCBE XGBoost XGBoostWCE
75.84 ± 1.45 75.83 ± 1.57 75.49 ± 1.45 75.49 ± 1.71 75.47 ± 1.52 75.02 ± 1.67 74.69 ± 1.76 74.69 ± 1.64 74.63 ± 1.61 74.61 ± 1.31
1 2 3 4 5 6 7 8 9 10
Multi-class
10
1 2 3 4 5 6 7 8 9 10
In the binary setting, SelfPacedEnsemble achieves the highest F1-score of 73.88, while UnderBagging attains the best G-mean of 89.32. In the multiclass setting, SMOTE and XGBoostCost rank first in F1-score and G-mean, with scores of 75.84 and 83.91, respectively. A notable observation is that many top-performing methods are based on XGBoost or its imbalance-aware variants, highlighting the strong effectiveness of gradient boosting combined with imbalance handling strategies. In addition, several resampling-based methods such as SMOTE and SMOTETomek remain highly competitive, particularly in the multi-class setting. For clarity, we report only the top 10 methods in Table 2. Complete results are provided in Table B.16 and Table B.16 in Appendix B. 4.1.2. Family-level Aggregation To obtain a broader understanding of method behavior, we further analyze performance at the family-level by grouping methods into data-level, algorithm-level, and ensemble-based approaches. The results are summarized in Table 3. For binary classification, algorithm-level methods achieve the best overall performance, obtaining the highest average F1-score and G-mean together with the lowest standard deviation. This indicates that directly incorporating imbalance awareness into the model learning generally leads to more stable and effective performance improvements. Ensemble-based methods rank second and include several highly competitive algorithms such as SelfPacedEnsemble and BalanceCascadeEnsemble, although their performance exhibits larger variability across datasets. In contrast, data-level methods achieve the weakest overall performance and show substantially larger variance within the family. In the multi-class setting, algorithm-level methods again achieve the best average F1-score, while data-level methods obtain a slightly higher average G-mean. However, the performance gap between these two families is relatively small. Notably, the baseline XGBoost already achieves strong performance, outperforming many imbalance-aware methods. This suggests that in multi-class tabular tasks, imbalance handling techniques often provide only marginal improvements over a strong baseline learner. Ensemble methods show the weakest overall performance in this setting, indicating limited scalability of ensemble-based imbalance handling for multi-class problems.
11
Table 3: Family-level performance comparison across binary and multi-class tasks in terms of F1-score and G-mean. We report average performance, standard deviation, and extreme values (max/min) within each family.
XGBoost Data-level Alg.-level Ensemble F1-score
Binary
Rank Avg. Std. Max Min
– 60.92 – 60.92 60.92
3 60.78 9.84 70.53 36.71
1 68.43 5.75 72.96 55.30
2 63.95 6.89 73.88 50.79
Multi-class
Rank Avg. Std. Max Min
– 74.63 – 74.63 74.63
2 70.51 5.67 75.84 58.47
1 71.23 5.00 75.83 60.11
3 63.61 7.36 73.11 50.44
G-mean
Binary
Rank Avg. Std. Max Min
– 67.17 – 67.17 67.17
3 77.04 6.94 87.27 65.95
1 82.52 3.15 86.22 77.05
2 82.15 6.29 89.32 71.41
Multi-class
Rank Avg. Std. Max Min
– 82.09 – 82.09 82.09
1 80.57 2.69 83.63 75.78
2 80.46 2.78 83.91 74.12
3 76.08 5.08 81.92 66.77
Summary for RQ1: Overall Performance Comparison Algorithm-level methods achieve the best overall performance and the most stable behavior across datasets. Ensemble-based methods are competitive in binary classification but exhibit larger variability, while data-level methods become more effective in multi-class settings. Overall, directly incorporating imbalance awareness into the learning process is generally more effective than modifying the data distribution alone. 12
4.2. Performance across Dataset Characteristics We evaluate how the performance of imbalanced learning methods varies across different dataset characteristics. Specifically, datasets are grouped according to sample size, feature dimensionality, imbalance ratio, and missing values, and performance is analyzed within each group. In the first three subsections, datasets containing missing values are excluded. 4.2.1. Sample Size Effects Sample size has a significant influence on the effectiveness of imbalanced learning methods. We group datasets into three regimes, less than 1k, 1k– 10k, and greater than 10k, and analyze performance at both the family level as shown in Fig. 3 and the individual method level as shown in Table 4. Here we report only the F1-score results; the corresponding G-mean results are provided in Fig. B.9 and Table B.17 in Appendix B.
Figure 3: Family-level F1-scores across sample size regimes. Each box shows the distribution of method performance within a family for each dataset size category. Results are reported separately for binary and multi-class tasks.
For binary classification, Fig. 3 shows that ensemble-based methods perform best on small datasets, with higher median performance and wider distributions, indicating strong but less stable behavior. As sample size increases, algorithm-level methods gradually dominate and achieve the best performance in large datasets. Data-level methods also show a clear improvement trend and gradually outperform ensemble-based methods. The individual rankings in Table 4 are consistent with these observations. For datasets smaller than 1k, ensemble methods such as SelfPacedEnsemble and 13
Table 4: Top five methods in each sample size regime ranked by F1-score for binary and multi-class tasks. Sample Size
<1k
1k–10k
>10k
Rank
Method
F1-score
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Binary AdaUCost BalanceCascadeEnsemble SelfPacedEnsemble SMOTEBoost SMOTEBagging SelfPacedEnsemble BalanceCascadeEnsemble XGBoostWCE XGBoostAWE XGBoostASL XGBoostASL XGBoostWCE XGBoostAWE XGBoostACE SelfPacedEnsemble
72.62 ± 8.65 72.48 ± 8.61 71.92 ± 9.23 71.49 ± 10.55 70.18 ± 9.68 74.13 ± 3.56 73.76 ± 3.47 73.14 ± 4.74 72.67 ± 4.49 72.34 ± 5.58 75.67 ± 1.86 75.15 ± 2.06 75.11 ± 1.71 74.93 ± 1.88 73.59 ± 1.51
Method
F1-score
Multi-class PolyfitSMOTE 88.59 ± 2.61 XGBoostCost 88.16 ± 2.96 SMOTEIPF 87.92 ± 2.80 SMOTE 87.76 ± 2.78 Lee 87.57 ± 3.16 XGBoostCost 74.05 ± 1.61 SMOTE 74.00 ± 1.38 SMOTEIPF 73.93 ± 1.43 Lee 73.91 ± 1.33 BorderlineSMOTE 73.81 ± 1.29 XGBoostCost 69.65 ± 0.59 SMOTE 68.66 ± 0.45 SMOTETomek 68.63 ± 0.39 BorderlineSMOTE 68.54 ± 0.49 XGBoostFL 68.08 ± 0.62
BalanceCascadeEnsemble rank among the top performers. In the medium regime, both ensemble-based and algorithm-level methods occupy top positions. For datasets larger than 10k, algorithm-level methods, particularly XGBoost-based variants, dominate the rankings, while ensemble methods become less competitive. Pure data-level methods appear less frequently among the top-performing methods in binary classification. For multi-class tasks, a different pattern is observed. Fig. 3 shows that data-level and algorithm-level methods achieve comparable performance across all sample size regimes, both outperforming ensemble-based methods. Datalevel methods exhibit particularly strong performance on small datasets, suggesting that resampling strategies are more effective in multi-class scenarios. Consistent trends are observed in Table 4, where XGBoostCost and SMOTEbased methods frequently rank among the top-performing approaches across all regimes. Unlike the binary setting, data-level methods appear regularly among the top ranks, while ensemble-based methods rarely do. Overall, sample size plays an important role in determining method effectiveness. Ensemble-based methods are more suitable for small datasets, while algorithm-level methods dominate in large-scale settings. Data-level 14
methods become increasingly competitive in multi-class tasks and moderateto-large datasets, but remain less consistent in binary classification. 4.2.2. Feature Dimensionality Feature dimensionality influences different method families in distinct ways. We group datasets into three regimes, less than 10, 10–50, and greater than 50 features, and analyze performance at both the family level as shown in Fig. 4 and the individual method level as shown in Table 5. Here we report only the F1-score results; the corresponding G-mean results are provided in Fig. B.10 and Table B.18 in Appendix B.
Figure 4: Family-level F1-scores across feature dimensionality regimes. Each box shows the distribution of method performance within a family for each feature group. Results are reported separately for binary and multi-class tasks.
For binary classification, Fig. 4 shows that algorithm-level methods consistently achieve the highest median performance across all feature regimes. Their advantage becomes more pronounced as dimensionality increases. Ensemble methods also perform well and frequently appear among the topperforming methods, although their overall distributions are less stable. In contrast, data-level methods exhibit large variability, including several severe outliers, indicating sensitivity to high-dimensional feature spaces. The rankings in Table 5 further support these observations. Across all feature regimes, the top-performing methods are dominated by ensemble-based and algorithm-level approaches, including SelfPacedEnsemble and multiple XGBoostbased variants, while data-level methods appear less frequently among the leading positions. 15
Table 5: Top five methods in each feature dimensionality regime ranked by F1-score for binary and multi-class tasks. Feature Dimension
<10
10–50
>50
Rank
Method
F1-score
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Binary SelfPacedEnsemble SMOTEBagging XGBoostASL XGBoostWCE Lee SelfPacedEnsemble BalanceCascadeEnsemble XGBoostWCE XGBoostAWE XGBoostACE BalanceCascadeEnsemble SelfPacedEnsemble XGBoostWCE XGBoostASL XGBoostAWE
65.67 ± 6.14 64.61 ± 6.65 63.50 ± 8.07 63.46 ± 7.83 62.70 ± 8.13 72.41 ± 3.11 71.89 ± 3.27 71.80 ± 4.15 71.63 ± 3.27 71.28 ± 5.40 79.01 ± 5.46 78.95 ± 5.85 77.34 ± 6.13 77.23 ± 7.28 76.60 ± 7.05
Method
F1-score
Multi-class XGBoostCost 75.38 ± 2.68 SMOTE 75.11 ± 2.28 SMOTEIPF 75.02 ± 2.33 Tomek 74.67 ± 2.83 Lee 74.52 ± 2.36 XGBoostCost 76.49 ± 1.35 BorderlineSMOTE 75.90 ± 0.96 Lee 75.85 ± 1.26 SMOBD 75.69 ± 1.11 SMOTEIPF 75.68 ± 1.23 SMOTE 77.15 ± 0.60 XGBoostCost 77.11 ± 0.66 SMOTETomek 76.94 ± 0.58 BorderlineSMOTE 76.92 ± 0.64 SMOBD 76.43 ± 0.55
For multi-class tasks, a different pattern emerges. Fig. 4 shows that datalevel and algorithm-level methods achieve comparable performance across all feature regimes, both outperforming ensemble-based methods. The performance gap between families increases as dimensionality grows, with ensemble methods showing the largest degradation. Table 5 shows that XGBoostCost and SMOTE-based methods consistently rank among the top-performing approaches across all feature regimes. Both data-level and algorithm-level methods maintain strong performance, although each family still contains several underperforming methods. Ensemble methods rarely appear among the top-performing methods and exhibit weaker robustness in high-dimensional settings. Overall, increasing feature dimensionality does not substantially alter the relative competitiveness between algorithm-level and data-level methods, but it consistently weakens the performance of ensemble-based approaches and increases variability within data-level methods. 4.2.3. Imbalance Severity Imbalance severity strongly influences the behavior of imbalanced learning methods. We group datasets into three regimes, less than 10, 10–50, and 16
greater than 50, and analyze performance at both the family level as shown in Fig. 5 and the individual method level as shown in Table 6. Here we report only the F1-score results; the corresponding G-mean results are provided in Fig. B.11 and Table B.19 in Appendix B.
Figure 5: Family-level F1-scores across imbalance severity regimes. Each box shows the distribution of method performance within a family for each imbalance group. Results are reported separately for binary and multi-class tasks.
Table 6: Top five methods in each imbalance severity regime ranked by F1-score for binary and multi-class tasks. Imbalance Ratio
<10
10–50
>50
Rank
Method
F1-score
Method
F1-score
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Binary SelfPacedEnsemble XGBoostASL XGBoostAWE XGBoostWCE BalanceCascadeEnsemble SelfPacedEnsemble BalanceCascadeEnsemble XGBoostASL XGBoostWCE SMOTEBagging SelfPacedEnsemble XGBoostAWE XGBoostFL XGBoostCBE XGBoostWCE
74.65 ± 2.68 74.61 ± 2.68 74.50 ± 2.53 74.43 ± 3.19 74.29 ± 2.13 71.13 ± 6.31 69.59 ± 8.75 69.11 ± 9.63 69.11 ± 8.08 67.28 ± 7.25 75.47 ± 5.13 74.64 ± 4.90 74.18 ± 4.14 73.61 ± 5.28 73.48 ± 5.08
Multi-class XGBoostCost SMOTE Lee SMOTETomek PolyfitSMOTE XGBoostCost SMOTE BorderlineSMOTE PolyfitSMOTE SMOTEIPF EditedNearestNeighbours XGBoost OneSidedSelection XGBoostFL XGBoostCBE
76.14 ± 1.70 75.75 ± 1.56 75.59 ± 1.62 75.49 ± 1.56 75.39 ± 1.45 70.89 ± 1.26 70.04 ± 0.85 69.93 ± 0.68 69.56 ± 0.54 68.89 ± 1.98 90.63 ± 0.66 90.27 ± 0.62 89.74 ± 1.29 89.64 ± 1.39 89.27 ± 1.09
17
For binary classification, Fig. 5 shows that algorithm-level methods consistently achieve the highest median performance across all imbalance regimes, with relatively narrow distributions indicating strong robustness. Ensemblebased methods also perform competitively, with SelfPacedEnsemble consistently ranking at the top across all regimes. In contrast, data-level methods underperform when the imbalance ratio is below 50, but become substantially more competitive in the high imbalance regime, where their median performance surpasses most ensemble-based methods. Nevertheless, data-level methods still exhibit large variability and contain several lowperforming cases. The individual rankings in Table 6 confirm these observations. SelfPacedEnsemble consistently ranks first across all binary imbalance regimes, followed by several XGBoost-based algorithm-level methods. Datalevel methods rarely appear among the top performers in low and moderate imbalance regimes, but become increasingly competitive when the imbalance ratio is high. For multi-class tasks, a different pattern is observed. Fig. 5 shows that algorithm-level and data-level methods achieve comparable performance across all imbalance regimes, both maintaining relatively high median performance. Ensemble-based methods, however, show reduced stability as imbalance severity increases, with wider distributions and lower medians. Consistent trends are observed in Table 6, where XGBoostCost and SMOTE-based methods frequently rank among the top-performing approaches in low and moderate imbalance regimes. In the high imbalance regime, several data-level methods such as ENN outperform the baseline, indicating strong effectiveness in extreme cases. At the same time, the baseline XGBoost model already achieves very high performance in this regime, suggesting that conclusions in the extreme imbalance setting may be influenced by the limited number of highly imbalanced datasets. Overall, increasing imbalance severity does not substantially change the relative advantage of algorithm-level methods in binary tasks, but improves the competitiveness of data-level methods in extreme cases. In multi-class settings, data-level and algorithm-level methods remain consistently strong, while ensemble-based methods exhibit reduced robustness as imbalance severity increases. 4.2.4. Missing Value Robustness We evaluate the compatibility of imbalanced learning methods with missing values based on their native compatibility and empirical performance. 18
The results for methods supporting missing values are summarized in Table 7. Table 7: Performance of methods that support missing values. We report F1-scores for both binary and multi-class tasks. Methods without native support are excluded.
Method
Binary
Multi-class
Algorithm-level XGBoostASL XGBoostACE XGBoostAWE XGBoostFL XGBoostWCE XGBoostCBE XGBoostCost DecisionTreeCost RandomForestCost
72.62 ± 5.48 72.07 ± 6.16 72.54 ± 4.57 70.14 ± 5.84 72.96 ± 4.91 69.58 ± 5.14 69.90 ± 4.90 61.35 ± 4.82 67.83 ± 4.83
72.55 ± 3.25 72.65 ± 2.76 71.66 ± 3.15 74.69 ± 1.76 74.61 ± 1.31 74.69 ± 1.64 75.83 ± 1.57 64.90 ± 2.04 70.65 ± 1.64
Ensemble SelfPacedEnsemble BalanceCascadeEnsemble UnderBagging OverBagging
73.88 ± 4.01 72.51 ± 4.55 61.55 ± 3.53 67.86 ± 4.70
70.45 ± 1.45 69.28 ± 1.47 71.10 ± 1.13 72.14 ± 1.32
Data-level methods generally do not support missing values because they rely heavily on operations such as distance computation and neighborhood search. For example, SMOTE and its variants require complete data for interpolation. As a result, preprocessing procedures such as mean imputation are typically required before applying data-level methods. Algorithm-level methods show significantly better compatibility with missing values. In particular, XGBoost-based methods benefit from sparsityaware split finding, allowing them to process incomplete data without explicit imputation. As shown in Table 7, these methods achieve strong and stable performance in both binary and multi-class settings, with XGBoostCost and XGBoostWCE ranking among the top-performing approaches. In contrast, simpler methods such as DecisionTreeCost and RandomForestCost remain compatible but exhibit weaker predictive performance. The only exception is LogRegCost, which lacks inherent mechanisms for handling missing values due to its linear formulation. 19
Ensemble-based methods exhibit mixed behavior. Some methods are compatible with missing values because their base learners, such as decisiontree-based models, can naturally process incomplete data. As shown in Table 7, SelfPacedEnsemble and BalanceCascadeEnsemble achieve competitive performance, particularly in binary tasks. However, other ensemble methods show more moderate performance, and their compatibility strongly depends on the underlying base learner and training framework. Overall, algorithm-level methods provide the most reliable performance in the presence of missing values, combining both compatibility and strong predictive ability. Ensemble-based methods can also be effective when built upon compatible base learners, while data-level methods require additional preprocessing and are less suitable in scenarios with substantial missingness. Summary for RQ2: Performance Across Dataset Characteristics The effectiveness of imbalanced learning methods is strongly dependent on dataset characteristics. Algorithm-level methods demonstrate consistently strong and stable performance across different regimes, particularly in the presence of missing values. In contrast, data-level methods become more competitive in multi-class and highly imbalanced scenarios. Ensemble-based methods perform well on small datasets but exhibit reduced robustness as sample size, feature dimensionality, or imbalance severity increases. These results further demonstrate that no single method is universally optimal, and that method effectiveness is highly regime-dependent. 4.3. Scalability and Computational Efficiency With the increasing prevalence of large-scale datasets, computational efficiency becomes an important factor in practical method selection. In this section, we evaluate whether different algorithms can maintain computational efficiency under increasing sample size, feature dimensionality, and class number. This analysis complements predictive performance comparison by incorporating efficiency considerations. We generate synthetic datasets using the sklearn [63] dataset generator while minimizing interference from unrelated data characteristics. Three groups of datasets are constructed by independently varying sample size, feature dimensionality, and class number. When a parameter is not under 20
study, it is fixed to a baseline configuration with 10000 samples, 10 features, and 2 classes, with an imbalance ratio of 1:9. The results are presented in Fig. 6, Fig. 7, and Fig. 8, which illustrate training time under different scaling conditions. 4.3.1. Scalability across Sample Sizes We evaluate scalability by measuring training time under increasing sample sizes from 1k to 100k samples, as shown in Table 8 and Fig. 6. Table 8: Running time comparison across different sample sizes. Runtime values are reported as log10 (seconds) and averaged over five independent runs. We report family-level average, minimum, and maximum runtime. Larger values indicate lower computational efficiency.
XGBoost Data-level Alg.-level Ensemble 1k Samples
Rank Avg. Min Max
– -1.24 -1.24 -1.24
2 -0.71 -1.36 0.12
1 -1.20 -2.80 -0.62
3 -0.57 -1.14 -0.23
100k Samples
Rank Avg. Min Max
– -0.40 -0.40 -0.40
3 2.90 -0.17 4.13
1 -0.27 -1.42 -0.01
2 1.02 0.04 1.33
Distinct scalability patterns can be observed across different families. Algorithm-level methods exhibit the most stable scaling behavior, with training time remaining close to the XGBoost baseline across both regimes. Most XGBoost-based variants show only moderate growth as sample size increases, indicating strong scalability on large datasets. In contrast, simpler costsensitive methods such as DecisionTreeCost and RandomForestCost exhibit larger increases in training time. Data-level methods show a much steeper increase in computational cost. While their training time is comparable to the baseline on small datasets, the gap widens substantially as sample size increases. This behavior mainly arises because these methods largely rely on neighborhood search. Methods such as SMOTE require pairwise comparisons within the minority class, leading to a sharp increase in computational cost as the number of samples grows. More complex methods such as InstanceHardnessThreshold and 21
Baseline Data-level Alg-level Ensemble 1k samples 100k samples
XGB TL ENN NCR IHT CC CNN AK NM OSS SMO BLS PFS SIPF Lee SBD SENN ST ASL ACE AWE FL WCE CBE XC LRC DTC RFC SPE BCE BRF EE RUS UBA OBO SBO OBA SBA AC AUB ASB 10 3
10 2
10 1
101
100
102
Training Time (seconds, log scale)
103
104
Figure 6: Training time (log10 (seconds)) of different methods under increasing sample sizes. Each method is evaluated on datasets with 1k and 100k samples. Bar length indicates training time on a logarithmic scale. Colors represent method families, and color intensity indicates dataset size.
22
CondensedNearestNeighbor are particularly expensive because they rely on repeated classifier-based sample evaluation during the resampling process. Ensemble methods consistently incur higher computational cost than algorithm-level methods due to repeated model training and resampling procedures. Their training time increases steadily with dataset scale, although the overall growth remains more controlled than that of the most expensive data-level methods. Another notable observation is the variability within families. Algorithmlevel methods remain tightly clustered, indicating stable efficiency across different implementations. In contrast, data-level and ensemble methods exhibit substantially larger variation, suggesting that scalability strongly depends on the underlying resampling or ensemble design. Overall, algorithm-level methods provide the best scalability with respect to sample size, while several data-level and ensemble methods become increasingly expensive as dataset scale grows. 4.3.2. Scalability across Feature Dimensions We evaluate scalability with respect to feature dimensionality by increasing the number of features from 50 to 500, as shown in Table 9 and Fig. 7. Compared with sample size scaling, the impact of feature dimensionality varies more substantially across method families. Table 9: Running time comparison across different feature dimensions. Runtime values are reported as log10 (seconds) and averaged over five independent runs. We report family-level average, minimum, and maximum runtime. Larger values indicate lower computational efficiency.
XGBoost Data-level Alg.-level Ensemble 50 Features
Rank Avg. Min Max
– -0.58 -0.58 -0.58
2 0.51 -0.58 1.39
1 -0.48 -1.64 -0.33
3 0.70 -0.29 1.10
500 Features
Rank Avg. Min Max
– 0.26 0.26 0.26
2 1.50 0.03 2.55
1 0.33 -0.53 0.64
3 1.66 -0.17 2.17
Algorithm-level methods exhibit stable training time as dimensionality increases. Their growth remains moderate and close to the XGBoost baseline 23
Baseline Data-level Alg-level Ensemble 50 features 500 features
XGB TL ENN NCR IHT CC CNN AK NM OSS SMO BLS PFS SIPF Lee SBD SENN ST ASL ACE AWE FL WCE CBE XC LRC DTC RFC SPE BCE BRF EE RUS UBA OBO SBO OBA SBA AC AUB ASB 10 1
101
100
Training Time (seconds, log scale)
102
Figure 7: Training time (log10 (seconds)) of different methods under increasing feature dimensionality. Each method is evaluated on datasets with 50 and 500 features. Bar length represents training time on a logarithmic scale. Colors represent method families, and color intensity indicates feature dimensionality.
24
across both regimes, indicating strong robustness to high-dimensional data. In contrast, simple linear models such as LogRegCost show a noticeable increase in training time as dimensionality grows, reflecting their dependence on full feature utilization. Data-level methods show substantially higher sensitivity to feature dimensionality. Many methods experience clear increases in computational cost in the high-dimensional regime, especially those relying on neighborhood search and distance computation. Methods such as InstanceHardnessThreshold, CondensedNearestNeighbor, and SMOTE-IPF become considerably more expensive as dimensionality increases, reflecting the cost of distance computation in high-dimensional spaces. Meanwhile, simpler methods such as TomekLinks and NearMiss remain relatively efficient. Ensemble methods are generally the most computationally expensive across both regimes. Their overhead becomes more pronounced in high-dimensional settings because of repeated model training and repeated resampling procedures. However, scalability differs substantially within this family. Methods such as BalancedRandomForest and EasyEnsemble remain relatively efficient, indicating variability in ensemble design efficiency. Overall, algorithm-level methods demonstrate the best scalability with respect to feature dimensionality, while data-level and ensemble methods incur higher computational costs as dimensionality increases, with ensemble methods showing the largest overall overhead. 4.3.3. Scalability across Class Numbers We evaluate scalability with respect to class number by increasing the number of classes from 2 to 20, as shown in Table 10 and Fig. 8. Compared with sample size and feature dimensionality, the impact of increasing class number on training time is less consistent across method families. Algorithm-level methods remain among the most computationally efficient approaches. Most XGBoost-based variants exhibit only limited increases in training time as the number of classes grows, remaining close to the baseline across both regimes. Simpler methods such as LogRegCost and DecisionTreeCost are particularly efficient, although their predictive performance is generally weaker than that of XGBoost-based methods. Ensemble methods exhibit mixed scalability behavior. Methods such as SelfPacedEnsemble, BalanceCascadeEnsemble, BalancedRandomForest, and EasyEnsemble maintain relatively stable training time as the number of classes increases. In contrast, several bagging-based oversampling meth25
Table 10: Running time comparison across different class numbers. Runtime values are reported as log10 (seconds) and averaged over five independent runs. We report family-level average, minimum, and maximum runtime. Larger values indicate lower computational efficiency.
XGBoost Data-level Alg.-level Ensemble 2 Classes
Rank Avg. Min Max
– -1.35 -1.35 -1.35
3 0.24 -1.29 1.26
1 -1.15 -2.12 -0.66
2 -0.09 -0.88 0.33
20 Classes
Rank Avg. Min Max
– 0.34 0.34 0.34
3 1.55 0.16 2.63
1 0.11 -0.67 0.50
2 0.81 0.05 1.49
ods remain computationally expensive across both class regimes, resulting in substantially larger overhead than other families. Data-level methods show the largest variability. Simpler undersampling methods such as EditedNearestNeighbors and NearMiss remain highly efficient, while methods involving iterative classifier-based procedures during the resampling process tend to incur substantially higher computational costs. In particular, InstanceHardnessThreshold, CondensedNearestNeighbor, and SMOTE-IPF consistently exhibit high computational cost across both binary and multi-class settings. Overall, increasing class number has a smaller impact on training time than increasing sample size or feature dimensionality. Nevertheless, algorithmlevel methods remain the most computationally stable family, while datalevel and ensemble methods show greater variability depending on the underlying resampling or ensemble strategy.
26
Baseline Data-level Alg-level Ensemble 2 classes 20 classes
XGB TL ENN NCR IHT CC CNN AK NM OSS SMO BLS PFS SIPF Lee SBD SENN ST ASL ACE AWE FL WCE CBE XC LRC DTC RFC SPE BCE BRF EE RUS UBA OBO SBO OBA SBA AC AUB ASB 10 2
10 1
100
101
Training Time (seconds, log scale)
102
Figure 8: Training time (log10 (seconds)) of different methods under increasing class numbers. Each method is evaluated on datasets with 2 and 20 classes. Bar length represents training time on a logarithmic scale. Colors indicate method families, while color intensity represents the number of classes.
27
Summary for RQ3: Scalability and Computational Efficiency Algorithm-level methods exhibit the strongest computational scalability and the most stable training efficiency as sample size, feature dimensionality, and the number of classes increase. By contrast, data-level methods can become increasingly expensive in large-scale and highdimensional settings, particularly when they involve auxiliary classifierbased iterative resampling mechanisms. Ensemble methods generally incur the highest computational overhead because they require repeated model training. These results highlight the importance of considering computational efficiency alongside predictive performance when selecting imbalanced learning methods for practical applications. 4.4. Practical Recommendations Based on the empirical findings from the preceding analyses, we summarize practical recommendations for selecting imbalanced learning methods under different dataset characteristics and computational constraints. The main takeaway is that method selection should be regime-aware rather than based on a single globally best algorithm. Since XGBoost remains highly competitive in many settings, especially for multi-class tasks, we recommend using it as a strong starting baseline before adopting more complex imbalance-specific methods. Table 11 summarizes the recommended method families and representative algorithms under different scenarios. Table 11: Practical recommendations for selecting imbalanced learning methods under different data conditions. Data condition
Recommended family
Representative methods
General tasks Small binary Large-scale High-dimensional Multi-class Severe imbalance Missing values
Baseline Ensemble-based Algorithm-level Algorithm-level Data-/Algorithm-level Data-/Algorithm-level Algorithm-level
XGBoost SelfPacedEnsemble, BalanceCascadeEnsemble XGBoostASL, XGBoostCost XGBoostWCE, XGBoostCost SMOTE, XGBoostCost EditedNearestNeighbours, XGBoostFL XGBoostASL, XGBoostACE
For binary classification, ensemble-based methods such as SelfPacedEnsemble and BalanceCascadeEnsemble are strong choices for small datasets, but 28
their computational cost can be high. When scalability is important, algorithmlevel methods such as XGBoostASL and XGBoostWCE provide a better balance between predictive performance and efficiency. For large-scale or high-dimensional datasets, algorithm-level methods are generally preferred, while distance-based resampling methods should be used with caution due to their higher computational cost and instability in high-dimensional spaces. For multi-class classification, data-level methods become more competitive. SMOTE and its variants often achieve strong performance, especially under moderate or severe imbalance. XGBoostCost is also a reliable and efficient choice across different regimes. Under extreme imbalance, data-level methods such as EditedNearestNeighbours and OneSidedSelection may be worth trying, although algorithm-level methods remain more stable when computational efficiency is a concern. For datasets with missing values, algorithm-level methods are generally recommended because many XGBoost-based methods can handle missing values natively. In contrast, most data-level methods require preprocessing before resampling. Overall, method selection should depend jointly on predictive performance, dataset characteristics, and computational constraints. Ensemble-based methods are useful for small binary tasks, data-level methods become more competitive in multi-class and highly imbalanced settings, and algorithm-level methods provide the most reliable overall trade-off across robustness, scalability, and missing-value handling. 5. Conclusions This paper presented TILBench, a systematic empirical study of imbalanced learning methods for tabular data. Rather than evaluating methods only by their average performance, TILBench examines how different method families behave from three perspectives: predictive performance, sensitivity to dataset characteristics, and computational scalability. Our analysis shows that the effectiveness of imbalanced learning methods is strongly regime-dependent. Algorithm-level methods often provide strong and stable performance, especially when scalability or missing values are important. Data-level methods are not uniformly superior, but they become competitive in multi-class and severely imbalanced settings. Ensemble-based methods can achieve strong performance on small binary datasets, but their computational cost and variability make them less attractive in large-scale or high-dimensional settings. These results suggest that the relative value of 29
each method family depends on the interaction between data properties and computational constraints. The main lesson from TILBench is that method selection for tabular imbalanced learning should be regime-aware. A strong GBDT baseline should first be considered, and more specialized imbalance-handling methods should be selected according to the dataset scale, dimensionality, imbalance severity, missingness, and available computational budget. We hope that TILBench provides a useful empirical reference for practitioners and motivates future work on imbalanced learning methods that are not only accurate, but also robust and scalable across diverse tabular data regimes. Appendix A. Implementation Details Table A.12 summarizes the datasets used in this study. Table A.13 shows the abbreviations of all methods. Table A.14 presents the hyperparameter search ranges for all methods. Methods using XGBoost as the base learner share the same hyperparameter settings as the baseline model; therefore, only additional method-specific hyperparameters are reported to avoid redundancy. We also provide the corresponding open-source implementation links in Table A.15. Appendix B. Supplementary Results We also provide the complete performance results in Table B.16 and Table B.16, together with the corresponding G-mean visualizations in Fig. B.9, Fig. B.10, and Fig. B.11. References [1] H. He, E. A. Garcia, Learning from imbalanced data, IEEE Transactions on knowledge and data engineering 21 (9) (2009) 1263–1284. 1
https://imbalanced-learn.org/stable/index.html https://smote-variants.readthedocs.io/en/latest/index.html 3 http://scikit-learn.org/stable/ 4 https://github.com/Luojiaqimath/ClassbalancedLoss4GBDT 5 https://xgboost.readthedocs.io/en/stable/ 6 https://imbalanced-ensemble.readthedocs.io/en/latest/ 2
30
Table A.12: Dataset description. #S means the sample number; #F indicates the feature number; #C gives the number of class; #M is the number of samples with missing values; IR is the imbalanced ratio. For binary and multi-class classification, IR is the ratio of the most frequent class to the least frequent class. Dataset name with an asterisk (*) is an abbreviation. Full names are: (1) o-s-i: online-shoppers-intention (2) I-A: Internet-Advertisements (3) P-D-H: Pulsar-Dataset-HTRU2 (4) C-C-F-D: CreditCardFraudDetection (5) G-P-S-P: GesturePhaseSegmentationProcessed (6) j-c-2-r-e-c: jungle_chess_2pcs_raw_endgame_complete. Name
#S
#F
#C
#M
IR
Name
#S
#F
#C
#M
IR
8192 48842 12330 3279 1458 336 6435 17898 531 7797 360 3772 937 34780 76000 5100 284807
32 14 17 1558 37 7 36 8 93 617 90 52 49 300 170 36 30
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
0 0 0 0 0 0 0 0 0 0 0 0 0 0 75244 0 0
2.22 3.18 5.46 6.14 7.19 8.60 9.28 9.92 10.80 12.00 14.00 15.33 21.85 34.45 54.27 67.00 577.88
178 1473 160 3186 9873 101766 44819 625 50789 58310 83733
13 9 4 180 32 49 6 4 19 180 54
3 3 3 3 5 3 3 3 3 10 4
0 0 0 0 0 0 0 0 1 0 0
1.48 1.89 2.10 2.16 2.96 4.83 5.32 5.88 7.45 9.41 22.83
breast-w ada kc2 churn kick bank-marketing page-blocks sick_euthyroid vowel car_eval_34 us_crime sick arrhythmia yeast_me2 mammography dis protein_homo
699 4147 522 5000 72983 45211 5473 3163 990 1728 1994 3772 452 1484 11183 3772 145751
9 48 21 20 32 16 10 42 12 21 100 29 278 8 6 29 74
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
16 0 0 0 383 0 0 0 0 0 0 3772 0 0 0 0 0
Binary 1.90 bank32nh adult 3.03 3.88 o-s-i* I-A* 6.07 7.13 pc4 7.55 ecoli 8.77 satimage 9.80 P-D-H* 10.00 spectrometer 11.90 isolet 12.29 libras_move 15.33 thyroid_sick 17.08 oil 28.09 webpage 42.01 APSFailure 64.03 Satellite 111.46 C-C-F-D*
MiceProtein contraceptive eucalyptus splice satimage fabert newthyroid dermatology connect-4 flare steel-plates-fault kropt
1080 1473 736 3190 6430 8237 215 358 67557 1066 1941 28056
77 9 19 60 36 800 5 34 42 19 27 6
8 3 5 3 6 7 3 6 3 6 7 18
528 0 95 0 0 0 0 0 0 0 0 0
Multi-class 1.43 wine 1.89 cmc 2.04 hayes-roth 2.16 dna 2.45 G-P-S-P* 3.84 Diabetes130US j-c-2-r-e-c* 5.00 5.55 balance 6.90 okcupid-stem 7.70 volkert 12.24 jannis 168.63
[2] G. Haixiang, L. Yijing, J. Shang, G. Mingyun, H. Yuanyue, G. Bing, Learning from class-imbalanced data: Review of methods and applications, Expert Systems with Applications 73 (2017) 220–239. [3] A. Fernández, S. García, M. Galar, R. C. Prati, B. Krawczyk, F. Herrera, Learning from imbalanced data sets, Springer (2018).
31
Table A.13: Method abbreviations used in figures. Family
Abbreviation
Baseline
XGB TL ENN NCR IHT CC CNN AK NM OSS SMO BLS PFS SIPF Lee SBD SENN ST ASL ACE AWE
Data-level
Alg.-level
Method
Family
XGBoost TomekLinks EditedNearestNeighbours NeighbourhoodCleaningRule InstanceHardnessThreshold ClusterCentroids CondensedNearestNeighbour AllKNN NearMiss OneSidedSelection SMOTE BorderlineSMOTE PolyfitSMOTE SMOTEIPF Lee SMOBD SMOTEENN SMOTETomek XGBoostASL XGBoostACE XGBoostAWE
Alg.-level
Ensemble
Abbreviation FL WCE CBE XC LRC DTC RFC SPE BCE BRF EE RUS UBA OBO SBO OBA SBA AC AUB ASB
Method XGBoostFL XGBoostWCE XGBoostCBE XGBoostCost LogRegCost DecisionTreeCost RandomForestCost SelfPacedEnsemble BalanceCascadeEnsemble BalancedRandomForest EasyEnsemble RUSBoost UnderBagging OverBoost SMOTEBoost OverBagging SMOTEBagging AdaCost AdaUCost/AdaUBoost AsymBoost
Figure B.9: Family-level G-mean scores across sample size regimes. Each box shows the distribution of method performance within a family for each imbalance group. Results are reported separately for binary and multi-class tasks. Larger values indicate lower computational efficiency.
[4] B. Krawczyk, Learning from imbalanced data: open challenges and future directions, Progress in Artificial Intelligence 5 (4) (2016) 221–232. [5] H. Zhu, G. Liu, M. Zhou, Y. Xie, A. Abusorrah, Q. Kang, Optimizing weighted extreme learning machines for imbalanced classification and 32
Table A.14: The hyperparameters involved in training are given. Methods in the datalevel family, as well as those involving XGBoost in the algorithm-level family, share the same four hyperparameters as the base model, thus we leave out the repeated parts. Category
Algorithm
Hyperparameter
Type
Range/Values
Base Model
XGBoost
max_depth alpha lambda eta
int float float float
[2, 10] [1e − 8, 0.1] [0.5, 2.0] [0.05, 0.3]
Data-level Methods
TomekLinks EditedNearestNeighbours NeighbourhoodCleaningRule InstanceHardnessThreshold ClusterCentroids CondensedNearestNeighbour AllKNN NearMiss OneSidedSelection SMOTE BorderlineSMOTE PolyfitSMOTE SMOTEIPF Lee SMOBD SMOTEENN SMOTETomek
sampling_strategy n_neighbors n_neighbors sampling_strategy sampling_strategy n_neighbors n_neighbors n_neighbors n_neighbors k_neighbors k_neighbors order n_neighbors n_neighbors eta1 k_neighbors k_neighbors
int int int int int int int int int int int float int int
auto [2, 8] [2, 8] auto auto [2, 8] [2, 8] [2, 8] [2, 8] [3, 10] [3, 10] [1, 3] [3, 10] [3, 10] [0.4, 1.0] [3, 10] [3, 10]
r1 r2 m m r1 m r1 r1 b C max_depth min_samples_split min_samples_leaf n_estimators max_depth min_samples_split min_samples_leaf
categorical categorical float float categorical float categorical categorical float float int int int int int int int
XGBoostASL XGBoostACE XGBoostAWE
Algorithm-level Methods
XGBoostFL XGBoostWCE XGBoostCBE XGBoostCost LogRegCost DecisionTreeCost
RandomForestCost
[0.0, 0.1] [0.5, 1.0, 2.0] [0.05, 0.2] [0.05, 0.2] [2.0, 3.0, 5.0] [0.05, 0.2] [0.5, 1.0, 2.0] [2.0, 3.0, 5.0] [0.05, 0.999] [1e − 6, 1e3] [2, 10] [2, 10] [1, 5] [20, 200] [2, 10] [2, 10] [1, 5]
application to credit card fraud detection, Neurocomputing 407 (2020) 50–62. [6] L. I. Santos, M. O. Camargos, M. F. S. V. D’Angelo, J. B. Mendes, 33
Table A.14: Hyperparameters for different methods(continued) Category
Algorithm SelfPacedEnsemble BalanceCascadeEnsemble BalancedRandomForest EasyEnsemble RUSBoost UnderBagging
Ensemble-based Methods
OverBoost SMOTEBoost OverBagging SMOTEBagging AdaCost AdaUCost AsymBoost
Hyperparameter
Type
Range/Values
n_estimators k_bins n_estimators n_estimators max_depth min_samples_split min_samples_leaf n_estimators n_estimators learning_rate n_estimators n_estimators learning_rate n_estimators k_neighbors learning_rate n_estimators n_estimators k_neighbors n_estimators learning_rate n_estimators learning_rate n_estimators learning_rate
int int int int int int int int int float int int float int int float int int int int float int float int float
[20, 200] [2, 10] [20, 200] [20, 200] [2, 10] [2, 10] [1, 5] [20, 200] [20, 200] [0.5, 2] [20, 200] [20, 200] [0.5, 2] [20, 200] [3, 10] [0.5, 2] [20, 200] [20, 200] [3, 10] [20, 200] [0.5, 1.5] [20, 200] [0.5, 1.5] [20, 200] [0.5, 1.5]
Table A.15: Open-source Python tools for the methods used in the paper.
Category
Approach
Python Packages
Data-level
Over-sampling Under-sampling Hybrid
imbalanced-learn1 , smote-variants2 imbalanced-learn imbalanced-learn
Algorithm-level
Cost-sensitive Loss modification
scikit-learn3 , XGBoost5 gbdtCBL4
Ensemble
Resampling Cost-incorporated
imbalanced-learn, imbalanced-ensemble6 imbalanced-ensemble
E. E. C. de Medeiros, A. L. S. Guimarães, R. M. Palhares, Decision tree and artificial immune systems for stroke prediction in imbalanced data, 34
Table B.16: Complete performance results for binary classification tasks. Rank
Method
F1-score
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
SelfPacedEnsemble XGBoostWCE XGBoostASL XGBoostAWE BalanceCascadeEnsemble XGBoostACE SMOTEBagging Lee XGBoostFL XGBoostCost BorderlineSMOTE XGBoostCBE PolyfitSMOTE SMOTE OverBagging RandomForestCost SMOTETomek SMOTEIPF SMOTEENN AdaCost AsymBoost SMOTEBoost AdaUCost OverBoost NeighbourhoodCleaningRule SMOBD EditedNearestNeighbours UnderBagging DecisionTreeCost AllKNN XGBoost TomekLinks CondensedNearestNeighbour OneSidedSelection BalancedRandomForest LogRegCost InstanceHardnessThreshold EasyEnsemble RUSBoost NearMiss ClusterCentroids
73.88 ± 4.01 72.96 ± 4.91 72.62 ± 5.48 72.54 ± 4.57 72.51 ± 4.55 72.07 ± 6.16 70.93 ± 4.73 70.53 ± 5.97 70.14 ± 5.84 69.90 ± 4.90 69.71 ± 5.72 69.58 ± 5.14 69.12 ± 6.28 67.95 ± 6.46 67.86 ± 4.70 67.83 ± 4.83 67.75 ± 6.05 65.64 ± 5.48 65.50 ± 5.07 65.19 ± 5.83 65.19 ± 5.83 64.48 ± 5.53 64.33 ± 4.45 63.79 ± 5.43 63.34 ± 12.74 62.35 ± 10.62 61.73 ± 17.58 61.55 ± 3.53 61.35 ± 4.82 61.23 ± 16.80 60.92 ± 12.79 60.21 ± 13.09 59.71 ± 7.00 59.20 ± 13.33 57.07 ± 3.54 55.30 ± 3.35 54.90 ± 11.30 53.77 ± 3.45 50.79 ± 8.50 37.77 ± 5.51 36.71 ± 3.91
Method
G-mean score
UnderBagging BalancedRandomForest SMOTEENN BalanceCascadeEnsemble AdaUCost EasyEnsemble XGBoostCost SelfPacedEnsemble SMOTEIPF SMOTETomek SMOTE LogRegCost XGBoostAWE OverBoost BorderlineSMOTE DecisionTreeCost XGBoostASL XGBoostWCE SMOTEBoost RandomForestCost Lee XGBoostACE SMOTEBagging CondensedNearestNeighbour PolyfitSMOTE InstanceHardnessThreshold XGBoostFL XGBoostCBE OverBagging NearMiss AsymBoost AdaCost NeighbourhoodCleaningRule SMOBD ClusterCentroids RUSBoost AllKNN EditedNearestNeighbours XGBoost TomekLinks OneSidedSelection
89.32 ± 2.12 88.34 ± 2.38 87.27 ± 3.21 87.20 ± 2.72 86.74 ± 3.43 86.70 ± 2.40 86.22 ± 3.39 85.97 ± 3.14 85.00 ± 3.58 84.92 ± 3.99 84.89 ± 3.98 84.68 ± 3.16 84.61 ± 3.84 84.58 ± 4.62 84.48 ± 3.82 84.40 ± 3.55 83.87 ± 4.99 83.66 ± 3.86 83.56 ± 4.28 82.73 ± 4.61 82.37 ± 4.57 80.61 ± 5.88 80.61 ± 3.86 79.87 ± 5.09 77.80 ± 5.34 77.58 ± 14.30 77.37 ± 5.10 77.05 ± 4.63 75.81 ± 3.97 74.76 ± 4.76 73.86 ± 5.66 73.86 ± 5.66 72.84 ± 13.64 71.61 ± 10.74 71.41 ± 3.98 71.41 ± 7.59 71.14 ± 18.36 71.05 ± 19.26 67.17 ± 13.36 66.66 ± 14.31 65.95 ± 14.48
Binary
Expert Systems with Applications 191 (2022) 116221. [7] Y. Zhang, X. Li, L. Gao, L. Wang, L. Wen, Imbalanced data fault 35
Table B.16: Complete performance results for multi-class classification tasks. Rank
Method
F1-score
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41
SMOTE XGBoostCost SMOTETomek BorderlineSMOTE Lee TomekLinks XGBoostFL XGBoostCBE XGBoost XGBoostWCE OneSidedSelection SMOTEIPF PolyfitSMOTE SMOBD SMOTEBagging XGBoostACE XGBoostASL OverBagging NeighbourhoodCleaningRule XGBoostAWE EditedNearestNeighbours UnderBagging RandomForestCost SelfPacedEnsemble BalanceCascadeEnsemble BalancedRandomForest AllKNN CondensedNearestNeighbour DecisionTreeCost SMOTEENN NearMiss InstanceHardnessThreshold LogRegCost AdaUCost AsymBoost SMOTEBoost AdaCost OverBoost ClusterCentroids EasyEnsemble RUSBoost
75.84 ± 1.45 75.83 ± 1.57 75.49 ± 1.45 75.49 ± 1.71 75.47 ± 1.52 75.02 ± 1.67 74.69 ± 1.76 74.69 ± 1.64 74.63 ± 1.61 74.61 ± 1.31 74.42 ± 1.92 74.15 ± 1.83 73.92 ± 1.36 73.74 ± 1.56 73.11 ± 1.34 72.65 ± 2.76 72.55 ± 3.25 72.14 ± 1.32 71.75 ± 3.01 71.66 ± 3.15 71.39 ± 2.78 71.10 ± 1.13 70.65 ± 1.64 70.45 ± 1.45 69.28 ± 1.47 68.03 ± 1.77 67.39 ± 2.66 67.00 ± 3.39 64.90 ± 2.04 64.59 ± 2.42 62.91 ± 1.78 61.63 ± 2.96 60.11 ± 1.55 59.79 ± 3.17 59.55 ± 2.14 59.54 ± 3.17 59.48 ± 2.18 58.97 ± 2.65 58.47 ± 1.62 55.06 ± 1.78 50.44 ± 6.00
Method
G-mean score
XGBoostCost SMOTE SMOTETomek BorderlineSMOTE Lee TomekLinks SMOTEIPF XGBoostFL XGBoostCBE XGBoostWCE XGBoost PolyfitSMOTE OneSidedSelection UnderBagging SMOBD SelfPacedEnsemble SMOTEBagging RandomForestCost NeighbourhoodCleaningRule EditedNearestNeighbours XGBoostACE XGBoostASL BalanceCascadeEnsemble BalancedRandomForest OverBagging XGBoostAWE SMOTEENN AllKNN CondensedNearestNeighbour DecisionTreeCost NearMiss InstanceHardnessThreshold ClusterCentroids AdaUCost LogRegCost OverBoost SMOTEBoost AsymBoost AdaCost EasyEnsemble RUSBoost
83.91 ± 1.16 83.63 ± 1.00 83.44 ± 0.94 83.23 ± 1.26 83.08 ± 1.04 82.60 ± 1.29 82.49 ± 1.11 82.15 ± 1.29 82.15 ± 1.19 82.12 ± 0.95 82.09 ± 1.19 82.08 ± 0.94 81.99 ± 1.41 81.92 ± 0.87 81.76 ± 1.19 81.43 ± 1.05 81.26 ± 1.08 81.20 ± 1.25 80.88 ± 2.15 80.84 ± 1.86 80.78 ± 2.22 80.66 ± 2.56 80.57 ± 1.16 80.50 ± 1.36 80.30 ± 1.07 79.94 ± 2.51 78.46 ± 1.75 78.17 ± 2.00 78.10 ± 2.45 77.60 ± 1.45 76.79 ± 1.41 76.38 ± 2.14 75.78 ± 1.15 74.19 ± 2.18 74.12 ± 1.14 73.73 ± 1.98 73.49 ± 2.11 71.99 ± 1.68 71.96 ± 1.70 70.88 ± 1.43 66.77 ± 3.28
Multi-class
diagnosis of rotating machinery using synthetic oversampling and feature learning, Journal of manufacturing systems 48 (2018) 34–50. 36
Table B.17: Top five methods in each sample size regime ranked by G-mean score for binary and multi-class tasks. Sample Size
<1k
1k–10k
>10k
Rank
Method
G-mean score
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Binary EasyEnsemble UnderBagging BalancedRandomForest BalanceCascadeEnsemble SMOTEBoost UnderBagging BalancedRandomForest SMOTEENN BalanceCascadeEnsemble XGBoostCost UnderBagging AdaUCost BalancedRandomForest OverBoost XGBoostCost
89.25 ± 4.94 88.42 ± 4.95 86.96 ± 5.27 86.50 ± 6.09 84.80 ± 7.91 89.94 ± 1.91 89.31 ± 1.71 88.51 ± 2.67 88.26 ± 2.08 87.11 ± 3.02 89.12 ± 0.73 88.44 ± 1.07 87.72 ± 1.12 87.57 ± 1.82 87.56 ± 1.48
Method
G-mean score
Multi-class PolyfitSMOTE 92.25 ± 1.79 XGBoostCost 92.08 ± 1.95 SMOTE 91.81 ± 1.77 SMOTEIPF 91.70 ± 1.77 UnderBagging 91.35 ± 1.66 XGBoostCost 82.63 ± 1.16 SMOTE 82.44 ± 1.01 SMOTEIPF 82.43 ± 0.87 BorderlineSMOTE 82.26 ± 0.96 SMOTETomek 82.26 ± 0.96 XGBoostCost 80.14 ± 0.48 SMOTETomek 78.65 ± 0.34 SMOTE 78.59 ± 0.34 BorderlineSMOTE 78.41 ± 0.43 SMOBD 78.05 ± 0.37
Table B.18: Top five methods in each feature dimensionality regime ranked by G-mean score for binary and multi-class tasks. Feature Dimension
<10
10–50
>50
Rank
Method
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Binary BalancedRandomForest UnderBagging EasyEnsemble LogRegCost SMOTEENN UnderBagging BalancedRandomForest SMOTEENN BalanceCascadeEnsemble AdaUCost UnderBagging BalanceCascadeEnsemble SelfPacedEnsemble SMOTEENN EasyEnsemble
G-mean score 89.97 ± 2.39 89.28 ± 4.09 87.83 ± 3.58 87.77 ± 3.20 87.34 ± 4.72 88.26 ± 1.80 87.80 ± 1.68 86.40 ± 2.38 86.15 ± 2.23 85.82 ± 2.46 91.51 ± 2.46 91.07 ± 3.32 89.20 ± 3.82 88.89 ± 4.10 88.65 ± 3.23
Method
G-mean score
Multi-class XGBoostCost 82.98 ± 1.86 SMOTE 82.29 ± 1.59 SMOTEIPF 82.09 ± 1.47 SMOTETomek 81.86 ± 1.45 Tomek 81.61 ± 2.29 XGBoostCost 84.60 ± 0.90 BorderlineSMOTE 83.77 ± 0.68 PolyfitSMOTE 83.75 ± 0.74 SMOTEIPF 83.69 ± 0.72 SMOTE 83.62 ± 0.84 SMOTE 85.51 ± 0.41 XGBoostCost 85.45 ± 0.58 SMOTETomek 85.45 ± 0.44 BorderlineSMOTE 85.25 ± 0.55 PolyfitSMOTE 84.95 ± 0.55
[8] M. R. Smith, T. Martinez, C. Giraud-Carrier, Instance hardness: A measure of difficulty for an instance based on classification error, Machine 37
Table B.19: Top five methods in each imbalance severity regime ranked by G-mean score for binary and multi-class tasks. Imbalance Ratio
<10
10–50
>50
Rank
Method
G-mean score
Method
G-mean score
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Binary UnderBagging BalancedRandomForest SMOTEENN XGBoostCost AdaUCost UnderBagging BalancedRandomForest BalanceCascadeEnsemble EasyEnsemble SelfPacedEnsemble UnderBagging EasyEnsemble BalanceCascadeEnsemble BalancedRandomForest SMOTEENN
88.00 ± 1.79 87.34 ± 1.74 87.13 ± 1.82 86.90 ± 2.16 86.72 ± 2.07 90.05 ± 3.23 88.98 ± 3.44 88.80 ± 4.92 87.62 ± 3.70 87.05 ± 4.67 92.62 ± 1.67 91.27 ± 1.66 90.48 ± 2.51 90.33 ± 1.90 89.65 ± 3.31
Multi-class XGBoostCost SMOTE SMOTETomek PolyfitSMOTE Lee XGBoostCost SMOTEIPF SMOTE SMOTETomek SelfPacedEnsemble XGBoostCost EditedNearestNeighbours XGBoost SMOTETomek OneSidedSelection
83.92 ± 1.20 83.32 ± 1.09 83.12 ± 1.03 83.06 ± 1.00 82.98 ± 1.09 81.61 ± 1.04 81.01 ± 0.67 80.87 ± 0.49 80.62 ± 0.36 80.60 ± 0.71 95.01 ± 0.56 94.80 ± 0.42 94.56 ± 0.32 94.50 ± 0.53 94.41 ± 0.73
Figure B.10: Family-level G-mean scores across feature dimensionality regimes. Each box shows the distribution of method performance within a family for each imbalance group. Results are reported separately for binary and multi-class tasks. Larger values indicate lower computational efficiency.
Learning 95 (3) (2014) 337–362. [9] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: synthetic minority over-sampling technique, Journal of artificial intelligence research 16 (2002) 321–357. 38
Figure B.11: Family-level G-mean scores across imbalance severity regimes. Each box shows the distribution of method performance within a family for each imbalance group. Results are reported separately for binary and multi-class tasks. Larger values indicate lower computational efficiency.
[10] H. Han, W.-Y. Wang, B.-H. Mao, Borderline-smote: a new oversampling method in imbalanced data sets learning, in: International conference on intelligent computing, Springer, 2005, pp. 878–887. [11] J. Luo, Y. Yuan, S. Xu, Improving gbdt performance on imbalanced datasets: An empirical study of class-balanced loss functions, Neurocomputing 634 (2025) 129896. [12] Q. Xu, S. Lu, W. Jia, C. Jiang, Imbalanced fault diagnosis of rotating machinery via multi-domain feature extraction and cost-sensitive learning, Journal of Intelligent Manufacturing 31 (6) (2020) 1467–1481. [13] W. Liu, H. Fan, M. Xia, M. Xia, A focal-aware cost-sensitive boosted tree for imbalanced credit scoring, Expert Systems with Applications 208 (2022) 118158. [14] Z. Liu, W. Cao, Z. Gao, J. Bian, H. Chen, Y. Chang, T.-Y. Liu, Selfpaced ensemble for highly imbalanced massive data classification, in: 2020 IEEE 36th international conference on data engineering (ICDE), IEEE, 2020, pp. 841–852. [15] G. Karakoulas, J. Shawe-Taylor, Optimizing classifers for imbalanced training sets, Advances in neural information processing systems 11 (1998). 39
[16] P. Viola, M. Jones, Fast and robust classification using asymmetric adaboost and a detector cascade, Advances in neural information processing systems 14 (2001). [17] A. A. Khan, O. Chaudhari, R. Chandra, A review of ensemble learning and data augmentation models for class imbalanced problems: Combination, implementation and evaluation, Expert Systems with Applications 244 (2024) 122778. [18] G. Kovács, An empirical comparison and evaluation of minority oversampling techniques on a large number of imbalanced datasets, Applied Soft Computing 83 (2019) 105662. [19] Z. Liu, Z. Li, Z. Yang, T. Wei, J. Kang, Y. Zhu, H. Hamann, J. He, H. Tong, Climb: Class-imbalanced learning benchmark on tabular data, arXiv preprint arXiv:2505.17451 (2025). [20] I. Tomek, Two modifications of cnn. (1976). [21] D. L. Wilson, Asymptotic properties of nearest neighbor rules using edited data, IEEE Transactions on Systems, Man, and Cybernetics (3) (2007) 408–421. [22] J. Laurikkala, Improving identification of difficult small classes by balancing class distribution, in: Conference on artificial intelligence in medicine in Europe, Springer, 2001, pp. 63–66. [23] S. Gazzah, N. E. B. Amara, New oversampling approaches based on polynomial fitting for imbalanced data sets, in: 2008 the eighth iapr international workshop on document analysis systems, IEEE, 2008, pp. 677–684. [24] G. E. Batista, R. C. Prati, M. C. Monard, A study of the behavior of several methods for balancing machine learning training data, ACM SIGKDD explorations newsletter 6 (1) (2004) 20–29. [25] G. E. Batista, A. L. Bazzan, M. C. Monard, et al., Balancing training data for automated annotation of keywords: a case study., Wob 3 (2003) 10–18.
40
[26] W. Liu, H. Fan, M. Xia, C. Pang, Predicting and interpreting financial distress using a weighted boosted tree-based tree, Engineering Applications of Artificial Intelligence 116 (2022) 105466. [27] J. Luo, Y. Quan, S. Xu, Robust-gbdt: leveraging robust loss for noisy and imbalanced classification with gbdt, Knowledge and Information Systems 67 (12) (2025) 12361–12381. [28] C. Wang, C. Deng, S. Wang, Imbalance-xgboost: leveraging weighted and focal losses for binary label-imbalanced classification with xgboost, Pattern recognition letters 136 (2020) 190–197. [29] L. M. Manevitz, M. Yousef, One-class svms for document classification, Journal of machine Learning research 2 (Dec) (2001) 139–154. [30] X.-Y. Liu, J. Wu, Z.-H. Zhou, Exploratory undersampling for classimbalance learning, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39 (2) (2008) 539–550. [31] C. Chen, A. Liaw, L. Breiman, et al., Using random forest to learn imbalanced data, University of California, Berkeley 110 (1-12) (2004) 24. [32] N. V. Chawla, A. Lazarevic, L. O. Hall, K. W. Bowyer, Smoteboost: Improving prediction of the minority class in boosting, in: Knowledge Discovery in Databases: PKDD 2003: 7th European Conference on Principles and Practice of Knowledge Discovery in Databases, CavtatDubrovnik, Croatia, September 22-26, 2003. Proceedings 7, Springer, 2003, pp. 107–119. [33] W. Fan, S. J. Stolfo, J. Zhang, P. K. Chan, Adacost: misclassification cost-sensitive boosting, in: Icml, Vol. 99, 1999, pp. 97–105. [34] B. Nikpour, F. Rahmati, B. Mirzaei, H. Nezamabadi-pour, A comprehensive review on data-level methods for imbalanced data classification, Expert Systems with Applications 295 (2026) 128920. [35] S. H. Khan, M. Hayat, M. Bennamoun, F. A. Sohel, R. Togneri, Costsensitive learning of deep feature representations from imbalanced data, IEEE Transactions on Neural Networks and Learning Systems 29 (8) (2018) 3573–3587. 41
[36] I. Araf, A. Idri, I. Chairi, Cost-sensitive learning for imbalanced medical data: a review., Artificial Intelligence Review 57 (4) (2024). [37] S. Rezvani, X. Wang, A broad review on class imbalance learning techniques, Applied Soft Computing 143 (2023) 110415. [38] G. Aguiar, B. Krawczyk, A. Cano, A survey on learning from imbalanced data streams: taxonomy, challenges, empirical study, and reproducible experimental framework, Machine learning 113 (7) (2024) 4165–4243. [39] B. Zhu, B. Baesens, A. Backiel, S. K. Vanden Broucke, Benchmarking sampling techniques for imbalance learning in churn prediction, Journal of the Operational Research Society 69 (1) (2018) 49–65. [40] J. Xiao, Y. Wang, J. Chen, L. Xie, J. Huang, Impact of resampling methods and classification models on the imbalanced credit scoring problems, Information Sciences 569 (2021) 508–526. [41] T. Wongvorachan, S. He, O. Bulut, A comparison of undersampling, oversampling, and smote methods for dealing with imbalanced classification in educational data mining, Information 14 (1) (2023) 54. [42] J. Vanschoren, J. N. Van Rijn, B. Bischl, L. Torgo, Openml: networked science in machine learning, ACM SIGKDD Explorations Newsletter 15 (2) (2014) 49–60. [43] T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, in: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785–794. [44] V. Borisov, T. Leemann, K. Seßler, J. Haug, M. Pawelczyk, G. Kasneci, Deep neural networks and tabular data: A survey, IEEE transactions on neural networks and learning systems (2022). [45] Y. Gorishniy, I. Rubachev, V. Khrulkov, A. Babenko, Revisiting deep learning models for tabular data, Advances in Neural Information Processing Systems 34 (2021) 18932–18943. [46] L. Grinsztajn, E. Oyallon, G. Varoquaux, Why do tree-based models still outperform deep learning on typical tabular data?, in: Thirtysixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022. 42
[47] W.-C. Lin, C.-F. Tsai, Y.-H. Hu, J.-S. Jhang, Clustering-based undersampling in class-imbalanced data, Information Sciences 409 (2017) 17– 26. [48] P. Hart, The condensed nearest neighbor rule (corresp.), IEEE transactions on information theory 14 (3) (1968) 515–516. [49] I. Tomek, An experiment with the edited nearest-nieghbor rule. (1976). [50] I. Mani, I. Zhang, knn approach to unbalanced data distributions: a case study involving information extraction, in: Proceedings of workshop on learning from imbalanced datasets, Vol. 126, ICML, 2003, pp. 1–7. [51] M. Kubat, Addressing the curse of imbalanced training sets: one-sided selection, in: Proceedings of the 14th international conference on machine learning, Morgan Kaufmann, 1997, pp. 179–186. [52] J. A. Sáez, J. Luengo, J. Stefanowski, F. Herrera, Smote–ipf: Addressing the noisy and borderline examples problem in imbalanced classification by a re-sampling method with filtering, Information Sciences 291 (2015) 184–203. [53] J. Lee, N.-r. Kim, J.-H. Lee, An over-sampling technique with rejection for imbalanced class learning, in: Proceedings of the 9th international conference on ubiquitous information management and communication, 2015, pp. 1–6. [54] Q. Cao, S. Wang, Applying over-sampling technique based on data density and cost-sensitive svm to imbalanced learning, in: 2011 International conference on information management, innovation management and industrial engineering, Vol. 2, IEEE, 2011, pp. 543–548. [55] T. Ridnik, E. Ben-Baruch, N. Zamir, A. Noy, I. Friedman, M. Protter, L. Zelnik-Manor, Asymmetric loss for multi-label classification, in: Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 82–91. [56] T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal loss for dense object detection, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 2980–2988. 43
[57] Y. Sun, A. K. Wong, M. S. Kamel, Classification of imbalanced data: A review, International journal of pattern recognition and artificial intelligence 23 (04) (2009) 687–719. [58] Y. Cui, M. Jia, T.-Y. Lin, Y. Song, S. Belongie, Class-balanced loss based on effective number of samples, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9268– 9277. [59] C. Seiffert, T. M. Khoshgoftaar, J. Van Hulse, A. Napolitano, Rusboost: A hybrid approach to alleviating class imbalance, IEEE transactions on systems, man, and cybernetics-part A: systems and humans 40 (1) (2009) 185–197. [60] R. Maclin, D. Opitz, An empirical evaluation of bagging and boosting, AAAI/IAAI 1997 (1997) 546–551. [61] S. Wang, X. Yao, Diversity analysis on imbalanced data sets by using ensemble models, in: 2009 IEEE symposium on computational intelligence and data mining, IEEE, 2009, pp. 324–331. [62] T. Akiba, S. Sano, T. Yanase, T. Ohta, M. Koyama, Optuna: A nextgeneration hyperparameter optimization framework, in: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 2623–2631. [63] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al., Scikit-learn: Machine learning in python, the Journal of machine Learning research 12 (2011) 2825–2830.
44