Commit-Aware Learning-Based Test Case Prioritization for Continuous Integration Lorenzo Abbondante[0009−0007−6561−1370] Gerardo Canfora[0000−0003−0049−1279]
arXiv:2604.25363v1 [cs.SE] 28 Apr 2026
Univerity of Sannio, Dept. of Engineering, Benevento - Italy
Abstract. Regression testing in Continuous Integration (CI) pipelines is increasingly costly due to the growing size and execution frequency of test suites. Test Case Prioritization (TCP) mitigates this problem by reordering tests to expose faults earlier. However, most existing techniques rely primarily on historical execution data and coverage metrics, neglecting the rich structural information contained in code changes. This paper proposes a commit-aware, learning-based TCP method that combines structural properties of version-control diffs, test coverage relations, and historical execution behavior into a unified predictive model. Given a new commit, the method estimates the probability that each test suite will reveal at least one failure and prioritizes test execution accordingly. We evaluate our method on five Defects4J projects using a leave-oneproject-out cross-project validation setting. Results show that the commitaware TCP significantly outperform noncommit-aware baselines in both classification and prioritization effectiveness. Our findings show that including commit structural semantics substantially enhances regression fault detection and enables robust, generalizable learning-based TCP in CI environments. Keywords: Regression Testing · Test Case Prioritization · Continuous Integration.
1
Introduction
In modern Agile and DevOps environments, software systems evolve continuously through rapid and incremental development cycles. Continuous Integration (CI) and Continuous Deployment (CD) pipelines automate the process of building, testing, and deploying new code as soon as it is committed to the shared repository. Although these practices significantly improve responsiveness and shorten delivery times, they also increase the frequency with which large regression test suites must be executed. In large-scale industrial projects, CI pipelines may execute thousands of test cases multiple times per day under strict time and resource constraints. Executing the full test suite after every commit becomes prohibitively expensive, and organizations face the need for intelligent strategies that optimize testing efficiency without compromising the ability to detect regressions early. Test Case Prioritization (TCP) addresses this challenge
by reordering test execution so that test cases with the highest probability of revealing faults are executed earlier [38, 53]. Over the past two decades, many researchers have investigated TCP strategies. Early approaches relied primarily on static heuristics and structural coverage data [12, 13, 38]. Subsequent work introduced search-based and metaheuristic optimizationssuch as Genetic Algorithmsthat treat TCP as a multi-objective problem balancing fault detection, coverage, and cost [25, 44]. More recently, the advent of AI and deep learning has opened new possibilities for data-driven TCP, where models learn to predict the fault-revealing potential of tests based on historical and structural features [6, 17, 43, 54]. These AI-based methods outperform many traditional heuristics but often rely heavily on historical execution data, overlooking the contextual information contained in the code changes that trigger each CI cycle. Indeed, although version-control repositories encode rich signals about code evolution, very few TCP techniques leverage the semantics of commits. Every commit conveys structural change informationsuch as the number of altered files, the added or removed lines, and the churn levelthat has long been recognized in defect prediction research as a strong predictor of fault-proneness. However, this knowledge has been underutilized in prioritizing the tests executed after each change. We address this gap by introducing a commit-aware, machine-learning-based TCP method that integrates version control diffs, test coverage information, and historical outcomes into a unified predictive model. We address the hypothesis that commit structural characteristics combined with coverage and execution history provide stronger signals for test failure prediction than historical data alone. The intuition is that tests that exercise recently modified or high-churn components are more likely to detect regressions. By learning the relationships between code evolution and test failures, the proposed approach bridges the traditional “black-box” (based on execution history) and “white-box” (based on coverage measures) paradigms into a strategy suitable for CI environments. In addition to improving responsiveness to current changes, commit-aware TCP helps mitigate cold-start and data sparsity. In fact, history-based methods degrade when many tests have never failed or when the history is short or noisy. The features related to the current diff provide useful signals even with a limited history. The remainder of this paper is structured as follows. Section 2 discusses related work. Section 3 introduces the proposed commit-aware method, while Section 4 describes the experimental methodology. Section 5 reports and analyzes the results, and Section 6 concludes the paper.
2
Related work
TCP has been extensively studied in the context of regression testing and CI, with the goal of accelerating fault detection by reordering the execution of the tests [38, 53]. The majority of prior work can be grouped into three main
paradigms: traditional heuristic-based approaches, search-based/metaheuristic optimization, and learning-based techniques.
2.1
Traditional and Search-Based TCP
Early TCP research focused on deterministic heuristics based on structural coverage and execution cost [38, 12, 13]. Whilst these approaches demonstrated that coverage-driven ordering can significantly accelerate fault detection, they often fail to adapt to evolving code bases and CI dynamics. Subsequent work formulated TCP as an optimization problem, applying metaheuristics such as Genetic Algorithms, Particle Swarm Optimization, and Firefly algorithms to balance multiple objectives including fault detection, execution cost, and severity [25, 5, 46, 21]. While effective in controlled settings, these methods are scarcely scalable and require expensive re-optimization when project characteristics change. Efforts to improve traditional deterministic methods include multi-objective scoring systems that balance effectiveness and resource constraints [42, 41], as well as the Accelerated Greedy Additional algorithm, which specifically targets the efficiency bottleneck of greedy approaches by reducing computational complexity [24]. Information-Retrieval and similarity-based techniques introduce lightweight alternatives that prioritize test diversity and change relevance [33, 15, 36]. Historybased and risk-driven models further exploit execution logs, commit metadata, and defect-prediction metrics to guide prioritization [15, 35, 26, 30, 3, 10]. Largescale industrial studies show that although sophisticated heuristics can be beneficial, simple history- and time-based strategies remain surprisingly competitive in practice [27, 11, 50].
2.2
Learning-Based TCP
Learning-based TCP methods treat prioritization as a predictive ranking problem, learning from historical execution data, coverage, and code metrics [34, 22]. Early industrial approaches applied supervised classifiers to predict failing tests based on execution history and change-aware features [9, 28, 14, 32], while later works employed ensemble models and automated hyperparameter tuning to improve robustness [20]. While some authors focus on model training, others define tools to collect the features evaluated in their approach [52]. More recent research explores reinforcement learning and learning-to-rank formulations, enabling adaptive prioritization under CI feedback loops [6, 45, 8, 40, 4, 37]. Deep learning approaches further model long-term execution patterns and sequential dependencies using neural and recurrent architectures [43, 7, 2, 16, 39, 49, 47, 29]. Transfer learning and clustering-based strategies have been proposed to mitigate data scarcity and volatility across projects [31, 48, 51, 19], while others have framed TCP as a Learning-to-Rank task using SVMRank on natural-language descriptions [23].
Despite these advances, most learning-based approaches rely predominantly on execution history and coverage signals, while structural commit semantics remain weakly exploited, particularly in cross-project generalization settings. The method proposed in this paper directly addresses this limitation by embedding commit-level diff semantics into a unified learning framework and evaluating it under cross-project validation.
3
The method
This section introduces our commit-aware, learning-based TCP method. The goal is to predict, for each test suite and each commit, the probability of observing at least one failure and to prioritize tests accordingly. Whilst traditional TCP focuses on historical data or code coverage, our approach comprises structural code change information extracted from version-control diffs. The underlying hypothesis is that tests exercising recently modified or high-churn components are more likely to expose regressions, and that commit-level change properties provide predictive signals complementary to historical behavior. Given a commit ct , the framework builds a feature vector for each test suite that encodes: (i) historical execution behavior, (ii) structural coverage relations between tests and production code, and (iii) commit-level change relevance signals. A supervised machine-learning model estimates the probability that a given test suite will fail under ct , and test suites are ranked in descending order of predicted failure probability. 3.1
Design Rationale
The proposed method combines black-box historical signals with white-box structural information derived from coverage and code evolution. Commit-level metrics capture change magnitude and locality, which are well-known correlates of fault-proneness in defect prediction research. Historical execution features encode temporal behavioral tendencies of test suites, while coverage features approximate their structural exposure to production code. By combining these three dimensions, the model can reason about both what changed in the system and which tests are structurally and behaviorally most affected by the change. This hybrid representation is particularly suitable for CI environments, where historical data may be sparse or noisy and purely history-based methods suffer from cold-start effects. Commit-aware features provide informative signals even when execution history is limited. 3.2
Feature Representation
Each test suite under a given commit is represented by a compact set of features. All features are computed using only information available prior to executing the current test run, ensuring leakage-free learning. The complete formal definition
of features and the process to compute them is provided in the replication package [1].
Commit-Aware (Diff-Based) Features Commit-aware features capture the structural properties of code changes introduced by the current commit. From the version-control diff, we extract basic change metrics such as: the number of modified files, added lines, and removed lines. These metrics are originally defined at file or class level. To obtain testlevel representations, we associate each test suite with the production classes it covers, using a heuristic mapping [1], and propagate diff information from modified classes to their corresponding test suites. If a test suite covers multiple modified components, diff metrics are aggregated. The resulting features approximate the structural relevance of each test suite with respect to the current change, capturing both the size of the change and its proximity to the code exercised by the test.
Historical Features These features summarize the past behavior of test suites. We consider: (i) the previous pass rate, representing the most recent execution outcome, and (ii) a weighted historical pass rate, computed over multiple past versions. The weighted pass rate combines temporal recency with commit similarity, assigning higher importance to recent executions and to past commits with similar change intensity. This feature estimates the expected reliability of a test suite under the current commit.
Coverage Features Coverage features describe the structural relationship between test suites and production code. We compute: (i) the suite coverage ratio, indicating how much of the system is exercised by the test suite, and (ii) a combined coverage–diff signal, capturing the extent to which a test suite covers recently modified components. This combination allows the model to identify test suites that are both structurally broad and directly exposed to the latest code changes.
3.3
Machine Learning Models
We evaluate two representative supervised learning models with complementary characteristics: gradient-boosted decision trees (XGBoost) and a feedforward neural network (MLP). XGBoost is well suited for imbalanced tabular data and provides interpretable feature importance, while MLPs can capture non-linear interactions among heterogeneous features. Both models are trained to estimate the probability that a test suite will fail under a given commit.
3.4
Training and Test Prioritization
Models are trained using a leave-one-project-out strategy to evaluate crossproject generalization. Details are provided in the replication package [1]. At prediction time, each model outputs a failure probability for each test suite associated with the current commit. Test suites are then prioritized in descending order of predicted failure probability, producing the final execution order evaluated in the experimental phase.
4
Experimental setup
This section describes the experimental methodology adopted to evaluate the proposed commit-aware TCP approach. We report research questions, dataset and data processing strategies, evaluation metrics, design and analyzes. Our automated data collection pipeline, detailed implementation artifacts and scripts are provided in the publicly available replication package [1]. 4.1
Research Questions
We address the following research questions: – RQ1: Can commit-aware features improve test failure prediction? – RQ2: Does improved failure prediction translate into more effective test case prioritization? – RQ3: How do different supervised learning models compare when using the same commit-aware feature set? RQ1 evaluates the predictive contribution of commit-aware features using classification metrics. RQ2 assesses whether improved prediction leads to faster and more effective fault detection. RQ3 compares the behavior of different learning models under identical feature representations, allowing us to isolate modelspecific effects from feature-driven improvements. 4.2
Metrics
To answer RQ1, we evaluate failure prediction performance using Precision– Recall AUC (PR-AUC) and F1-score. These metrics are appropriate for highly imbalanced classification problems, as they emphasize performance on the minority (failing) class. To answer RQ2, we evaluate test case prioritization effectiveness using: (i) Average Percentage of Faults Detected (APFD), that measures the average rate of fault detection per percentage of test suite execution [12], and; (ii) Speedup, measuring the reduction in First-Time-To-Failure relative to chronological execution. RQ3 is addressed by comparing learning models using the same feature set across both prediction (PR-AUC, F1) and prioritization (APFD, Speedup) metrics.
4.3
Dataset
We evaluate the proposed approach on five projects from the Defects4J benchmark [18], namely Math, Lang, Jsoup, Time, JacksonDatabind. Defects4J provides real-world Java faults, reproducible regression testing scenarios, and versioned project histories, and is widely used in regression testing research. Each project consists of multiple faulty program versions, associated test suites, and execution outcomes. Heuristic mapping Commit-aware features are originally defined at file or class level, while prediction is performed at test-suite level. To bridge this gap, we apply a heuristic mapping, based on similarity of file names, that associates each test suite with the production classes it covers. Diff-based metrics computed on modified classes are then propagated to the corresponding test suites and aggregated when multiple classes are involved. This mapping enables the estimation of change relevance at test level while remaining scalable and lightweight. While simple, the heuristic has proven sufficiently robust: we manually checked a sample of 415 test-class mappings taken from the Math project and observed that 89.16% of test suites are linked to at least one Java class. Dataset imbalance The dataset exhibits strong class imbalance, as failing test suites represent a small fraction of the total observations. To mitigate this issue, we apply standard resampling techniques during training. Specifically, we applied SMOTE oversampling and random undersampling for the NN-based model, and the scale_pos_weight parameter tuning for the decision-tree-based model. These strategies prevent learning models from being biased toward the majority class and improve sensitivity to failure instances. Data imputation In some cases, the heuristic mapping was unable to associate the code changes with the test suites for each commit. To address missing diff-metric values arising from imperfect mappings between production classes and test suites, we applied a data imputation strategy that leverages an Exponentially Weighted Moving Average mechanism with local and global fallbacks to replace null values with statistically coherent estimates. This ensured that the learning models received consistent feature vectors and preserved the structural integrity of positive examples without exacerbating the existing class imbalance. 4.4
Design
We use a leave-one-out cross-project strategy: five different clusters each with four training projects and a test project. In the evaluation of the results, we considered only four out of the five folds. The fold where the Time project represents the test set was kept out of the evaluation as this project, due to its structure, is an outlier with all datapoints in the positive class.
First, we train the models including all the selected features, and evaluate the classification performance and the prioritization scores. Then, to quantify the contribution of diff-based information, we performed an ablation study in which we retrain the models after removing all commit-aware features. The goal of the ablation study is to evaluate whether the inclusion of commitaware features produces statistically significant improvements in both classification and prioritization performance. We test the following hypotheses: – H01 (classification): the removal of diff-based features does not change classification performance; – H02 (prioritization): removing diff-based features does not change the effectiveness of TCP. Statistical Analysis We apply the Wilcoxon signed-rank test for statistical significance and Cliff’s delta for effect size. As descriptive statistics, we report Q1, median (Q2), and Q3.
5
Results and discussion
This section presents the experimental results, addressing the 3 research questions. 5.1
RQ1: commit-aware feature effectiveness on test failure prediction
The ablation study highlights the critical role of the diff features in the classification performance of both the XGBoost and the MLP models. When these features are removed, performances degrade across all four projects, as shown in table 1. Specifically, F1 and PR-AUC values drop from high-performance levels to near-zero values, indicating that the models lose their discriminative power without these specific inputs. The analysis of the bootstrap quartiles reveals large differences in performance stability between the two experimental configurations. With diff-based features, the models exhibit a robust statistical profile, characterized by high median values and narrow interquartile ranges across the validation folds. This indicates that the predictive performance is not only elevated but also highly consistent, with minimal variance in the model’s output. In contrast, the removal of these features outlines a collapse of the distribution. In the ablation scenario, the entire quartile spread shifts toward zero, and the upper bounds fail to reach meaningful classification thresholds. The absence of overlap between the distributions of the two scenarios confirms that the diff-based signals are not merely additive enhancements but are the foundation required for the models to achieve any discriminative capability. Statistical analysis in table 2 confirms this impact. The Cliff’s delta for both models and all classification metrics is 1.000, meaning a large effect size and
demonstrating that the with-features configuration consistently outperforms the baseline across all folds. Furthermore, the Wilcoxon signed-rank test exhibit a pvalue of 0.0625 for all metrics. While slightly above the traditional 0.05 threshold, this value represents the minimum possible p-value for a sample size of n = 4, thus providing the strongest statistical evidence achievable for our experimental setup.
Answer to RQ1: Based on the experimental results and statistical analysis, the hypothesis H01 can be rejected. The ablation of diff-based features leads to significant and systematic performance degradation in the classification task. This finding is further supported by the feature importance chart shown in figure 1: the diff-based features are the ones that most influence the decisionmaking process of the model.
Table 1. Classification report for both models on each fold. For each metric we report the Q1, Q2 and Q3 values. The number of bootstrap replications is 5000 per fold. Metric Diff features
XGBoost
MLP
Lang F1
Yes No
(0.922, 0.939, 0.956) (0.000, 0.022, 0.039)
(0.845, 0.870, 0.891) (0.126, 0.151, 0.180)
PR-AUC
Yes No
(0.857, 0.891, 0.923) (0.024, 0.028, 0.033)
(0.898, 0.923, 0.951) (0.057, 0.074, 0.094)
JacksonDatabind F1
Yes No
(0.909, 0.941, 0.962) (0.018, 0.034, 0.051)
(0.815, 0.857, 0.900) (0.000, 0.000, 0.000)
PR-AUC
Yes No
(0.964, 0.980, 0.996) (0.014, 0.018, 0.023)
(1.000, 1.000, 1.000) (0.052, 0.072, 0.095)
Jsoup F1
Yes No
(0.863, 0.879, 0.895) (0.000, 0.000, 0.000)
(0.291, 0.325, 0.359) (0.046, 0.061, 0.076)
PR-AUC
Yes No
(0.908, 0.928, 0.946) (0.069, 0.076, 0.083)
(0.329, 0.364, 0.400) (0.050, 0.055, 0.060)
Math F1
Yes No
(0.882, 0.895, 0.908) (0.011, 0.015, 0.019)
(0.743, 0.760, 0.778) (0.015, 0.019, 0.023)
PR-AUC
Yes No
(0.791, 0.817, 0.841) (0.009, 0.010, 0.011)
(0.452, 0.477, 0.504) (0.007, 0.009, 0.010)
Fig. 1. XGBoost feature importance on Lang test set
5.2
RQ2: improved prediction effectiveness on fault detection
The analysis of the prioritization quartiles reveals a distinct behavioral shift compared to the total collapse observed in the classification task, as shown in table 3. The APFD Gain demonstrates a surprising resilience in the absence of diff-based features. This indicates that the models retain a good ability to order test suites effectively by leveraging residual signals, such as historical pass rates or coverage data. However, this resilience does not translate to the Speedup metric. Here, the removal of diff-based features causes a drastic degradation. This dichotomy suggests that, while the models can achieve a generally good sorting quality without diff features, they lose the ability to identify the first failure immediately. This dichotomy is statistically confirmed by table 2: the Cliffs δ for APFD gain indicates only a moderate effect size compared to the maximum separation seen in classification, whereas the Speedup metric reveals a large Mean δ Confidence Interval, confirming that, while general ordering is robust, the ability to rapidly detect failures is strictly dependent on diff-based features. Answer to RQ2: The evidence regarding H02 does not support rejection, as the prioritization performance gap is narrower. Although the removal of diff-based features severely impacts the time-to-first-failure (Speedup), the resilience of the APFD Gain, characterized by overlapping distributions, demonstrates that the overall efficiency of the test ordering is not entirely compromised. 5.3
RQ3: model comparison
The comparative analysis of the bootstrap distributions definitively establishes XGBoost as the more robust and reliable model. In classification tasks (table 1), XGBoost demonstrates superior stability, characterized by consistently narrower
Table 2. Cliff’s δ and Wilcoxon p-value and mean δ CI (low-high interval) for XGBoost and MLP metrics on each fold prediction. Number of bootstrap samples: n = 5000; alternative hypothesis for Wilcoxon signed-rank test: alternative=’greater’; number of pairs: p = 4. Mean δ represents the per-fold difference between the two mean metric values from the with and without features scenario. Model
Metric
Cliff δ
p-value
Mean δ CI (low-high)
F1 PR-AUC XGB APFD gain Speedup
1.000 1.000 0.500 0.750
0.0625 0.0625 0.0625 0.0625
0.877-0.912 0.820-0.928 0.053-0.169 258.750-1622.643
F1 PR-AUC MLP APFD gain Speedup
1.000 1.000 0.375 0.500
0.0625 0.0625 0.0625 0.0625
0.383-0.821 0.388-0.889 0.140-0.509 0.232-116.418
interquartile ranges and significantly higher median values across challenging projects. Regarding prioritization (table 3), the distinction becomes nuanced: MLP occasionally achieves higher peak efficiency, as seen in the Math project, where its upper quartile Speedup reaches higher values. However, this peak performance is inconsistent, as the MLP fails to maintain this advantage in other contexts, dropping well below XGBoost. Answer to RQ3: Whilst the neural network can sporadically outperform the ensemble method, XGBoost exhibits better stability. It offers the predictive consistency required for a reliable CI environment, avoiding the performance fluctuations inherent to the MLP architecture, while ensuring operational advantages in terms of interpretability and implementation efficiency compared to an NNbased model.
6
Conclusion
We introduced a commit-aware, learning-based approach for Test Case Prioritization in Continuous Integration environments. By integrating structural information extracted from version-control diffs with historical execution data and coverage metrics, our method explicitly accounts for the characteristics of the current code change when prioritizing test execution. Empirical evaluation on five Defects4J projects shows that commit-level signals play a central role in enabling accurate failure prediction. The ablation study demonstrates that the removal of diff-based features leads to a systematic collapse of classification performance, confirming that structural properties of code changes convey essential information about the likelihood of test failure. Whilst prioritization metrics exhibit greater resilience to removal of commitaware features, models equipped with these features consistently achieve faster
Table 3. Prioritization metrics for both models on each fold. For each metric we report the Q1, Q2 and Q3 values. The number of bootstrap replications is 5000 per fold. Metric
Diff features
XGBoost
MLP
Lang APFD Gain
Yes No
(0.506, 0.553, 0.599) (0.453, 0.503, 0.552)
(0.501. 0.547, 0.594) (0.477, 0.522, 0.571)
Speedup
Yes No
(809, 983, 1173) (62, 79, 99)
(801, 975, 1168) (127, 161, 202)
JacksonDatabind APFD Gain
Yes No
(0.664, 0.711, 0.753) (0.607, 0.654, 0.697)
(0.659, 0.709, 0.751) (0.596, 0.644, 0.685)
Speedup
Yes No
(504, 663, 745) (20, 25, 31)
(568, 648, 728) (14, 16, 18)
Jsoup APFD Gain
Yes No
(0.541, 0.566, 0.589) (0.326, 0.360, 0.392)
(0.440, 0.466, 0.492) (-0.112, -0.073, -0.033)
Speedup
Yes No
(36, 43, 51) (6, 7, 8)
(15, 19, 23) (6, 9, 14)
Math APFD Gain
Yes No
(0.459, 0.482, 0.505) (0.334, 0.363, 0.392)
(0.460, 0.485, 0.510) (0.395, 0.422, 0.450)
Speedup
Yes No
(1864, 2149, 2455) (88, 103, 121)
(3273, 4168, 5218) (78, 101, 126)
fault detection and more effective execution orders. Across cross-project scenarios, tree-based models, particularly XGBoost, prove more robust and stable than neural models. This study also highlights important limitations. Internal threats: data preprocessing, such as class rebalancing and selective imputation, while necessary for effective learning, may move the setting away from a production-like CI pipeline. Structural threats: the approach relies on a heuristic mapping between production classes and test suites, which enables scalable automation but may introduce noise when testcode relationships are complex. External threats: the evaluation is limited to Defects4J and results cannot be directly generalized to industrial pipelines with different scales and workflows. In our study, models are trained offline in a static cross-project setting, whereas real CI environments evolve over time.
Despite these limitations, our results demonstrate that commit-aware TCP is both feasible and effective. Future work will extend the evaluation to industrial datasets, explore richer semantic representations of commits, and investigate online and incremental learning strategies to improve adaptability to long-running CI pipelines.
References 1. Abbondante, L., Canfora, G.: Replication package for commit-aware learningbased test case prioritization for continuous integration. Zenodo (2026). https://doi.org/https://doi.org/10.5281/zenodo.19355771 2. Abdelkarim, M., ElAdawi, R.: Tcp-net++: Test case prioritization using end-toend deep neural networks - deployment analysis and enhancements. In: IEEE International Conference On Artificial Intelligence Testing. pp. 99–106. IEEE (2023). https://doi.org/10.1109/AITEST58265.2023.00024 3. Ahmed, F.S., Majeed, A., Khan, T.A., Bhatti, S.N.: Value-based cost-cognizant test case prioritization for regression testing. PLOS ONE 17(5), 1–26 (05 2022). https://doi.org/10.1371/journal.pone.0264972 4. Alrakban, N.A., Alrashoud, M., Abdullah-Al-Wadud, M.: Optimizing test case prioritization with meta deep reinforcement learning in continuous integration. IEEE Access 13, 172435–172455 (2025). https://doi.org/10.1109/ACCESS.2025.3617387 5. Ashraf, E., Mahmood, K., Khan, T.A., Ahmed, S.: Value based pso test case prioritization algorithm. International Journal of Advanced Computer Science and Applications 8(1) (2017). https://doi.org/10.14569/IJACSA.2017.080149 6. Bagherzadeh, M., Kahani, N., Briand, L.C.: Reinforcement learning for test case prioritization. IEEE Trans. Software Eng. 48(8), 2836–2856 (2022). https://doi.org/10.1109/TSE.2021.3070549 7. Behera, A., Acharya, A.A.: An effective gru-based deep learning method for test case prioritization in continuous integration testing. Procedia Computer Science 258, 4070–4083 (2025). https://doi.org/https://doi.org/10.1016/j.procs.2025.04.658, international Conference on Machine Learning and Data Engineering 8. Bertolino, A., Guerriero, A., Miranda, B., Pietrantuono, R., Russo, S.: Learning-torank vs ranking-to-learn: strategies for regression testing in continuous integration. In: ICSE ’20: 42nd International Conference on Software Engineering. pp. 1–12. ACM (2020). https://doi.org/10.1145/3377811.3380369 9. Busjaeger, B., Xie, T.: Learning for test prioritization: an industrial case study. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. pp. 975–980. ACM (2016). https://doi.org/10.1145/2950290.2983954 10. Chen, Z., Chen, J., Wang, W., Zhou, J., Wang, M., Chen, X., Zhou, S., Wang, J.: Exploring better black-box test case prioritization via log analysis. ACM Trans. Softw. Eng. Methodol. 32(3), 72:1–72:32 (2023). https://doi.org/10.1145/3569932 11. Cheng, R., Wang, S., Jabbarvand, R., Marinov, D.: Revisiting test-case prioritization on long-running test suites. In: Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. pp. 615–627. ACM (2024). https://doi.org/10.1145/3650212.3680307 12. Elbaum, S.G., Malishevsky, A.G., Rothermel, G.: Test case prioritization: A family of empirical studies. IEEE Trans. Software Eng. 28(2), 159–182 (2002). https://doi.org/10.1109/32.988497
13. Graves, T.L., Harrold, M.J., Kim, J., Porter, A.A., Rothermel, G.: An empirical study of regression test selection techniques. In: Proceedings of the 1998 International Conference on Software Engineering. pp. 188–197. IEEE Computer Society (1998). https://doi.org/10.1109/ICSE.1998.671115 14. Hajri, I., Goknil, A., Pastore, F., Briand, L.C.: Automating system test case classification and prioritization for use case-driven testing in product lines. Empir. Softw. Eng. 25(5), 3711–3769 (2020). https://doi.org/10.1007/S10664-020-09853-4 15. Hemmati, H., Fang, Z., Mäntylä, M.V.: Prioritizing manual test cases in traditional and rapid release environments. In: 8th IEEE International Conference on Software Testing, Verification and Validation. pp. 1–10. IEEE Computer Society (2015). https://doi.org/10.1109/ICST.2015.7102602 16. Jabbar, E., Hemmati, H., Feldt, R.: Investigating execution trace embedding for test case prioritization. In: 23rd IEEE International Conference on Software Quality, Reliability, and Security. pp. 279–290. IEEE (2023). https://doi.org/10.1109/QRS60937.2023.00036 17. Jabbar, E., Zangeneh, S., Hemmati, H., Feldt, R.: Test2vec: An execution trace embedding for test case prioritization. CoRR abs/2206.15428 (2022). https://doi.org/10.48550/ARXIV.2206.15428 18. Just, R., Jalali, D., Ernst, M.D.: Defects4j: a database of existing faults to enable controlled testing studies for java programs. In: International Symposium on Software Testing and Analysis, ISSTA. pp. 437–440. ACM (2014). https://doi.org/10.1145/2610384.2628055 19. Karatayev, A., Ogorodova, A., Shamoi, P.: Fuzzy inference system for test case prioritization in software testing. CoRR abs/2404.16395 (2024). https://doi.org/10.48550/ARXIV.2404.16395 20. Khan, M.A., Azim, A., Liscano, R., Smith, K., Chang, Y., Seferi, G., Tauseef, Q.: An end-to-end test case prioritization framework using optimized machine learning models. In: IEEE International Conference on Software Testing, Verification and Validation. pp. 1–8. IEEE (2024). https://doi.org/10.1109/ICSTW60967.2024.00014 21. Khatibsyarbini, M., Isa, M.A., Jawawi, D.N.A., Hamed, H.N.A., Suffian, M.D.M.: Test case prioritization using firefly algorithm for software testing. IEEE Access 7, 132360–132373 (2019). https://doi.org/10.1109/ACCESS.2019.2940620 22. Khatibsyarbini, M., Isa, M.A., Jawawi, D.N.A., Shafie, M.L.M., Wan-Kadir, W.M.N., Hamed, H.N.A., Suffian, M.D.M.: Trend application of machine learning in test case prioritization: A review on techniques. IEEE Access 9, 166262–166282 (2021). https://doi.org/10.1109/ACCESS.2021.3135508 23. Lachmann, R., Schulze, S., Nieke, M., Seidl, C., Schaefer, I.: System-level test case prioritization using machine learning. In: 15th IEEE International Conference on Machine Learning and Applications. pp. 361–368. IEEE Computer Society (2016). https://doi.org/10.1109/ICMLA.2016.0065 24. Li, F., Zhou, J., Li, Y., Hao, D., Zhang, L.: AGA: an accelerated greedy additional algorithm for test case prioritization. IEEE Trans. Software Eng. 48(12), 5102– 5119 (2022). https://doi.org/10.1109/TSE.2021.3137929 25. Li, Z., Harman, M., Hierons, R.M.: Search algorithms for regression test case prioritization. IEEE Trans. Software Eng. 33(4), 225–237 (2007). https://doi.org/10.1109/TSE.2007.38 26. Liang, J., Elbaum, S.G., Rothermel, G.: Redefining prioritization: continuous prioritization for continuous integration. In: Proceedings of the 40th International Conference on Software Engineering. pp. 688–698. ACM (2018). https://doi.org/10.1145/3180155.3180213
27. Luo, Q., Moran, K., Poshyvanyk, D., Penta, M.D.: Assessing test case prioritization on real faults and mutants. In: 2018 IEEE International Conference on Software Maintenance and Evolution. pp. 240–251. IEEE Computer Society (2018). https://doi.org/10.1109/ICSME.2018.00033 28. Machalica, M., Samylkin, A., Porth, M., Chandra, S.: Predictive test selection. In: Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice. pp. 91–100. IEEE / ACM (2019). https://doi.org/10.1109/ICSE-SEIP.2019.00018 29. Mahdieh, M., Mirian-Hosseinabadi, S., Etemadi, K., Nosrati, A., Jalali, S.: Incorporating fault-proneness estimations into coverage-based test case prioritization methods. Inf. Softw. Technol. 121, 106269 (2020). https://doi.org/10.1016/J.INFSOF.2020.106269 30. Mahdieh, M., Mirian-Hosseinabadi, S., Mahdieh, M.: Test case prioritization using test case diversification and fault-proneness estimations. Autom. Softw. Eng. 29(2), 50 (2022). https://doi.org/10.1007/S10515-022-00344-Y 31. Mamata, R., Azim, A., Liscano, R., Smith, K., Chang, Y., Seferi, G., Tauseef, Q.: Test case prioritization using transfer learning in continuous integration environments. In: IEEE/ACM International Conference on Automation of Software Test. pp. 191–200. IEEE (2023). https://doi.org/10.1109/AST58925.2023.00023 32. Marijan, D.: Comparative study of machine learning test case prioritization for continuous integration testing. Softw. Qual. J. 31(4), 1415–1438 (2023). https://doi.org/10.1007/S11219-023-09646-0 33. Miranda, B., Cruciani, E., Verdecchia, R., Bertolino, A.: FAST approaches to scalable similarity-based test case prioritization. In: Proceedings of the 40th International Conference on Software Engineering. pp. 222–232. ACM (2018). https://doi.org/10.1145/3180155.3180210 34. Pan, R., Bagherzadeh, M., Ghaleb, T.A., Briand, L.C.: Test case selection and prioritization using machine learning: a systematic literature review. Empir. Softw. Eng. 27(2), 29 (2022). https://doi.org/10.1007/S10664-021-10066-6 35. Paterson, D., Campos, J., Abreu, R., Kapfhammer, G.M., Fraser, G., McMinn, P.: An empirical study on the use of defect prediction for test case prioritization. In: 12th IEEE Conference on Software Testing, Validation and Verification. pp. 346–357. IEEE (2019). https://doi.org/10.1109/ICST.2019.00041 36. Peng, Q., Shi, A., Zhang, L.: Empirically revisiting and enhancing ir-based test-case prioritization. In: ISSTA ’20: 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. pp. 324–336. ACM (2020). https://doi.org/10.1145/3395363.3397383 37. Qian, Z., Yu, Q., Zhu, H., Liu, J., Fu, T.: Reinforcement learning for test case prioritization based on lleed k-means clustering and dynamic priority factor. Inf. Softw. Technol. 179, 107654 (2025). https://doi.org/10.1016/J.INFSOF.2024.107654 38. Rothermel, G., Untch, R.H., Chu, C., Harrold, M.J.: Prioritizing test cases for regression testing. IEEE Trans. Software Eng. 27(10), 929–948 (2001). https://doi.org/10.1109/32.962562 39. da Roza, E.A., Lima, J.A.P., Silva, R.C., Vergilio, S.R.: Machine learning regression techniques for test case prioritization in continuous integration environment. In: IEEE International Conference on Software Analysis, Evolution and Reengineering. pp. 196–206. IEEE (2022). https://doi.org/10.1109/SANER53432.2022.00034 40. da Roza, E.A., do Prado Lima, J.A., Vergilio, S.R.: On the use of contextual information for machine learning based test case prioritization in continuous integration development. Inf. Softw. Technol. 171, 107444 (2024). https://doi.org/10.1016/J.INFSOF.2024.107444
41. Samad, A., Mahdin, H., Kazmi, R., Ibrahim, R.: Regression test case prioritization: A systematic literature review. International Journal of Advanced Computer Science and Applications 12(2) (2021). https://doi.org/10.14569/IJACSA.2021.0120282 42. Samad, A., Mahdin, H.B., Kazmi, R., Ibrahim, R., Baharum, Z.: Multiobjective test case prioritization using test case effectiveness: Multicriteria scoring method. Sci. Program. 2021, 9988987:1–9988987:13 (2021). https://doi.org/10.1155/2021/9988987 43. Sharif, A., Marijan, D., Liaaen, M.: Deeporder: Deep learning for test case prioritization in continuous integration testing. In: IEEE International Conference on Software Maintenance and Evolution. pp. 525–534. IEEE (2021). https://doi.org/10.1109/ICSME52107.2021.00053 44. Singhal, S., Kumar, S., Kumar, R., Mohanty, M.: Multi-objective fault-coverage based regression test selection using enhanced aco_tcsp. Mathematics 11(5), 1101 (2023). https://doi.org/10.3390/math11051101 45. Spieker, H., Gotlieb, A., Marijan, D., Mossige, M.: Reinforcement learning for automatic test case prioritization and selection in continuous integration. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. pp. 12–22. ACM (2017). https://doi.org/10.1145/3092703.3092709 46. Su, W., Li, Z., Wang, Z., Yang, D.: A meta-heuristic test case prioritization method based on hybrid model. In: 2020 International Conference on Computer Engineering and Application. pp. 430–435 (2020). https://doi.org/10.1109/ICCEA50009.2020.00099 47. Vescan, A., Gaceanu, R.D., Szederjesi-Dragomir, A.: Neural network-based test case prioritization in continuous integration. In: 38th IEEE/ACM International Conference on Automated Software Engineering. pp. 68–77. IEEE (2023). https://doi.org/10.1109/ASEW60602.2023.00014 48. Vescan, A., Serban, C.: Towards a new test case prioritization approach based on fuzzy clustering analysis. In: IEEE International Conference on Software Maintenance and Evolution. pp. 786–788. IEEE (2020). https://doi.org/10.1109/ICSME46990.2020.00091 49. Vescan, A., Tiutin, C.: Test case prioritization based on neural networks classification: A replication study and hyper-parameter optimization using taguchi methods. IEEE Access 13, 118082–118095 (2025). https://doi.org/10.1109/ACCESS.2025.3586144 50. Wang, H., Yu, R., Wang, D., Du, Y., Zhao, Y., Chen, J., Wang, Z.: An empirical study of test case prioritization on the linux kernel. Autom. Softw. Eng. 32(2), 49 (2025). https://doi.org/10.1007/S10515-025-00522-8 51. Wang, X., Zhang, S.: Cluster-based adaptive test case prioritization. Inf. Softw. Technol. 165, 107339 (2024). https://doi.org/10.1016/J.INFSOF.2023.107339 52. Yaraghi, A.S., Bagherzadeh, M., Kahani, N., Briand, L.C.: Scalable and accurate test case prioritization in continuous integration contexts. IEEE Trans. Software Eng. 49(4), 1615–1639 (2023). https://doi.org/10.1109/TSE.2022.3184842 53. Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw. Test. Verification Reliab. 22(2), 67–120 (2012). https://doi.org/10.1002/STV.430 54. Zhao, Y., Hao, D., Zhang, L.: Revisiting machine learning based test case prioritization for continuous integration. In: IEEE International Conference on Software Maintenance and Evolution. pp. 232–244. IEEE (2023). https://doi.org/10.1109/ICSME58846.2023.00032