arXiv CS open access

RiskBlend: A Multi-Signal Framework for Test Input Prioritization in Machine Learning Regression Testing

2026 · arxiv_cs

arXiv CS · Papers · 2026

software-architecture software-engineering testing

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

arXiv:2608.27704v1 [cs.LG] 27 Aug 2026

RiskBlend: A Multi-Signal Framework for Test Input Prioritization in Machine Learning Regression Testing 1st Madhusudan Srinivasan

2nd Namith Nishal Raphael

Department of Computer Science East Carolina University Greenville, USA srinivasanm23@ecu.edu

Department of Computer Science East Carolina University Greenville, USA

Abstract—When machine learning classifiers are retrained, inputs that were correctly classified by the previous model version may be misclassified by the updated version: regression faults that are costly to detect because verifying predictions against ground truth requires human annotation, expert review, or expensive simulation rather than cheap model inference. Test input prioritization addresses this by ranking inputs so that a limited verification budget surfaces as many regression faults as possible. Existing approaches rely predominantly on single-model confidence scores, which do not exploit how predictions, decision boundaries, and local neighborhoods change between model versions. We propose RiskBlend, a classifier-agnostic prioritization framework that fuses four complementary risk signals, historical failure patterns, prediction shift, decision-boundary shift, and neighborhood change, into a single score via validation-learned APFD-squared weighting. Across four datasets, five classifiers, four regression-update scenarios, and 15 random seeds (1,200 configurations per dataset), RiskBlend achieves the highest average APFD in all 80 classifier–scenario combinations, with margins of up to 0.32 APFD over the strongest baseline. Confidencebased methods remain competitive only on linear classifiers over sparse categorical features, a limitation we attribute to featurespace geometry. These results demonstrate that cross-version behavioral signals are essential complements to model-output confidence for robust prioritization in ML regression testing. Index Terms—Regression Testing, Test input prioritization, Machine learning

I. I NTRODUCTION AND M OTIVATION Machine learning (ML) classifiers have become a critical component of modern software systems. They are increasingly deployed in high-stakes domains such as finance, security, and especially medicine, where predictive models support tasks that include disease diagnosis, prediction of patient readmission, and risk stratification. In such settings, ensuring the reliability of ML models is essential, as incorrect or biased predictions can directly impact safety, equity, and trust. ML systems differ fundamentally from traditional software in that their behavior depends not only on the program logic but also on the continuously evolving data used to train them. Organizations regularly collect new data, refine feature pipelines, and train classifiers to improve accuracy or mitigate bias. Each such update requires regression testing

and verifying that the updated model does not introduce new failures on inputs the previous version handled correctly. Unlike classical software regression testing, where executing a test and verifying its outcome are a single inseparable step, ML regression testing decouples these two costs. Running the updated model on a test input is computationally cheap, requiring only a single forward pass. However, determining whether that prediction is correct requires a comparison against ground truth, which is rarely available for free: it demands human annotation, delayed outcome observation, or expensive simulation. This asymmetry means that the dominant cost of ML regression testing is not model inference but oracle invocation, the process of verifying predictions against ground truth. To motivate this problem, consider a hospital using an ML model to predict the risk of readmission from the ICU. After retraining with new patient data, the updated model must be verified against the previous version in thousands of records. Although prediction generation is computationally cheap, verifying each prediction requires the physician’s review, often taking 10–15 minutes per case. With limited verification resources, only a small subset of records can be checked. Randomly selecting cases wastes valuable expert time, as most cases reveal no regression, whereas prioritizing high-risk input increases the likelihood of detecting newly introduced faults before deployment. This motivates the need for prioritization of test input in ML regression testing. Previous work on ML test prioritization includes uncertainty, transformation, and mutation-based approaches. DeepGini [4] ranks inputs using prediction uncertainty but is based on a single-model confidence signal. Metamorphic-relation prioritization [10] orders fault-revealing transformations, but requires domain-specific MR design, while MLPrior [3] combines mutation-based features and learning-to-rank at additional mutation cost. Other neural-network approaches [1], [8] rely similarly primarily on confidence, uncertainty, or surprise measures. In contrast, R ISK B LEND targets regression testing by combining four classifier-agnostic signals that capture historical and cross-version behavior. Their weights are learned from validation regression faults, avoiding classifier-specific

tuning and mutation generation while prioritizing inputs likely to expose newly introduced faults. Here are the main contributions of this work: 1) We propose RiskBlend, a novel regression-aware test input prioritization framework for machine learning systems that prioritizes test instances using blended risk signals to improve early regression fault detection. 2) We develop a validation-driven weight learning mechanism that automatically learns the contribution of each risk signal by optimizing APFD on a validation set, eliminating the need for manually tuned weights and improving adaptability across datasets and classifiers. 3) We perform a comprehensive empirical evaluation on four benchmark datasets, four regression-update scenarios, five classifier families, and multiple random seeds. II. BACKGROUND A. Regression Testing for Machine Learning Unlike traditional software, the behavior of machine learning (ML) systems depends not only on program logic but also on the data used for training. Updates caused by retraining, feature engineering, or data augmentation can shift decision boundaries and change model predictions in ways that are often difficult to anticipate. Regression testing is therefore essential to ensure that an updated model version does not introduce new faults compared to the previous version. Let Vk denote the original (previous) model version and Vk+1 denote the updated model version after retraining or modification. In this work, we define a regression fault as a test instance for which the original model predicts correctly, but the updated model predicts incorrectly. Formally, for a test instance x with ground-truth label y, a regression fault occurs when f (Vk , x) = y ∧ f (Vk+1 , x) ̸= y

(1)

where f (V, x) denotes the prediction made by model version V on input x. Thus, a regression fault represents a newly introduced failure caused by the model update, rather than simply a disagreement between the two model versions. B. Test Input Prioritization Test input prioritization is a regression testing technique that orders test instances by their estimated fault-revealing potential so that instances most likely to expose regressions are verified first. Formally, given a candidate test set T = {x1 , x2 , . . . , xn } and an oracle budget B ≪ n, the goal is to produce a ranking π over T such that verifying the top-B ranked instances against ground truth recovers as many of the true regression faults in T as possible. Effective prioritization reduces the time-to-feedback for fault detection and allows the verification effort to be concentrated where it is most likely to matter, without requiring the full test suite to be labeled.

III. R ELATED W ORK Test input prioritization has been widely studied for deep learning systems, but remains comparatively underexplored for classical ML classifiers under regression-testing conditions. We review the most relevant approaches and position R ISK B LEND with respect to mutation-based, confidencebased, diversity-based, and multi-signal prioritization methods. The work most closely related to ours is MLPrior by Dang et al. [3], a mutation-based test input prioritization framework for classical ML classifiers. MLPrior ranks test instances using mutation-derived features, decision-boundary proximity, and a learning-to-rank model to estimate misclassification likelihood. It addresses several limitations of coverage- and confidence-based prioritization, but operates in a single-model setting and focuses on generic misclassification. In contrast, R ISK B LEND is designed specifically for regression testing under model evolution, prioritizing inputs likely to reveal newly introduced faults between Vk and Vk+1 . RiskBlend explicitly models cross-version prediction change, decisionboundary movement, neighborhood change, and historical failure information, while avoiding the need to generate artificial mutants. GraphRank [13] similarly combines heterogeneous modelaware and model-agnostic attributes to prioritize test inputs through an iteratively trained binary classifier. Although this multi-signal design is conceptually related to RiskBlend, GraphRank is tailored to graph-structured data, relies on graph-topological features and an active-learning loop, and does not model regression faults or cross-version evolution. In the broader DNN testing literature, Kim et al. [6] propose Surprise Adequacy, which measures how much a test input deviates from training-time behavior using neuron activation traces. This idea reflects the broader intuition that atypical inputs may be more fault-revealing, but it depends on internal DNN representations. Wang et al. [11] propose PRIMA, which uses model and input mutation analysis to construct features for a learning-to-rank model. PRIMA therefore requires generating many artificial mutants, whereas RiskBlend derives its cross-version signals directly from the naturally available model versions. Among confidence-based approaches, DeepGini [4] ranks inputs using the Gini impurity of the predicted probability distribution and has shown strong APFD performance over coverage-based alternatives. However, it relies on uncertainty from a single model version. DATIS [7] extends this direction by combining uncertainty with distance-based information from training neighborhoods and inter-test diversity to reduce redundant selections. RiskBlend also incorporates neighborhood and boundary-sensitive information, but differs in that these signals are explicitly version-aware and are used to detect regression faults rather than generic misclassifications. Other approaches combine multiple objectives or sampling criteria. MOTS [5] jointly optimizes uncertainty and diversity for selecting DNN test inputs for retraining, while SSOA [12] uses stratified sampling based on predictive confidence to

estimate model accuracy under limited labeling budgets. These methods demonstrate the value of combining complementary signals, but their objectives differ from regression fault prioritization. R ISK B LEND instead learns how to combine classifieragnostic historical and cross-version signals so that inputs most likely to expose newly introduced faults are ranked earlier under a limited oracle budget. IV. P ROPOSED A PPROACH This section presents RiskBlend, a regression-aware test prioritization framework for machine learning (ML) classification systems. Given two versions of a model, Vk and Vk+1 , the objective is to prioritize test cases that are most likely to expose regression faults, i.e., instances that were correctly classified by Vk but incorrectly classified by Vk+1 . RiskBlend assigns a priority score to each test case by combining two complementary risk dimensions. The risk signals driving this ranking are described below. 1) History Risk (rhist ): History Risk estimates the likelihood that a test instance belongs to a region of the input space that has previously exhibited regression failures. The intuition is that regression faults tend to cluster in specific feature-space regions rather than being uniformly distributed; test instances near previously observed failures are therefore more likely to expose additional faults after a model update. RiskBlend identifies the locations of regression-faults using a labeled validation set Xval kept in the Vk+1 training partition. Because Xval carries ground-truth labels y, both model versions can be evaluated and compared without oracle invocation. Specifically, a validation instance z ∈ Xval is classified as a regression fault if Vk predicts it correctly but Vk+1 does not: Freg = {z ∈ Xval : Vk (z) = y ∧ Vk+1 (z) ̸= y} .

(2)

The set Freg thus represents the validated, label-confirmed locations in the feature space where the model update introduced new misclassifications. For each unlabeled test instance x ∈ Xtest , RiskBlend computes its minimum distance to the regression-fault region: dmin (x) = min d(x, z). z∈Freg

(3)

where d(·, ·) is the Euclidean distance calculated in the standardized feature space of Vk+1 . Test instances closer to Freg are more likely to reside in fault-prone regions of the updated model’s decision surface. The history risk score is defined as: dmin (x) − Dlo , (4) rhist (x) = 1 − Dhi − Dlo where dmin (x) is the minimum distance from test instance x to the regression-fault region Freg (as defined above), and Dlo = minx∈Xtest dmin (x) and Dhi = maxx∈Xtest dmin (x) are the minimum and maximum of dmin (·) taken over all test instances, so that rhist (x) ∈ [0, 1]. A higher score indicates proximity to previously confirmed regression-fault locations, and such instances are ranked earlier in the prioritization order. If Freg = ∅ (no regression faults are observed in the validation set), rhist is set to zero for all test instances and the remaining three signals carry the full prioritization weight.

2) Regression-Aware Impact Risk (rimpact ): The Impact Risk component estimates how strongly a test instance is affected by the transition from the original model version Vk to the updated model version Vk+1 . It uses three classifier-agnostic signals derived only from the predicted class probabilities of each model. a) Prediction Shift (rpred ).: The prediction-shift signal measures the absolute change in the predicted positive-class probability between the model versions for each test instance. Both Vk and Vk+1 are applied to the same test instance x ∈ Xtest using the feature representation of their respective versions to produce the predicted positive-class probabilities PVk (x) and PVk+1 (x). Training data or ground-truth labels are not used. The prediction-shift score is: rpred (x) = PVk+1 (x) − PVk (x) ,

(5)

Large values indicate that retraining substantially altered Vk+1 ’s confidence on x, suggesting that the decision boundary may have shifted near that input and increasing the likelihood of a regression fault. b) Boundary Shift (rboundary ).: The boundary-shift signal captures changes in classification behavior near the decision boundary between model versions. Like the prediction-shift signal, it is computed entirely from the predicted probabilities of both Vk and Vk+1 on each test instance x ∈ Xtest , using the representation of features of each version. Training data, training labels, or ground-truth test labels are not accessed. It combines three sub-components: 1) Prediction shift rpred (x) (Equation 5): magnitude of probability movement. 2) Boundary crossing: a binary indicator that captures whether the prediction for a test input changes from one side of the decision threshold to the other between model versions Vk and Vk+1 . Here, we use a default decision threshold of 0.5, meaning that predictions with probability greater than or equal to 0.5 are classified as positive, while those below 0.5 are classified as negative. The indicator is set to 1 if the predicted class changes between versions and to 0 otherwise. 3) Near-boundary movement: a continuous measure of how close the predicted probability is to the decision boundary, weighted by the magnitude of shift: 



  m(x) = maxc(x), (1 − 2 min(d1 , d2 )) rpred (x) , {z } | boundary nearness

(6) where d1 = |PVk (x)−0.5| and d2 = |PVk+1 (x)−0.5| are the distances of each prediction version from the threshold. The boundary-nearness factor 1 − 2 min(d1 , d2 ) ∈ [0, 1] is maximized when either prediction is exactly at the boundary and zero when both predictions are far from it.

The boundary-shift score combines prediction shift and boundary movement with equal weighting: rboundary (x) = minmax 0.5 minmax(rpred (x)) (7) + 0.5 minmax(m(x)) where minmax(·) denotes the min-max normalization to [0, 1] in all test instances. Inputs that both cross the decision boundary between versions and whose predictions lie close to the threshold receive the highest boundary-shift scores, as they represent the strongest evidence of a shifted decision surface. c) Neighbourhood Change (rneigh ).: The neighbourhoodchange signal measures how the local training neighbourhood of a test instance changes between Vk and Vk+1 . It is calculated using k-nearest neighbors (k = 15) applied separately to the training sets Vk and Vk+1 and captures three distinct aspects of neighborhood change. For each test instance x, let NVk (x) and NVk+1 (x) denote its k nearest neighbors in the Vk and Vk+1 training sets, respectively, retrieved via Euclidean distance in the standardized feature space of each version. Three sub-signals are then computed: 1) Class-composition shift (sclass ): : this measures how much the local class distribution around a test instance changes between the original and updated model. Specifically, we examine whether the proportion of neighboring instances belonging to the positive class changes after the model update. A large shift in class-composition suggests that the local decision boundary around the test instance has changed significantly, indicating a higher risk of regression. 2) Centroid shift (scent ): the Euclidean distance between the centroid of the neighborhood of x in version Vk and the centroid of its neighborhood in version Vk+1 . This captures the geometric displacement of the local training context around x. 3) Density shift (sdens ): measures how much the local neighborhood around a test instance changes between the model versions Vk and Vk+1 . It is computed as the absolute difference between the mean distance from the test instance to its k-nearest neighbors in the training data of the two model versions. A larger density shift indicates that the instance has become more isolated or has moved to a substantially different region of the training distribution. The three sub-signals are combined with fixed weights reflecting their relative informativeness: rneigh (x) = minmax 0.50 minmax(sclass (x)) + 0.30 minmax(scent (x)) + 0.20 minmax(sdens (x))

(8)

where minmax(·) normalizes each sub-signal to [0, 1] over the test set before combination. Class-composition shift receives the highest weight (0.50) as it most directly reflects

local label distribution changes that affect the classifier’s decision surface. The centroid shift (0.30) captures the geometric displacement of the local context, and the density shift (0.20) captures the changes of distributional sparsity . 3) Final RiskBlend Score: The final prioritization score is computed as a weighted combination of the four RiskBlend signals: RiskBlend(x) = wh rhist (x) + wb rboundary (x) + wn rneigh (x) + wp rpred (x) (9) subject to the constraints that wh , wb , wn , and wp are non-negative and normalized to sum to one. To determine the contribution of each signal, RiskBlend individually evaluates the effectiveness of regression fault detection of each component on a validation set that contains regression faults introduced during the transition from model version Vk to Vk+1 . The APFD achieved by each signal is converted into a normalized weight, assigning greater influence to signals that demonstrate stronger regression-fault detection capabilities. This validation-driven weighting strategy allows RiskBlend to adapt automatically to different datasets and model updates without relying on manually specified parameters or gridsearch tuning. After learning the weights, each test instance is assigned a RiskBlend score according to Equation 9. The test instances are then ranked in descending order, ensuring that inputs exhibiting strong historical fault-proneness, substantial behavioral change, and significant local distributional shifts are prioritized earlier. By integrating complementary risk signals through validation-guided weighting, RiskBlend enables the early detection of regression fault. V. E VALUATION A. Experimental Setup To assess the effectiveness of the proposed RiskBlend framework, we conducted experiments across multiple datasets, regression-update scenarios, classifiers, and random seeds to ensure statistical robustness and generalizability. We selected four benchmark datasets1 from the socioeconomic, healthcare and finance domains (Table I) for three reasons: they represent diverse real-world applications where regression faults can have significant impact, contain heterogeneous tabular features (both numerical and categorical) for realistic structured-data evaluation, and are widely used in prior machine learning and fairness studies, enabling comparison with existing approaches. For each data set, 80% of the samples were used for training and 20% for testing. Two consecutive versions of the model were constructed: Vk , trained on the original data and feature set, and Vk+1 , produced under one of the four regressionupdate scenarios described in the following. In our setting, 1 Adult: https://archive.ics.uci.edu/dataset/2/adult; Bank Marketing: https://archive.ics.uci.edu/dataset/222/bank+marketing; Hospital Readmission: https://www.kaggle.com/datasets/vanpatangan/readmission-dataset; Default of Credit Card Clients: https://www.kaggle.com/datasets/uciml/ default-of-credit-card-clients-dataset.

each test case corresponds to a single labeled test instance from the held-out test partition. Thus, test input prioritization V refers to ranking individual test samples in Xtestk+1 according to their estimated regression-fault revealing potential. We evaluated RiskBlend using five classifiers representing major machine learning families: XGBoost (ensemble boosting), Decision Tree (rule-based learning), Logistic Regression (linear models), Naive Bayes (probabilistic models) and KNearest Neighbors (instance-based learning). This selection enables us to evaluate whether RiskBlend is generalized across classifiers with substantially different decision boundaries and learning behavior. We focus on tabular machine learning systems rather than image-based deep neural networks because RiskBlend is designed around structured-data risk signals which are directly measurable in tabular feature space. Extending RiskBlend to complex image or multimodal models would require embedding-based representations and deep-model-specific risk signals, which we consider important future work. We generate Vk+1 through four maintenance scenarios designed to reflect common patterns of model evolution in deployed ML systems, rather than injecting faults directly at the label or prediction level. Feature-only update (feature only): Vk+1 is retrained on the same training samples as Vk but with an enriched feature space. For each data set, a fixed set of domainrelevant engineered attributes is appended to the original feature set; the set of new features is determined once per data set based on domain knowledge and is kept constant across all seeds and classifiers. No new training samples are added. The new features are appended to both Vk+1 training and test sets; Vk retains the original feature set only. This scenario isolates the effect of feature-space enrichment on prediction behavior: a test instance that Vk classified correctly may be misclassified by Vk+1 because the additional features shift the learned decision boundary, even though the underlying sample has not changed. • Moderate regression (moderate regression): Vk+1 is retrained with an augmented training set consisting of the original training samples used for Vk plus 20% % additional samples. The additional samples are generated by bootstrapping the original training set and perturbing each numerical feature independently with zero-mean Gaussian noise scaled to 0.6 × σf , where σf denotes the per-feature standard deviation of the original training data. The perturbed values are cut to [minf , 1.5 × maxf ] to preserve the distributional plausibility. In this scenario, no engineered features or preprocessing transformations are introduced. This setup models a common maintenance pattern in which a classifier is retrained using newly collected data from a mildly drifted population. • Strong regression (strong regression): Vk+1 is retrained with an augmented training set consisting of the original Vk training samples plus 30% additional samples, combined with the same dataset-specific engineered features •

introduced in the feature-only scenario. The additional samples are generated using the same bootstrapping and Gaussian perturbation mechanism as in the moderate scenario, but with a higher shift strength of 0.9 × σf , producing a more aggressively drifted augmentation. A stronger preprocessing pipeline change is also applied to training and test data Vk+1 , transforming a larger set of numerical features than in the moderate scenario. Vk sees neither the additional samples, the stronger pipeline transformations, nor the engineered features. This scenario emulates significant model adaptation, such as occurs when an organization retrains on data from a substantially different population or operating environment while also updating its preprocessing pipeline. • Pipeline drift (pipeline drift): Vk+1 is retrained on the same training samples and original feature set as Vk , with no new data and no engineered features added. The sole change is a moderate preprocessing pipeline modification applied to both the Vk+1 training and test partitions, consisting of the same monotonic transformations used in the moderate regression scenario. Vk retains the original untransformed features throughout. This scenario isolates the effect of preprocessing changes on model behavior, decoupled from any data distribution shift or featurespace expansion: any regression faults observed here are attributable solely to the change in how existing features are normalized rather than to new information entering the model. The augmentation ratios (20% and 30%) and drift strengths (0.6σf and 0.9σf ) were selected to create measurable but noncatastrophic regression behavior. Pilot experiments showed that smaller perturbations often produced too few regression faults for a meaningful APFD evaluation, whereas substantially larger perturbations caused severe model degradation that no longer reflected realistic maintenance updates. These settings consistently yielded moderate fault densities suitable for the regression testing evaluation. Across all scenarios and datasets, updates produced measurable but non-catastrophic behavioral changes. Each combination of data sets and scenario was evaluated in five classifiers that mirror the multiclassifier protocol of Dang et al. et al. [2]. For each combination (dataset, scenario, classifier, seed), Vk and Vk+1 were trained independently in their respective training sets. A held-out 20% waiting for the Vk+1 training partition was reserved as a validation set for the RiskBlend signal-weight learning step; the remaining 80% was used to train Vk+1 . Prioritization quality was measured on the shared Vk+1 test set using APFD. To ensure statistical robustness, each experiment was repeated in 15 random seeds (42, 44, . . . , 70); all reported results are means over seeds. The complete evaluation spans 4 datasets × 4 scenarios × 5 classifiers × 15 seeds = 1,200 experimental configurations per dataset, yielding 4,800 configurations in total. Information leakage does not occur because weight learning uses only a heldout validation partition drawn from the training data available

TABLE I S UMMARY OF DATASETS U SED FOR E VALUATION . Dataset

Description

Features

Samples

Domain

Bank Marketing

Predicts whether a customer will subscribe to a term deposit based on marketing campaign data Predicts whether a person earns more than $50K based on census data hospital readmission prediction from clinical, demographic, and hospitalization features Predicts whether a credit card client will default on the next payment based on demographic and financial attributes

16 (numeric + categorical)

45,211

Finance

14 (numeric + categorical) 49 (mixed)

48,842

Socioeconomic

101,766

Healthcare

30,000

Finance

Adult Income Hospital Readmission Credit Card Default

during model update, not from the final test set used for evaluation. Ground-truth labels for this validation partition are assumed to already be available as part of the historical labeled data used during retraining, which is realistic in supervised ML pipelines. The final prioritized test ranking is generated without accessing any labels from the test set. B. Baseline Methods We compare RiskBlend with four representative baselines that cover uncertainty-based, distance-aware, mutation-based, and random prioritization strategies. All methods were implemented in Python 3.11 using standard libraries (NumPy, Pandas, scikit-learn, and XGBoost) and evaluated using identical random seeds, data set splits, and sampling settings to ensure fair comparison. The baseline methods are as follows: 1) DeepGini: DeepGini [4] prioritizes test instances based on prediction uncertainty. The intuition is that test instances with uncertain predictions are more likely to reveal model failures. Let pi = (pi (1), . . . , pi (C)) denote the predicted class probability vector for the test instance xi , where C is the number of classes. The DeepGini score is computed as rDG (xi ) = 1 −

C X

pi (c)2

(10)

c=1

Higher scores indicate greater uncertainty, corresponding to more uniform class probabilities. Test instances are prioritized in descending order of their DeepGini scores. The DeepGini prioritization procedure consists of the following steps: 1) Train or load the updated version of the model Vk+1 . V 2) Execute Vk+1 on each test instance xi ∈ Xtestk+1 to obtain the predicted class probability vector pi . 3) Compute the DeepGini score rDG (xi ) for each test instance using Equation 10. 4) Sort all test instances in descending order of rDG (xi ). 5) Execute or inspect the test instances according to this ranked order. For binary classification, DeepGini is equivalent to a monotonic transformation of the distance from the decision boundary.

23 (numeric + categorical)

2) DATIS-Tabular (Distance-Aware Test Prioritization): DATIS-Tabular is a distance-aware test prioritization baseline adapted for tabular machine learning systems [7]. Unlike purely confidence-based approaches such as DeepGini, DATIS estimates fault-revealing potential using distance-based neighborhood information and redundancy elimination rather than relying solely on prediction probabilities. In our regression testing setting, DATIS operates on the updated model version V Vk+1 and prioritizes unlabeled test instances from Xtestk+1 . The training data used to build Vk+1 serve as the reference set for distance computation and neighborhood analysis. The prioritization process consists of two stages: 1) All features are normalized, and each training and test instance is represented as a numerical feature vector suitable for distance-based comparison. Each test instance V xi ∈ Xtestk+1 is first passed through Vk+1 to obtain its predicted label. DATIS then computes the distances between xi and its k nearest training neighbors. Test instances whose predicted label receives weak support from nearby training samples are assigned higher uncertainty scores, indicating a greater likelihood of misclassification and thus higher fault-revealing potential. 2) After computing the uncertainty scores for all test instances, DATIS ranks them in descending order of uncertainty. To avoid selecting highly similar test cases that may expose the same fault, DATIS performs redundancy elimination using pairwise distances among candidate test instances. Instances that are too close to already selected ones are treated as redundant and deprioritized. 3) MLPrior (Mutation-Based Prioritization): MLPrior is a mutation-based test input prioritization baseline for machine learning classifiers [3]. Estimates the fault-proneness of test instances by analyzing their sensitivity to model-level and input-level perturbations. In our regression testing setting, MLPrior operates on the updated model version Vk+1 and V prioritizes test instances from Xtestk+1 . The method constructs three groups of features for each test instance: • Original input features: Original feature representations derived from the training and test data. • Model-mutation features: Features obtained by perturbing the classifier behavior, such as modifying decision

thresholds or predicted probability distributions, to simulate model-level mutations. • Data-mutation features: Features generated by perturbing selected input attributes to simulate input-level mutations. These features are combined to train a secondary ranking model using supervised learning, implemented with an XGBoost-based gradient-boosted decision tree ensemble, to estimate the probability that a test instance will be misclassified by Vk+1 . The predicted probability of misclassification is used as the prioritization score, where higher scores indicate a greater potential to reveal faults. The test instances are then ranked in descending order of this score to construct the prioritized test suite. C. Evaluation Metric We evaluate prioritization effectiveness using the Average Percentage of Faults Detected (APFD), which measures how early regression faults appear in the prioritized ranking. Given a test set of n inputs containing m regression faults, let Ti denote the position of the i-th fault in the ranked list and the index at which the i-th regression fault is first encountered when inspecting the inputs in priority order. APFD is defined as: Pm Ti 1 . (11) APFD = 1 − i=1 + n×m 2n The APFD ranges from 0 to 1, with higher values indicating that regression faults are concentrated earlier in the ranking. In our setting, the APFD is calculated over the entire Vk+1 test set averaged over 15 independent runs. VI. R ESULTS This section evaluates the effectiveness of RiskBlend in prioritizing test instances for regression testing of machine learning models. We investigate a research question. • RQ1: How effective is RiskBlend in terms of APFD compared to baseline prioritization methods? • RQ2: How does each individual risk signal contribute to RiskBlend’s prioritization effectiveness, and which signals are most informative across different regressionupdate scenarios and classifier families? A. RQ1: Overall Prioritization Effectiveness (APFD) To assess the overall effectiveness of prioritization, we computed the APFD for RiskBlend and all baseline methods in the four datasets. APFD measures how early faults appear in the prioritized test sequence, with higher values indicating faster fault detection. Tables II and III report the average APFD at full test-suite coverage across all four datasets, four regression-update scenarios, and five classifiers (80 classifier– scenario cells in total, each averaged over 15 seeds). RiskBlend achieves the highest APFD in all 80 cells in all four datasets and all baseline methods, with total data set-level means of 0.9557 (Bank), 0.9452 (Adult), 0.9238 (Hospital) and 0.8599 (Credit Card).

The advantage of RiskBlend over the strongest baseline per cell ranges from +0.003 (Adult, pipeline drift, Naı̈ve Bayes, where APFD differs marginally from DeepGini: 0.999 vs. 0.996) to +0.317 (Credit Card, moderate regression, KNN, where the next-best baseline reaches only 0.583). Across all datasets, RiskBlend leads DeepGini, the strongest confidencebased baseline by 0.014–0.223 APFD on average per data set, with the largest data set-level gaps observed in Hospital (+0.157) and Credit Card (+0.224), and smaller but consistent gaps in Bank (+0.135) and Adult (+0.132). RiskBlend’s gains are most pronounced on Decision Tree and KNN classifiers, where confidence-based methods lose discriminative power after version transitions. For example, on the Bank dataset with Decision Tree, DeepGini achieves 0.754–0.772 across scenarios, while RiskBlend holds at 0.937– 0.949 (a gain of +0.16–+0.19). In Hospital KNN, RiskBlend reaches 0.861–0.916 against DeepGini’s 0.554–0.626 (+0.25– +0.29). In linear and probabilistic classifiers (Logistic Regression, Naı̈ve Bayes), RiskBlend continues to outperform all baselines, although margins are narrower in datasets with sparse categorical features (Adult, Bank) where well-calibrated probability outputs from these classifiers provide a stronger confidence signal. RiskBlend maintains its advantage across all four regression-update scenarios. The pipeline drift scenario where only preprocessing transformations change and no new data or features are added yields some of the highest RiskBlend APFD values (e.g., 0.9933 on Hospital NB, 0.9276 on Credit KNN), demonstrating that prediction-shift and boundary-shift signals are particularly effective at detecting the altered decision boundaries that arise from preprocessing changes alone. Under strong regression (the most aggressive distributional shift), RiskBlend’s absolute APFD is modestly lower than under feature-only updates but still leads all baselines by substantial margins (e.g., +0.147 over DeepGini on Hospital XGB, +0.248 on Credit KNN). MLPrior and DATIS both trail RiskBlend on every cell. In Bank and Adult where these baselines are strongest, benefiting from larger training sets for MLPrior’s misclassification, and they achieve overall means of 0.826–0.854, compared to RiskBlend’s 0.945–0.956. On Hospital and Credit Card, where both methods rely on smaller datasets and fewer training misclassifications, they degrade more sharply (0.552–0.675), while RiskBlend remains stable (0.860–0.924). Random selection consistently achieves approximately 0.50 across all settings, confirming that the test inputs contain genuine regression faults that are non-trivially distributed across the test suite. All reported APFD values are averaged across 15 independent experimental runs using fixed random seeds (42, 44, 46, . . . , 70), each controlling the train/validation split of Vk+1 ’s training data, the weight-learning step, and the stochastic components of the MLPrior ranker and Random baseline. This repeated-measures design accounts for stochastic variation arising from data partitioning and model initialization. Answer to RQ1: RiskBlend achieves the highest APFD

Dataset

Scenario

]

Classifier RiskBlend DeepGini DATIS MLPrior Random Dataset Scenario

Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital Hospital

Feature Only Feature Only Feature Only Feature Only Feature Only Moderate Reg Moderate Reg Moderate Reg Moderate Reg Moderate Reg Drift Drift Drift Drift Drift Strong Reg Strong Reg Strong Reg Strong Reg Strong Reg

XGB TREE LR NB KNN XGB TREE LR NB KNN XGB TREE LR NB KNN XGB TREE LR NB KNN

0.9752 0.8047 0.9722 0.9794 0.8857 0.9704 0.8181 0.9668 0.9827 0.8781 0.9797 0.8234 0.9839 0.9933 0.9155 0.9509 0.8156 0.9617 0.9564 0.8610

0.8790 0.5468 0.9150 0.9229 0.5915 0.8585 0.5448 0.9122 0.9314 0.5800 0.9033 0.5583 0.9559 0.9843 0.6261 0.8043 0.5368 0.8731 0.8570 0.5539

0.7031 0.6226 0.7040 0.6068 0.6115 0.6967 0.6247 0.7000 0.6458 0.6114 0.6774 0.6139 0.6682 0.7067 0.6209 0.6649 0.6206 0.6710 0.7075 0.6103

0.6655 0.5301 0.8498 0.7520 0.5725 0.6798 0.5262 0.8578 0.7996 0.5641 0.6997 0.5374 0.8446 0.7161 0.6099 0.6726 0.5240 0.8197 0.7279 0.5523

0.5048 0.4988 0.5047 0.5017 0.5003 0.4953 0.5001 0.4998 0.5028 0.4990 0.5013 0.4952 0.4976 0.5222 0.4971 0.5012 0.5029 0.5017 0.4983 0.5010

Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit Credit

XGB TREE LR NB KNN XGB TREE LR NB KNN XGB TREE LR NB KNN XGB TREE LR NB KNN

0.8586 0.7938 0.9064 0.8851 0.9198 0.8565 0.7943 0.8856 0.8685 0.9000 0.8870 0.8249 0.8987 0.9179 0.9276 0.8283 0.7771 0.8626 0.8187 0.7864

0.7068 0.5261 0.6986 0.7277 0.6725 0.6139 0.5083 0.6931 0.6795 0.5825 0.6929 0.5234 0.7501 0.7636 0.6555 0.5844 0.5223 0.6139 0.6904 0.5176

0.5619 0.5473 0.4792 0.6066 0.5139 0.5718 0.5277 0.5221 0.6019 0.5442 0.5576 0.5402 0.5484 0.5634 0.4925 0.6108 0.5861 0.5484 0.5739 0.5387

0.6252 0.5235 0.6304 0.6303 0.5776 0.5756 0.4921 0.5817 0.6308 0.5331 0.6000 0.5336 0.5892 0.6564 0.5681 0.5584 0.5227 0.5352 0.5779 0.5037

0.4600 0.4858 0.4741 0.4778 0.4827 0.4213 0.5023 0.4649 0.5015 0.5035 0.4662 0.5084 0.4806 0.4477 0.4755 0.4749 0.4829 0.5245 0.4732 0.4965

TABLE II P ER - SCENARIO APFD RESULTS FOR H OSPITAL AND C REDIT C ARD DATASETS .

in all 80 classifier scenario cells across all four datasets, outperforming confidence-based, mutation-based and distancebased baselines in every configuration. Gains are most pronounced on tree-based and distance-based classifiers, and under pipeline-drift and strong-regression scenarios, confirming that multi-signal prioritization capturing cross-version behavioral change provides robust fault detection across diverse regression testing conditions. B. RQ2: Signal Contribution and Informative Signals To answer RQ2, we analyze the validation-learned signal weights produced by RiskBlend’s APFD-squared weighting mechanism across all four datasets, scenarios, and classifiers. For each experimental configuration, the weight assigned to each signal reflects its estimated contribution to regressionfault detection in the held-out validation partition. We report the mean weights averaged on 15 seeds. Table IV shows the mean weight per-dataset. The boundary shift dominates in three of four datasets (Adult: 0.349, Bank: 0.324, Credit: 0.375). The hospital is the exception, where history risk receives the highest weight (0.366), closely followed by boundary shift (0.309). This reflects the structured clinical feature space of the hospital, where the historical misclassification pat-

Classifier RiskBlend DeepGini DATIS MLPrior Random

Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank Bank

Feature Only XGB Feature Only TREE Feature Only LR Feature Only NB Feature Only KNN Moderate Reg XGB Moderate Reg TREE Moderate Reg LR Moderate Reg NB Moderate Reg KNN Drift XGB Drift TREE Drift LR Drift NB Drift KNN Strong Reg XGB Strong Reg TREE Strong Reg LR Strong Reg NB Strong Reg KNN

0.9899 0.9490 0.9687 0.9475 0.9888 0.9835 0.9405 0.9298 0.9421 0.9653 0.9805 0.9365 0.9353 0.9398 0.9637 0.9837 0.9387 0.9239 0.9443 0.9628

0.9696 0.7722 0.9181 0.8786 0.9295 0.9326 0.7540 0.8549 0.7456 0.7177 0.8859 0.7611 0.8018 0.8577 0.6980 0.9286 0.7654 0.7964 0.6937 0.7569

0.9152 0.8873 0.8553 0.8833 0.9175 0.8480 0.8621 0.8327 0.8384 0.8546 0.8309 0.8586 0.8149 0.8166 0.8558 0.8605 0.8600 0.8166 0.8111 0.8518

0.9412 0.7196 0.9202 0.9158 0.8690 0.9149 0.7117 0.8857 0.8123 0.7419 0.8809 0.7224 0.8832 0.8454 0.7441 0.9149 0.7154 0.8604 0.7973 0.7213

0.4929 0.4951 0.5072 0.5006 0.4961 0.5032 0.5039 0.5001 0.4985 0.5028 0.4992 0.5028 0.5029 0.4992 0.5021 0.5079 0.5004 0.5033 0.4969 0.4991

Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult

0.9916 0.9211 0.8764 0.9955 0.9675 0.9901 0.9198 0.8460 0.9960 0.9600 0.9897 0.9253 0.9184 0.9991 0.9709 0.9767 0.9144 0.8427 0.9668 0.9350

0.9616 0.7545 0.6904 0.9902 0.8315 0.9384 0.7532 0.6815 0.7740 0.8243 0.9681 0.7765 0.7213 0.9959 0.8450 0.8794 0.7340 0.7095 0.6924 0.7398

0.8568 0.8250 0.8124 0.8922 0.8597 0.8448 0.8266 0.8042 0.9608 0.8745 0.8502 0.8122 0.7797 0.8933 0.8545 0.7853 0.8086 0.7847 0.8124 0.8387

0.9062 0.6974 0.8502 0.8116 0.7999 0.8938 0.7038 0.8269 0.8600 0.7879 0.9166 0.7359 0.7877 0.8503 0.8308 0.8417 0.6814 0.8014 0.7549 0.7489

0.5240 0.4995 0.5020 0.5036 0.5079 0.5091 0.5057 0.5004 0.5008 0.5010 0.5096 0.4992 0.4999 0.4839 0.5055 0.5052 0.5040 0.4980 0.5046 0.5045

TABLE III P ER - SCENARIO APFD RESULTS FOR BANK AND A DULT DATASETS .

terns of Vk in training data (captured by history risk) provide a particularly strong signal for identifying regression-prone test inputs. Across all four datasets, neighborhood change is consistently the weakest signal, ranging from 0.099 (Hospital) to 0.142 (Adult). TABLE IV M EAN VALIDATION - LEARNED WEIGHTS PER SIGNAL AND DATASET. B OLD INDICATES THE DOMINANT SIGNAL PER DATASET. Dataset

History

Boundary

Pred. Shift

Neigh.

Adult Bank Credit Hospital

0.214 0.230 0.209 0.366

0.349 0.324 0.375 0.309

0.295 0.306 0.280 0.225

0.142 0.141 0.136 0.099

Table V complements the learned-weight analysis by quantifying the actual contribution of each signal to the prioritization performance through a leave-one-signal-out (LOSO) ablation study. Unlike the validation-learned weights, which indicate how strongly the optimizer relies on each signal during weight learning, the LOSO analysis measures the performance degradation after removing an individual signal while renormalizing the remaining weights. Across all four datasets,

TABLE V L EAVE - ONE - SIGNAL - OUT (LOSO) ABLATION STUDY. E ACH VALUE REPORTS THE MEAN APFD DROP (∆APFD) AFTER REMOVING ONE R ISK B LEND SIGNAL . L ARGER POSITIVE VALUES INDICATE GREATER CONTRIBUTION TO REGRESSION - FAULT DETECTION . R ESULTS ARE AVERAGED OVER ALL SCENARIOS , CLASSIFIERS , AND SEEDS . Removed Signal

Adult

Bank

Hospital

Credit

Boundary Shift History Risk Neighborhood Change Prediction Shift

0.0469 0.0042 −0.0003 −0.0042

0.0328 0.0021 −0.0006 −0.0025

0.0686 0.0169 −0.0012 −0.0125

0.0749 0.0232 −0.0030 −0.0233

the removal of the Boundary Shift signal results in the largest reduction in APFD, with mean drops ranging from 0.0328 (Bank) to 0.0749 (Credit Card), confirming that changes in the decision boundary between Vk and Vk+1 provide the strongest evidence to identify regression-prone test inputs. Removing History Risk produces a smaller but consistently positive APFD reduction, particularly for the Hospital (0.0169) and Credit Card (0.0232) datasets, indicating that historical fault patterns provide complementary information beyond boundary changes. In contrast, removing Neighborhood Change has virtually no effect on APFD across all datasets, while removing Prediction Shift slightly improves APFD on average. These results suggest that the information captured by the prediction shift is largely redundant with the Boundary Shift signal, whereas Neighborhood Change contributes only marginal additional information under the considered regression-testing scenarios. Answer to RQ2: The boundary shift is the most influential component of R ISK B LEND, receiving the highest average learned weight (w̄ = 0.349) and producing the largest APFD reduction in the leave-one-signal-out ablation across all datasets. The prediction shift is consistently assigned the second-highest weight (w̄ = 0.277), while the history risk provides a stable complementary contribution (w̄ = 0.255), particularly for the hospital data set. The neighborhood change receives the lowest average weight (w̄ = 0.130) and has the least impact on the APFD, indicating that its information is largely redundant with the other signals. The learned weights remain stable across the regression scenarios (σ < 0.05), demonstrating that the validation-based weighting strategy is consistently generalized across the datasets and classifier families. C. Statistical Significance To assess whether RiskBlend’s APFD improvements are statistically significant, we applied the Wilcoxon signed-rank test, pairing RiskBlend against each baseline by (scenario, seed, classifier) per dataset, yielding N = 300 paired observations per comparison (4 scenarios × 15 seed × 5 classifiers). The p-values are reported with the Bonferroni correction for four simultaneous comparisons. The effect size is measured by Cliff’s δ, with thresholds |δ| < 0.147 negligible, < 0.330 small, < 0.474 medium, and ≥ 0.474 large [9].

All 16 comparisons in four datasets are statistically significant at p < 0.001 after Bonferroni correction, with large effect sizes in 15 of 16 cases. Against Random, RiskBlend achieves near-complete stochastic dominance across all datasets (δ ≥ 0.993, ∆APFD ≥ +0.380). Against DATIS-Tabular, effect sizes are also uniformly large (δ ≥ 0.973). Against MLPrior, the effect sizes are large in all datasets (δ ≥ 0.630, ∆APFD ≥ +0.130). Against DeepGini, the strongest baseline effect sizes are large in Hospital (δ = 0.567), Credit Card (δ = 0.898) and Bank (δ = 0.887). The single exception is RiskBlend vs. DeepGini in Adult (δ = 0.084, negligible, ∆APFD = +0.077), where DeepGini remains competitive in linear classifiers with sparse categorical features, reducing the practical margin despite statistical significance. In Bank, RiskBlend exceeded every baseline in all 300 paired observations, yielding the maximum possible Wilcoxon statistic (W = 45,150) for all four comparisons. The replication package is publicly available on Figshare.2 . VII. D ISCUSSION The results confirm that the combination of multiple risk signals produces more robust prioritization than any singlesignal strategy under diverse regression-update conditions. RiskBlend achieves the highest APFD in all 80 classifier scenario cells, with the largest gains on tree-based and distancebased classifiers, where confidence-based methods lose discriminative power after version transitions. Even on wellcalibrated classifiers where confidence is strongest, RiskBlend remains competitive, demonstrating that multi-signal blending does not sacrifice performance in favorable conditions while substantially improving it in adverse ones. Confidence-based methods assume that model uncertainty reliably reflects fault risk after a version update. Our results show this weakens for Decision Tree classifiers across all datasets, where DeepGini APFD falls to 0.547–0.623 while RiskBlend maintains 0.816– 0.858. The weight analysis (RQ2) explains this: The boundary shift and the prediction shift jointly receive ≈62% of the ensemble weight, compensating for the reduced informativeness of the single-version confidence after a version transition. MLPrior performs well on larger datasets, but degrades on smaller ones (Credit Card), exposing a data-volume sensitivity that RiskBlend’s validation-learned weighting does not share. No single signal consistently dominates across all configurations; rather, signal importance adapts to the dataset and the classifier via validation-learned weighting without classifierspecific tuning. Limitations include sensitivity to the heldout validation partition for weight learning and reduced effectiveness when regression faults are very sparse. Future work includes extension to multi-class and regression settings, and online weight adaptation across model version sequences. Regression fault densities varied substantially across datasets, classifiers, and regression-update scenarios, with average fault rates ranging from approximately 3.6% (Bank) to 14.0% (Credit Card). This variability reflects differences in 2 https://doi.org/10.6084/m9.figshare.32964359

classifier behavior and update severity, resulting in both easy and challenging prioritization settings. Consequently, APFD values, particularly near-ceiling scores (e.g., ≥ 0.98), should be interpreted in the context of the underlying regression fault density. Table VI reports the average wall-clock execution time of each prioritization method, averaged over all classifiers, random seeds, and regression-update scenarios. The reported time measures only the prioritization stage before oracle verification, excluding model training, dataset preparation, and manual labeling. Random and DeepGini incur negligible overhead (less than 0.001 s). DATIS requires between 0.62 s and 5.30 s across datasets. RiskBlend completes in 0.32–8.95 s, and MLPrior in 0.20–17.63 s. Relative to MLPrior, RiskBlend is substantially faster on Bank (4.94 s vs. 17.63 s) but slower on Adult (8.95 s vs. 1.16 s), Hospital (2.39 s vs. 0.76 s), and Credit Card (0.32 s vs. 0.20 s). These results indicate that RiskBlend’s computational cost is dataset-dependent, and while it avoids MLPrior’s worst-case overhead on Bank, it does not consistently offer lower overhead than the mutationbased baseline. TABLE VI AVERAGE EXECUTION TIME ( SECONDS ) PER PRIORITIZATION METHOD ACROSS DATASETS ( MEAN OVER ALL CLASSIFIERS , SEEDS , SCENARIOS Method

Adult

Bank

Hospital

Credit

Random DeepGini MLPrior DATIS RiskBlend

0.000199 0.000102 1.1590 4.0762 8.9482

0.000222 0.000120 17.6336 5.3016 4.9425

0.000164 0.000098 0.7584 1.5293 2.3929

0.000128 0.000038 0.2043 0.6218 0.3152

VIII. T HREATS TO VALIDITY Internal validity: Signal weights are learned from a heldout 20% validation partition of Vk+1 ’s training data. If this partition is small or unrepresentative of the regression-fault distribution, learned weights may not generalize to the test set. We mitigate this by averaging results over 15 random seeds, which stabilizes weight estimates across partitioning choices. External validity: We evaluate RiskBlend in four benchmark datasets that span the financial, socioeconomic, and healthcare domains in five classifier families and four regression-update scenarios. Although this covers a broad range of conditions, results may not generalize to unstructured data (images, text), deep learning models, or production pipelines with continuous data drift. regression-update scenarios were constructed to reflect realistic maintenance patterns; however, real-world update distributions may differ in ways not captured by our four scenarios. IX. C ONCLUSION This work presented RiskBlend, a classifier-agnostic framework for prioritization of test input in ML regression testing. RiskBlend fuses four generic risk signals, historical failure patterns, prediction shift, decision-boundary shift, and

neighborhood change, into a single score via APFD-squared weighting learned from validation, requiring only no classifierspecific tuning. Evaluated across four datasets, five classifiers, and four regression-update scenarios (1,200 configurations per dataset), RiskBlend achieves the highest APFD in all 80 classifier–scenario cells, with statistically significant improvements over all baselines. The Signal-weight analysis identifies the boundary shift as the most informative signal (w̄ = 0.339), with stable weights between scenarios and classifiers, confirming that the behavioral signals in crossversion are essential complements to the confidence of the single-model for the prioritization of the robust regression test ML. R EFERENCES [1] Taejoon Byun, Vaibhav Sharma, Abhishek Vijayakumar, Sanjai Rayadurgam, and Darren Cofer. Input prioritization for testing neural networks. In 2019 IEEE International Conference On Artificial Intelligence Testing (AITest), pages 63–70. IEEE, 2019. [2] Xuan Dang, Yinghua Li, Mike Papadakis, Jacques Klein, Tegawendé F. Bissyandé, and Yves Le Traon. Test input prioritization for graph neural networks. IEEE Transactions on Software Engineering, 50(6), 2024. [3] Xuan Dang, Yinghua Li, Mike Papadakis, Jacques Klein, Tegawendé F. Bissyandé, and Yves Le Traon. Test input prioritization for machine learning classifiers. IEEE Transactions on Software Engineering, 50(3):413–442, 2024. [4] Yang Feng, Qingkai Shi, Xinyu Gao, Jun Wan, Chunrong Fang, and Zhenyu Chen. DeepGini: Prioritizing massive tests to enhance the robustness of deep neural networks. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pages 177–188. ACM, 2020. [5] Yao Hao, Zhiqiu Huang, Hongjing Guo, and Guohua Shen. Test input selection for deep neural network enhancement based on multipleobjective optimization. In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 534–545. IEEE, 2023. [6] Jinhan Kim, Robert Feldt, and Shin Yoo. Guiding deep learning system testing using surprise adequacy. In Proceedings of the 41st International Conference on Software Engineering (ICSE), pages 1039–1049. IEEE Press, 2019. [7] Zhong Li, Zhengfeng Xu, Ruihua Ji, Minxue Pan, Tian Zhang, Linzhang Wang, and Xuandong Li. Distance-aware test input selection for deep neural networks. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). ACM, 2024. [8] Zhonghao Pan, Shan Zhou, Jianmin Wang, Jinbo Wang, Jiao Jia, and Yang Feng. Test case prioritization for deep neural networks. In 2022 9th International conference on dependable systems and their applications (DSA), pages 624–628. IEEE, 2022. [9] Jeanine Romano, Jeffrey D Kromrey, Jesse Coraggio, and Jeff Skowronek. Appropriate statistics for ordinal level data: Should we really be using t-test and cohen’sd for evaluating group differences on the nsse and other surveys. In annual meeting of the Florida Association of Institutional Research, volume 177, 2006. [10] Madhusudan Srinivasan and Upulee Kanewala. Improving early fault detection in machine learning systems using data diversity-driven metamorphic relation prioritization. Electronics, 13(17), 2024. [11] Zan Wang, Hanmo You, Junjie Chen, Yingyi Zhang, Xuyuan Dong, and Wenbin Zhang. Prioritizing test inputs for deep neural networks via mutation analysis. In Proceedings of the 43rd IEEE/ACM International Conference on Software Engineering (ICSE), pages 397–409. IEEE, 2021. [12] Pengcheng Wu et al. Stratified random sampling for neural network test input selection. Information and Software Technology, 163, 2023. [13] Lichuang Yang, Daojing He, Yu Li, et al. Toward efficient testing of graph neural networks via test input prioritization. Automated Software Engineering, 2025.

RiskBlend: A Multi-Signal Framework for Test Input Prioritization in Machine Learning Regression Testing

Analyze & explore related research in Conceptio

Related documents