arXiv:2606.06265v1 [cs.CR] 4 Jun 2026
Robust Ensemble of Selectively Strengthened and Augmented Predictors Parsa Memarzadehsaghezi∗
Zahra Hashemi∗
Ontario Tech University Oshawa, Ontario, Canada [email protected]
Ontario Tech University Oshawa, Ontario, Canada [email protected]
Pooria Madani
Mehran Ebrahimi
Ontario Tech University Oshawa, Ontario, Canada [email protected]
Ontario Tech University Oshawa, Ontario, Canada [email protected]
Abstract Evasion attacks present a significant challenge to the robustness of machine learning (ML)-based classifiers, particularly in critical applications such as fraud detection and cybersecurity. Although existing defense mechanisms are effective in some settings, they often suffer from limited generalizability and do not systematically improve model robustness across diverse attack scenarios. To address these limitations, we introduce Robust Ensemble of Selectively Strengthened and Augmented Predictors (RESSAP), a novel framework that transforms a single classifier into an ensemble of robust classifiers. Each classifier in the ensemble is trained on a carefully selected subset of features, where feature selection is guided by a resilience metric that accounts for both feature importance and robustness. During inference, a random subset of these classifiers is used to make predictions, increasing unpredictability and improving resistance to adversarial manipulation. In addition, noise-based data augmentation is applied during training to strengthen decision boundaries and improve generalization. Our experimental results demonstrate that RESSAP significantly improves robustness against adversarial evasion attacks while maintaining strong accuracy on clean data. Overall, this model-agnostic framework provides a scalable and flexible defense strategy for enhancing the security of machine learning systems without requiring major changes to existing architectures.
CCS Concepts • Security and privacy → Systems security; • Computing methodologies → Machine learning.
Keywords Adversarial machine learning, robust machine learning models, evasion attacks, classifier randomization, ensemble learning ∗ These authors contributed equally to this work.
This work is licensed under a Creative Commons Attribution 4.0 International License. IWSPA ’26, Frankfurt am Main, Germany © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2609-5/2026/06 https://doi.org/10.1145/3806007.3810966
ACM Reference Format: Parsa Memarzadehsaghezi, Zahra Hashemi, Pooria Madani, and Mehran Ebrahimi. 2026. Robust Ensemble of Selectively Strengthened and Augmented Predictors. In Proceedings of the 12th ACM International Workshop on Security and Privacy Analytics (IWSPA ’26), June 23–25, 2026, Frankfurt am Main, Germany. ACM, New York, NY, USA, 7 pages. https://doi.org/10. 1145/3806007.3810966
1
Introduction
Despite the remarkable success of machine learning (ML) in classification tasks, these models often exhibit blind spots—regions in the input space where their predictions become unreliable. Such blind spots arise naturally due to the limited scope of training data, the high dimensionality of feature spaces, and the inherent approximations made by learning algorithms. As a result, even wellperforming models may misclassify inputs that fall just outside familiar patterns, leaving them vulnerable to deliberate manipulation in adversarial settings, commonly known as adversarial evasion attacks [3, 6, 13]. These attacks exploit blind spots in learned decision boundaries, causing input instances to be misclassified at runtime without affecting the training process. For example, in financial fraud detection systems that rely on anomaly detection models, an adversary may split a large transaction into several smaller ones, each carefully crafted to remain below the model’s learned detection thresholds. In doing so, the attacker can mimic legitimate spending behaviour and evade detection by exploiting the model’s limited exposure to such subtle manipulations during training. Such evasion attacks are especially concerning because they exploit the same generalization mechanisms that learning algorithms rely on to form decision boundaries from training data, making them difficult to detect and potentially costly in high-stakes applications [18]. In adversarial settings, attackers often have the ability to interact with a deployed ML model either through direct query access or by observing its outputs. This interaction enables them to probe the model’s behaviour and iteratively refine their inputs in order to craft instances (i.e., adversarial examples) that evade detection or cause misclassification [10, 14]. Even without access to the internal details of the classification model, repeated interaction with the system can reveal enough information about its decision boundaries to construct highly effective adversarial examples [4, 19]. Despite extensive research on defending against adversarial examples, there remains a lack of systematic frameworks for evaluating
IWSPA ’26, June 23–25, 2026, Frankfurt am Main, Germany
Parsa Memarzadehsaghezi, Zahra Hashemi, Pooria Madani, and Mehran Ebrahimi
and improving the robustness of learning-based detection systems against evasion attacks [2, 17]. Many existing defenses are evaluated under narrowly defined attack scenarios, which limits their generalizability and practical applicability. In addition, many proposed methods require substantial changes to the training process or restrict the choice of learning algorithms, making them impractical for many data scientists and domain experts [13, 21]. Consequently, there is a strong need for a more systematic, model-agnostic defense strategy that can improve robustness without compromising flexibility, particularly within modern Sec-DevOps pipelines, where the security and robustness of learning-based assets (e.g., models) must be continuously evaluated and improved. Among the defense strategies explored in the literature, introducing randomness at inference time (e.g., stochastically flipping the classification label) has emerged as a promising approach for hindering adversarial evasion attacks [5, 11, 12, 16]. Randomization can be applied to input data, model parameters, or decision outputs to reduce an attacker’s ability to exploit consistent model behaviour. By making ML model responses less predictable, these techniques increase the difficulty of reliably crafting successful evasion attempts. However, if applied naively, such randomness can also degrade the classification accuracy of benign inputs. Therefore, any randomized defense must carefully balance robustness and classification accuracy in order to remain practically effective [16]. To address this challenge, we propose Robust Ensemble of Selectively Strengthened and Augmented Predictors (RESSAP), a novel model-agnostic framework that transforms an existing trained ML model into an ensemble of specialized sub-models through featurespace subsetting. At inference time, a random subset of these submodels is selected to produce predictions, thereby obtaining the benefits of randomization-based defense without compromising overall classification accuracy. Applying this framework within existing ML pipelines significantly enhances the robustness of trained models against adversarial evasion attacks by increasing the number of queries required to craft successful adversarial examples [2, 5, 12, 16]. This increased query cost, introduced through randomization, forms the basis of our evaluation, where we demonstrate improved robustness with minimal impact on classification accuracy. The key contributions of this work are as follows:
the experimental setup and presents the results and analysis. Finally, Section 6 concludes the paper and outlines directions for future work.
(1) We propose RESSAP, a novel ensemble framework that enhances adversarial robustness by combining feature-level diversity with classifier-level randomization. (2) We introduce a new feature selection approach that combines permutation importance and robustness into a unified resilience metric for guiding the formation of diverse feature subsets. (3) We validate the effectiveness of our approach through experiments on synthetic datasets, demonstrating significant improvements over traditional models in both clean accuracy and attack resistance. The remainder of the paper is organized as follows. Section 2 reviews related work on evasion attacks and defense strategies. Section 3 defines the adversarial threat model. Section 4 presents our proposed RESSAP framework, including feature selection, data augmentation, ensemble training, and inference. Section 5 describes
2 Related Work 2.1 Evasion Attacks The vulnerability of machine learning models to adversarial manipulation has been extensively studied in recent years [3]. These studies primarily aim to expose model weaknesses and assess the security risks that arise when models operate in adversarial environments. Among the different attack settings, evasion attacks at test time have received significant attention as a critical research direction. Goodfellow et al. [6] introduced one of the earliest gradientbased attacks for neural networks, known as the Fast Gradient Sign Method (FGSM). Their method perturbs inputs in the direction of the loss gradient to generate adversarial examples with minimal computational effort. This simple but effective approach highlighted the sensitivity of deep models to small, targeted perturbations and established the foundation for many subsequent white-box attack methods. Early work by Nelson et al. [14] introduced one of the first frameworks for computing near-optimal evasion attacks against convex classifiers. Their multi-line search algorithm probes a classifier’s decision boundary along strategically chosen feature directions, combined with binary search, in order to find minimally perturbed inputs that evade detection. Kantchelian et al. [8] later showed that even ensemble models such as random forests and boosted trees are highly vulnerable to evasion attacks. They formulated the problem of evading tree ensembles as an optimization task for generating minimal perturbations that can fool all trees in the ensemble. Their results demonstrated that tree ensembles, like linear models, can be effectively compromised by carefully crafted adversarial instances. Alzantot et al. [1] observed that earlier black-box attacks often required excessive numbers of queries, particularly when building substitute models or estimating gradients. To address this, they introduced GenAttack, a gradient-free black-box attack based on a genetic algorithm that evolves adversarial inputs. Their method achieved competitive success rates on image classifiers while substantially reducing query overhead. Chen and Gu [4] proposed RayS, a hard-label black-box attack that searches over discrete input directions to locate decision boundaries. RayS removes the need for gradient or confidence information and reduces the number of required queries through a prioritydriven ray search strategy. More recently, Shi et al. [19] introduced CISA, an adaptive iterative attack that dynamically adjusts sampling and step sizes based on the estimated distance to the classifier’s decision surface. Their method further improves query efficiency while maintaining strong attack performance in limited-feedback black-box settings.
2.2
Defense Against Evasion Attacks
Defending classifiers against evasion attacks has been approached from both feature-centric and ensemble-based perspectives. At the
Robust Ensemble of Selectively Strengthened and Augmented Predictors
feature level, Zhang et al. [21] proposed an adversary-aware feature selection strategy for training robust classifiers. Their method explicitly optimizes the feature set by trading off nominal accuracy for improved resistance to evasion, producing models that maintain strong performance while becoming more difficult to evade through feature manipulation. Kołcz and Teo [9] introduced a related feature-weighting approach that adjusts the importance of individual features based on robustness. Their method reweights features using the output of an initial classifier and improves resilience in spam detection tasks with relatively low computational cost. Although designed for linear models, their work highlights the value of feature-level adjustments in mitigating adversarial impact. Another feature-focused line of work considers adversarial example detection rather than direct prevention. Huang et al. [7] proposed a model-agnostic detection scheme based on injecting random input perturbations. By analyzing the variance of a model’s outputs under these perturbations, their method can flag inputs that cause abnormal output fluctuations as likely adversarial, without modifying the underlying classifier. Ensemble methods have also shown promise as a defense strategy. Smutz and Stavrou [20] leveraged the intrinsic diversity of ensembles to detect evasion attempts. They proposed monitoring the level of agreement among an ensemble’s base classifiers: when an input causes substantial disagreement (i.e., low consensus), the system outputs an “uncertain” label rather than a standard prediction. This mutual-agreement analysis forces an attacker to fool multiple diverse models simultaneously, thereby increasing the difficulty of a successful evasion attempt and enabling many attacks to be identified as outliers in the agreement measure. Pang et al. [15] introduced a more formal approach to ensemble robustness by explicitly promoting prediction diversity. Their Adaptive Diversity Promoting (ADP) regularizer encourages ensemble members to produce distinct non-top-ranked predictions, thereby improving robustness by reducing the transferability of adversarial examples across models. Although effective, their approach primarily emphasizes output-level diversity and does not explore feature-level or model-specific diversity strategies. In addition to detection and diversity-based methods, randomized smoothing has emerged as a powerful ensemble-inspired defense. Cohen et al. [5] showed that averaging a base classifier’s predictions over multiple noisy copies of an input, generated using Gaussian noise, produces a “smoothed” classifier with certified robustness guarantees. This approach reduces the effect of any single adversarial perturbation, making the overall classifier provably robust up to a quantifiable perturbation radius.
3
Threat Model
We consider a black-box evasion setting in which an adversary interacts with a deployed machine learning model by submitting inputs and observing the predicted class labels. Our threat model is based on the following assumptions: • Model-Agnostic Deployment: We assume that an initial machine learning model, denoted as 𝑐𝑙 𝑓0 , has already been trained by the defender prior to deployment. Our defense
IWSPA ’26, June 23–25, 2026, Frankfurt am Main, Germany
strategy is model-agnostic in the sense that it does not impose restrictions on the architecture or training procedure of 𝑐𝑙 𝑓0 . Instead, it operates as a post-training enhancement step, consistent with secure development and operations (SecDevOps) practices. The pretrained model and its associated training data are provided as inputs to our framework, which then applies its defense mechanisms before deployment in adversarial environments. • Binary Classification Assumption: We focus on binary classification tasks (e.g., malicious vs. non-malicious), where the negative class represents benign inputs and the positive class represents malicious ones. Nevertheless, without loss of generality, the proposed framework can be extended to multiclass settings and is compatible with a broad range of underlying machine learning models. • Adversarial Goal: The adversary begins with a malicious input 𝑥 𝐴 that is correctly classified as belonging to the positive (i.e., malicious) class. The adversary’s objective is to find a perturbed instance 𝑥 ∗ such that the classifier outputs the negative (i.e., benign) class label. • Query Access: The adversary can submit any syntactically valid input to the deployed model and observe the corresponding output label. Although query access is unrestricted in principle, the adversary seeks to minimize the number of queries because repeated interactions may increase detection risk or operational cost. • No Training Control: The adversary has no influence over the training data, training procedure, or feature selection process. All adversarial manipulations take place at inference time, with the goal of discovering an evading instance 𝑥 ∗ . • Attack Methodology: We assume that the adversary employs a Multi-Line Search (MLS) strategy constrained by the ℓ2 norm [14]. This attack efficiently probes decision boundaries by querying along weighted feature directions. However, the proposed framework is not tied to this specific attack and can, in principle, be evaluated under other querybased attack strategies as well. This threat model emphasizes the generality and practicality of our defense framework. Rather than relying on knowledge of a specific classifier or attack formulation, the framework is designed to increase the difficulty of reliably estimating decision boundaries and identifying an optimal evading instance 𝑥 ∗ .
4
Proposed Model: RESSAP
The overall architecture of our proposed model, Robust Ensemble of Selectively Strengthened and Augmented Predictors (RESSAP), is depicted in Fig. 1. Our framework begins by accepting a pretrained base classifier, denoted as 𝑐𝑙 𝑓0 , and a training dataset. This model is assumed to have been developed initially by the data science and/or ML engineering team and is now ready to be assessed for susceptibility to adversarial evasion attacks. The ultimate goals of the framework are twofold: (1) to measure the susceptibility of 𝑐𝑙 𝑓0 to optimal evasion attacks and quantify it based on the number of adversarial queries required to successfully generate an evading instance, and (2) to create a randomized ensemble classifier using the provided training data to increase the adversarial evasion cost,
IWSPA ’26, June 23–25, 2026, Frankfurt am Main, Germany
Parsa Memarzadehsaghezi, Zahra Hashemi, Pooria Madani, and Mehran Ebrahimi
Figure 1: The RESSAP architecture. i.e., increasing the number of queries required to craft a successful evading instance. Multiple weak classifiers are needed to create the ultimate randomized classification system, which is the core promise of this framework. We achieve this by first creating multiple subsets of the feature space and training a weak classifier for each subset (i.e., bag). Then, at runtime, 𝑘 subsets are selected at random to handle a given classification query. In this context, 𝑐𝑙 𝑓0 is used to compute three feature evaluation metrics based on importance, robustness, and resilience, which guide the construction of 𝑡 diverse feature subsets. Next, the training data is vertically split (based on the feature subsets created in the previous step) into multiple bags, with each bag containing the training instances corresponding to a specific feature subset. To improve the generalizability and robustness of the weak classifiers, we then apply noise-based data augmentation within each bag. Finally, a weak classifier is trained on each augmented bag. This modular structure enables diversity and randomness in both feature representation and model decision-making, thereby making the system significantly harder to bypass through adversarial evasion. In the following subsections, we describe each component of the framework in more detail.
4.1
Feature Selections and Subset Generations
The goal of our feature selection process is to construct subsets that are both informative and resilient to adversarial evasion attacks. By identifying features that are significantly discriminatory while remaining stable under adversarial perturbation, we ensure that each classifier in the ensemble is built on a robust and diverse foundation. In this component, features are selected based on two primary metrics: (1) feature importance, denoted as 𝐹𝑆𝐼 , and (2) feature robustness, denoted as 𝐹𝑆𝑅. To make the selection process more objective, we introduce a new composite metric, 𝐹𝑆𝑅𝐼 , which integrates both measures to assess the overall resilience of each feature. 4.1.1 Feature Importance - 𝐹𝑆𝐼 . This metric measures how crucial each feature is for the base model 𝑐𝑙 𝑓0 . Feature importance using feature value permutation is done by measuring the drop in a model’s performance when the values of a specific feature are randomly shuffled across all instances in the dataset. This permutation breaks the relationship between that feature and the target
variable, allowing us to observe how much the model relied on it to make accurate predictions. A larger drop in performance indicates higher importance of the feature, while little or no change suggests the feature is less relevant. The normalized feature importance for each feature 𝑗 is computed as 1 Í𝑁 𝑖=1 (𝑎 − 𝑎𝑖 𝑗 ) 𝐹𝑆𝐼 ( 𝑗) = Í𝑑 𝑁 1 Í𝑁 , (1) 𝑖=1 (𝑎 − 𝑎𝑖𝑘 ) 𝑘=1 𝑁 where 𝑎 is the baseline classification accuracy of 𝑐𝑙 𝑓0 ; 𝑎𝑖 𝑗 is the accuracy after permuting feature 𝑗 in the 𝑖-th repetition, 𝑁 is the number of repetitions, and 𝑑 is the total number of features. 4.1.2 Feature Robustness - 𝐹𝑆𝑅. This metric evaluates the robustness of individual features based on the pretrained base classifier 𝑐𝑙 𝑓0 , by quantifying the change in its performance (e.g., accuracy) when each feature in the training set is perturbed with small random noise. Specifically, we first compute a normalized sensitivity term across features based on the average absolute performance change, and then convert it into a robustness-oriented score by subtracting it from one. This provides a relative robustness measure in which higher values indicate lower sensitivity to perturbation. 1 𝑁
𝑁 Í
𝑎 − 𝑎𝑖 𝑗
𝑖=1
𝐹𝑆𝑅( 𝑗) = 1 −
𝑑 𝑁 Í 1 Í 𝑁 𝑘=1
(2) |𝑎 − 𝑎𝑖𝑘 |
𝑖=1
Equation (2) defines 𝐹𝑆𝑅( 𝑗), the normalized robustness score of feature 𝑗, 𝑎 is the baseline accuracy on the original dataset, 𝑎𝑖 𝑗 is the accuracy after perturbing feature 𝑗 in the 𝑖-th repetition using random noise, 𝑁 is the number of perturbation repetitions, and 𝑑 is the total number of features. The term is subtracted from one to reflect robustness rather than sensitivity, since the numerator measures how much the model’s performance changes when feature 𝑗 is perturbed. The only difference between 𝐹𝑆𝑅 and 𝐹𝑆𝐼 lies in how the features are altered. Instead of permuting the feature values as in 𝐹𝑆𝐼 , we perturb them by adding small random noise. Additionally, we take the absolute value of the performance change because in robustness analysis we are interested in the magnitude of sensitivity regardless of whether the performance increases or decreases. 4.1.3 Feature Resilience - 𝐹𝑆𝑅𝐼 . To fully capture the strength of each feature, we compute a resilience score that combines both
Robust Ensemble of Selectively Strengthened and Augmented Predictors
IWSPA ’26, June 23–25, 2026, Frankfurt am Main, Germany
its importance and robustness. These metrics are derived entirely by observing how 𝑐𝑙 𝑓0 ’s performance changes under controlled perturbations of the training instances. The resilience score for a feature 𝑗 is then computed by summing its normalized importance and robustness scores: 𝐹𝑆𝑅𝐼 ( 𝑗) = 𝐹𝑆𝐼 ( 𝑗) + 𝐹𝑆𝑅( 𝑗)
(3)
A higher 𝐹𝑆𝑅𝐼 ( 𝑗) value indicates that feature 𝑗 is both highly influential to 𝑐𝑙 𝑓0 ’s decision-making and robust to adversarial or random perturbations. Conversely, features with lower resilience are either less critical, more vulnerable to noise, or both. Subset Generation: Once the feature scores are computed using 𝑐𝑙 𝑓0 , we dynamically generate feature subsets based on these scores. Each subset 𝑠𝑙 is a selected subset of features from the original feature space, and its size |𝑠𝑙 | is determined by: 𝑙 ·𝑑 |𝑠𝑙 | = , for 𝑙 = 1, 2, . . . , 𝑡 (4) 𝑡 where 𝑑 is the total number of features, 𝑡 is the total number of subsets (and classifiers), and ⌈·⌉ denotes the ceiling function to ensure an integer number of features. Subsets are constructed by selecting the top-ranked features according to three ranking metrics: 𝐹𝑆𝑅𝐼 , 𝐹𝑆𝐼 , and 𝐹𝑆𝑅. To prioritize the selection of more resilient and informative features while maintaining diversity: half of the subsets ( 𝑡2 ) are constructed using features ranked highest by 𝐹𝑆𝑅𝐼 ; a quarter of the subsets ( 𝑡4 ) are constructed using features ranked highest by 𝐹𝑆𝐼 ; and the remaining quarter ( 𝑡4 ) are constructed using features ranked highest by 𝐹𝑆𝑅. We primarily emphasize resilience (features that are both important and robust) to encourage stronger adversarial resistance, while also including subsets focusing solely on importance (to support clean accuracy) and robustness (to support stability under perturbations).
4.2
4.3
Building upon the feature subsets constructed from the selection metrics, we apply data augmentation to simulate natural variations and minor adversarial perturbations within each bag, encouraging classifiers to learn more stable and generalizable decision boundaries. Data augmentation is applied to each selected feature subset. Let X ∈ R𝑛×𝑑 denote the original training dataset with 𝑛 samples and 𝑑 features. Let 𝑠𝑙 ∈ 𝑆 denote the set of feature subsets created in the previous step. Let 𝑋𝑠𝑙 ∈ R𝑛× |𝑠𝑙 | denote the training dataset sliced based on features in 𝑠𝑙 . Then data augmentation for each training-set split (i.e., bag) is defined as 𝝐 ∼ N (0, 𝜎),
where N is the Gaussian distribution, and 𝜎 is the noise level controlling the intensity of the augmentation. This process mirrors the perturbations used in the computation of 𝐹𝑆𝑅, ensuring that classifiers trained on these training subsets are exposed to realistic variations and potential adversarial alterations. This step not only simulates natural data variations but also improves the model’s resilience by reducing sensitivity to minor adversarial modifications.
Classifiers
Following the data augmentation step, we train an ensemble of 𝑡 classifiers, each on a distinct, augmented feature subset S𝑙 . Using an ensemble of classifiers trained on diverse subsets enhances model heterogeneity—a critical factor in defending against adversarial attacks. This structure compels adversaries to bypass multiple, independently trained decision boundaries, significantly increasing the complexity of successful evasion. In our proposed model, each classifier 𝑓𝑙 , where 𝑙 ∈ {1, . . . , 𝑡 }, is trained on its corresponding augmented subset 𝑠𝑙 : 𝑓𝑙 = train X̃𝑠𝑙 , y (5) The ensemble collaboratively strengthens the model’s resilience by incorporating diverse perspectives rooted in feature importance, robustness, and resilience.
4.4
Data Augmentation
X̃𝑠𝑙 = X𝑠𝑙 + 𝝐,
Figure 2: Decision boundary of 𝑐𝑙 𝑓0 on our mock data.
Aggregation and Prediction
At inference time, 𝑘 classifiers are selected at random to form the prediction set 𝑃. For each selected classifier 𝑓𝑙 , the model computes the class probability vector using only the features in 𝑠𝑙 : p𝑙 = 𝑓𝑙 x𝑠𝑙 , for 𝑙 ∈ 𝑃 (6) The aggregated probability vector is obtained via summation: ∑︁ p= p𝑙 (7) 𝑙 ∈𝑃
The final predicted class is given by: 𝑦ˆ = arg max p(𝑐)
(8)
𝑐
where p(𝑐) denotes the aggregated probability for class 𝑐.
5 Experiments, Results, and Discussion 5.1 Experimental Setup We evaluate our models using a mock dataset consisting of 𝑛 = 600 instances and 𝑑 = 10 features. The dataset is synthetically generated by first creating a 600×10 matrix of uniformly distributed random values, which is then scaled by column-specific factors
IWSPA ’26, June 23–25, 2026, Frankfurt am Main, Germany
Parsa Memarzadehsaghezi, Zahra Hashemi, Pooria Madani, and Mehran Ebrahimi
Table 1: Results on the Mock Dataset (Accuracy %). Model SVM RESSAP-without Feature Selection RESSAP-without Random Classifier Selection RESSAP-without Data Augmentation RESSAP
Clean Data (Accuracy) 94.2 91.7 96.5 97.2 96.5
derived from additional uniform random numbers and their tangent transformations. Each feature thus becomes uniformly distributed over a unique range. Binary labels are assigned by computing the ℓ2 norm of each sample and thresholding at the median value, ensuring a balanced classification target. The distribution of the dataset is depicted in Fig. 2. For simplicity, we set the number of classifiers equal to the number of features, i.e., 𝑡 = 𝑑, and the number of selected classifiers as 𝑘 = 𝑡/3. Exploring the effect of tuning this parameter is left for future work. To evaluate robustness under adversarial conditions, we apply a query-based evasion strategy using the Multi-Line Search (MLS) method [14]. This algorithm iteratively probes the classifier along weighted feature directions, searching for the minimum-cost perturbation that causes misclassifications. Its efficient binary search mechanism ensures consistent adversarial query generation across all model variants. We employ two evaluation methods: (1) counting the number of queries required to fool the models, and (2) measuring the success rate of adversarial attacks generated by the MLS attack algorithm. We compare our results against a baseline Support Vector Machine (SVM) classifier as 𝑐𝑙 𝑓0 .
5.2
Adversarial Attack (Success Rate) 100 51.45 35.63 36.80 32.49
Table 2: Success Rate Based on Number of Queries Compared to the SVM Classifier. Model Variant RESSAP w/o Feature Selection RESSAP w/o Random Classifier Selection RESSAP w/o Data Augmentation RESSAP
Success Rate (%) 61.41 65.50 68.26 78.08
Results and Discussion
In our first result, Table 1 reports (i) the accuracy on clean data and (ii) the success rate of adversarial attacks across different model configurations. As shown, the complete RESSAP model substantially reduces the adversarial attack success rate compared to the baseline SVM (i.e., the initial non-robust 𝑐𝑙 𝑓0 ), while maintaining high accuracy on clean data. Moreover, the ablation results—in which one key component is removed at a time—highlight the contribution of each mechanism to overall robustness. In particular, omitting feature selection, random classifier selection, or data augmentation consistently leads to weaker robustness. Overall, these results indicate that each component of RESSAP plays an important role in improving resistance to adversarial evasion. Table 2 summarizes robustness from a complementary perspective based on the number of queries required to flip predictions. For each instance, we apply the MLS attack and record the number of queries needed to evade both the baseline SVM and each RESSAP variant. We then compute the percentage of cases in which a given RESSAP variant requires more queries than the SVM. As shown, all RESSAP variants require more queries than the baseline in a majority of cases, and the full RESSAP configuration achieves the highest percentage. This result further supports the claim that RESSAP increases the query cost of successful evasion. Figure 3 provides a scatter plot comparing the number of queries required per instance to evade RESSAP versus the baseline SVM.
Figure 3: Query Count Comparison: RESSAP vs. SVM (Scatter Plot).
Each point corresponds to a single instance; the x-axis shows the number of queries for RESSAP and the y-axis shows the number of queries for SVM. The diagonal identity line 𝑦 = 𝑥 indicates equal query cost between the two models. Points below this line correspond to instances for which RESSAP requires more queries than SVM (i.e., higher robustness). As observed, most instances fall below the identity line, indicating that RESSAP is generally more resistant to adversarial perturbations. However, approximately 20% of instances lie on or above the line, suggesting that for a subset of cases RESSAP offers comparable or slightly lower robustness than the baseline. Figure 4 presents box plots of the query counts required to successfully evade the RESSAP model and the baseline SVM. As shown, the RESSAP model exhibits a higher median query count, indicating that, on average, an adversary must issue more queries to craft a successful evasion. In addition, the interquartile range (IQR) for RESSAP is narrower than that of the baseline, which suggests more
Robust Ensemble of Selectively Strengthened and Augmented Predictors
Figure 4: Query Count Comparison: RESSAP vs. SVM (Box Plot). consistent robustness across instances. In contrast, the baseline model shows greater variability, including outliers with very low query requirements, highlighting its susceptibility to rapid exploitation. Overall, these results suggest that RESSAP not only increases the difficulty of evasion in most cases, but also reduces variability in the attack cost across different instances. In summary, our experimental results indicate that combining resilient feature selection, perturbation-based data augmentation, and randomized classifier selection can substantially reduce the success rate of adversarial evasion while maintaining strong clean-data performance. RESSAP leverages an ensemble of classifiers trained on diverse feature subsets and introduces randomness at inference time, making it more difficult for an adversary to reliably estimate decision boundaries and construct effective evading instances.
6
Conclusion and Future Work
In this paper, we introduced RESSAP, a robust ensemble framework that combines feature-level selection, data augmentation, and classifier randomization to strengthen classifiers against adversarial evasion attacks. Our experimental evaluation shows that RESSAP improves robustness against adversarial evasion while maintaining strong accuracy on benign inputs. However, we acknowledge that the current evaluation is limited to a synthetically generated dataset, which may not fully capture the complexity of real-world applications. In addition, we have not yet compared RESSAP against alternative robust architectures that are specifically designed for adversarial settings. Future work will therefore focus on evaluating the proposed framework on diverse real-world datasets, conducting more extensive comparisons with state-of-the-art robust architectures, and refining the feature selection process to further improve adversarial robustness.
References [1] Moustafa Alzantot, Yash Sharma, Supriyo Chakraborty, Huan Zhang, Cho-Jui Hsieh, and Mani B. Srivastava. 2019. GenAttack: Practical Black-box Attacks with Gradient-Free Optimization. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO ’19). ACM, 1111–1119. doi:10.1145/3321707. 3321749 [2] Battista Biggio, Igino Corona, Giorgio Fumera, Giorgio Giacinto, and Fabio Roli. 2011. Bagging Classifiers for Fighting Poisoning Attacks in Adversarial Classification Tasks. In Multiple Classifier Systems (MCS 2011). 350–359.
IWSPA ’26, June 23–25, 2026, Frankfurt am Main, Germany
doi:10.1007/978-3-642-21557-5_37 [3] Battista Biggio and Fabio Roli. 2018. Wild Patterns: Ten Years after the Rise of Adversarial Machine Learning. Pattern Recognition 84 (2018), 317–331. doi:10. 1016/j.patcog.2018.07.023 [4] Jinghui Chen and Quanquan Gu. 2020. RayS: A Ray Searching Method for Hardlabel Adversarial Attack. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’20). ACM, 1739–1747. doi:10.1145/3394486.3403225 [5] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. 2019. Certified Adversarial Robustness via Randomized Smoothing. In Proceedings of the 36th International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 97). PMLR, 1310–1320. https://proceedings.mlr.press/v97/cohen19c.html [6] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations (ICLR). https://arxiv.org/abs/1412.6572 [7] Bo Huang, Yi Wang, and Wei Wang. 2019. Model-Agnostic Adversarial Detection by Random Perturbations. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19). 4689–4696. doi:10.24963/ijcai.2019/ 651 [8] Alex Kantchelian, J. D. Tygar, and Anthony D. Joseph. 2016. Evasion and Hardening of Tree Ensemble Classifiers. In Proceedings of the 33rd International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 48). PMLR, 2387–2396. https://proceedings.mlr.press/v48/kantchelian16.html [9] Aleksander Kołcz and Choon-Hui Teo. 2009. Feature Weighting for Improved Classifier Robustness. In Proceedings of the 6th Conference on Email and Anti-Spam (CEAS 2009). 1–8. https://users.cecs.anu.edu.au/~chteo/pub/KolTeo09.pdf [10] Daniel Lowd and Christopher Meek. 2005. Adversarial Learning. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’05). ACM, 641–647. doi:10.1145/1081870.1081950 [11] Pooria Madani, Natalija Vlajic, and Ivo Maljevic. 2022. Randomized Moving Target Approach for MAC-Layer Spoofing Detection and Prevention in IoT Systems. Digital Threats: Research and Practice 3, 4 (2022), 35:1–35:24. [12] Pooria Madani, Natalija Vlajic, and Shadi Sadeghpour. 2020. MAC-Layer Spoofing Detection and Prevention in IoT Systems: Randomized Moving Target Approach. In CPSIOTSEC@CCS (Joint Workshop on CPS&IoT Security and Privacy). 71–80. [13] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In International Conference on Learning Representations (ICLR). https: //arxiv.org/abs/1706.06083 [14] Blaine Nelson, Benjamin Rubinstein, Ling Huang, Anthony Joseph, Shing-hon Lau, Steven Lee, Satish Rao, Anthony Tran, and J. D. Tygar. 2010. NearOptimal Evasion of Convex-Inducing Classifiers. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (AISTATS) (Proceedings of Machine Learning Research, Vol. 9). PMLR, 549–556. https: //proceedings.mlr.press/v9/nelson10a.html [15] Tianyu Pang, Kun Xu, Chao Du, Ning Chen, and Jun Zhu. 2019. Improving Adversarial Robustness via Promoting Ensemble Diversity. In Proceedings of the 36th International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 97). PMLR, 4970–4979. https://proceedings.mlr.press/v97/ pang19a.html [16] Rafael Pinot, Raphael Ettedgui, Geovani Rizk, Yann Chevaleyre, and Jamal Atif. 2020. Randomization Matters: How to Defend against Strong Adversarial Attacks. In Proceedings of the 37th International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 119). PMLR, 7717–7727. https: //proceedings.mlr.press/v119/pinot20a.html [17] Benjamin I. P. Rubinstein, Blaine Nelson, Ling Huang, Anthony D. Joseph, Shinghon Lau, Satish Rao, Nina Taft, and J. D. Tygar. 2009. ANTIDOTE: Understanding and Defending against Poisoning of Anomaly Detectors. In Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement (IMC ’09). ACM, 1–14. doi:10.1145/1644893.1644895 [18] Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. 2016. Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16). ACM, 1528–1540. doi:10.1145/2976749.2978392 [19] Yucheng Shi, Yahong Han, Qinghua Hu, Yi Yang, and Qi Tian. 2023. QueryEfficient Black-Box Adversarial Attack With Customized Iteration and Sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 2 (2023), 2226– 2245. doi:10.1109/TPAMI.2022.3169802 [20] Charles Smutz and Angelos Stavrou. 2016. When a Tree Falls: Using Diversity in Ensemble Classifiers to Identify Evasion in Malware Detectors. In Network and Distributed System Security Symposium (NDSS). https://www.ndsssymposium.org/wp-content/uploads/2017/09/when-tree-falls-using-diversityensemble-classifiers-identify-evasion-malware-detectors.pdf [21] Fei Zhang, Patrick P. K. Chan, Battista Biggio, Daniel S. Yeung, and Fabio Roli. 2016. Adversarial Feature Selection Against Evasion Attacks. IEEE Transactions on Cybernetics 46, 3 (2016), 766–777.