Large-scale Testing Global Optimization Methods with Black-box Adversarial Attacks

Large-scale Testing Global Optimization Methods with Black-box Adversarial Attacks Wojciech Zarzecki and Jarosław Arabas

arXiv:2608.13296v1 [cs.LG] 13 Aug 2026

Warsaw University of Technology, Institute of Computer Science wojciech.zarzecki.stud@pw.edu.pl, jaroslaw.arabas@pw.edu.pl

Abstract. Existing global optimization benchmark suites are of a moderate size and are based on a small number of analytical functions that date back even to the 1970s. This causes a risk of biasing the development of global optimization methods. We argue that the tasks related to the black-box adversarial attack (BBAA) can serve as valuable global optimization benchmark in many-dimensional space. We demonstrate the efficiency of several types of evolutionary algorithms and other metaheuristics in solving example BBAA problems. Thus, we take a step towards convergence of global optimization methods to the challenges and needs that arise in the modern machine learning field.

1

Introduction

1.1

Evolution of global optimization benchmarking methods

Development of global optimization methods is facilitated by benchmark suites. Experimental analysis of global optimization methods dates back at least to the 1970’s, with the book by Dixon and Szegö [13] being perhaps the most commonly known. The original idea of the authors was to define the test objective functions that are easy to compute, with the known position and value of the global optimum. At the same time, the landscape of the test functions was designed to model the difficulties that were expected to be observed in realworld optimization problems. The advent of evolutionary computation has broadened the pool of test functions. As a result, there are several famous test functions whose origins date back to the 1980’s or earlier, including functions by Ackley, Griewank, Rastrigin, Rosenbrock, Schaffer, and Schwefel (the names are given to honour their inventors), to name a few. The year 2005 brought the inspiring competition at the CEC conference that was based on the CEC’2005 benchmarking suite [31] that comprised 25 optimization problems. The number of dimensions was 10, 30, and 50. The benchmark suite came with the computer code that implemented the optimization problems to be run by the author of the optimization method. The benchmark suite came with the standardized testing methodology definition. Subsequent years brought a series of CEC benchmark suites [34,26,8,30,27,28,29], with the dimension number not exceeding 100.

2

W. Zarzecki and J. Arabas

Another advance towards the reproducibility of results was the COCO suite [20,19]. The benchmarking process was performed by the COCO software, and the authors of the optimization method had to implement it as a function to be run by the benchmarking procedure. The number of dimensions did not exceed 40. In CEC and COCO, the benchmarking can be performed many times for the same method, so the optimization method can be tuned for maximum performance for the whole suite. To avoid this, the BBComp competition [39] provided access to the server that allowed the user a maximum number of objective function evaluations. In this way, the organizers planned to avoid the risk of “overfitting” the optimization methods to the benchmarking suite. In this competition, the maximum number of dimensions was 64. Large-scale versions of the CEC suite [36,37] and of COCO [14] were also proposed, where the dimension number was at most 1000 and 640, respectively. In the benchmarking suites from the CEC and COCO families, the optimization problems were defined with the use of combinations of a few types of objective functions that were introduced in much earlier literature. In contrast to these “synthetic” problems, the CEC real-world optimization suite [12] was based on problems taken directly from practical applications. The maximum number of dimensions was only 30. One possible application of optimization methods is the field of Machine Learning (ML). In ML, in particular in the training process of neural networks, standard procedures are gradient-based, since the gradient and the value of the loss function can be computed at comparable cost. The number of dimensions in typical neural models is enormous. Therefore, the ML optimization problems, where the global optimization methods can be considered, are rather related to tuning hyperparameters of the learning process - cf. the black-box optimization challenge at NeurIPS’2020 [45], where the number of parameters to optimize is rather moderate. 1.2

Adversarial attack problem

Progress in machine learning and computer vision has resulted in a broader application of machine learning models, which has naturally attracted attention to the safety and robustness of these mechanisms. One of the key aspects in this area is the adversarial attack. Its goal is a perturbation of an input image that is imperceptible to a human observer but changes the decision of the model. Figure 1 illustrates an example successful black-box attack where a subtle, optimized perturbation is added to an original image of a horse. As illustrated in Figure 1, the input remains benign to the human eye while causing a wrong decision of the attacked model [43]. Such attacks are worth attention for two main reasons. First, they reveal the scope of possible threats against vision systems, ranging from physical-world attacks against visual classifiers [15] to attacks targeting medical deep learning systems [16]. Second, they have potential for benign applications such as

Testing global optimization methods with adversarial attacks.

3

Fig. 1: Example adversarial attack: an image that is originally classified as a horse, after adding the noise, is classified as a cat

preserving private or copyrighted data, for example, by watermarking deep neural networks through backdooring [1] or by protecting images from malicious diffusion-based editing [9]. Adversarial attacks can be distinguished into model-informed and modelagnostic approaches. The model-informed types of attack make use of the information about the disturbance-related sensitivity of the classifier, e.g., the gradient of the loss function [17], [33], [5]. Following Costa et. al[11], we call attacks from the first group white-box attacks and from the latter — black-box attacks. One of the approaches for black-box attacks is gradient estimation, represented by [7], where a coordinate-wise finite-difference gradient estimation applied to the objective from [5] is used to optimize the disturbance pattern. This result was improved by the GenAttack method [2], which used a fitnessproportionate selection GA on class-probability feedback. AutoZOOM [44] is another improvement to ZOO that reduces the number of queries needed to perform the attack by 93% thanks to the use of the autoencoder-compressed perturbation space. Ilyas et al. [23] use the NES gradient estimation and demonstrate achieving 2–3 orders of magnitude fewer queries in comparison to the pixel-wise finite differences. Some methods try to rely only on the final decision of the model, eg. the chosen class, without accessing the output probability distribution across all classes. One of them is Boundary Attack [4] that consists in performing the random walk along the decision boundary starting from a large adversarial perturbation obtained via progressively added Gaussian noise. This approach was refined in [6], where the authors perform the Monte Carlo boundary gradient estimation and couple it with the binary search. Other methods [38,24] rely on the assumption that the distributions of training data are known. Liu et. al[32] suggested sharing the adversarial perturbations for models that were trained using data from the same distribution. However, the most prominent direction to perform the black-box attack is the application of some optimization methods to find perturbations that change

4

W. Zarzecki and J. Arabas

the decision of the attacked model. The Square Attack [3] used a random search to find adversarial perturbations based on the fitness function constructed with the model’s output probability distribution. This idea was further developed as SimBA [18], where the procedure basis is changed from the Cartesian to the discrete cosine. Other methods use more sophisticated metaheuristics. In PSO Attack [35], the Particle Swarm Optimization searches the perturbation space, and the authors report a 99.6% success rate on CIFAR-10 with fewer queries than ZOO. EvoBA [22] is an application of an Evolution Strategy to minimize L0 via sparse pixel-level mutations. Some methods limit the perturbation to one pixel only; for example, One Pixel [42] is an application of Differential Evolution to find a single modified pixel sufficient to flip the class. The efficiency of Genetic Algorithms, Differential Evolution, and CMA-ES for the one-pixel attacks on CIFAR-10 was compared by Clare et al. [10]. Zheng et. al [48] proposed a benchmark for different black-box adversarial attacks, comparing various attacks techniques; however, the impact of the chosen optimization method for score-based attacks remains under-explored. 1.3

Scope of the paper

We focus on the Black-Box Adversarial Attack (BBAA), in which only the model output probability distribution can be accessed. We allow for the perturbation of all pixels of the original image.We define the BBAA as an optimization problem that is aimed at finding the perturbation of the original image that results in misclassification and is similar to the noise with the smallest possible variance. We formulate the local search method to demonstrate that the optimization problem is multimodal. For this reason, the BBAA problem can be considered a benchmark to test global optimization methods. We provide an efficiency comparison of several example metaheuristic methods, including the classical Evolutionary Algorithm, four versions of Differential Evolution, Grey Wolf Optimizer, and the INFO (Efficient Optimizer based on Weighted Mean of Vectors) method. Furthermore, we give possibility to test new methods, proving a framework at https://anonymous.4open.science/r/black-box-adversarial-attacks/ Under the same address, the reader can find detailed information that would exceed the page limit of this text.

2

Optimization task related to the adversarial attack problem

We consider the classification function c : [0, 1]n → {1, . . . , k} and the model p : [0, 1]n × RN → P ({1, ..., k}), where P ({1, ..., k}) is the space of probability distributions over the space of indices 1,...,k. We assume that the model assumes N tunable real parameters. Given an input vector x, the model assigns each class j the probability pj (x, θ). The class whose probability is highest is considered the winning class that is yielded by the model:

Testing global optimization methods with adversarial attacks.

5

ĉ(x, θ) = arg max pj (x, θ); j=1,...,k

While performing BBAA, we assume that the model parameters have been tuned to maximize the classification accuracy. The adversarial attack to the input vector x consists of defining a disturbance vector δ(x) ∈ [−ε, ε]n , where ε > 0 is a hyperparameter controlling the maximum perturbation magnitude. The disturbance is added to the original input, and it is expected that the classifier’s output becomes improper: ĉ(clip(x + δ), θ) ̸= c(x) where   0 clip(y) = y   1

y<0 0≤y≤1 y>1

For the notation brevity, in further text we simply write p(x) and δ instead of p(x, θ) and δ(x). The adversarial attack should be hard to recognize for the human — it is desired that the degree of disturbance is smallest. We model this by the minimization of the squared L2 norm of δ. Hence, the adversarial attack for a particular image x is formulated as a bound-constrained minimization problem with the objective function defined as: L(δ) = − log pc(x) (x + δ) − α∥δ∥22

(1)

where c(x) is the ground-truth class and α is a hyperparameter to control the importance of the L2 norm of the perturbation. The feasible set is a hypercube [−ε, ε]n . The optimization operates in continuous space: the perturbation vector is generated by the optimizer, added element-wise to the original normalized image, and the perturbed image is clipped to the range [0, 1] to ensure valid pixel values. The attack for each image is treated as a separate optimization problem. The adversarial attack is performed only for these input vectors that are properly classified when no disturbance is added.

3

Is adversarial attack a global optimization problem?

If the classification model is nonlinear with respect to x, it can be expected that the objective function (1) is multimodal. In this section we validate this claim.

6

3.1

W. Zarzecki and J. Arabas

Datasets and models

The experiments are conducted on two standard image classification benchmarks, CIFAR-10 [25] and ImageNet [40]. In their raw form, images are represented as tensors with pixel intensities in the range [0, 255]. During preprocessing, the images are converted to floating-point tensors normalized to the range [0, 1]. CIFAR-10 images are three-channel (RGB) tensors of shape 3 × 32 × 32. ImageNet images are in higher resolution and vary in size; we follow the authors’ recommended procedure before passing them to the attacked classification model. First, we resize input images to a tensor of shape 256 × 256 × 3 and then crop the central pixel to a tensor of size 224 × 224 × 3 — such a tensor is also the input for the adversarial attack procedure. Before conducting the adversarial attacks, we trained two classification models to serve as the black-box attack targets. When performing an attack, we access only the predicted probabilities for the target classes. For ImageNet, the model was a standard pre-trained Resnet-18 [21], while for CIFAR-10, we trained a model based on standard convolutional architectures, similar to VGG [41]. To ensure reproducibility, we provide complete information about the architectural and training details in the linked archive. 3.2

Local search applied for the adversarial attack

We define a local optimization method, the Stochastic Growth Attack with Binary Search Refinement (SGA-BSR), designed to identify a minimal perturbation δ through a sequential two-phase process. The images are represented in the pixel space as flattened vectors of size 3×32×32 (CIFAR-10) or 224×224×3 (ImageNet). As described in Algorithm 1, the SGA-BSR first initiates a Stochastic Growth phase. The perturbation vector δ is initialized to zero. During each of the N iterations, the algorithm randomly selects a single pixel index idx and increments its value by a step size η, while ensuring the value remains within the valid pixel range [0, ε] via a clipping function. This modification is accepted only if it results in a non-decreasing objective function value L(δtmp ) ≥ L(δ). This greedy, stochastic progression continues until the model pθ fails to correctly identify the ground-truth class c(x), at which point the growth phase terminates. Once a successful misclassification has been achieved, the algorithm enters the Refinement phase to minimize the magnitude of the disturbance while maintaining the misclassification. The algorithm identifies the set S of all indices that were modified during the first phase. For each modified index in S, a binary search is performed between zero and the current perturbation value δ[idx]. The search seeks to find the smallest possible increment best_val that still results in an incorrect classification. By isolating and minimizing the contribution of each perturbed pixel in sequence, the SGA-BSR effectively identifies a local optimum that satisfies the misclassification constraint with a significantly reduced total perturbation magnitude.

Testing global optimization methods with adversarial attacks.

7

Algorithm 1 Stochastic Growth Attack with Binary Search Refinement 1: Input: Original image x, ground-truth c(x), model pj , iterations K, step size η, limit ε=1 2: Output: Adversarial image xadv 3: Initialize perturbation δ ← 0 4: Phase 1: Stochastic Growth 5: for i = 1 to K do 6: Choose a random index idx ∈ {1, . . . , C × H × W } 7: δtmp ← δ 8: δtmp [idx] ← clip(δ[idx] + η, 0, ε) 9: if L(δtmp ) ≥ L(δ) then 10: δ ← δtmp 11: end if 12: if ĉ(clip(x + δ)) ̸= c(x) then 13: break ▷ Target misclassified 14: end if 15: end for 16: Phase 2: Refinement 17: Identify indices S = {idx | δ[idx] > 0} 18: for each idx ∈ S do 19: low ← 0, high ← δ[idx], best_val ← high 20: while low ≤ high do 21: mid ← ⌊(low + high)/2⌋ 22: δtest ← δ 23: δtest [idx] ← mid 24: if ĉ(clip(x + δtest )) ̸= c(x) then 25: best_val ← mid, high ← mid − 1 26: else 27: low ← mid + 1 28: end if 29: end while 30: δ[idx] ← best_val 31: end for 32: return xadv ← x + δ

We execute SGA-BSR on each image using 1000 different random seeds. If the underlying problem were strictly unimodal (a simple local optimization task), attacks originating from different seeds would consistently converge to the exact same optimal perturbation. However, our empirical results demonstrate that the resulting perturbations, as well as their quality, are different between independent optimization runs. Figure 2 presents heatmaps of the mean and standard deviation of perturbations yielded by each run of Algorithm 1 for two randomly selected example images. In both cases, it can be observed that for some image regions, the mean disturbance is zero or nearly zero, and the standard deviation is also zero. Yet there are regions of the image for which both the mean and standard deviation of the disturbance are nonzero, which indicates that a diversity of pixels is involved

8

W. Zarzecki and J. Arabas

Fig. 2: Pixel-wise mean and standard deviation of the perturbation δ obtained over 100 independent SGA-BSR runs, shown for two example CIFAR-10 images. Regions with a nonzero standard deviation indicate pixels that are used only by some runs. This is evidence that different seeds yield different successful perturbation vectors; therefore, the attack problem is multimodal.

in successful attacks, but there is no need to use all of them to change the model class. In other words, independent runs of SGA-BSR yield different alternative disturbance vectors δ. Figure 3 displays the boxplots of the objective function values across the runs for each image under attack, for CIFAR-10 and ImageNet sets. For many images, the SGA-BSR yielded a variety of results which differed both in the objective function value and the perturbation magnitude. This evidences that the adversarial attack problem has many different local optima, since SGA-BSR is a local optimization method.

4

Application of global optimization methods to perform black-box adversarial attack

Knowing that the BBAA is a global optimization problem, we test several global optimization methods to check their efficiency in performing the attack. Section 4.1 describes the details of experiments, while Section 4.2 presents results for CIFAR-10 and for ImageNet. 4.1

Outline of the experiment

For each image x, we perform 100 independent runs of the optimization method. Each method has the same population size, and in each iteration can test each member of the population only once. The initial population of perturbations generated randomly within the admissible range of [−ε, ε]n . Each run of the optimization process is terminated after exceeding the admissible limit of perturbations (which means that the run was unsuccessful). From each run we record the number of tested perturbations, the attack success rate, and the normalized

Testing global optimization methods with adversarial attacks.

9

Fig. 3: Per-image variance of SGA-BSR solutions across 1000 independent runs (α = 0.1). For each image, Algorithm 1 is run with 100 random seeds. Boxplots for objective function value: top two panels: CIFAR-10, bottom: ImageNet. Columns are images, boxes are IQR over seeds, whiskers 1.5 IQR, dots outliers.

disturbance strength defined as ||δ||2 strength(δ) = √ n

(2)

where n is the vector length. All optimizers were run with their default hyperparameter values as provided by the mealpy library [46]. The two settings shared across every optimizer are the population size Npop = 500 and the maximum number of iterations T = 500. Optimizer-specific defaults are listed in the archive accompanying the paper. SADE, GWO, and INFO require no additional hyperparameters beyond the shared ones, as they perform internal self-adaptation or use fixed algorithmic rules. 4.2

Results

Table 1 summarizes the results obtained for the CIFAR-10 set. Each row corresponds to one combination of optimizer, regularization weight α, and maximum pixel disturbance ε. We report the fraction of attacks that succeed in flipping the label (Suc. R.), the number of queries until the first successful perturbation averaged over successful runs only (First Succ.), the strength of successful perturbations, and the best objective function value observed in each run. Results

10

W. Zarzecki and J. Arabas

are reported in the form of pairs (mean ± standard deviation). For each (α, ε) block, the best objective value across optimizers is highlighted in bold, and the second best is underlined. The pixel perturbation strength ε is the single most decisive factor for the attack success on CIFAR-10: at ε = 0.01 every optimizer is effectively blocked, while at ε = 0.1 and ε = 0.2 most methods flip the majority of images. The regularization weight α has a non-monotonic effect: a moderate value of α = 0.1 helps most optimizers by pulling the population toward perturbations that both cross the decision boundary and keep a tighter perturbation strength, and in particular lifts GWO from near-zero success, while pushing α up to 10 or 100 gives no further efficiency inrease. Table 1: Results of adversarial attacks on the CIFAR-10-based classifier Optim.

α

ε Suc. R. (↑) First Succ. (↓)

L2

Obj. fn. (↑)

DE GEN GWO INFO JADE SADE SHADE

0.0 0.0 0.0 0.0 0.0 0.0 0.0

0.01 0.01 0.01 0.01 0.01 0.01 0.01

2.60% 2.60% 0.00% 1.30% 1.30% 2.60% 2.60%

1.0 ± 0.0 1.0 ± 0.0 0.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0

0.0071 ± 0.0002 0.0057 ± 0.0001 0.0000 ± 0.0000 0.0052 ± 0.0000 0.0055 ± 0.0000 0.0062 ± 0.0001 0.0060 ± 0.0005

0.22 ± 0.26 0.18 ± 0.23 0.19 ± 0.24 0.20 ± 0.24 0.21 ± 0.26 0.21 ± 0.26 0.21 ± 0.26

DE GEN GWO INFO JADE SADE SHADE

0.0 0.0 0.0 0.0 0.0 0.0 0.0

0.1 0.1 0.1 0.1 0.1 0.1 0.1

28.57% 24.68% 1.30% 25.97% 27.27% 28.57% 25.97%

1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0

0.0708 ± 0.0044 0.0565 ± 0.0019 0.0335 ± 0.0000 0.0563 ± 0.0047 0.0614 ± 0.0050 0.0610 ± 0.0043 0.0623 ± 0.0059

0.75 ± 0.95 0.41 ± 0.69 0.21 ± 0.25 0.45 ± 0.66 0.60 ± 0.75 0.63 ± 0.78 0.59 ± 0.73

DE GEN GWO INFO JADE SADE SHADE

0.0 0.0 0.0 0.0 0.0 0.0 0.0

0.2 0.2 0.2 0.2 0.2 0.2 0.2

59.74% 57.14% 10.39% 53.25% 54.55% 55.84% 55.84%

1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0

0.1381 ± 0.0085 0.1110 ± 0.0056 0.0658 ± 0.0006 0.1107 ± 0.0078 0.1198 ± 0.0090 0.1208 ± 0.0074 0.1189 ± 0.0103

2.47 ± 2.60 1.69 ± 2.11 0.34 ± 0.47 1.62 ± 2.05 1.88 ± 2.08 1.95 ± 2.23 1.82 ± 2.05

DE GEN GWO INFO JADE SADE SHADE

0.1 0.1 0.1 0.1 0.1 0.1 0.1

0.01 0.01 0.01 0.01 0.01 0.01 0.01

3.90% 2.60% 0.00% 1.30% 2.60% 1.30% 2.60%

1.0 ± 0.0 1.0 ± 0.0 0.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0

0.0070 ± 0.0001 0.0057 ± 0.0001 0.0000 ± 0.0000 0.0056 ± 0.0000 0.0059 ± 0.0004 0.0059 ± 0.0000 0.0057 ± 0.0003

0.18 ± 0.27 0.15 ± 0.23 0.17 ± 0.25 0.19 ± 0.24 0.18 ± 0.26 0.18 ± 0.26 0.18 ± 0.26

DE GEN

0.1 0.1 0.1 0.1

59.74% 61.04%

14.9 ± 24.6 11.2 ± 29.7

0.0742 ± 0.0054 0.0569 ± 0.0018

0.67 ± 1.03 0.59 ± 0.64

continued on next page. . .

Testing global optimization methods with adversarial attacks.

11

continued on next page. . . Optim.

α

ε Suc. R. (↑) First Succ. (↓)

L2

Obj. fn. (↑)

GWO INFO JADE SADE SHADE

0.1 0.1 0.1 0.1 0.1

0.1 0.1 0.1 0.1 0.1

32.47% 25.97% 50.65% 37.66% 40.26%

16.4 ± 25.8 1.1 ± 0.5 8.9 ± 12.6 11.8 ± 30.4 6.3 ± 11.6

0.0888 ± 0.0141 0.0556 ± 0.0017 0.0633 ± 0.0068 0.0630 ± 0.0064 0.0606 ± 0.0056

0.32 ± 0.76 0.37 ± 0.55 0.50 ± 0.70 0.36 ± 0.76 0.53 ± 0.62

DE GEN GWO INFO JADE SADE SHADE

0.1 0.1 0.1 0.1 0.1 0.1 0.1

0.2 0.2 0.2 0.2 0.2 0.2 0.2

74.03% 97.40% 62.34% 53.25% 81.82% 66.23% 89.61%

5.4 ± 23.5 4.2 ± 6.7 9.4 ± 34.9 1.0 ± 0.0 6.2 ± 21.8 4.8 ± 23.3 15.4 ± 42.1

0.1432 ± 0.0092 0.1115 ± 0.0050 0.1468 ± 0.0475 0.1095 ± 0.0082 0.1228 ± 0.0111 0.1265 ± 0.0118 0.1178 ± 0.0123

2.23 ± 2.43 2.63 ± 1.60 1.43 ± 2.13 1.22 ± 1.71 1.85 ± 1.92 1.58 ± 2.17 1.93 ± 1.81

DE GEN GWO INFO JADE SADE SHADE

1.0 1.0 1.0 1.0 1.0 1.0 1.0

0.01 0.01 0.01 0.01 0.01 0.01 0.01

2.60% 2.60% 0.00% 2.60% 1.30% 2.60% 2.60%

1.0 ± 0.0 1.0 ± 0.0 0.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0

0.0068 ± 0.0000 0.0057 ± 0.0001 0.0000 ± 0.0000 0.0055 ± 0.0001 0.0054 ± 0.0000 0.0055 ± 0.0000 0.0059 ± 0.0005

−0.18 ± 0.27 −0.10 ± 0.22 −0.23 ± 0.39 0.19 ± 0.23 −0.13 ± 0.28 −0.14 ± 0.28 −0.06 ± 0.24

DE GEN GWO INFO JADE SADE SHADE

1.0 1.0 1.0 1.0 1.0 1.0 1.0

0.1 0.1 0.1 0.1 0.1 0.1 0.1

49.35% 81.82% 64.94% 23.38% 53.25% 33.77% 57.14%

17.1 ± 40.7 11.9 ± 25.1 13.5 ± 34.1 1.0 ± 0.0 20.0 ± 37.8 1.8 ± 2.3 25.5 ± 43.7

0.0752 ± 0.0057 0.0568 ± 0.0021 0.0790 ± 0.0221 0.0535 ± 0.0029 0.0629 ± 0.0102 0.0613 ± 0.0084 0.0496 ± 0.0078

−2.87 ± 1.69 −0.58 ± 1.61 −3.34 ± 2.16 −0.29 ± 0.78 −2.19 ± 1.71 −2.91 ± 1.11 −0.89 ± 1.57

DE GEN GWO INFO JADE SADE SHADE

1.0 1.0 1.0 1.0 1.0 1.0 1.0

0.2 0.2 0.2 0.2 0.2 0.2 0.2

74.03% 92.21% 79.22% 54.55% 74.03% 68.83% 76.62%

8.3 ± 31.0 13.2 ± 41.0 9.1 ± 22.4 1.0 ± 0.0 12.2 ± 45.5 5.6 ± 18.8 17.4 ± 43.0

0.1478 ± 0.0093 0.1113 ± 0.0051 0.1542 ± 0.0445 0.1050 ± 0.0078 0.1277 ± 0.0180 0.1216 ± 0.0151 0.1006 ± 0.0141

−4.30 ± 3.79 0.85 ± 3.24 −4.98 ± 4.43 −1.79 ± 2.27 −2.72 ± 3.69 −4.49 ± 3.05 −0.52 ± 3.21

DE 10.0 0.01 GEN 10.0 0.01 GWO 10.0 0.01 INFO 10.0 0.01 JADE 10.0 0.01 SADE 10.0 0.01 SHADE 10.0 0.01

3.90% 5.19% 5.19% 1.30% 3.90% 3.90% 3.90%

1.3 ± 0.5 5.8 ± 5.1 11.5 ± 11.3 1.0 ± 0.0 7.0 ± 8.5 1.0 ± 0.0 34.3 ± 47.1

0.0068 ± 0.0002 0.0053 ± 0.0001 0.0069 ± 0.0022 0.0055 ± 0.0000 0.0053 ± 0.0005 0.0058 ± 0.0003 0.0043 ± 0.0009

−3.77 ± 0.27 −2.69 ± 0.27 −5.22 ± 0.62 0.16 ± 0.36 −3.34 ± 0.35 −3.44 ± 0.32 −2.37 ± 0.29

10.0 0.1 10.0 0.1 10.0 0.1

49.35% 51.95% 66.23%

17.9 ± 41.6 14.8 ± 27.5 13.7 ± 24.6

0.0687 ± 0.0061 0.0537 ± 0.0024 0.0730 ± 0.0218

−38.66 ± 2.64 −27.99 ± 1.40 −45.51 ± 12.28

DE GEN GWO

continued on next page. . .

12

W. Zarzecki and J. Arabas continued on next page. . .

Optim.

α

ε Suc. R. (↑) First Succ. (↓)

L2

Obj. fn. (↑) −6.66 ± 11.85 −33.11 ± 5.37 −33.96 ± 3.72 −24.79 ± 3.01

INFO 10.0 JADE 10.0 SADE 10.0 SHADE 10.0

0.1 0.1 0.1 0.1

24.68% 48.05% 31.17% 38.96%

1.0 ± 0.0 28.9 ± 47.7 2.0 ± 4.4 28.4 ± 63.5

0.0501 ± 0.0030 0.0556 ± 0.0121 0.0535 ± 0.0054 0.0430 ± 0.0076

DE 10.0 GEN 10.0 GWO 10.0 INFO 10.0 JADE 10.0 SADE 10.0 SHADE 10.0

0.2 0.2 0.2 0.2 0.2 0.2 0.2

72.73% 75.32% 77.92% 54.55% 72.73% 68.83% 72.73%

5.8 ± 23.8 16.1 ± 53.2 6.1 ± 8.1 1.1 ± 0.5 10.5 ± 35.6 4.0 ± 12.6 23.0 ± 69.7

0.1283 ± 0.0100 −73.99 ± 5.51 0.1063 ± 0.0059 −54.33 ± 3.66 0.1123 ± 0.0385 −74.33 ± 27.87 0.0965 ± 0.0086 −29.47 ± 27.23 0.0984 ± 0.0160 −59.53 ± 10.71 0.1049 ± 0.0128 −63.63 ± 9.24 0.0822 ± 0.0143 −47.43 ± 7.80

DE 100.0 0.01 GEN 100.0 0.01 GWO 100.0 0.01 INFO 100.0 0.01 JADE 100.0 0.01 SADE 100.0 0.01 SHADE 100.0 0.01

3.90% 3.90% 5.19% 2.60% 3.90% 2.60% 3.90%

1.3 ± 0.5 46.0 ± 63.6 8.8 ± 5.4 1.0 ± 0.0 11.7 ± 13.7 1.0 ± 0.0 54.0 ± 74.2

0.0070 ± 0.0004 0.0052 ± 0.0001 0.0077 ± 0.0014 0.0054 ± 0.0000 0.0055 ± 0.0008 0.0055 ± 0.0001 0.0050 ± 0.0010

DE 100.0 0.1 GEN 100.0 0.1 GWO 100.0 0.1 INFO 100.0 0.1 JADE 100.0 0.1 SADE 100.0 0.1 SHADE 100.0 0.1

48.05% 33.77% 64.94% 20.78% 44.16% 35.06% 37.66%

20.1 ± 49.8 22.8 ± 53.7 10.9 ± 14.1 1.0 ± 0.0 23.0 ± 43.0 1.7 ± 3.4 25.7 ± 70.1

0.0680 ± 0.0057 −390.13 ± 22.44 0.0513 ± 0.0018 −290.15 ± 1.61 0.0720 ± 0.0217 −460.48 ± 121.11 0.0498 ± 0.0030 −58.47 ± 114.74 0.0548 ± 0.0113 −334.58 ± 47.72 0.0540 ± 0.0055 −344.92 ± 35.42 0.0444 ± 0.0072 −255.27 ± 25.61

DE 100.0 0.2 GEN 100.0 0.2 GWO 100.0 0.2 INFO 100.0 0.2 JADE 100.0 0.2 SADE 100.0 0.2 SHADE 100.0 0.2

74.03% 63.64% 79.22% 51.95% 72.73% 68.83% 71.43%

6.9 ± 35.2 12.7 ± 47.6 9.7 ± 20.0 1.0 ± 0.0 8.2 ± 23.9 3.3 ± 8.4 10.9 ± 38.0

0.1273 ± 0.0111 −752.37 ± 54.10 0.1006 ± 0.0048 −578.90 ± 3.73 0.1131 ± 0.0410 −750.02 ± 282.10 0.0958 ± 0.0087 −285.93 ± 276.88 0.0993 ± 0.0168 −608.23 ± 101.65 0.1047 ± 0.0129 −641.97 ± 84.39 0.0843 ± 0.0139 −492.17 ± 68.39

−39.40 ± 1.01 −28.85 ± 0.26 −54.46 ± 3.31 −0.60 ± 4.76 −35.38 ± 1.54 −36.19 ± 1.63 −25.71 ± 1.45

Across optimizers, INFO behaves like a greedy local search and plateaus at the lowest success rates, whereas DE, GEN, JADE, and SHADE spend more queries yet discover markedly stronger adversarial directions. The smallest perturbation strength is achieved by GEN and SHADE, while DE and GWO drift toward the corners of the admissible area, probably because they have no selfadaptation of parameters like SADE, JADE, or SHADE. For ImageNet, the dependence of the results quality on ε mirrors that of CIFAR-10. Yet, low ε values make the attack much more difficult than for CIFAR-10, and no optimizer succeeds at ε = 0.01. On the other hand, at ε = 0.1, most attacks are successful, and several optimizers obtain even complete success at ε = 0.2. The smallest disturbance strength was obtained by GEN, INFO, and

Testing global optimization methods with adversarial attacks.

13

SHADE. Assuming α = 0.1 helped nearly all optimizers to obtain good results, provided that the budget was large enough. The most striking beneficiary is again GWO, which, without regularization, stalls near a 10-15% success rate, but then it catches up to the other population-based methods. Figure 4 presents examples of successful attacks for an example image. Table 2: Results of adversarial attacks on the ImageNet-based classifier Optim. α

ε Suc. R. (↑) First Succ. (↓)

L2

Obj. fn. (↑)

DE 0.0 0.01 GEN 0.0 0.01 GWO 0.0 0.01 INFO 0.0 0.01 JADE 0.0 0.01 SADE 0.0 0.01 SHADE 0.0 0.01

0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00%

0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0

0.0000 ± 0.0000 0.34 ± 0.49 0.0000 ± 0.0000 0.21 ± 0.32 0.0000 ± 0.0000 0.25 ± 0.36 0.0000 ± 0.0000 0.25 ± 0.36 0.0000 ± 0.0000 0.32 ± 0.46 0.0000 ± 0.0000 0.32 ± 0.47 0.0000 ± 0.0000 0.32 ± 0.47

DE 0.0 0.1 GEN 0.0 0.1 GWO 0.0 0.1 INFO 0.0 0.1 JADE 0.0 0.1 SADE 0.0 0.1 SHADE 0.0 0.1

60.00% 55.00% 10.00% 50.00% 55.00% 55.00% 55.00%

1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0

0.0686 ± 0.0041 2.26 ± 2.09 0.0552 ± 0.0025 1.58 ± 1.76 0.0331 ± 0.0000 0.46 ± 0.82 0.0567 ± 0.0072 1.49 ± 1.75 0.0610 ± 0.0058 1.82 ± 1.80 0.0596 ± 0.0035 1.85 ± 1.81 0.0585 ± 0.0072 1.82 ± 1.84

DE 0.0 0.2 GEN 0.0 0.2 GWO 0.0 0.2 INFO 0.0 0.2 JADE 0.0 0.2 SADE 0.0 0.2 SHADE 0.0 0.2

100.00% 85.00% 15.00% 85.00% 90.00% 90.00% 90.00%

1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0 1.1 ± 0.5 1.0 ± 0.0 1.0 ± 0.0 1.0 ± 0.0

0.1325 ± 0.0095 5.61 ± 2.97 0.1063 ± 0.0064 4.18 ± 2.85 0.0648 ± 0.0003 1.15 ± 1.61 0.1036 ± 0.0055 4.11 ± 2.77 0.1142 ± 0.0118 4.70 ± 2.84 0.1170 ± 0.0086 4.74 ± 2.84 0.1167 ± 0.0105 4.59 ± 2.84

DE 0.1 0.01 GEN 0.1 0.01 GWO 0.1 0.01 INFO 0.1 0.01 JADE 0.1 0.01 SADE 0.1 0.01 SHADE 0.1 0.01

0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00%

0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0 0.0 ± 0.0

0.0000 ± 0.0000 0.06 ± 0.49 0.0000 ± 0.0000 0.01 ± 0.30 0.0000 ± 0.0000 −0.04 ± 0.46 0.0000 ± 0.0000 0.25 ± 0.36 0.0000 ± 0.0000 0.08 ± 0.47 0.0000 ± 0.0000 0.07 ± 0.47 0.0000 ± 0.0000 0.12 ± 0.44

DE 0.1 0.1 GEN 0.1 0.1 GWO 0.1 0.1 INFO 0.1 0.1 JADE 0.1 0.1 SADE 0.1 0.1 SHADE 0.1 0.1

80.00% 90.00% 90.00% 50.00% 80.00% 80.00% 80.00%

1.7 ± 1.7 4.5 ± 8.4 5.5 ± 3.1 1.0 ± 0.0 7.0 ± 11.5 16.5 ± 50.9 14.2 ± 27.3

0.0725 ± 0.0041 0.43 ± 2.34 0.0555 ± 0.0024 2.02 ± 2.25 0.0832 ± 0.0180 0.46 ± 2.32 0.0537 ± 0.0022 0.37 ± 1.06 0.0637 ± 0.0078 0.63 ± 2.02 0.0620 ± 0.0055 0.13 ± 2.10 0.0528 ± 0.0051 0.79 ± 1.79

0.1 0.2

100.00%

1.0 ± 0.0

0.1396 ± 0.0088 1.32 ± 2.55

DE

continued on next page. . .

14

W. Zarzecki and J. Arabas continued on next page. . . Optim. α

ε Suc. R. (↑) First Succ. (↓)

GEN 0.1 0.2 GWO 0.1 0.2 INFO 0.1 0.2 JADE 0.1 0.2 SADE 0.1 0.2 SHADE 0.1 0.2

100.00% 100.00% 85.00% 100.00% 100.00% 95.00%

(a) original class: sombrero

1.4 ± 1.2 3.6 ± 1.4 1.0 ± 0.0 2.9 ± 5.7 1.9 ± 3.7 1.1 ± 0.4

L2

Obj. fn. (↑)

0.1069 ± 0.0061 3.90 ± 2.13 0.1474 ± 0.0457 1.47 ± 1.95 0.1022 ± 0.0079 0.48 ± 2.17 0.1229 ± 0.0181 1.54 ± 2.16 0.1225 ± 0.0135 1.02 ± 2.87 0.1039 ± 0.0177 1.69 ± 2.24

(b) ε = 0.1: jigsaw puzzle

(c) ε = 0.2: jigsaw puzzle

Fig. 4: Example image from ImageNet with two successful attack examples for different ε values

At matched (ε, α), the attack is uniformly easier on ImageNet than on CIFAR-10, which we attribute to the much higher input dimensionality and the much larger label set that statistically shrinks the margin to the nearest competing class. The relative ranking of methods remains stable across datasets: GEN and SHADE dominate by achieving high success rates while keeping low disturbance strength, GWO performs worst whenever the objective lacks a regularization signal, and INFO stays the queryefficient but low-ceiling option.

5

Discussion and Future Work

We demonstrated that the BBAA is a demanding global optimization task — it is multimodal, and the search space has very many dimensions. We provided results of several ready-for-use optimizers, without any parameter tuning. The next step would be to broaden the portfolio of optimization methods and to tune them to achieve better efficiency. In future work, we plan to extend the evaluation formula to attacks targeted at falsely assigning a specific class. We also plan to add more classifiers and datasets, and to reformulate the objective function (1) by replacing or complementing the L2 regularization term with a perceptual loss [47] that better reflects human-visible distortion.

Testing global optimization methods with adversarial attacks.

15

Acknowledgment We gratefully acknowledge Polish high-performance computing infrastructure PLGrid (HPC Center: ACK Cyfronet AGH) for providing computer facilities and support within computational grant no. PLG/2025/018167.

References 1. Adi, Y., Baum, C., Cisse, M., Pinkas, B., Keshet, J.: Turning your weakness into a strength: Watermarking deep neural networks by backdooring. In: 27th USENIX security symposium (USENIX Security 18). pp. 1615–1631 (2018) 2. Alzantot, M., Sharma, Y., Chakraborty, S., Zhang, H., Hsieh, C.J., Srivastava, M.B.: Genattack: Practical black-box attacks with gradient-free optimization. In: Proceedings of the genetic and evolutionary computation conference. pp. 1111– 1119 (2019) 3. Andriushchenko, M., Croce, F., Flammarion, N., Hein, M.: Square attack: a queryefficient black-box adversarial attack via random search. In: European conference on computer vision. pp. 484–501. Springer (2020) 4. Brendel, W., Rauber, J., Bethge, M.: Decision-based adversarial attacks: Reliable attacks against black-box machine learning models (2017) 5. Carlini, N., Wagner, D.A.: Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP) pp. 39–57 (2016), https: //api.semanticscholar.org/CorpusID:2893830 6. Chen, J., Jordan, M.I., Wainwright, M.J.: Hopskipjumpattack: A query-efficient decision-based attack. In: 2020 ieee symposium on security and privacy (sp). pp. 1277–1294. IEEE (2020) 7. Chen, P.Y., Zhang, H., Sharma, Y., Yi, J., Hsieh, C.J.: Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models (2017), https://api.semanticscholar.org/CorpusID:2179389 8. Chen, Q., Liu, B., Zhang, Q., Liang, J., Suganthan, P., Qu, B.: Problem definitions and evaluation criteria for CEC’2015 special session on bound constrained singleobjective computationally expensive numerical optimization (2014) 9. Choi, J.S., Lee, K., Jeong, J., Xie, S., Shin, J., Lee, K.: Diffusionguard: A robust defense against malicious diffusion-based image editing (2024) 10. Clare, L., Marques, A., Correia, J.: A comparative analysis of evolutionary adversarial one-pixel attacks. In: International Conference on the Applications of Evolutionary Computation (Part of EvoStar). pp. 147–162. Springer (2024) 11. Costa, J.C., Roxo, T., Proença, H., Inacio, P.R.M.: How deep learning sees the world: A survey on adversarial attacks & defenses. IEEE Access 12, 61113–61136 (2024) 12. Das, S., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2011 competition on testing evolutionary algorithms on real world problems. Tech. rep. (2010) 13. Dixon, L.C.W., Szegő, G.P. (eds.): Towards Global Optimisation 2. North-Holland Publishing Company, Amsterdam (1978) 14. Elhara, O., Varelas, K., Nguyen, D., Tusar, T., Brockhoff, D., Hansen, N., Auger, A.: COCO: the large scale black-box optimization benchmarking (bbob-largescale) test suite. arXiv preprint arXiv:1903.06396 (2019) 15. Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., Song, D.: Robust physical-world attacks on deep learning visual classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1625–1634 (2018)

16

W. Zarzecki and J. Arabas

16. Finlayson, S.G., Chung, H.W., Kohane, I.S., Beam, A.L.: Adversarial attacks against medical deep learning systems. arXiv preprint arXiv:1804.05296 (2018) 17. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014) 18. Guo, C., Gardner, J.R., You, Y., Wilson, A.G., Weinberger, K.Q.: Simple black-box adversarial attacks. vol. abs/1905.07121 (2019), https://api.semanticscholar. org/CorpusID:86541092 19. Hansen, N., Finck, S., Ros, R., Auger, A.: Real-Parameter Black-Box Optimization Benchmarking 2009: Noisy Functions Definitions. Research Report RR-6869, INRIA (2009), https://inria.hal.science/inria-00369466 20. Hansen, N., Ros, R., Auger, A.: Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions. Tech. rep. (2009), https://api. semanticscholar.org/CorpusID:270969128 21. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 770–778 (2015), https://api.semanticscholar.org/CorpusID:206594692 22. Ilie, A., Popescu, M., Stefanescu, A.: Evoba: An evolution strategy as a strong baseline for black-box adversarial attacks. In: International Conference on Neural Information Processing. pp. 188–200. Springer (2021) 23. Ilyas, A., Engstrom, L., Athalye, A., Lin, J.: Black-box adversarial attacks with limited queries and information. In: International Conference on Machine Learning (2018), https://api.semanticscholar.org/CorpusID:5046541 24. Ilyas, A., Engstrom, L., Madry, A.: Prior convictions: Black-box adversarial attacks with bandits and priors (2018) 25. Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009) 26. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2014 special session and competition on single objective realparameter optimization. Tech. rep., Zhengzhou University and Nanyang Technological University (2013) 27. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2019 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2018) 28. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2020 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2020) 29. Liang, J.J., Qu, B.Y., Suganthan, P.N.: Problem definitions and evaluation criteria for the CEC’2022 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2022) 30. Liang, J.J., Qu, B.Y., Suganthan, P.N., Chen, Q.: Problem definitions and evaluation criteria for the CEC’2017 competition on single objective real-parameter optimization. Tech. rep., Nanyang Technological University (2017) 31. Liang, J.J., Suganthan, P.N., Deb, K.: Problem definitions and evaluation criteria for the CEC’2005 special session on real-parameter optimization. Tech. rep., Nanyang Technological University (2005) 32. Liu, Y., Chen, X., Liu, C., Song, D.: Delving into transferable adversarial examples and black-box attacks. arXiv preprint arXiv:1611.02770 (2016) 33. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)

Testing global optimization methods with adversarial attacks.

17

34. Mallipeddi, R., Suganthan, P.N., Pan, Q., Tasgetiren, M.F.: Problem definitions and evaluation criteria for the CEC’2013 special session on real-parameter optimization. Tech. rep., Nanyang Technological University (2013) 35. Mosli, R., Wright, M., Yuan, B., Pan, Y.: They might not be giants: Crafting blackbox adversarial examples with fewer queries using particle swarm optimization (2019) 36. Omidvar, M.N., Li, X., Tang, K., Mei, Y., Yao, X.: Benchmark functions for the CEC’2012 special session and competition on large scale global optimization. Tech. rep., Nanyang Technological University (2012) 37. Omidvar, M.N., Li, X., Tang, K., Yao, X.: Benchmark functions for the CEC’2018 competition on large scale global optimization. Tech. rep., Nanyang Technological University (2018) 38. Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia conference on computer and communications security. pp. 506–519 (2017) 39. Ruhr University Bochum, Institute for Neural Computation: Black-box optimization competition (bbcomp). https://www.ini.rub.de/PEOPLE/glasmtbl/ projects/bbcomp/index.html 40. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. International journal of computer vision 115(3), 211–252 (2015) 41. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014), https://api.semanticscholar.org/CorpusID:14124313 42. Su, J., Vargas, D.V., Sakurai, K.: One pixel attack for fooling deep neural networks. IEEE Transactions on Evolutionary Computation 23(5), 828–841 (2019) 43. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing properties of neural networks (2013) 44. Tu, C.C., Ting, P.S., Chen, P.Y., Liu, S., Zhang, H., Yi, J., Hsieh, C.J., Cheng, S.M.: Autozoom: Autoencoder-based zeroth order optimization method for attacking black-box neural networks. In: AAAI Conference on Artificial Intelligence (2018), https://api.semanticscholar.org/CorpusID:44079102 45. Turner, R., Eriksson, D., McCourt, M., Kiili, J., Laaksonen, E., Xu, Z., Guyon, I.: Bayesian optimization is superior to random search for machine learning hyperparameter tuning: Analysis of the black-box optimization challenge 2020. In: NeurIPS 2020 competition and demonstration track. pp. 3–26. PMLR (2021) 46. Van Thieu, N., Mirjalili, S.: Mealpy: An open-source library for latest metaheuristic algorithms in python. Journal of Systems Architecture (2023). https: //doi.org/10.1016/j.sysarc.2023.102871 47. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 586–595 (2018), https://api. semanticscholar.org/CorpusID:4766599 48. Zheng, M., Yan, X., Zhu, Z., Chen, H., Wu, B.: Blackboxbench: A comprehensive benchmark of black-box adversarial attacks (2025), https://arxiv.org/abs/ 2312.16979

Related documents

• The Many Faces of Adversarial Machine Learning#199591
• Budget-Aware Adaptive Adversarial Patches for Black-Box Object Detection#287079
• Adversarial Deepfake Generation and an Investigation of Purification-Based Adversarial Detection#410964
• VeriScale: Adversarial Test-Suite Scaling for Verifiable Code Generation#216886
• Large Language Models for Code: Security Hardening and Adversarial Testing#296434