arXiv:2606.27215v1 [cs.AI] 25 Jun 2026
Vulnerability of Natural Language Classifiers to Evolutionary Generated Adversarial Text Manjinder Singh Computing Science & Mathematics University of Stirling, Stirling, UK Alexander E. I. Brownlee Computing Science & Mathematics University of Stirling, Stirling, UK [email protected] Mohamed Elawady Computer and Information Sciences University of Strathclyde, Glasgow, UK
Abstract Deep learning models have achieved impressive performance across various fields but remain vulnerable to adversarial inputs, particularly in NLP, where such attacks can have significant real-world consequences. Adversarial attacks often involve small, semantically similar token replacements to fool NLP models, and recent methods have become more precise by targeting specific vulnerable words, often by exploiting some level of access to the model’s internal structure. This paper proposes GAversary, a hybrid Genetic Algorithm (GA) to generate adversarial attacks on natural language models. The GA is able to treat the target model as a black box, requiring only the logit value output by the model to guide the search. GAversary differs from GAs previously proposed for this problem, by using GloVe embeddings to propose word replacements (the mutation operator) to improve the semantic similarity of the adversarial examples. GAversary is applied to several benchmark data sets and well-known target models. GAversary is able to substantially reduce the target model’s accuracy on test data compared to the BAE and A2T attacks compared against (in the best case, reducing a 76.8% accuracy to 5.8%, compared to BAE’s 27.6%). The trade-off is that GAversary perturbs just under twice as many words as the other two methods, with a
1
slightly lower semantic similarity to the original text, with around a 5% increase in run-time.
Keywords: Adversarial examples, Adversarial machine learning, Genetic algorithm, Machine learning, Natural language processing
1
Introduction
Deep learning models such as deep neural networks (DNNs) have been applied to a vast range of fields including speech recognition, natural language processing (NLP) and computer vision. In many areas they have performed as well if not better than human experts [1]. However, they are vulnerable to adversarial inputs; perturbations that may be imperceptible to humans but can cause DNNs to misclassify the output [2]. Automated text analysis through NLP models is increasingly used to auto-generate summaries which drive, for example, news recommender websites, with direct implications for reputation, reliability, and branding [3]. For example, NLP models can be used to inform investors on buy/sell decisions based on the prevailing sentiment within these reviews [4, 5]. Thus, adversarial attacks on NLP models could have major impact, such as triggering poor investment decisions and the corresponding disruption of local economies. The overall goal for an adversarial system is to generate an attack, in the form of changes to an original text such that the original text is misclassified, having the least number of modifications to the original text, and maintaining semantic similarity after those modifications. A variety of approaches have been demonstrated in the literature. Generally these rely on replacing tokens in the examples [6]. Typically such approaches rely on generating a few synonym token replacements (usually using some form of semantic similarity), and choosing the one replacement that has the biggest influence on the classification probability for the targeted model [7, 8]. Recently these approaches have become more targeted, identifying the most vulnerable words, and using context to select suitable replacements [9, 10]. However, the search process each of these methods follows generally amounts to a variation of local or some other single-point search. Genetic Algorithms (GAs) perform well on combinatorial optimization problems where there is a large search space [11, 12]. Even with only the combination of synonyms (ignoring sentence structure), Natural Language has such a vast space to explore, yet there are few examples of the application of GAs to generating adversarial examples. 2
The advantage of GAs is that they can treat the target problem as a black box, which makes them particularly suitable for an attack scenario in which the inner workings of the targeted model are unavailable. As GAs follow a search process that simply seeks inferences from the target model and can be terminated at any arbitrary point, they also have the potential to generate attacks more quickly than GANs. Thus, this paper presents GAversary, which uses a Genetic Algorithm to target several well-known pre-trained models. The other recent attempts to use GAs to target NLP models rely on operators that work at random and do not exploit the targeting and context awareness of Bert-Attack [9, 10] and others. More recently [13], GloVe has been used to guide the search as GAversary does, but they limit the search to a filtered set of ‘influential’ words, whereas GAversary allows the search to explore a wider range of possible replacements. Another approach [14] also used GloVe, and targeted a wider range of words for replacement, but used GloVe to identify semantically similar words as replacements, rather than GAversary which uses the surrounding context to identify replacements. The paper’s major contribution then is to draw on the GloVe embedding model to guide the genetic operators towards contextually-plausible equivalent word replacements. In this setting, the GA still treats the target model as a black box; the use of GloVe is independent of whatever model is being targeted. This paper’s research questions are as follows: 1. Can a genetic algorithm guided by GloVe embeddings generate adversarial examples for a pre-existing NLP model such that the model’s prediction accuracy is significantly reduced for those examples? 2. What is the performance of the proposed algorithm compared to existing methods, in both reducing accuracy of the target model, and computational efficiency? 3. How different do the examples need to be from the original text for the model to misclassify them for GAversary? Thorough experimentation compares GAversary to many models, alternative techniques, and data sets, to answer these questions. GAversary seeks the best of previous work: like Bert-Attack it is fast as it only needs to run target model, and uses GloVe for semantic similarity. However, while GAversary also uses nothing but the logit value from the target model (the same as Bert-Attack), that approach ranks words by sensitivity, then iterates over the ranking making substitutions exhaustively. In contrast, 3
GAversary picks words at random to avoid local optima, and guides the overall search using a genetic algorithm. Previous GA-based approaches either do not use the guidance from the target model in the way GAversary does, or do not include the context of words being replaced. Thus our key novelty is proposing a hybrid approach getting the best of both BertAttack’s guided mutations to the target text, and the Genetic Algorithm’s high level efficient search. The main contributions of this work, then, are: (1) GAversary, a hybrid approach to generating adversarial text using a genetic algorithm search guided by GloVe for semantic similarity, and (2) experimental results showing that the approach reduces accuracy more than other state-of-the-art techniques. The paper continues in Section 2 by reviewing related work in adversarial attacks, with a focus on natural language. Section 3 introduces GAversary, including the major components of its genetic algorithm framework. Section 4 describes the computational experiments and discusses the results, then Section 5 concludes the paper and discusses future work.
2
Related Work
Adversarial attacks were first applied in the field of computer vision to image classifier models, with the term ‘adversarial example’ first introduced in [15]. It was noticed that the researchers could cause the DNN to misclassify images by introducing small perturbations to the images. Application of ‘noise’ to the images led to misclassification by the DNN even when there was no perceptible difference to human observers. As neural networks have increasingly become more popular in a diverse range of fields within the AI space and specifically within the NLP field, the need to ensure the robustness of the models and the predictions they generate has become more urgent. The majority of approaches proposed for attacking NLP models focus on replacing tokens in the examples [6]. These attacks can be applied into different levels: sentence, word, character or a mixture. The present work focuses on word-level attack recipes. TextBugger [7] is the earliest work which used different levels of sentence modification (i.e. substitution of characters, replacement of words, and insertion of typo letters) to produce adversarial examples. However, Alzantot et al. [14] developed a genetic algorithm focusing on words replacement in which the mutation operator uses Euclidean distance in the GloVe embedding space to identify semantically similar words as candidate substitution words in the original text. To ensure the correctness of the text semantics,
4
a filter exploiting the Google 1 billion words language model [16] is used to exclude words that do not fit in the surrounding context. A fast variant of Alzantot’s method [17] used different NLP methods ensuring robust word substitutions through Interval Bound Propagation (IBP) [18]. TextFooler [19] checked important words in both semantic and grammatical way before considering these words as possible substitutes until the model’s prediction changes, without understanding how the used text generation method works. In the applications of BERT [20] (pre-trained masked language model), a method called BERT-Attack [9] was introduced to identify less important words in input text in order to replace them with acceptable alternatives generated by BERT, satisfying the semantic and grammatical constraints. Another method BAE [21] was developed based on BERT’s masked language model (BERT-MLM) to perform text replacements and insertions while considering grammatical and contextual aspects. CLARE [10] used RoBERTa [22] to introduce three perturbation techniques (Replace, Insert, and Merge) using a mask-then-infill procedure, generating grammatically adversarial examples. A2T [23] used DistilBERT [24] to build an attack algorithm determining most substitutional candidates based on gradient-based word importance ranking and generating semantically correct adversarial replacements based on counter-fitted word embeddings. Inspired by input data perturbations in the computer vision domain [25] to strength-up the loss functions for better image classification, GBDA [26] and TPGD [27] adapted these perturbations into the NLP domain by investigating the latent space and optimising the gradients of the components of the model used. In recent years, different methods (i.e. SemAttack [28], TextHoaxer [29], TextHacker [30], ATGSL [31]) were developed based on the previous work to find most suitable words inside the text for replacement and to substitute these words with candidates while persevering the text semantics and avoiding the grammar errors. In one of of the closest proposals to GAversary, [13] proposed a genetic algorithm based approach building on the work of [32]. In [32], clustering among the embedding vectors was used to identify synonyms for each word. The GA selected replacements from these clusters as its mutation operator; the crossover was the same as that of GAversary in that it simply took a random combination of the replacements from both parents. [13] built on this idea to exploit GloVe, as GAversary does, to guide the search for replacement words. GloVe embeddings were used to identify the 10 closest synonyms to each word, capped to a maximum Euclidean distance of 0.5 from the original word. From that list, replacement words were selected 5
in descending order of frequency in the dictionary. However, rather than choosing the words to be replaced as part of the search, they adopted what they call word influence to identify all words to be replaced. This is, essentially, the sensitivity of the target to each word in the input; determined by removing each word and observing the affect on the model output. A part-of-speech constraint was applied to maintain consistency. Their fitness function included cosine similarity to the original text, and their initial GA population was made by already replacing the most influential words with synonyms. The key differences to GAversary are that GAversary allows changes to everything except stopwords, they always change all of the influential words whereas GAversary might ignore some, and GAversary places no constraints on part-of-speech/semantic similarity.
3
Methodology
The present work seeks to investigate the creation of adversarial examples against an NLP model. This research area, compared to similar work with image models, is relatively new and so this project will aim to combine work already done in several similar areas [33, 34, 8, 35, 36]. Overall there are two aims when generating the adversarial examples: (1) the examples should be incorrectly classified; and (2) the examples should be as similar to the original text as possible. The proposal is to use a GA to evolve a population of adversarial examples. GAs are a search-based framework for optimisation and have several components that can be constrained to guide the search to meet the aims noted above. GAs find a solution(s) based on the concept of fitness, i.e., a relative measure of quality comparing one solution to another. GAs maintain a population of solutions for the problem at hand, and in each iteration select fitter solutions, which are recombined (a.k.a. crossover) and mutated to generate a new population of solutions. This new population replaces the original one and the process repeats until some termination criterion is met: usually a solution of sufficient quality is found or a maximum number of solutions have been evaluated. The components listed above are brought together into an algorithm as illustrated in the workflow in Figure 1. This figure is annotated with the sub-steps for the two stages that interact with the model, described shortly. The remainder of this section will describe the implementation of the major components of the GA for generating adversarial examples. The overall framework is referred to as GAversary. The aim is to maintain the grammatical and syntactical structure of the
6
Apply candidate text to target model
Initialization
Fitness =model logit
Fitness assignment No
Untested candidate texts in population?
Selection
Yes Crossover Choose a word to replace at random
Mutation
Call GloVe for closest 4 replacements Apply each mutated sentence to target model; keep that with largest logit change
Yes
No New population complete?
Stop
Figure 1: GA Workflow input text, so attempting to not only replace words with their synonyms, but to minimise the number of words replaced. The concept of fitness and the implementation of each operator is designed to reflect this strategy. A black-box approach is taken that assumes access to only the input and output of the model. GAversary was implemented within the TextAttack framework, and the code artefact is publicly available [37].
3.1
Representation
An adversarial example is simply a variation of a piece of original text. The GA’s job is to find such examples: so each “solution” in the population is one possible variation of the text. As such each example could be represented as the full text but this is obviously rather wasteful of memory and, as crossover and mutation operations are applied to generate new solutions, could be computationally costly as well. Consequently a compact represen7
2, apples
4, highly
2, apples
4, most
5, delicious
1, red 4, highly
5, delicious
Figure 2: Four solutions using the compact representation. Each solution is a different colour for clarity. Solutions are a list of replacements for words in the original text: in this example the text was “green grapes are very tasty” so, for example, the top solution represents the adversarial text “green apples are highly tasty”. tation for solutions is adopted that only stores the changes to be made to the original text to make the adversarial example. Each solution is a list of (location, replacement) pairs specifying which word in the original text shall be replaced and what word to replace it with. Solutions may be any length from zero (unchanged) up to the number of words in the original text (completely replacing the phrase). Some examples are given in Figure 2. This representation is less computationally intensive, and easier to manipulate, than storing the full mutated variants of the text.
3.2
Initialization
The GA begins by generating an initial population P0 of size S. To generate each member of P0 , the algorithm starts with an empty solution (i.e., no change to the original text) and applies the mutation operator described in Section 3.5 five times. Consequently, each solution in P0 represents up to five word changes to the original text. Following some preliminary tuning experiments, the experiments had S = 60. The solutions in P0 will then be evaluated by applying the word replacements to the original text, and passing the text to the NLP model for classification. The fitness for each solution is then determined as described in Section 3.3.
3.3
Fitness and Selection
The fitness measure targets the first aim: incorrect classification, and is follows the default TextAttack Untargeted Classification implementation. There are only two possible classifications for the text: negative, and positive. The NLP model assigns a score to each class, with the class having the higher score being the prediction. The proximity to misclassification 8
is simply the difference between these two scores, if the true class still has the higher score. The algorithm seeks to minimise this difference until the correct class no longer has the higher score. A fitness proportionate selection [38] operation uses this concept of fitness to choose parents for the remaining GA operators. Each solution is allocated a probability of selection, by taking its fitness divided by the total fitness of all solutions in the population. Solutions are then selected according to this probability so that they are chosen roughly in proportion with their fitness. TextAttack’s implementation of fitness proportionate selection is used, which uses a softmax function to normalise the probability when sampling parents.
3.4
Recombination / Crossover
Two parent solutions chosen by selection are recombined into a single new offspring following a simple uniform crossover [38]. Each replacement from the two parents has a 50% probability of being copied into the offspring; where the two parents both have a replacement for one location in the text, one is chosen, with an equal probability of the either parent’s replacement being copied. Recombination is repeated until S solutions are created (i.e., enough to replace the present population).
3.5
Mutation
Each solution within the GA’s population is a set of individual word replacements. The mutation operator is designed to allow for exploration of new parts of the search space, while achieving the second overall aim by guiding the search towards examples semantically similar to the original text. The operator simply adds a word replacement to the set of replacements in the solution. A target word w for replacement is selected uniformly at random from the original text, excluding the default NLTK stop-words (e.g., the, and, to, a ...). If w is already marked for replacement in the solution, the existing replacement is updated. To choose a replacement for w, a set of “most likely” replacements for w is identified. Similar to [13, 14] GAversary uses the pre-trained counterfitted Global Vectors for Word Representation (GloVe) embedding [34] that is included with the TextAttack library1 to translate the words into numeric vectors. GloVe [39] tries to capture the sense that certain words are more likely to be seen in context of others. For example, “lettuce” is likely to 1
Specifically the WordEmbedding.counterfitted GLOVE embedding() function
9
The
quick
brown
fox
jumped
over
the
lazy
dog
The
quick
brown
fox
?
over
the
lazy
dog
Figure 3: Masking is used in the mutation operator, so the GloVe can identify the best fitting replacement words for a given context. In this case, the GloVe embedding is estimated for the location ‘?’ and the words with the closest GloVe embeddings in Euclidean space to that target are selected as possible replacements for ‘jumped’. be seen in the context of “salad” but not with “train”. Context is implied through measuring the spatial distance between these word vectors, whereby the shorter the distance between word vectors the stronger the relationship between the actual words. In the approach of [13, 14], w was converted to a vector using GloVe, then the closest words to w in the Euclidean space served as possible replacements. GAversary applies a mask. The four words (or fewer if near an end of the text) before and after w are provided as the local context window for GloVe (Figure 3); w is masked and the closest max neighbours to the GloVe embedding for the space left by w are determined. The final choice of word to replace w is the one that takes the model prediction furthest toward the target label, given all the other replacements already applied. The idea is that this will lead to more plausible word replacements that fit better with the surrounding context.
3.6
Stopping Criteria
There are two possible exit conditions for the algorithm. The first is the discovery of a population member that, after querying the sentiment model, is equal to the label and so is an adversarial example. The second condition is that the algorithm has run for a set number of generations. The maximum number of generations was determined after experimentation. Initially, the number of generations was set at 100. After multiple trials it was noted that after 20 generations there was no improvement in the adversarial examples generated.
10
4
Experiments and Results
4.1
Evaluation Metrics
Several measures are used to compare the approaches. When the model is applied to the test set, the standard measures TP, TN, FP, FN are used for true/false positive/negative classifications, with a bar (e.g., T P ) for these measures when applied to the attacked test set. The measures are defined as: Accuracy Standard classification accuracy; % of correct predictions on the test set after attack has been applied; Acc =
100(T P + T N ) TP + TN + FP + TN
(1)
Original Accuracy Standard classification accuracy; % of correct predictions on the original unaltered test set; Accorg =
100(T P + T N ) TP + TN + FP + TN
(2)
% Perturbed Words Mean (over test set) % of words changed from original in each adversarial example; if the test set S contains |S| documents, each document si ∈ S in the original test set contains |si | words, and H(si , si ) is the Hamming distance between the original document si and the corresponding adversarial example si , then this measure is: P |S| 100H(si ,si ) i=1 si
PW =
|S|
(3)
Semantic Similarity Mean (over test set) USE similarity between original text and adversarial example: SS =
|S| X
U SE(si , si )
(4)
i=1
Query Number QN The number of queries of target model to generate adversarial example set Computation Time CT Wall-clock time to generate adversarial example set 11
Parameter Max generations Population size Perturbations Temperature for softmax used selection Crossover Rate
Value 30 20 5 0.3 100%
Table 1: Major configuration settings for GAversary
4.2
Experimental Settings
Using TextAttack framework [40], the proposed algorithm is compared against the state-of-art methods (BAE [21], A2T [23]), which are the best candidates to compare against in terms of black-box text replacement. The following hardware specifications are used: GPU (GeForce RTX 2080), CPU (Intel(R) Core(TM) i9-9900K), and RAM (16GB). To validate the performance of the proposed work, two popular datasets were used: (1) Movie Reviews (MR) [41]: binary classification on movie reviews (positive, negative) with data split of 9K train & 1K test samples and its average sentence length is 20 words, (2) AG-News [42]: multi-class classification on news articles (business, science, sports, world) with data split of 120K train and 7.6K test samples and its average sentence length is 43 words. Besides that, three common text classification models were used: WordCNN [43], WordLSTM [44], BERT [45]. These pretrained models are all available in the TextAttack framework. The major configuration parameters of the genetic algorithm are detailed in Table 1; the full implementation of GAversary is available in the paper’s accompanying artefact [37].
4.3
Quantitative results
Tables 2 and 3 report the performance of the three models on the original test data and the test data with each attack applied. All the approaches reduce classification accuracy by a substantial amount on both data sets and with all three models. BAE performs slightly better than A2T, and GAversary reduces accuracy further still. However, this reduction in accuracy comes at a cost. GAversary typically perturbs just under twice as many words as the other two methods. Semantic similarity of the text compared to the original is also slightly lower with GAversary than with the other methods, and GAversary requires more queries of the model (just under 2x that of BAE and 6–8x that of
12
Models
Metric
WordCNN
Acc PW SS QN
WordLSTM
Acc PW SS QN
BERT
Acc PW SS QN
Adv Attack
Accorg BAE
A2T
GAversary
76.83
27.58 12.85 0.85 59.25
44.28 13.68 0.86 15.56
5.82 21.37 0.83 91.72
77.86
24.95 11.75 0.86 57.23
38.56 11.37 0.88 14.45
7.04 20.42 0.84 88.83
84.24
36.49 13.73 0.85 63.07
59.76 12.03 0.87 17.73
19.51 20.74 0.84 132.63
Table 2: Performance evaluation - MR A2T). Thus GAversary adds to the trade-off: for slightly more change to the text, the explorative power of the GA is able to greatly reduce accuracy. The additional model queries do not feed through to greatly increased run times for GAversary. Table 4 shows the wall clock run times in seconds for the three approaches. GAversary is the fastest of the three approaches on both the WordCNN and WordLSTM models, and approximately 5% longer running then BAE on BERT. In practical terms, GAversary is preferable where effectiveness is critical, when trying to maximally expose model vulnerability and where text fidelity is less important. This could include security evaluation contexts for high-stakes domains (e.g., finance, law, medicine) where maximum robustness testing may be preferred even if perturbations are unrealistic. In such offline evaluation settings the additional number of queries does not matter as much; but this could in future be tackled by adding early stopping or response caching.
4.4
Qualitative results
Illustrative examples of the output generated by each method are given in Tables 5 to 9. The extent of changes made by GAversary is often more than that of the other methods, but typically more subtle, with fewer changes to 13
Models
Metric
WordCNN
Acc PW SS QN
WordLSTM
Acc PW SS QN
BERT
Acc PW SS QN
Adv Attack
Accorg BAE
A2T
GAversary
91.57
73.93 6.66 0.92 113.6
82.22 8.52 0.91 26.96
56.89 13.10 0.88 203.79
91.63
73.53 6.69 0.92 115.96
77.49 7.59 0.91 24.6
62.08 12.74 0.88 204.91
95.14
81.55 7.28 0.93 122.63
82.86 7.70 0.91 27.23
73.17 12.26 0.88 218.45
Table 3: Performance evaluation - AG-News
Models
Adv Attack BAE A2T GAversary
WordCNN
WordLSTM
BERT
666 117 66
492 116 98
1484 400 1547
Table 4: Computation time (in seconds) over test sub-set of MR
14
the literal meaning of the original. With the example for the BERT model on the MR data (Table 5), GAversary used two additional replacements than BAE; but BAE changed “portray” to the less natural “appear” rather than “describes”; BAE also replaced “powerful” with “short”, rather than the much closer “emphatic”. Similarly GAversary did not replace “filmmaking” with “cinematographic” as A2T did. Nevertheless, in this example GAversary’s replacements were less grammatically sound than those of BAE; though remained comparable with A2T. On the WordCNN classifier with the same data (Tables 6 and 7), GAversary replaced “tears” with the more appropriate “crying” rather than “greed”, replaced “melodrama” with “melodramadic” vs “shame”, and replaced “mr” with “mister” rather than “love”. However, GAversary also replaced “the” with “per”: a mistake that was observed several times in the full results for both GAversary and A2T. In general the examples featuring comma separated lists rather than full sentences seemed to lead to better results for all attacks, perhaps because the context matters less in these cases. For BERT on the AG-News set (Table 8), GAversary uses one more substitution than BAE: but both are subtle: replacement of “Congress” with “Senate” rather than an insertion to create “Congress senate”, and replacement of “final” with “ultimate”, which works well in this context. GAversary uses fewer substitutions than A2T, which also made a factual change by replacing “Wednesday” with “Thursday”. For WordCNN on AGNews (Table 9), GAversary replaced “Games” with “Gaming” rather than the less appropriate “time” (common to both BAE and A2T). GAversary did introduce an additional change: “unclear” was replaced with the similar “nebulous”. In general, the adversarial examples from all methods replaced proper nouns and abbrevations, e.g. ag-news “T N” being replaced with random other initials and “Melbourne” being replaced with other Australian cities. This is a possible flaw in only using similarity in the embedding space. Clearly proper nouns such as these are used in similar ways across a corpus so will have similar embeddings, but are not interchangeable. Named Entity Recognition [46] represents a suitable direction for future research to address this issue.
15
Attack
Text
Original
it uses some of the figures from the real-life story to portray themselves in the film . the result is a powerful , naturally dramatic piece of low-budget filmmaking .
BAE
it uses some of the figures from the real-life story to appear themselves in the film . the result is a short , naturally dramatic piece of low-budget filmmaking .
A2T
it uses some of the figures from the real-life story to portray themselves in the film . the result is a emphatic , evidently prodigious piece of low-budget cinematographic .
GAversary
it uses some of the numerals from the real-life story to describes themselves in the film . the result is a emphatic , naturally whopping piece of low-budget filmmaking .
Table 5: Qualitative examples of GAversary attacks against BAE and A2T on the BERT classifier over MR dataset. (Blue: modification candidate, Red and Cyan: modified and non-modified words for each attack.)
Attack
Text
Original
passion , melodrama , sorrow , laugther , and tears cascade over the screen effortlessly . . .
BAE
passion , shame , humor , laugther , and greed cascade over the pages effortlessly . . .
A2T
fervor , melodrama , bereavement , laugther , and sobs cascade over the screen effortlessly . . .
GAversary
passion , melodramatic , sorrow , laugther , and crying cascade over per screen effortlessly . . .
Table 6: Qualitative examples of GAversary attacks against BAE and A2T on the WordCNN classifier over MR dataset. (Blue: modification candidate, Red and Cyan: modified and non-modified words for each attack.)
16
Attack
Text
Original
skillful as he is , mr . shyamalan is undone by his pretensions .
BAE
skillful as he is , love . shyamalan is undone by his pretensions .
A2T
crafty as he is , mister . shyamalan is undone by his pretensions .
GAversary
skillful as he is , mister . shyamalan is conquered by his pretensions .
Table 7: Qualitative examples of GAversary attacks against BAE and A2T on the WordCNN classifier over MR dataset. (Blue: modification candidate, Red and Cyan: modified and non-modified words for each attack.) Attack
Text
Original
Congress Passes Bill Allowing Space Tours (AP) AP -Outer space could become the final frontier of tourism under legislation passed Wednesday by the Senate to regulate commercial human spaceflight.
BAE
Congress senate Bill Allowing Space Tours (AP) AP -Outer space could become the final frontier of tourism under legislation passed Wednesday by the Senate to regulate commercial human spaceflight.
A2T
Congress Passes Bills Allowing Space Visits (AP) AP -Outside space could become the final frontier of tourism under legislation passed Thursday by the Senate to regulate commercial human spaceflight.
GAversary
Senate Passes Bill Allowing Space Tours (AP) AP -Outer space could become the ultimate frontier of tourism under legislation passed Wednesday by the Senate to regulate commercial human spaceflight.
Table 8: Qualitative examples of GAversary attacks against BAE and A2T on the BERT classifier over AG-News dataset. (Blue: modification candidate, Red and Cyan: modified and non-modified words for each attack.)
17
Attack
Text
Original
A #39;new Greece #39; beams after success of Games As Greeks get a boost, it remains unclear if success will mean higher stature in Europe. By Peter Ford Staff writer of The Christian Science Monitor.
BAE
A #39;new Greece #39; beams after success of time As Greeks get a boost, it remains unclear if success will mean higher stature in Europe. By Peter Ford Staff writer of The Christian Science Monitor.
A2T
A #39;new Greece #39; beams after success of time As Greeks get a boost, it remains unclear if success will mean higher stature in Europe. By Peter Ford Staff writer of The Christian Science Monitor.
GAversary
A #39;new Greece #39; beams after success of Gaming As Greeks get a boost, it remains nebulous if success will mean higher stature in Europe. By Peter Ford Staff writer of The Christian Science Monitor.
Table 9: Qualitative examples of GAversary attacks against BAE and A2T on the WordCNN classifier over AG-News dataset. (Blue: modification candidate, Red and Cyan: modified and non-modified words for each attack.)
18
5
Conclusion
This paper has proposed GAversary, a hybrid approach to generating adversarial text seeking the best of previous work: a search based approach using a genetic algorithm that only needs the logit value output from the model, guided by GloVe for semantic similarity. In practice GAversary reduces accuracy even more than the BAE and A2T approaches , but at the cost of needing 2x-8x as many queries of the target model and perturbing around twice as many words on average. However, the resulting examples show only a slight reduction in semantic similarity over those generated by the other approaches, and only around 5% increase in run time despite the greater number of model queries. There are several lines of potential future work. The algorithm could draw further on BERT to use the context of words being perturbed to further improve the semantic similarity of the replacement text. A simple local search could be applied during each replacement, taking the top-n most likely words according to BERT given the word’s context, testing the resulting text for each replacement before deciding which one to keep (this is known as a memetic algorithm [47]). Amending the objective function for the GA to include a penalty for replacements could also improve the similarity of the adversarial examples to the original text. It would be interesting to try to identify where proper nouns and abbreviations appear to prevent their replacement. Clearly there is also scope for drawing from more recent large language models i.e. LLaMA [48], Mixtral [49]), but at a potential increase in computational cost.
References [1] Timnit Gebru, Jonathan Krause, Yilun Wang, Duyun Chen, Jia Deng, Erez Lieberman Aiden, and Li Fei-Fei. Using deep learning and google street view to estimate the demographic makeup of neighborhoods across the united states. Proceedings of the National Academy of Sciences, 114(50):13108–13113, 2017. [2] Samuel Dodge and Lina Karam. A study and comparison of human and deep learning recognition performance under visual distortions. In 2017 26th international conference on computer communication and networks (ICCCN), pages 1–7. IEEE, 2017. [3] Myrthe Reuver, Antske Fokkens, and Suzan Verberne. No NLP task
19
should be an island: Multi-disciplinarity for diversity in news recommender systems. In Proceedings of the EACL Hackashop on News Media Content Analysis and Automated Report Generation, pages 45–55, Online, April 2021. Association for Computational Linguistics. [4] M Ángeles López-Cabarcos, Ada M Pérez-Pico, Maria Luisa López Perez, et al. Investor sentiment in the theoretical field of behavioural finance. Economic research-Ekonomska istraživanja, 33(1):2101–2119, 2020. [5] Brian M Lucey and Michael Dowling. The role of feelings in investor decision-making. Journal of economic surveys, 19(2):211–237, 2005. [6] Tom Roth, Yansong Gao, Alsharif Abuadbba, Surya Nepal, and Wei Liu. Token-modification adversarial attacks for natural language processing: A survey. arXiv preprint arXiv:2103.00676, 2021. [7] Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. Textbugger: Generating adversarial text against real-world applications. arXiv preprint arXiv:1812.05271, 2018. [8] Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50–56. IEEE, 2018. [9] Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. Bert-attack: Adversarial attack against bert using bert. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6193–6202, 2020. [10] Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, MingTing Sun, and Bill Dolan. Contextualized perturbation for textual adversarial attack. arXiv preprint arXiv:2009.07502, 2020. [11] Edward J Anderson and Michael C Ferris. Genetic algorithms for combinatorial optimization: the assemble line balancing problem. ORSA Journal on Computing, 6(2):161–173, 1994. [12] Fouad Bennis and Rajib Kumar Bhattacharjya. Nature-Inspired Methods for Metaheuristics Optimization: Algorithms and Applications in Science and Engineering, volume 16. Springer, 2020.
20
[13] Shijun Ye, Pengcheng Zhang, Hai Dong, and Shunhui Ji. Heuristicword-selection genetic algorithm for generating natural language adversarial examples. In 2021 IEEE International Conference on Artificial Intelligence Testing (AITest), pages 39–40. IEEE, 2021. [14] Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2890–2896, 2018. [15] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013. [16] Ciprian Chelba, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants, Phillipp Koehn, and Tony Robinson. One billion word benchmark for measuring progress in statistical language modeling. arXiv preprint arXiv:1312.3005, 2013. [17] Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. Certified robustness to adversarial word substitutions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4129–4142, 2019. [18] Sven Gowal, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli. On the effectiveness of interval bound propagation for training verifiably robust models. arXiv preprint arXiv:1810.12715, 2018. [19] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018–8025, 2020. [20] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171– 4186, 2019. 21
[21] Siddhant Garg and Goutham Ramakrishnan. Bae: Bert-based adversarial examples for text classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6174–6181, 2020. [22] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019. [23] Jin Yong Yoo and Yanjun Qi. Towards improving adversarial training of nlp models. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 945–956, 2021. [24] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019. [25] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, D. Erhan, Ian J. Goodfellow, and Rob Fergus. Intriguing properties of neural networks. CoRR, abs/1312.6199, 2013. [26] Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. Gradient-based adversarial attacks against text transformers. arXiv preprint arXiv:2104.13733, 2021. [27] Lifan Yuan, Yichi Zhang, Yangyi Chen, and Wei Wei. Bridge the gap between cv and nlp! a gradient-based textual adversarial attack framework. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7132–7146, 2023. [28] Boxin Wang, Chejian Xu, Xiangyu Liu, Yu Cheng, and Bo Li. Semattack: Natural textual attacks via different semantic spaces. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 176–205, 2022. [29] Muchao Ye, Chenglin Miao, Ting Wang, and Fenglong Ma. Texthoaxer: Budgeted hard-label adversarial attacks on text. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3877– 3884, 2022. [30] Zhen Yu, Xiaosen Wang, Wanxiang Che, and Kun He. Texthacker: Learning based hybrid local search algorithm for text hard-label adver22
sarial attack. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 622–637, 2022. [31] Guoyi Li, Bingkang Shi, Zongzhen Liu, Dehan Kong, Yulei Wu, Xiaodan Zhang, Longtao Huang, and Honglei Lyu. Adversarial text generation by search and learning. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. [32] Xiaosen Wang, Jin Hao, and Kun He. Natural language adversarial attacks and defenses in word level. In ArXiv preprint arXiv:1909.06723v1, 2019. [33] Alec Go, Richa Bhayani, and Lei Huang. Twitter sentiment classification using distant supervision. CS224N project report, Stanford, 1(12):2009, 2009. [34] Nikola Mrkšić, Diarmuid O Séaghdha, Blaise Thomson, Milica Gašić, Lina Rojas-Barahona, Pei-Hao Su, David Vandyke, Tsung-Hsien Wen, and Steve Young. Counter-fitting word vectors to linguistic constraints. arXiv preprint arXiv:1603.00892, 2016. [35] Bin Liang, Hongcheng Li, Miaoqiang Su, Pan Bian, Xirong Li, and Wenchang Shi. Deep text classification can be fooled. arXiv preprint arXiv:1704.08006, 2017. [36] Zhengli Zhao, Dheeru Dua, and Sameer Singh. Generating natural adversarial examples. arXiv preprint arXiv:1710.11342, 2017. [37] Alexander E. I. Brownlee and M. and Singh. Data and processing scripts for the paper “vulnerability of natural language classifiers to evolutionary generated adversarial text”, 2025. URL - TBC on publication [Online; accessed 7-March-2025]. [38] Sean Luke. Essentials of Metaheuristics. Lulu, second edition, 2013. Available for free at http://cs.gmu.edu/∼sean/book/metaheuristics/. [39] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014. [40] John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. Textattack: A framework for adversarial attacks, data 23
augmentation, and adversarial training in nlp. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 119–126, 2020. [41] Bo Pang and Lillian Lee. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In ACL, 2005. [42] Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. Advances in neural information processing systems, 28, 2015. [43] Yoon Kim. Convolutional neural networks for sentence classification. In EMNLP, 2014. [44] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997. [45] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. [46] Zara Nasar, Syed Waqar Jaffry, and Muhammad Kamran Malik. Named entity recognition and relation extraction: State-of-the-art. ACM Computing Surveys (CSUR), 54(1):1–39, 2021. [47] Pablo Moscato and Michael G Norman. A memetic approach for the traveling salesman problem implementation of a computational ecology for combinatorial optimization on message-passing systems. Parallel computing and transputer applications, 1:177–186, 1992. [48] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [49] Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024.
24