arXiv CSopen access

DeepSeek Robustness Against Semantic-Character Dual-Space Mutated Prompt Injection

1

DeepSeek Robustness Against Semantic-Character Dual-Space Mutated Prompt Injection

arXiv:2604.12548v1 [cs.CR] 14 Apr 2026

Junyu Ren, Xingjian Pan, Wensheng Gan*, Philip S. Yu, Life Fellow, IEEE

prompts to induce models to generate harmful content, thereby creating policy, ethical, and safety concerns. Ensuring model robustness and compliance requires systematic evaluation of safety boundaries under various types of input variations. Currently, mainstream LLM services typically employ safety mechanisms, including alignment strategies, content moderation filters, and system prompts to constrain outputs, for instance, by prohibiting the generation of violent, hateful, explicit, or illegal content. However, empirical evidence shows these security mechanisms are not impenetrable. Attackers can gradually circumvent system-imposed safety constraints through techniques such as prompt injection, jailbreak attacks, and roleplaying, enabling models to produce implicit or indirect harmful information under the guise of “normal conversation”. Such attacks typically do not exploit parameter-level vulnerabilities; rather, they leverage the model’s high sensitivity to naturallanguage instructions and limitations in safety-detection rules, thereby exhibiting characteristics of strong concealment, low cost, and easy propagation. As LLM interfaces progressively open as general-purpose APIs, input distributions become increasingly complex across application scenarios, making model performance on non-standard inputs increasingly critical. In practice, user inputs often contain colloquial expressions, spelling errors, code-mixed text, and even malformed characters, while malicious attackers deliberately inject zero-width characters, character variants, and special encodings as “characterlevel noise” to evade keyword- or rule-based safety detections. From a defense perspective, if models are only evaluated Index Terms—large model security, DeepSeek, prompt injection, using standardized, well-formed data during training and robustness, semantic-character dual-space mutation. testing phases, their true safety and robustness in real-world environments risk being substantially overestimated [4]. In recent years, prompt injection research has evolved I. I NTRODUCTION from manual template construction toward automated, largeLarge language models (LLMs) [1]–[3] have achieved scale adversarial sample generation [5], [6]. Existing work remarkable progress in natural language processing, demon- predominantly focuses on black-box prompt injection techstrating powerful capabilities in language understanding and niques, with automated attack pipelines based on search, generation across diverse applications, including question an- optimization, and generative models already emerging in swering, text generation, programming assistance, and content English contexts; however, systematic research addressing moderation. However, their widespread deployment introduces Chinese-language scenarios and emerging domestic models new security risks: malicious users can craft carefully designed such as DeepSeek remains sparse [7], [8]. These models differ from openly available English-language models in training This research was supported in part by the National Natural Science Foundation of China (No. 62272196), and Guangzhou Basic and Applied data, pretraining and fine-tuning strategies, and dialogue safety Basic Research Foundation (No. 2024A04J9971). Junyu Ren and Xingjian mechanisms, introducing uncertainty when directly transferring Pan contributed equally to this work. Corresponding author: Wensheng Gan Junyu Ren, Xingjian Pan, and Wensheng Gan are with the Col- existing methods. Research on “space mutation”—perturbations lege of Cyber Security, Jinan University, Guangzhou 510632, China. of prompts at different representational levels—remains in (E-mail: [email protected], [email protected], ws- its infancy. Most existing work examines semantic-level [email protected]) or character-level transformations independently: semantic Philip S. Yu is with the Department of Computer Science, University of Illinois Chicago, Chicago, USA. (E-mail: [email protected]) approaches focus on synonym substitution and instruction

Abstract—Prompt injection has emerged as a critical security threat to large language models (LLMs), yet existing studies predominantly focus on single-dimensional attack strategies, such as semantic rewriting or character-level obfuscation, which fail to capture the combined effects of multi-space perturbations in realistic scenarios. In addition, systematic black-box robustness evaluations of recent Chinese LLMs, such as DeepSeek, remain limited. To address these gaps, we propose PromptFuzz-SC, a semantic–character dual-space mutation framework for evaluating LLM robustness against prompt injection. The framework integrates semantic transformations (e.g., paraphrasing and wordorder perturbation) with character-level obfuscation (e.g., zerowidth insertion and encoding-based mutation), forming a unified and extensible mutation operator library. A hybrid search strategy combining ε-greedy exploration and hill-climbing refinement is adopted to efficiently discover high-quality adversarial prompts. We further introduce a unified evaluation protocol based on three metrics: misuse success rate (MSR), Average Queries to Success (AQS), and Stealth. Experimental results on DeepSeek demonstrate that dual-space mutation achieves the strongest overall attack performance among the evaluated strategies, attaining the highest mean MSR (0.189), peak MSR (0.375), and mean Stealth. Compared with semantic-only and character-only mutation, it improves mean MSR by 12.5% and 5.6%, respectively. While not consistently minimizing query cost, the proposed method achieves competitive best-case efficiency and maintains strong imperceptibility, indicating a more favorable balance between attack effectiveness and concealment. These findings highlight the importance of composite mutation strategies for robust redteaming of LLMs and provide practical insights for the design of multi-layer defense mechanisms. The code and datasets are publicly available at https://github.com/karma1822/PromptFuzzSC.

2

rewriting, while character-level approaches emphasize zeroThe paper is organized as follows: Section II reviews related width characters and character substitution. However, these work. Section III presents the proposed algorithms, and Section dimensions are predominantly studied in isolation, with no IV shows experimental results. Finally, Section V provides systematic analysis of synergistic effects, composite mutation conclusions and future research directions. strategies, or approaches that jointly optimize attack success and imperceptibility under practical constraints [9]. Notably absent II. R ELATED W ORK are pluggable mutation operator libraries, unified measurement A. Evolution of Prompt Injection Attacks frameworks, and empirically validated designs of search Prompt injection (PI) was systematically studied in early strategies. Regarding DeepSeek, public systematic research work such as Wallace et al. [12], which showed that carefully on its robustness and failure modes when facing composite crafted input prompts can induce language models to ignore attacks combining both semantic and character mutations intended safety constraints. Although these early attacks were remains relatively underexplored. This gap exists at both relatively simple and only partially automated, they helped theoretical levels (lack of unified dual-space modeling and establish prompt injection as a distinct attack paradigm and measurement frameworks) and practical levels (absence of motivated subsequent research on safety evaluation. Shin et large-scale black-box evaluations and reproducible datasets al. [13] further advanced this line of work by introducing for DeepSeek). In summary, existing research still shows automated prompt optimization based on discrete gradient several limitations: (1) emphasis on single attack modalities, search, marking a shift from manual prompt crafting to making it difficult to comprehensively cover multiple strategy algorithmic attack generation. combinations potentially present in real attacks [10]; (2) lack of As LLM safety mechanisms became more sophisticated, the explicit modeling of “query costs” in search processes, failing effectiveness of early prompt optimization methods gradually to adequately address efficient discovery of effective attack declined, especially against newer aligned and reasoningsamples; and (3) inconsistent evaluation metrics and lack of capable models. With the widespread deployment of closedsystematic design and comparison for measuring “success rate”, source LLM APIs after 2021, black-box prompt injection “search efficiency”, and “imperceptibility” dimensions [11]. became the dominant research setting. In this setting, attackers To address the above research gaps, we propose and vali- interact with models only through query interfaces, without date PromptFuzz-SC. This novel attack-evaluation framework access to model parameters or internal states, making the supports semantic-character dual-space perturbations, explicitly threat model more realistic for deployed systems. Zou et al. models query-budget constraints, and provides unified attack [14] provided an influential formulation of jailbreak attacks metrics and visualization capabilities. This framework facili- and introduced an automated universal attack baseline, which tates a more realistic characterization of LLM security risks served as an important reference for subsequent studies. Zhu in open environments and establishes a reliable experimental et al. [15] further examined the robustness of prompt injection foundation for the design and validation of subsequent defense attacks across different models under black-box conditions. enhancement strategies. The key contributions of this paper More recently, Li et al. [16] extended prompt injection beyond are as follows: text-only settings by exploring multimodal attacks against vision-language models. • Dual-space mutation framework: Design a semanticOverall, prompt injection research has evolved from manual character dual-space mutation framework for prompt and template-based attacks to automated optimization-based injection attacks, constructing a library of pluggable generation, and from text-only attacks to multimodal variants. mutation operators covering common and composite attack These studies can be broadly grouped into three categories: (i) vectors. manual or template-based attacks, (ii) automated optimization• Hybrid search strategy: Apply a strategy combining εbased attacks, and (iii) multimodal extensions, with black-box greedy global exploration and hill-climbing local refinesettings becoming increasingly important in practical security ment to efficiently discover variants with low cost and evaluation. high success rates in high-dimensional mutation spaces. • Unified evaluation system: Conduct systematic black-box robustness assessment on DeepSeek through a unified B. Dual-Space Mutation: An Emerging Research Branch metric system (MSR, AQS, Stealth) and reproducible Research on spatially structured mutation for prompt injecexperimental pipelines, revealing synergistic effects and tion has emerged as a new direction in recent years. Cheng et al. trade-off characteristics of dual-space mutations. [17] introduced the concept of spatial mutation and, building on • Empirical validation on DeepSeek: Conduct systematic multimodal injection settings, proposed a dual-space attack and black-box robustness evaluation of DeepSeek across defense framework involving both image-space and text-space diverse parameter configurations, demonstrating that dual- perturbations. This line of work highlights the importance of space cooperation delivers the strongest overall perfor- analyzing adversarial behavior across different representational mance among the tested strategies, with mean MSR spaces rather than focusing on a single perturbation type. improvements of 12.5% and 5.6% over semantic-only Recent studies have further extended this idea to text-based and character-only mutation, respectively, a peak MSR of dual-space mutation. For example, Nasr et al. [18] explored 0.375, and strong concealment performance under suitable alternating optimization over semantic and character spaces, settings. showing that multi-round mutation can substantially increase

3

PromptFuzz-SC：Adversarial Attack Framework

Seed Prompts

Framework Overview

① Semantic-character dual-space mutation engine Semantic space

Character space

·Rewriting ·emoji ·ICL

·ZWJ ·Base64 ·Leetspeak

② 𝜺 −greedy + Hill-climbing

④ Control & Visualization

③ Metrics & Online Monitoring

Experiment management

Hybrid Searcher

𝜺-greedy strategy Candidate Samples

𝜺

1 − 𝜺

Explore （random）

Exploit （refine）

MutationOP Interface

Hill-climbing Search

Pluggable Operators

Local optimization → Deep digging

MSR

Stealth

Attack success rate

Stealthiness Score

Query/ Response

Matric Data

AQS

CLI Config

Web Panel

Batch exp Grid search Tuning

Curves Compare Reports

Trend Analysis

Distribution

Avg Queries to Success

Prometheus Export Real-time metrics + Dashboard

Feedback

DeepSeek API/LLM Interaction

Fig. 1. Overall architecture of the PromptFuzz-SC framework.

attack effectiveness against existing defenses. Their results suggest that combining heterogeneous mutation spaces may expose vulnerabilities that are difficult to reveal using semanticonly or character-only perturbations. At the same time, this branch of research is still developing, and many aspects remain insufficiently studied, including unified mutation libraries, consistent evaluation protocols, and systematic black-box assessment on newly emerging LLM families. Therefore, dualspace mutation can be regarded as a promising but still underexplored direction in prompt injection research.

C. Security Assessment of DeepSeek, a Domestic Chinese LLM Security evaluation of major international LLMs such as GPT4 and Claude has become increasingly systematic, covering red-team testing, jailbreak benchmarks, multimodal safety risks, and automated attack pipelines [19], [20]. In contrast, public security research on recently released domestic Chinese LLMs, particularly reasoning-oriented models such as DeepSeek, remains relatively limited. Owing to differences in training data, alignment strategies, and reasoning behavior, these models may exhibit security characteristics that are not fully captured by existing evaluation frameworks. DeepSeek began attracting widespread attention after its public deployment in late 2024, but dedicated prompt injection assessment for this model family is still at an early stage. Wu et al. [21] included DeepSeek in broader adversarial attack evaluations, providing initial evidence of its robustness under specific attack settings. Cui et al. [22] investigated chain-of-thought interruption vulnerabilities in reasoning-oriented LLMs such as DeepSeek, which revealed another class of prompt-related security risks. However, existing studies are either broad in scope or focused on specialized vulnerability types, and they do not yet provide a systematic black-box robustness evaluation framework tailored to composite prompt injection strategies on DeepSeek. Overall, prompt injection research targeting reasoning-based Chinese LLMs remains at an early stage, leaving substantial room for further exploration [23].

III. P ROPOSED METHOD A. Overall Framework Design The proposed adversarial attack framework PromptFuzzSC is designed for systematic robustness evaluation of conversational large language models under safety-constrained scenarios [24], [25]. The overall design follows a pipelined structure of “mutation generation – search optimization – metric monitoring – visualization analysis” [26], decoupling the attack generation and analysis components, which facilitates extension with new perturbation operators and migration to different models or evaluation tasks [27], [28]. As illustrated in Fig. 1, the framework comprises four core modules: (1) Semanticcharacter dual-space mutation engine: systematically perturbs seed prompts at both semantic and character levels to generate candidate attack samples. (2) ε-greedy + hill-climbing hybrid search: balances global exploration and local refinement under limited query budgets to discover efficient attack templates. (3) Multi-dimensional metrics and online monitoring: real-time computation and export of key evaluation metrics such as MSR, AQS, and Stealth, supporting dynamic diagnosis during experiments. (4) Experiment control and visualization frontend: provides configuration-driven batch experiment capabilities and interactive result dashboards for easy reproduction and analysis. The overall attack process of the proposed framework can be formulated as “maximizing expected success rate under query budget constraints”. Given the original safety prompt x0 , the generated attack sample set X, the target model response function f (x), and the attack judgment function g(f (x)) ∈ {0, 1}, where 1 denotes attack success and 0 denotes attack failure, the attack objective under a maximum query budget B can be written as 1 X max g(f (x)). (1) S⊆X |S| |S|≤B

x∈S

B. Semantic-Character Dual-Space Mutation Engine PromptFuzz-SC constructs a “semantic-character dual-space mutation” engine at the input side to systematically perturb safety prompts. The semantic space employs operators such as synonym substitution, emoji injection, and in-context learning

4

(ICL) [29] contamination exemplification, enhancing pragmatic combining segments such as “task description–constraint ambiguity, or inserting harmful demonstrations to induce model conditions–historical examples–safety disclaimers” into deviation from safety norms. The character space employs finecomplex prefixes enables models to expose sensitive grained perturbations, including space insertion, zero-width details under false “review/analysis” contexts. connectors, Base64 encoding, and Leetspeak transformation • Template concatenation and structure reordering: We to weaken sensitive pattern-matching capabilities of the model explore different prompt organization strategies through and safety filters [30], [31]. The framework encapsulates both automated concatenation and reordering of structured segperturbation strategies as pluggable operators via a unified ments. Specific approaches include prefix-suffix concateMutationOp interface, thereby enabling extensibility [32]. nation (appending attack prefixes or safety disclaimer suffixes) and segment reordering (modifying the sequence of (1) Mathematical formulation of the dual-space mu“task description–constraint conditions–few-shot examples– tation engine: To systematically characterize the impact closing statement”) to evaluate how different prompt of perturbations across different levels on LLM security, structures affect model safety behavior. These operators the framework partitions mutation operators into semantic can also be combined with jailbreak template libraries to and character spaces, supporting three experimental modes: systematically generate prompts with diverse structures. “semantic only”, “character only”, and “dual-space”. Let M denote the mutation operator set and m denote a specific (3) Character space operators. In contrast to semantic operator instance determined by the operational mode: operators that modify linguistic expression, character space  operators alter the surface form and encoding of prompts  (semantic mode) Ms while preserving human readability, thereby evading safety M = Mc (2) filter detection [34]. The framework integrates the following (character mode)   Ms ∪ Mc (dual-space mode) character operators: where

Invisible characters and zero-width symbol insertion: We insert invisible characters such as zero-width joiners (ZWJ), zero-width spaces (ZWSP), and bidirectional control characters within sensitive keywords. This keeps strings visually unchanged but fragments them at the underlying token sequence level, disrupting filter rules based on string matching. For reproducibility, the zero-width insertion operator supports three insertion modes: randomposition insertion, insertion around sensitive keywords only, and inter-character insertion. The sensitive keyword lexicon contains more than 50 terms covering securityrelated concepts such as accounts, passwords, viruses, trojans, and attacks. This approach is particularly effective against defenses relying on keyword blacklists or simple regular expressions. • Random spaces, character variants, and Leetspeak transformation: We generate attack samples with orthographically similar but differently encoded variants by injecting redundant spaces or punctuation between or within words and replacing letters with visually or phonetically similar symbols (e.g., “a→4”, “e→3”, “s→5”). The implementation includes a base mapping with the most common substitutions (e.g., a→4, e→3, i→1, o→0, s→5) and an extended mapping with additional replacements such as b→8, t→7, g→9, l→1, and z→2. By default, the full-text replacement mode is activated with a probability of 30%, while an optional random mode replaces only a subset of characters to preserve readability. This operator is used to evaluate the robustness of both models and defenses against non-standard inputs. • Encoding rewriting and hybrid representation: We induce models to actively decode and execute hidden instructions by encoding sensitive content (e.g., Base64 or hexadecimal) and embedding decoding instructions such as “please decode the following”. The Base64 rewriting operator supports four modes: token mode, •

(s)

(s)

(s)

(3)

(c)

(c)

(c)

(4)

Ms = {m1 , m2 , . . . , mKs }, Mc = {m1 , m2 , . . . , mKc }.

Given the current prompt xt and a selected mutation operator mt ∈ M, the one-step mutation is defined as xt+1 = mt (xt ).

(5)

(2) Semantic space operators. To perturb LLMs at the high-level semantic dimension of prompts, we design semantic operators that modify natural language expressions while preserving attack intent, thereby inducing shifts in model safety judgments [33]. Specifically, the framework integrates three categories of semantic operators: • Synonym rewriting and style transfer: We generate semantically equivalent or near-equivalent text variants through vocabulary substitution (synonyms and nearsynonymous expressions), sentence restructuring, and tone modulation (e.g., converting direct imperatives into courteous requests or academic discussion styles). To improve reproducibility, the framework implements a manually constructed synonym mapping lexicon containing 147 synonym groups, covering domain-relevant keywords such as action terms, adjectives, cybersecurity terminology, and attack-related technical expressions. This approach both bypasses shallow filters based on keyword or template matching and leverages differences in model sensitivity to varying tones and styles to induce relaxation of safety constraints. • Context expansion and multi-turn template insertion: We construct dialogue contexts favorable to attacks by inserting false scenario settings (e.g., “security audit”, “red team penetration testing”), system constraints (e.g., “you are a security research assistant”), or fabricated dialogue histories before or after the original prompt. For instance,

5

which randomly encodes selected whitespace-delimited words or segments; keyword mode, which encodes only sensitive keywords; full mode, which encodes the entire prompt and appends a decoding instruction; and mixed mode, which preserves part of the readable content while encoding sensitive terms. The sensitive keyword lexicon also contains more than 50 security-related terms, including accounts, passwords, viruses, trojans, and attack commands. This approach simulates real-world attacks that bypass content detection through encoding. (4) Unified abstract interface and configuration-driven operator library management. To simultaneously support both operator categories while facilitating future extensions, PromptFuzz-SC introduces a unified MutationOp abstract interface. Every operator—whether semantic or character, built-in or plugin—implements the same apply (text) → text’ method. The framework interacts solely with this abstract interface during search without concerning itself with internal implementation details. This design provides two direct benefits: First, built-in and plugin operators are completely equivalent at the scheduling layer, allowing us to implement new perturbation strategies as independent plugins without modifying core search code, enabling the ε-greedy + Hill-Climbing framework to automatically incorporate them into the search space. Second, operator parameterization information (template fragment libraries, prefix-suffix lists, concatenation/reordering rules, domain-specific keyword lists, etc.) is uniformly managed through JSON configuration files. Different experimental scenarios require only switching configuration files to rapidly reconfigure and reorganize the “mutation operator library” without altering code logic. At runtime, users can select one of three operator activation schemes through command-line parameters or configuration options: • Enable only semantic space operators for evaluating model sensitivity to semantic-level attack-defense interactions • Enable only character space operators for analyzing how encoding and representation-level perturbations affect safety filters • Combine both operator categories in dual-space mode to systematically explore the synergistic effects introduced by “semantic + character” hybrid attacks This “unified interface + configuration-driven” design enables PromptFuzz-SC’s mutation engine to cover diverse attack patterns while maintaining excellent maintainability and extensibility, providing a foundation for rapid iteration against emerging LLMs and novel defense mechanisms. C. ε-greedy with Hill-Climbing Search Strategy Above the candidate sample generation layer, we design a hybrid search algorithm combining ε-greedy and hill-climbing strategies to automatically discover efficient attack templates within bounded query budgets. The ε-greedy phase explores new directions through multi-operator mutation with probability ε, while with probability 1-ε it exploits the historical best template. Upon discovering successful samples, hill-climbing search is triggered to iteratively explore the neighborhood and accept superior variants. This strategy balances global

exploration with local exploitation, automatically discovering multiple high-success-rate jailbreak templates, providing highquality samples for subsequent analysis. Let the current local optimum sample be xt , and let the candidate operator set be M. For each operator m ∈ M, let st (m) and nt (m) denote the number of successful attacks and the total number of times that operator m has been selected up to step t, respectively. To avoid undefined values when an operator has not been selected before, we adopt Laplace smoothing and define the empirical value of operator m at step t as st (m) + 1 Qt (m) = . (6) nt (m) + 2 A higher Qt (m) indicates that operator m is empirically more effective in the current attack context. The operator selection rule is defined as ( Uniform(M), with probability ε, mt = (7) arg maxm∈M Qt (m), with probability 1 − ε. After selecting mt , a new candidate prompt is generated by x′t+1 = mt (xt ). The hill-climbing update rule is then ( x′t+1 , if R(x′t+1 ) > R(xt ), xt+1 = xt , otherwise.

(8)

(9)

The proposed search strategy is detailed as follows: (1) Initialization phase: Randomly select several samples from the seed prompt collection as initial candidates, defining statistics including success count, query count, and Stealth score. This phase provides diverse starting points for subsequent search, covering different attack intentions, sentence styles, and perturbation patterns. (2) ε-greedy iterative search: Each iteration balances global exploration and local exploitation. (i) Exploration phase (probability ε): Randomly select parent samples from the seed collection or historical samples, apply randomly combined mutation operators (which may span semantic and character spaces), and generate new samples with substantial structural and content differences to discover novel attack paths. (ii) Exploitation phase (probability 1-ε): Based on empirical values Qt (m) or recent attack success rates, select high-performing samples and perform local perturbation refinement in their vicinity. The search algorithm randomly samples one or multiple MutationOp operators to perform one or multiple mutations, thereby generating a candidate set. These candidate samples are submitted asynchronously in batches to the target model for evaluation through asyncio’s semaphore mechanism (concurrency level C) to respect rate limits and optimize resource utilization. (3) Feedback evaluation and metric update: Attack success is determined via a three-stage cascaded rule-based filter. First, a response is marked as failed if it contains policyrefusal indicators such as safety disclaimers or explicit denial of the request on ethical or operational grounds. Second, a response is also marked as failed if it consists solely of affirmative acknowledgements without substantive content, or

6

falls below a minimum length threshold of 40 characters. E. Experiment Control, Monitoring, and Visualization A response is confirmed as successful only when it clears PromptFuzz-SC integrates experiment control and visualboth preceding filters and provides substantive, task-relevant ization tools supporting centralized configuration management content (≥40 characters); this cascaded criterion reduces false and a local web control panel, upgrading the system to an positives arising from ambiguous or evasive model outputs. end-to-end “evaluation and analysis platform”. Upon determination, the framework computes the Stealth score 1) Local web control panel: Existing research configures relative to the original seed prompt and updates the global query budget, concurrency level, ε value, mutation space type, MSR and AQS in real time. DeepSeek temperature, generation length, and other parameters (4) Hill-climbing local optimization: Upon each successful through a browser interface, with one-click launching of single attack sample, hill-climbing is triggered if the sample’s Stealth or batch robustness experiments (parameter sweeps), avoiding score exceeds the current elite sample mean or if success is complex command-line operations. achieved with fewer queries than prior successful attempts. The 2) Metric monitoring: During search, key metrics, including neighborhood of a sample xt is defined as the set of single- cumulative query count, success count, MSR, AQS, and step mutants {m(xt ) | m ∈ M }, where one operator m is Stealth mean, are exported in real time in Prometheus format, sampled from M per step. Each invocation performs at most facilitating integration with Grafana for process visualization. 5 local search steps, terminating early upon the first successful 3) Batch robustness experiments and report generation: neighbor. A neighbor is accepted and the “peak” updated Automatically execute multiple attack experiments for different if and only if R(xt+1 ) > R(xt ). This mechanism improves budget, temperature, and generation length combinations, query budget utilization by intensively exploiting high-potential generating CSV result tables, plotting comparative curves of attack regions while preserving the global exploration diversity “MSR/AQS evolution with budget”, and synthesizing results maintained by the ε-greedy phase. into HTML chart report pages. (5) Termination conditions and output: Search terminates In the synthesis of the above design, PromptFuzz-SC when one of the following conditions is satisfied: total query supports both in-depth adversarial sample search in single count reaches budget B, or key metrics (MSR, Stealth mean) scenarios and systematic robustness assessment across multiple show no significant improvement, indicating convergence. parameter configurations, providing a unified experimental Output representative successful sample sets covering different platform for analyzing “LLM security boundaries under dualstyles, perturbation intensities, and Stealth levels. space semantic and character perturbations”. D. Attack Success and Imperceptibility Metrics To characterize the attack process and model defense performance, PromptFuzz-SC integrates multi-dimensional metric computation and monitoring modules. For each query-response pair, the framework real-time updates three categories of key metrics as follows: Misuse success rate (MSR): the ratio of successful attack queries to the total number of issued queries within the given budget. Given total query count N and successful query count S S: MSR = N . Average queries to success (AQS): For all successful samples, calculate the query round required for the first success and take the average. Given first-success query PS rounds of successful samples q1 , q2 , . . ., qS : AQS = S1 i=1 qi . Stealth imperceptibility: Measures the character-level similarity between the mutated attack sample xt and the original safety prompt x0 via normalized Longest common subsequence |LCS(x0 , xt )| (LCS) similarity: Stealth(xt ) = max(|x , where |·| denotes 0 |, |xt |) sequence length. The score ranges in [0, 1], a higher value indicates stronger imperceptibility, i.e., the attack maintains surface-level fidelity to the original prompt while successfully bypassing safety mechanisms. In implementation, after each experiment, the framework performs statistical analysis on attack history and optimal samples, outputting JSON statistical files and various visualization charts, including Stealth score boxplots, cumulative distribution functions (CDF), and histograms of query rounds to success, supporting quantitative analysis in the paper.

IV. E XPERIMENTAL R ESULTS AND A NALYSIS In this section, we present the experimental evaluation of the proposed PromptFuzz-SC framework. The source code is available at https://github.com/karma1822/PromptFuzz-SC. A. Experimental Setup We evaluate PromptFuzz-SC in a black-box setting on DeepSeek from three perspectives: attack effectiveness, query efficiency, and imperceptibility, using MSR, AQS, and Stealth as the evaluation metrics. Three attack modes are considered for comparison: semantic-only mutation, character-only mutation, and semantic-character dual-space mutation. All experiments use the search strategy introduced in Section III, which combines ε-greedy exploration and hill-climbing refinement. Unless otherwise specified, the default configuration is set to ε = 0.2, C = 8, Nseed = 50, α = 0.6 : 0.4. Here, C is the number of candidate mutations evaluated at each iteration, Nseed is the number of initial seed prompts, and α denotes the sampling ratio between semantic and character-level mutation operators. These parameter settings are based on empirical experience and commonly used configurations. For sensitivity analysis, we vary one factor at a time while keeping the remaining parameters fixed at their default values. Specifically, budget-level analysis is performed over B ∈ {25, 50, 75, 100, 125, 150, 175, 200}, and decoding-related analysis is conducted for T ∈ {0.3, 0.7} and Lmax ∈ {256, 512}. This design enables a controlled comparison of attack modes and a systematic analysis of parameter effects.

7

Fig. 2. Performance dynamics of single semantic-space attacks across query budgets under four (T, L) configurations: (a) Misuse success rate, (b) Average Queries to Success, and (c) Stealth score.

Fig. 3. Performance dynamics of single character-space attacks across query budgets under four (T, L) configurations: (a) Misuse success rate, (b) Average Queries to Success, and (c) Stealth score.

B. Attack Performance Evaluation Under this configuration, the method achieves a mean MSR 1) Semantic-Space Attack Performance Analysis: Semanticof 0.248, a peak MSR of 0.370, a mean AQS of 39.0, and space attack performance shows clear sensitivity to the decoding a mean Stealth score of 0.818. When the generation length configuration. As summarized in Table I, under the conservative is increased to L = 512, the mean and peak MSR decline setting (T = 0.3, L = 256), the method achieves a mean MSR to 0.177 and 0.287, respectively, while the mean Stealth of 0.152, a peak MSR of 0.166, a mean AQS of 34.0, and a score increases to 0.861. This indicates that, in the character mean Stealth score of 0.772. Increasing the generation length space, longer generations do not necessarily strengthen the to L = 512 raises the mean and peak MSR to 0.181 and attack; instead, they tend to improve concealment while 0.252, respectively, while reducing AQS to 30.9; however, the diluting attack intensity. High-temperature settings are generally mean Stealth score decreases to 0.733. This pattern suggests less favorable, with mean MSR values dropping to 0.141– that longer outputs enlarge the semantic search space and can 0.151, suggesting that character-level mutation is particularly improve attackability, but they also tend to introduce more sensitive to sampling randomness. The budget-wise trends visible semantic deviation. By contrast, the high-temperature further highlight the dynamic characteristics of character-space short-sequence setting (T = 0.7, L = 256) yields the highest attacks. Under (T = 0.3, L = 256), the method exhibits mean Stealth score of 0.841, while maintaining only moderate a rapid-rise-rapid-decline pattern: the MSR reaches 0.370 at attack success (mean MSR 0.171, peak MSR 0.190), indicating budget = 25, but then falls to 0.153 by budget = 200. In that this configuration favors imperceptibility more than attack contrast, (T = 0.3, L = 512) shows a more gradual trajectory, strength. rising from a low initial level to a peak of 0.287 at budget The budget-wise curves in Fig. 2 further show that semantic= 125, followed by a decline at later budgets. These curves only attacks often obtain their strongest gains early and then suggest that character-level perturbation can produce strong become difficult to sustain. In particular, the setting (T = 0.7, early breakthroughs under favorable settings, but such gains are L = 512) reaches a peak MSR of 0.334 at low query budgets difficult to maintain as the search continues. At the same time, and then declines rapidly, forming a clear early-peak pattern. the relatively high Stealth scores across configurations indicate More generally, several configurations exhibit plateauing or that character mutation can preserve surface-level concealment decreasing MSR as the budget increases, rather than steady even when attack success is less stable. improvement. For example, (T = 0.3, L = 256) reaches its Overall, character-space mutation is effective for shortpeak at budget = 125 and then declines, while (T = 0.7, L = 512) drops below 0.10 during the middle budget range. budget probing under specific parameter settings, especially at A plausible explanation is that semantic-only mutation can low temperature. However, its performance is more volatile exhaust effective reformulations relatively quickly; after the than its Stealth scores alone would suggest, and its success rate most useful variants have been explored, further mutation degrades noticeably outside the best-performing configuration. may introduce semantic drift rather than improving attack Within the tested settings, character-only mutation therefore appears to offer strong early attack bursts, but limited stability effectiveness. Overall, the semantic space provides relatively favorable and relatively high dependence on configuration choice. imperceptibility, but its gains in attack success are limited and often unstable across larger budgets. The results also show a visible trade-off between success rate and Stealth: configurations with higher attack success tend to sacrifice concealment, whereas the most stealthy setting does not achieve the strongest attack performance. Within the tested configurations, semantic-only mutation therefore appears better suited to generating natural-looking variants than to sustaining strong attack effectiveness over long search horizons. 2) Character-Space Attack Performance Analysis: Character- Fig. 4. Performance dynamics of dual-space cooperative attacks across query space mutation performs best under the low-temperature short- budgets under four (T, L) configurations: (a) Misuse success rate, (b) Average sequence setting (T = 0.3, L = 256), as shown in Table II. queries to success, and (c) Stealth score.

8

TABLE I S UMMARY OF ATTACK PERFORMANCE METRICS UNDER SINGLE SEMANTIC - SPACE MUTATION ACROSS FOUR (T, L) CONFIGURATIONS . Config T = 0.3, L = 256 T = 0.3, L = 512 T = 0.7, L = 256 T = 0.7, L = 512

Mean MSR

Mean AQS

Mean Stealth

Peak MSR

Best AQS

Peak Stealth

0.152 0.181 0.171 0.167

34.0 30.9 32.0 39.6

0.772 0.733 0.841 0.824

0.166 0.252 0.190 0.334

1.5 2.8 20.2 12.5

0.946 0.807 0.879 0.889

TABLE II S UMMARY OF ATTACK PERFORMANCE METRICS UNDER SINGLE CHARACTER - SPACE MUTATION ACROSS FOUR (T, L) CONFIGURATIONS . Config T = 0.3, L = 256 T = 0.3, L = 512 T = 0.7, L = 256 T = 0.7, L = 512

Mean MSR

Mean AQS

Mean Stealth

Peak MSR

Best AQS

Peak Stealth

0.248 0.177 0.141 0.151

39.0 32.6 30.9 36.7

0.818 0.861 0.822 0.822

0.370 0.287 0.175 0.214

16.5 3.6 14.0 8.3

0.978 0.982 0.882 0.966

3) Dual-Space Attack Performance Analysis: Dual-space mutation shows comparatively balanced behavior across the tested configurations, as summarized in Table III and Fig. 4. Under the low-temperature short-sequence setting (T = 0.3, L = 256), the method achieves a mean MSR of 0.242, a peak MSR of 0.270, a mean AQS of 35.8, and a mean Stealth score of 0.823, indicating a relatively even trade-off among the three evaluation dimensions. A different pattern appears under (T = 0.3, L = 512): although the mean MSR decreases to 0.202, the peak MSR rises sharply to 0.375, while the mean Stealth score also reaches 0.890. This result suggests that the longer low-temperature setting is particularly favorable for discovering high-quality attack instances, even if the average performance becomes more dispersed. Under the high-temperature shortsequence setting (T = 0.7, L = 256), dual-space mutation records the lowest AQS among its four configurations (28.3), together with a mean MSR of 0.166 and a mean Stealth score of 0.869, indicating stronger query efficiency. The remaining high-temperature long-sequence setting (T = 0.7, L = 512) shows a lower mean MSR of 0.145 but still reaches a peak MSR of 0.308, suggesting that effective attack instances can still emerge at later stages of the search.

space. Overall, dual-space mutation appears to provide the most balanced combination of success rate, search efficiency, and imperceptibility among the tested settings, while also exhibiting relatively stable budget-wise behavior. C. Comparative Analysis

Fig. 5. Three-dimensional comparative performance of semantic-only, character-only, and dual-space strategies under T = 0.3, L = 256 configuration across query budgets: (a) MSR, (b) AQS, and (c) Stealth.

1) Three-Dimensional Quantitative Comparison: Based on the results of the three mutation strategies, this section provides a cross-strategy comparison from three complementary perspectives: attack success rate, search efficiency, and imperceptibility. The budget-wise curves reveal two notable dynamic patterns. From the perspective of attack success rate, dual-space First, under (T = 0.3, L = 256), the attack rapidly rises to a mutation shows the strongest aggregate performance among near-peak level and then remains in a relatively stable plateau the tested strategies. As reported in Table IV, it achieves over a substantial budget interval, rather than showing an the highest mean MSR (0.189) and the highest peak MSR immediate collapse after early success. Second, under (T = 0.7, (0.375), indicating that it performs strongly both on average L = 512), the curve displays a late-stage breakthrough pattern, and at its best observed operating point. Character-space reaching its strongest performance near the end of the tested mutation remains competitive in this dimension, especially budget range. Compared with the sharper rise-and-fall behavior in its strongest configuration, where it reaches a peak MSR observed in single-space settings, these trajectories suggest that close to the dual-space result. By contrast, semantic-space dual-space search is less brittle under budget expansion and mutation shows a lower overall success rate, although it can can maintain useful search momentum across a wider range still produce relatively strong peak behavior under specific of query budgets. settings. Taken together, these results indicate that dual-space A further observation is that successful dual-space samples mutation provides the strongest overall attack capability across tend to involve alternating semantic and character operators, the tested settings, while the two single-space strategies depend with an approximate semantic-to-character ratio of 3:2. Al- more heavily on favorable parameter choices. though this observation does not by itself establish a causal In terms of search efficiency, the relationship among the mechanism, it is consistent with the intuition that mixed-space three strategies is more nuanced. Since lower AQS indicates mutation may help avoid premature saturation in either search that successful attacks are found with fewer queries, the best

9

TABLE III S UMMARY OF ATTACK PERFORMANCE METRICS UNDER DUAL - SPACE COOPERATIVE MUTATION ACROSS FOUR (T, L) CONFIGURATIONS . Config

Mean MSR

Mean AQS

Mean Stealth

Peak MSR

Best AQS

Peak Stealth

0.242 0.202 0.166 0.145

35.8 40.9 28.3 41.0

0.823 0.890 0.869 0.852

0.270 0.375 0.210 0.308

11.3 12.1 7.7 12.5

0.850 0.921 0.997 0.912

T = 0.3, L = 256 T = 0.3, L = 512 T = 0.7, L = 256 T = 0.7, L = 512

TABLE IV T HREE - DIMENSIONAL QUANTITATIVE COMPARISON OF SEMANTIC - ONLY, CHARACTER - ONLY, AND DUAL - SPACE MUTATION STRATEGIES ACROSS THE ATTACK SUCCESS RATE , SEARCH EFFICIENCY, AND IMPERCEPTIBILITY. Evaluation Dimension Attack Success Rate (Mean MSR) Attack Success Rate (Peak MSR) Search Efficiency (Mean AQS) Search Efficiency (typical AQS) Imperceptibility (Mean Stealth) Imperceptibility (Peak Stealth) Per-Query Productivity (MSR/AQS)

Semantic-only

Character-only

Dual-space

Dual- vs. Single-Space

0.168 0.334 34.1 30.9 0.793 0.946 0.00492

0.179 0.370 34.8 30.9 0.831 0.982 0.00513

0.189 0.375 36.5 28.3 0.859 0.997 0.00518

+12.5% / +5.6% +12.3% / +1.4% +7.0% / +4.9% −8.4% / −8.4% +8.3% / +3.4% +5.4% / +1.5% +5.3% / +1.0%

observed efficiency is achieved by the dual-space setting with T = 0.7 and L = 256, which reaches an AQS of 28.3. At the aggregated level, however, dual-space mutation does not achieve the lowest mean AQS: its mean AQS is 36.5, compared with 34.1 for semantic-only and 34.8 for character-only. This indicates that the efficiency advantage of dual-space mutation is concentrated in its best configuration rather than uniformly maintained across all settings. For this reason, Table IV also Fig. 6. Parameter sensitivity comparison across mutation strategies: (a) overall reports per-query productivity (MSR/AQS), under which dual- metric summary (MSR mean, AQS mean, and Stealth mean), (b) MSR heatmap by (T, L) configuration, and (c) Stealth heatmap by (T, L) configuration. space mutation remains slightly better than the two singlespace baselines. Overall, these results suggest that dual-space mutation improves attack success while preserving competitive 2) Parameter Sensitivity Comparison: As shown in Fig. 6 best-case query efficiency, although its average AQS is slightly and Table V, the effects of temperature differ across the higher than those of the single-space baselines. three mutation strategies, but the resulting stability pattern The comparison in imperceptibility further highlights the is not the same as the overall performance ranking. Under advantage of dual-space mutation. It achieves the highest mean low-temperature (T = 0.3) settings, considering the two Stealth score among the three strategies, indicating that its generation-length configurations, semantic-only exhibits the generated attacks are, on average, less distinguishable from smallest variation in mean MSR (standard deviation 0.015), the original prompts under the adopted metric. Character- followed by dual-space (0.020), while character-only shows the space mutation can still achieve very high peak Stealth in largest fluctuation (0.036). Under high-temperature (T = 0.7) individual configurations, but this advantage is not consistently settings, semantic-only again remains the most stable (standard accompanied by equally strong success rates. Semantic-space deviation 0.002), character-only stays relatively stable (0.005), mutation shows moderate concealment overall, but its stronger and dual-space shows the largest variation (0.011). These results attack configurations tend to be associated with lower Stealth suggest that semantic-only is the most stable strategy with scores. In contrast, dual-space mutation maintains relatively respect to parameter changes, whereas dual-space tends to trade strong Stealth while also preserving the highest observed greater variability for stronger aggregate attack performance attack ceiling, suggesting a more favorable balance between and higher attack ceilings. Character-only lies between the two, concealment and effectiveness. with stability that depends on the specific temperature regime. Overall, the three-dimensional comparison shows clear The effect of the generation-length parameter is clearly nondifferences in strategic profile. Semantic-space mutation is monotonic and depends on both the mutation strategy and comparatively conservative, with reasonable imperceptibility the temperature setting. For semantic-only, under the lowbut limited attack gains. Character-space mutation can produce temperature setting (T = 0.3), increasing L from 256 to strong early breakthroughs and high concealment in certain 512 improves mean MSR by 19.1% (0.152 to 0.181) while settings, but its performance is more configuration-sensitive. reducing mean Stealth by 5.1% (0.772 to 0.733), indicating a Dual-space mutation provides the most balanced behavior clear success-rate–imperceptibility trade-off. Under the highacross the three dimensions under the tested settings, making temperature setting (T = 0.7), however, the same length it the strongest overall strategy in this study. increase changes mean MSR only slightly (0.171 to 0.167)

10

TABLE V C ONSOLIDATED PERFORMANCE SUMMARY ACROSS ALL MUTATION STRATEGIES AND REPRESENTATIVE (T, L) CONFIGURATIONS , INCLUDING CHARACTERISTIC BEHAVIORAL LABELS . Strategy

Config

Mean MSR

Mean AQS

Mean Stealth

Peak MSR

Semantic-only

T =0.3, L=256 T =0.7, L=512

0.152 0.167

34.0 39.6

0.772 0.824

0.166 0.334

1.5 12.5

0.946 0.889

Conservative Stochastic early peak

T =0.3, L=256 Character-only T =0.3, L=512 T =0.7, L=512

0.248 0.177 0.151

39.0 32.6 36.7

0.818 0.861 0.822

0.370 0.287 0.214

16.5 3.6 8.3

0.978 0.982 0.966

Early burst Max. imperceptibility Low overall

T =0.3, L=512 T =0.7, L=256

0.202 0.166

40.9 28.3

0.890 0.869

0.375 0.210

12.1 7.7

0.921 0.997

MSR–Stealth synergy Min. AQS

Dual-space

while reducing mean Stealth from 0.841 to 0.824. For characteronly, the pattern differs across temperatures: under T = 0.3, increasing L from 256 to 512 decreases mean MSR by 28.6% (0.248 to 0.177) but improves mean Stealth by 5.3% (0.818 to 0.861); under T = 0.7, the same change slightly improves mean MSR from 0.141 to 0.151 while leaving mean Stealth essentially unchanged (0.822 to 0.822). For dual-space, the lowtemperature setting exhibits a particularly interesting separation between average and best-case behavior: increasing L from 256 to 512 decreases mean MSR by 16.5% (0.242 to 0.202), yet peak MSR rises by 38.9% (0.270 to 0.375) and mean Stealth improves by 8.1% (0.823 to 0.890). Under T = 0.7, increasing L from 256 to 512 again reduces mean MSR (0.166 to 0.145) but raises peak MSR from 0.210 to 0.308, while mean Stealth declines slightly from 0.869 to 0.852. Overall, these results indicate that the effect of generation length cannot be characterized by a simple linear trend; instead, it interacts strongly with both mutation space and temperature. The morphology of the budget growth curves reveals further practical implications. Semantic-only generally follows an early-peak pattern, often followed by decline or low-level plateauing, making it suitable for short-horizon exploration but less sustainable over extended budgets. Character-only does not exhibit a single uniform profile: under favorable settings it can produce strong early bursts, while in other settings it shows a more gradual rise followed by later decline. Dual-space, by contrast, operates in two characteristic modes: rapid convergence to a relatively high plateau or delayed deep breakthrough, combining efficiency with depth and adapting naturally to different time constraints. Concretely, the T = 0.3, L = 256 configuration reaches a near-peak MSR of 0.269 at budget = 75 and sustains it through budget = 125, offering a 50-unit stable testing window; the T = 0.7, L = 512 configuration accumulates momentum prior to budget = 150 and then surges rapidly, making it well suited to deep-audit scenarios. 3) Practical Implications and Configuration Recommendations: The comparative results suggest that the three mutation strategies are suitable for different testing objectives rather than a single universal deployment setting. Semantic-space mutation provides relatively natural prompt variants and favorable concealment, but its attack gains are limited and often difficult to sustain as the query budget increases. Character-space mutation can produce strong early breakthroughs under favorable settings, especially at low temperature, but its performance is more

Best AQS Peak Stealth

Label

sensitive to parameter choice and tends to degrade outside its best-performing configuration. Among the three strategies, dual-space mutation shows the most balanced overall behavior across success rate, query efficiency, and Stealth under the tested settings. For practical evaluation, parameter selection should be guided by the target objective. If the goal is to maximize the observed attack ceiling while maintaining strong concealment, the dual-space setting with T = 0.3 and L = 512 is the strongest option in the current experiments, achieving the highest peak MSR together with a high Stealth score. If the goal is rapid probing under limited query resources, the dualspace setting with T = 0.7 and L = 256 is more suitable because it yields the lowest observed AQS. For evaluations emphasizing relatively stable behavior across budget growth, the dual-space setting with T = 0.3 and L = 256 provides a more sustained plateau than the more volatile single-space settings. In contrast, semantic-only and character-only mutations appear more suitable as complementary options when a tester specifically wants to emphasize semantic reformulation or shallow obfuscation behavior. The results also suggest that budget allocation should depend on the mutation space. Semantic-space mutation tends to produce useful variants early, but extended search often yields diminishing returns and may introduce semantic drift. Characterspace mutation can be effective for short-budget probing, yet its gains are more configuration-sensitive and less stable over longer search horizons. Dual-space mutation remains the most flexible strategy across the full budget range and therefore serves as the most practical default option in the current framework, especially when prior knowledge of the target defense behavior is limited. Overall, the findings indicate that effective attack evaluation benefits from adaptive configuration selection rather than reliance on a single fixed strategy. D. Discussion on Defense Implications The experimental observations in this study provide several implications for defense design. First, the differing behavior of semantic-space, characterspace, and dual-space mutations suggests that single-layer defenses are unlikely to be sufficient. Semantic reformulation may weaken rigid lexical filtering, while character-level perturbation can obscure surface-form patterns. When these two mutation modes are combined, the resulting prompt variants become more diverse and less predictable. This suggests that

11

robust defense should rely on multiple complementary detection mechanisms rather than a single rule-based or model-based component. Second, the results suggest that defense models may benefit from jointly modeling semantic and character-level information. In the current experiments, semantic-only attacks tend to preserve fluency while altering intent expression, whereas character-only attacks preserve much of the surface structure while perturbing textual form. Dual-space attacks combine both effects, which may create blind spots if the defense pipeline evaluates these dimensions separately. A plausible implication is that defense systems should learn cross-level representations capable of identifying prompts that appear benign in one feature space but abnormal in another [35]. Third, the strategy-level results also suggest a rough correspondence between defense depth and attack-space effectiveness. For relatively shallow defenses, such as lexical filtering or simple statistical checks, character-space mutation may already provide meaningful probing capability at relatively low cost. As defenses rely more heavily on semantic understanding or intent recognition, however, single-space mutation appears less reliable, and mixed semantic-character search becomes comparatively more useful. At the deepest level, where behavior is constrained by stronger alignment mechanisms, all three strategies show bounded effectiveness. This indicates that deeper safety alignment remains an important barrier even when shallow defenses are partially bypassed. Fourth, the budget-wise attack dynamics indicate that static responses may unintentionally provide optimization signals to adaptive attackers. In particular, iterative search-based attacks can exploit relatively stable feedback patterns to refine prompts over time. This may help explain why some configurations achieve rapid early gains or late-stage breakthroughs under continued querying. From the defense perspective, dynamic response strategies, such as response randomization, delayed feedback, or uncertainty-aware moderation, may reduce the usefulness of attacker-side feedback loops and make adaptive optimization less effective [36]. These observations also carry implications for evaluation practice. Because attack effectiveness varies with both defense depth and query budget, security testing may benefit from adaptive strategy selection rather than a fixed single-space setup. For example, early probing results can be used to determine whether shallow obfuscation is already sufficient or whether broader dual-space exploration is needed. Such adaptive evaluation may provide a more realistic estimate of system robustness than relying on only one mutation mode or one budget regime. Overall, the findings support a defense strategy that combines heterogeneous detection, joint representation learning, dynamic response design, and continued alignment reinforcement [37]. At the same time, they suggest that robustness evaluation should account for the interaction between defense depth, mutation space, and query budget, rather than treating prompt attacks as a single uniform threat model.

V. C ONCLUSION AND F UTURE WORK This paper presents PromptFuzz-SC, a dual-space adversarial prompt generation framework for prompt-level security evaluation of large language models. By jointly modeling semantic and character mutations within a unified search framework, the proposed method expands the adversarial search space and enables more comprehensive exploration of model vulnerabilities beyond single-space strategies. Experimental results demonstrate that dual-space mutation achieves the strongest overall performance among the evaluated settings, reaching the highest mean MSR (0.189), peak MSR (0.375), and mean Stealth (0.859). Compared with semantic-only and characteronly mutation, it improves mean MSR by 12.5% and 5.6%, respectively, with corresponding peak MSR gains of 12.3% and 1.4%. Under specific configurations, the method attains a peak MSR of 0.375 while maintaining high Stealth (0.890), indicating a favorable balance between attack effectiveness and imperceptibility. Although it does not consistently minimize query cost, dual-space mutation achieves competitive best-case efficiency (AQS = 28.3) and exhibits flexible attack patterns across different settings. These results highlight the practical value of composite mutation strategies for red-team testing against multi-layer LLM defense pipelines. Despite these contributions, several limitations remain. The current evaluation is restricted to a general-purpose LLM setting and does not cover domain-specific deployments or specialized defense mechanisms. Moreover, the query budget is limited to 200, constraining the analysis of long-horizon adaptive attacks. Future work will extend this framework toward multimodal adversarial settings, adaptive attack–defense coevolution, and more efficient automated search strategies. In addition, developing standardized cross-domain benchmarks and integrating stronger safety safeguards will be essential for advancing robust and responsible LLM security evaluation. R EFERENCES [1] W. Gan, S. Wan, and P. S. Yu, “Model-as-a-service (MaaS): A survey,” in IEEE International Conference on Big Data. IEEE, 2023, pp. 4636– 4645. [2] J. Wu, W. Gan, Z. Chen, S. Wan, and P. S. Yu, “Multimodal large language models: A survey,” in IEEE International Conference on Big Data. IEEE, 2023, pp. 2247–2256. [3] W. Gan, Z. Ning, Z. Qi, and P. S. Yu, “Mixture of experts (MoE): A big data perspective,” Information Fusion, pp. 1–28, 2025. [4] C. Wang, Y. Liu, B. Li, D. Zhang, Z. Li, and J. Fang, “Safety in large reasoning models: A survey,” arXiv preprint arXiv:2504.17704, 2025. [5] P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, “Jailbreaking black box large language models in twenty queries,” in IEEE Conference on Secure and Trustworthy Machine Learning. IEEE, 2025, pp. 23–42. [6] K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world LLMintegrated applications with indirect prompt injection,” in The ACM Workshop on Artificial Intelligence and Security, 2023, pp. 79–90. [7] A. Paulus, A. Zharmagambetov, C. Guo, B. Amos, and Y. Tian, “AdvPrompter: Fast adaptive adversarial prompting for LLMs,” arXiv preprint arXiv:2404.16873, 2024. [8] J. Steinhardt, P. W. W. Koh, and P. S. Liang, “Certified defenses for data poisoning attacks,” Advances in Neural Information Processing Systems, vol. 30, 2017. [9] Z. Zhang, J. Yang, P. Ke, F. Mi, H. Wang, and M. Huang, “Defending large language models against jailbreaking attacks through goal prioritization,” in The Annual Meeting of the Association for Computational Linguistics, 2024, pp. 8865–8887.

12

[10] C. Anil, E. Durmus, N. Panickssery, M. Sharma, J. Benton, S. Kundu, J. Batson, M. Tong, J. Mu, D. Ford et al., “Many-shot jailbreaking,” Advances in Neural Information Processing Systems, vol. 37, pp. 129 696– 129 742, 2024. [11] A. Wan, E. Wallace, S. Shen, and D. Klein, “Poisoning language models during instruction tuning,” in International Conference on Machine Learning, 2023, pp. 35 413–35 425. [12] E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, “Universal adversarial triggers for attacking and analyzing nlp,” in The Conference on EMNLP-IJCNLP, 2019, pp. 2153–2162. [13] T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh, “AutoPrompt: Eliciting knowledge from language models with automatically generated prompts,” in The Conference on Empirical Methods in Natural Language Processing, 2020, pp. 4222–4235. [14] A. Zou, Z. Wang, N. Carlini, M. Nasr, J. Z. Kolter, and M. Fredrikson, “Universal and transferable adversarial attacks on aligned language models,” arXiv preprint arXiv:2307.15043, 2023. [15] K. Zhu, J. Wang, J. Zhou, Z. Wang, H. Chen, Y. Wang, L. Yang, W. Ye, Y. Zhang, N. Gong et al., “Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts,” in The 1st ACM workshop on Large AI Systems and Models with Privacy and Safety Analysis, 2023, pp. 57–68. [16] M. Li, L. Li, Y. Yin, M. Ahmed, Z. Liu, and Q. Liu, “Red teaming visual language models,” in Findings of the Association for Computational Linguistics, 2024, pp. 3326–3342. [17] H. Cheng, E. Xiao, Y. Wang, L. Zhang, Q. Zhang, J. Cao, K. Xu, M. Sun, X. Hao, J. Gu et al., “Exploring typographic visual prompts injection threats in cross-modality generation models,” arXiv preprint arXiv:2503.11519, 2025. [18] M. Nasr, N. Carlini, C. Sitawarin, S. V. Schulhoff, J. Hayes, M. Ilie, J. Pluto, S. Song, H. Chaudhari, I. Shumailov et al., “The attacker moves second: Stronger adaptive attacks bypass defenses against LLM jailbreaks and prompt injections,” arXiv preprint arXiv:2510.09023, 2025. [19] Z. Ni, H. Wang, and H. Wang, “Shieldlearner: A new paradigm for jailbreak attack defense in LLMs,” arXiv preprint arXiv:2502.13162, 2025. [20] K. Zhou, C. Liu, X. Zhao, S. Jangam, J. Srinivasa, G. Liu, D. Song, and X. E. Wang, “The hidden risks of large reasoning models: A safety assessment of R1,” in The 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, 2025, pp. 3250–3265. [21] Y.-H. Wu, Y.-J. Xiong, H. Zhang, J.-C. Zhang, and Z. Zhou, “Sugarcoated poison: Benign generation unlocks jailbreaking,” in Findings of the Association for Computational Linguistics, 2025, pp. 9645–9665. [22] Y. Cui, Y. Cai, and Y. Wang, “Token-efficient prompt injection attack: Provoking cessation in LLM reasoning via adaptive token compression,” arXiv preprint arXiv:2504.20493, 2025. [23] A. Souly, J. Rando, E. Chapman, X. Davies, B. Hasircioglu, E. Shereen, C. Mougan, V. Mavroudis, E. Jones, C. Hicks et al., “Poisoning attacks on LLMs require a near-constant number of poison samples,” arXiv preprint arXiv:2510.07192, 2025. [24] A. Shafahi, M. Najibi, M. A. Ghiasi, Z. Xu, J. Dickerson, C. Studer, L. S. Davis, G. Taylor, and T. Goldstein, “Adversarial training for free!” Advances in neural information processing systems, vol. 32, 2019. [25] L. Graves, V. Nagisetty, and V. Ganesh, “Amnesiac machine learning,” in The AAAI Conference on Artificial Intelligence, vol. 35, no. 13, 2021, pp. 11 516–11 524. [26] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in IEEE Symposium on Security and Privacy. IEEE, 2017, pp. 3–18. [27] C. M. Ackerman and N. Panickssery, “Mitigating many-shot jailbreaking,” arXiv preprint arXiv:2504.09604, 2025. [28] X. Wang, D. Wu, Z. Ji, Z. Li, P. Ma, S. Wang, Y. Li, Y. Liu, N. Liu, and J. Rahmel, “SelfDefend: LLMs can defend themselves against jailbreaking in a practical manner,” in The 34th USENIX Security Symposium, 2025, pp. 2441–2460. [29] Q. Dong, L. Li, D. Dai, C. Zheng, J. Ma, R. Li, H. Xia, J. Xu, Z. Wu, B. Chang et al., “A survey on in-context learning,” in The Conference on Empirical Methods in Natural Language Processing, 2024, pp. 1107– 1128. [30] E. Perez, S. Huang, F. Song, T. Cai, R. Ring, J. Aslanides, A. Glaese, N. McAleese, and G. Irving, “Red teaming language models with language models,” in The Conference on Empirical Methods in Natural Language Processing, 2022, pp. 3419–3448. [31] Y. Gao, C. Xu, D. Wang, S. Chen, D. C. Ranasinghe, and S. Nepal, “Strip: A defence against trojan attacks on deep neural networks,” in

The 35th Annual Computer Security Applications Conference, 2019, pp. 113–125. [32] N. Varshney, W. Yao, H. Zhang, J. Chen, and D. Yu, “A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation,” arXiv preprint arXiv:2307.03987, 2023. [33] B. Wang, W. Chen, H. Pei, C. Xie, M. Kang, C. Zhang, C. Xu, Z. Xiong, R. Dutta, R. Schaeffer et al., “DecodingTrust: A comprehensive assessment of trustworthiness in GPT models,” Advances in Neural Information Processing Systems, vol. 36, pp. 31 232–31 339, 2023. [34] Z. Wei, Y. Liu, and N. B. Erichson, “Emoji attack: Enhancing jailbreak attacks against judge LLM detection,” arXiv preprint arXiv:2411.01077, 2024. [35] C. Guo, T. Goldstein, A. Hannun, and L. Van Der Maaten, “Certified data removal from machine learning models,” arXiv preprint arXiv:1911.03030, 2019. [36] B. Chen, W. Carvalho, N. Baracaldo, H. Ludwig, B. Edwards, T. Lee, I. Molloy, and B. Srivastava, “Detecting backdoor attacks on deep neural networks by activation clustering,” arXiv preprint arXiv:1811.03728, 2018. [37] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson et al., “Extracting training data from large language models,” in The 30th USENIX Security Symposium, 2021, pp. 2633–2650.