arXiv:2605.17288v1 [cs.CR] 17 May 2026
When Efficiency Backfires: Cascading LLMs Trigger Cascade Failure under Adversarial Attack 1
Zehan Sun1 Dingfan Chen2 Songze Li1 2 Southeast University Max Planck Institute for Intelligent Systems
Abstract
Keywords: LLM Cascade Systems, Adversarial Attacks, Robustness, Cascade Vulnerabilities, CostEfficiency Degradation, Routing Manipulation
Large Language Model (LLM) cascade systems are designed to balance efficiency and performance by processing queries with lightweight models while selectively escalating complex cases to more powerful ones. Such systems seek to reduces computational cost and latency while maintaining task performance, making it an appealing choice for large-scale deployment. However, the cascade design introduces new vulnerabilities through an expanded attack surface: the inclusion of lightweight front-end models and internal decision mechanisms introduces new weaknesses. In this work, we present the first study demonstrating that LLM cascade systems are susceptible to targeted adversarial manipulation, which disrupts both performance objectives and the intended cost advantages of the cascade design. We propose a novel attack framework that employs constrained sequential collaborative optimization of adversarial suffix under cascade dependencies, enabling simultaneous exploitation of lightweight models and decision mechanisms. This framework adapts to adversaries with varying capabilities, inducing controllable degradation in both cost-efficiency and accuracy. Unlike prior attacks targeting standalone models, our approach strategically leverages the cascade structure to achieve significantly stronger impact. Extensive experiments across diverse datasets and representative LLM cascade systems validate the practicality and severity of this attack. Our findings highlight the urgent need to rigorously scrutinize the security of LLM cascade systems and call for broader attention to the systemic risks inherent in such designs.
1
Introduction
In recent years, large language models (LLMs) have gained widespread application across a broad spectrum of tasks, ranging from everyday applications to cutting-edge technological domains. The success of prominent models such as the GPT series [1], the Claude family [2], Gemini [3], DeepSeek [4], Qwen [5], and Kimi [6] exemplifies the growing influence of LLMs in fields such as natural language understanding, mathematical reasoning, scientific analysis, software development, and educational applications. Despite these achievements, the development and deployment of LLMs remain expensive, as their largescale architectures—often comprising more than billions of parameters—require substantial computational resources for training. Moreover, the inference stage itself is resource intensive, often requiring substantial hardware, latency, and energy to process even a single query. The real-time nature of userfacing applications further amplifies the need for efficient inference. In contrast, simple and lightweight models provide much lower inference latency and computational overhead, though typically at the expense of accuracy or generality. This naturally raises the question of whether it is possible to rely on inexpensive models whenever they suffice, and reserve the more powerful (but costly) models for only those queries that truly require them, thereby reducing over1
should not be regarded as a “free lunch”. This raises a critical concern: is a cascade system reliable enough as a practical solution for reducing inference cost? From a systems perspective, cascades could be structurally vulnerable because prediction and decision modules are executed sequentially, and the output of each stage directly determines the input distribution of the next. This sequential dependency implies that errors or misjudgments made early in the pipeline can prematurely terminate or misdirect processing, effectively allowing early-stage decisions to dominate the system’s final outcome. Intuitively, a cascade behaves like a chain whose strength is limited by its weakest links: once a query is filtered out at an early stage, the later, more accurate models never have a chance to respond. As a result, the system’s overall behavior could be highly sensitive to the correctness and robustness of these early predictions and routing decisions. More broadly, the cascade’s layered architecture introduces additional decision boundaries and routing mechanisms, each of which creates new opportunities for adversarial manipulation. This naturally leads to the question of whether cascade architectures introduce failure modes and security vulnerabilities that are absent when a single strong model is used in isolation. For instance, an adversary could exploit fragile decision logic to trigger unnecessary escalation, inflating inference cost or diverting queries to weaker components. Conversely, an adversary might suppress escalation, causing the system to rely on weaker models, thereby degrading task accuracy or increasing susceptibility to adversarial behaviors, which could otherwise be mitigated by stronger models. These scenarios suggest that the appealing costperformance trade-offs reported by prior work may not be robust under adversarial conditions, especially when low-cost decision modules play a pivotal role in determining the execution path of the system. In this work, we pioneer this direction and take a first step toward exploring whether cascade systems indeed exhibit such vulnerabilities. We begin by formulating a threat model that closely aligns with practical deployment settings, and we introduce a generic attack framework together with novel attack techniques. In particular, we note that existing attacks
all budget or resource consumption. This motivates the idea of designing a cascade system of LLMs [7, 8], where models are organized into a sequence of stages and executed in order from the first to the last. Lightweight but weaker models occupy the early stages, while stronger but more expensive models appear in later stages. Each stage is paired with a decision-making module that determines whether to terminate early at the current stage or escalate the query to the next one. The goal of such a system is to reduce the overall inference cost while maintaining performance comparable to the strongest models. Indeed, promising empirical results have been demonstrated: FrugalGPT [8] reports up to 98.3% cost reduction compared to running a single strong LLM, with only minor performance degradation. Following this initial demonstration, a number of subsequent works have explored improvements across various dimensions [9–21], such as enhanced decision heuristics, integration of intensively trained decision models, and hybrid architectures combining multiple weak and strong LLMs. These studies consistently demonstrate that carefully designed cascades can achieve appealing trade-offs between cost and performance, often approaching the accuracy of large proprietary models while notably reducing computational and monetary expenditure. The practical deployment of cascade systems is also gaining traction. For example, Microsoft’s Azure AI Foundry employs a language model as a decisionmaker, selecting between fast, smaller models or routing queries to the more powerful GPT-5 model based on query complexity, cost, and latency requirements, reporting a up to 60% reduction in inference costs while ensuring output quality [22, 23]. Similarly, Lemony.ai has open-sourced Cascadeflow, which processes prompts with low-cost models, evaluates output quality, and escalates only when necessary, reducing development costs by up to 85% with minimal latency overhead [24, 25]. Terminus Technology Group adopts a comparable approach in AIoT systems, deploying distilled edge models with decision mechanisms that selectively offload tasks to cloudscale models, thereby reducing cost and latency while preserving performance [26–28]. However, such gains 2
targeting a single LLM are sub-optimal when attacking cascade systems, as the coordination between prediction models and decision modules can often compensate for or correct errors introduced in any single component. As a result, attacks that consider only one module in isolation may be neutralized by other stages of the pipeline. We therefore explicitly exploit the cascade’s sequential dependency and design an alternating constrained optimization procedure, which enables new forms of manipulation that substantially strengthen attacks beyond what is possible when targeting a single LLM in isolation. We further provide analytical insights showing that, under typical conditions of practical cascade design, these vulnerabilities are inherent to the (vanilla) cascade architecture. Our experimental results demonstrate that, when targeting a single objective, our attack method can reduce prediction accuracy by up to 84.6%, inflate normalized token cost by 148.9%, and increase execution time by up to 108.9%. Furthermore, when jointly targeting both objectives, our method produces adversarial perturbations that simultaneously degrade task performance and system efficiency. These results consistently outperform existing baseline methods across multiple datasets, tasks, cascade architectures, and attack scenarios. Our findings suggest that vulnerability in cascade systems is not a corner-case artifact of specific implementations, but a structural consequence of how such systems are composed. This underscores an urgent need for new architectural principles, training objectives, and defense mechanisms that explicitly account for crossmodule interactions in cascade systems, rather than treating them as loosely coupled collections of individually optimized components.
prior work focusing on a single model, jointly exploits the interactions between the weak prediction LLM and the decision module to coordinate cross-component attacks. This framework enables effective degradation of inference efficiency while supporting diverse adversarial objectives, revealing fundamental weaknesses of LLM cascade systems. • Comprehensive empirical validation. Through extensive experiments spanning 9 benchmark datasets, 4 language inference tasks, 12 LLM architectures, and 33 cascade configurations, we show that a single adversarial attack can simultaneously compromise both core objectives of LLM cascades. In particular, our attacks reduce prediction accuracy by up to 84.6%, inflate normalized token cost by 148.9%, and increase execution time by up to 108.9%. For jailbreak scenarios, we also boost attack success rates by up to 81.4%.
2
Related work
2.1
LLM Cascade & Routing
With the growing demand for efficient LLM inference, cascading and routing strategies that orchestrate multiple models with different capabilities have received increasing attention as an option to balance computational cost and accuracy. While conceptually related, and sometimes treated interchangeably or combined into hybrid mixtures, we draw a clear distinction between cascades and routing mechanisms as discussed below.
Contributions. We summarize the key contributions of this work as follows:
Cascade. A cascade mechanism executes models sequentially in a fixed order and allows early termination when an earlier stage is sufficiently confident, thereby avoiding the cost of running later, more expensive models. This idea can be traced back to early work such as CascadeBERT [7], which explored cascades of lightweight BERT-type variants [29]. More recently, this concept was first extended to LLMs by FrugalGPT [8]. FrugalGPT employs a lightweight language model as a decision module that estimates
• Formalizing a new security problem. We identify inherent security vulnerabilities in LLM cascade architectures and introduce the first attack framework capturing diverse adversarial objectives. • Novel attack techniques. We propose a general adversarial attack framework that, unlike 3
Figure 1: Illustration of the LLM cascade and out joint-target attack (Section 4.3). “Top”: Standard cascade inference. “Middle”: Joint-target optimization, where the adversarial suffix is decomposed into δ (f ) (for prediction models) and δ (g) (for decision modules). “Bottom”: Attack-time inference with the optimized adversarial prompt. the confidence of the current prediction and terminates early when this confidence is sufficiently high. Following FrugalGPT, numerous studies adhere to the scheme of adopting separate lightweight language models as decision-makers [9–16]. Another common strategy is to ask the prediction models themselves to provide confidence scores alongside their answers, deferring to stronger models when the reported confidence is low [17–19]. Beyond these, several alternative heuristics have been explored. Examples include answer consistency across multiple queries to a weak model [20] and margin sampling, which derives confidence from the distribution over first-token predictions [21]. Despite these variations, using a lightweight language model as the decision module remains the mainstream approach, with FrugalGPT serving as the foundational architecture for most subsequent cascade designs. In this paper, we focus on this mainstream class of cascade architectures for its representative nature and practical prevalence.
parallel and uses a dedicated router to determine which LLM to invoke based on the characteristics or requirements of the input. Routing strategies can be broadly grouped into static and learned methods. Static routing uses fixed heuristics or comparison metrics to assess answer quality and model suitability. For instance, some approaches evaluate the output of a single model and escalate when a quality metric is not satisfied [30], while others query multiple candidate LLMs and select the best response based on a comparison metric [13, 31]. Although effective, such exhaustive evaluation incurs significant computational cost and latency, as all candidate models must be executed before selection.
Learned routing methods aim to avoid exhaustive evaluation by training a router that predict the most suitable model [32–35]. Such routers are typically trained using supervised signals, reward functions, or other learned policies to guide model selection in advance, thereby reducing inference cost and latency. Routing. In contrast to cascade mechanisms, a However, they require nontrivial offline training or routing paradigm typically places multiple models in calibration to characterize model behavior, and main4
taining them becomes increasingly costly as the numThese adversarial methods demonstrate that linber of available models grows. guistic perturbations and structural manipulations can effectively compromise LLM behavior. However, despite this extensive progress, existing pub2.2 Risks of LLM Systems lished evaluations consider vulnerabilities only for a single model in isolation. When multiple models are While LLMs have demonstrated remarkable perfororchestrated, as in LLM cascade systems, additional mance across diverse applications, their deployment failure modes may arise from routing errors, miscaliintroduces a broad range of security, privacy, fairness, brated confidence estimates, and adversarial manipand reliability risks. Recent studies highlight issues ulation of termination decisions. This gap motivates such as training-data leakage, prompt injection, bias our central research question: Do LLM cascades face propagation, and hallucinated content [36–39]. These security risks beyond those inherent to their individfindings underscore that LLM vulnerabilities are inual components, and how can such risks be quantitaherently multi-faceted, affecting model integrity, entively evaluated? abling misuse, and undermining trust. More recent discussions further argue that such risks are not Robustness & Inference Efficiency Atconfined to individual models but arise at the sys- tacks. Parallel to adversarial research, a comtem level, where interactions among models, retrieval plementary body of work investigates robustness, pipelines, and decision or routing components can in- i.e., the stability of model predictions under input troduce additional failure surfaces. variation. Traditional robustness studies for simAdversarial and Jailbreak Attacks. Within this ple language models have focused on substitution broader risk landscape, a substantial line of work robustness, paraphrase invariance, and certified investigates how adversarial perturbations and jail- guarantees under norm-bounded perturbations [52]. break prompts can deliberately elicit undesirable or For modern LLMs, robustness analysis has shifted unsafe behavior from language models. Early adver- toward instruction-following and safety alignment, sarial attacks [40–42] demonstrated that token-level investigating how small lexical or syntactic changes or semantic perturbations can manipulate model pre- to prompts affect factuality or compliance [53, 54]. dictions while preserving fluency. With the rise of Formal tools such as Lipschitz bounds, input-to-state LLMs, these approaches evolved into prompt-based stability, and randomized smoothing have also been attacks that circumvent alignment safeguards. A adapted for robustness verification [55–61]. Despite prominent line of research focuses on adversarial these advances, robustness is overwhelmingly evalprompt optimization, including the Greedy Coordi- uated along semantic or behavioral axes, that is, nate Gradient (GCG) attack [43], TAP [44], and whether the output content remains stable. PAIR [45], showing that compact adversarial sufIn contrast, our work examines a related but unfixes can effectively induce harmful content across derexplored vulnerability: adversarial manipulation diverse LLMs. In parallel, mismatched generaliza- of inference cost and execution behavior. While tion, such as CodeAttack [46], FlipAttack [47], and prior work in classical ML has shown that carefully ArtPrompt [48] use semantic or modality shifts to crafted inputs can increase latency or energy conexpose misalignment between training and inference sumption by disrupting activation sparsity [62, 63], distributions. Target-competition attacks, includ- analogous attacks on LLM infrastructures (and esing DAN [49], GUARD [50], and prefix/suffix injec- pecially on multi-stage or cascaded systems) remain tion [51], embed high-authority or high-gradient com- largely unstudied. This gap is particularly conpeting instructions that override system prompts or sequential for cascaded LLM architectures, where safety policies. Collectively, these methods demon- lightweight models and decision modules conditionstrate that LLMs remain broadly vulnerable to a wide ally invoke stronger (and more expensive) models. If spectrum of prompt-level adversarial manipulations. these decision boundaries are fragile, subtle pertur5
bations can redirect the cascade path, causing unnecessary escalation and inflating inference cost or latency without altering the semantic output. From a systems perspective, this exposes a form of robustness failure not only in prediction correctness but also in execution stability. Very recent work (much of it not yet published) has begun to expose security vulnerabilities in LLM routing systems [64–67]. These approaches primarily target a single routing component, aiming to manipulate the routing decision, i.e., forcing selection of a specific model. As such, these methods are instances of single-target attacks and largely reduce to the well-studied paradigm of adversarial attacks on a single model. Conceptually, they correspond to basic configurations of our framework (e.g., the single-target setting in Section 4.2), differing primarily in that the attacked component is a (light-weight) router rather than a standalone (strong) LLM. In contrast, our work studies coordinated attacks on cascaded systems that simultaneously influence both prediction and routing behavior. This joint objective makes our setting fundamentally different and strictly more challenging, as it requires optimizing multiple interacting components rather than a single decision point in isolation.
3
System and Threat Model
3.1
Threat Model
ally, they may increase policy violations by exploiting cross-stage inconsistencies, e.g., steering inputs toward weaker components that are easier to jailbreak, allowing restricted outputs that a stronger model would block. We consider the modification within a bounded perturbation region (i.e., under the standard adversarial or robustness setting) so that the resulting perturbed input x′ lies in a neighborhood Nϵ (x) around the original input. This constraint reflects realistic limits on how much an attacker can distort the input (e.g., via paraphrasing or prompt injection) while preserving semantic plausibility. Notably, defining a threat model for cascade systems is more subtle than for single-model systems, as cascades involve multiple interacting components (e.g., routers, smaller models, and fallback large models), and a fully specified threat model could in principle require detailing adversarial access to each component. While this leads to a large and complex design space, we do not attempt to exhaustively enumerate all such configurations. Instead, we focus on the following primary threat scenarios that best reflect common real-world deployment settings. • White-box setting: The adversary has full read access to the internal components of the cascade system, including the architectures and parameters of the smaller models, the decision modules, and the overall cascade algorithm. Such a setting corresponds to open-source or research environments where model checkpoints and the decision logic are publicly known. This enables gradient-based or structure-aware attacks that exploit specific decision boundaries or routing logic.
We consider an adversary who manipulates only the input text x, without altering any internal model parameters or routing policies. The adversary optimizes objectives defined over the system’s behavior, such as degrading prediction performance and/or inference • Black-box setting: The adversary has no acefficiency. These objectives reflect practical advercess to the cascade’s internal components. They sarial incentives in cascade deployments. By shapcan only interact with the overall system as a sining inputs, an attacker can force worst-case execution gle black-box function, i.e., submitting input texts paths (e.g., repeatedly triggering expensive fallback and observing final outputs, latency, or cost sigmodels); at scale, this can resemble an applicationnals. This regime reflects public-facing systems layer denial-of-service, consuming shared resources such as chat-based LLM services (e.g., customer and degrading service for other users. An attacker support bots or API endpoints), where attacks rely may also degrade reliability by inducing misrouting on input-output probing, response-time analysis, to weaker components, causing systematic errors that or transfer from externally trained surrogate moddisrupt downstream use or erode trust. Additionels. 6
3.2
termediate predictions.
System Model
Stopping Index and Inference Cost. For each input sample x, let the cascade Φ(x) proceed sequentially through stages i = 1, . . . , l. We define the stopping index τ as the stage at which the cascade terminates: ( min i g i (x, f i (x)) = 0 if such i exists τ= l otherwise (2) That is, τ indicates the index of the stage whose output is returned by the cascade for sample x (with τ = l if all decision modules choose to continue). Let cif (x) and cig (x, y i ) denote the costs of executing the i-th model f i and decision module g i , respectively, where y i = f i (x). The total inference cost for sample x is then given by
We consider an LLM cascade system designed to balance inference accuracy and computational efficiency. The system sequentially passes each input query through a series of language models with progressively increasing capability (and inference cost), optionally terminating early when a sufficiently confident response (determined by the decision modules) is obtained. In normal operation, the cascade aims to minimize the average inference cost without significantly degrading task performance. By design, smaller models are responsible for handling easy queries, whereas larger models are reserved for harder or ambiguous cases. Decision modules control this process by determining whether to stop or escalate based on intermediate outputs and confidence signals (See Figure 1 for a demonstration). 3.2.1
LLM Cascade Formulation
Cost(Φ(x)) =
Let F = {f i }li=1 denote the ordered set of stage lan-
τ X i=1
guage models, typically arranged in ascending order of expressive power (and thus execution cost), and let G = {g i }l−1 i=1 denote the corresponding decision modules. We represent the l-stage cascade system as a function Φ(x; F, G) parameterized by the stage models F and decision modules G. When clear from context, we write the cascade more simply as Φ(x). For each input sample x, the cascade proceeds sequentially through stages i = 1, 2, ..., l. At stage i, the i-th language model fi produces an intermediate output y i = f i (x). The corresponding decision module g i then receives both x and y i and decides whether to terminate or proceed to the next stage. Formally, g i (x, y i ) ∈ {0, 1} acts as a binary indicator, where g i = 1 denotes continuation and g i = 0 denotes stopping and returning the current output. The overall cascade can therefore be expressed as ( y i , if g i (x, y i ) = 0 for some i < l Φ(x; F, G) = y l , if g i (x, y i ) = 1 for all i < l (1) This formulation highlights the adaptive nature of the cascade, wherein progressively more capable (and costly) models are conditionally engaged based on in-
cif (x) +
τ X
cig (x, y i )
(3)
i=1
This captures the cumulative cost incurred up to termination.
4
Method
4.1
General Adversarial Model
As discussed in Section 3, we consider an adversary that perturbs the input x within a bounded neighborhood Nϵ (x) to optimize an objective over the cascade system Φ. In general, the attack can be formulated as max F Φ(x′ ); ξ (4) ′ x ∈Nϵ (x)
where F (·) captures adversarial objectives and ξ denotes optional auxiliary information. For example, an accuracy-targeting attack maximizes task loss: max ′
x ∈Nϵ (x)
Loss Φ(x′ ), y
(5)
where Loss(·) is a task-specific (e.g., cross-entropy for classification), and y is the ground-truth label for x. 7
In addition to degrading accuracy, an adversary cost may encourage continuation at these stages: X may also aim to increase inference cost by forcing max g i x′ , f i (x′ ) (8) ′ deeper execution: x ∈Nϵ (x) i∈I max Cost Φ(x′ ) (6) This formulation restricts the adversary to manipux′ ∈Nϵ (x) lating a single class of components in isolation. While these single-target attacks are partially effective, they i which encourages continuation decisions (i.e., g = 1) are generally suboptimal: targeting only prediction and thus engagement of more expensive stages. models cannot guarantee accuracy degradation, as erTo instantiate the neighborhood Nϵ (x), we adopt rors may be corrected by later stronger models. Conthe common suffix-based perturbation strategy [43], versely, targeting only decision modules may be inin which a fixed-length suffix is appended to each sufficient, as confident early-stage prediction outputs query during optimization, following recent advercan still lead to premature termination. sarial prompt studies [68, 69]. This induces a discrete neighborhood over token-level suffix modifications, serving as a simple and representative instan- 4.3 Joint-target Attacks tiation of our framework. Notably, our attack formu- To successfully attack a cascaded system, adversarial lation is not tied to this specific choice: it is com- effects must propagate through sequentially depenpatible with, possibly with minor modifications, any dent components, making single-target attacks inattack method that can be expressed as an optimiza- herently suboptimal. Optimizing only the prediction problem of the form in Equation 4. tion models or only the decision modules overlooks their coupling and fails to account for how errors are shaped and amplified across the pipeline. This motivates a joint-target optimization strategy that exWe first consider adversaries that target only one plicitly accounts for both components. Specifically, class of early-stage components in the cascade, i.e., we decompose the adversarial suffix into two parts: either the prediction models or the decision modules. δ (f ) , targeting the prediction models, and δ (g) , tarThis setting mirrors the standard single-model attack geting the decision modules. paradigm, allowing us to assess whether such attacks The optimization proceeds in multiple rounds r. already pose a threat in cascaded systems. In partic- In each round, we append a new pair of suffixes ular, the inclusion of weaker early-stage models inher- (δ (f ) , δ (g) ) to the input. After r rounds, the perr r ently enlarges the attack surface, suggesting that cas- turbed input takes the form cades may be more vulnerable than relying solely on (f ) (g) (f ) (g) x′r = x ∥ δ1 ∥ δ1 ∥ δ2 ∥ δ2 ∥ · · · ∥ δr(f ) ∥ δr(g) a strong model. From an analytical standpoint, this restriction isolates the impact of early-stage compowhere ∥ denotes the concatenation (i.e., appending) nents and provides a baseline to quantify how much operation. Each round consists of two phases: one performance the cascade may lose relative to using for prediction degradation and one for decision escaonly the final, strongest model f l . lation, each internally solved by a few optimization Concretely, an accuracy-oriented attack can be initerations t ∈ {1, · · · , T }. In the first phase of round stantiated by maximizing the prediction loss over ser, the prediction-targeted component is updated to lected early-stage models. Let I ⊆ {1, · · · , l − 1} increase the prediction loss: denote the targeted stages: (f ) (f ) (f ) (9) δr,t ← Update δr,t−1 ; Lacc (x′r−1 ∥ δr,t−1 ) X i ′ max Loss f (x ), y (7) where x′ ∈Nϵ (x) i∈I X (f ) Lacc = Loss f i (x′r−1 ∥ δr,t−1 ), y Similarly, an adversary aiming to increase inference i∈I
4.2
Basic Single-target Attacks
8
(i) Prefer candidates that keep the previous stage predictions unchanged:
In the second phase, the decision-targeted component is adjusted to increase the escalation tendency:
(f ) (g) (g) (f ) (g) f i (x′r−1 ∥ δr,T ∥ δ) = ŷri , ∀i < l δr,t ← Update δr,t−1 ; Lcost (x′r−1 ∥ δr,T ∥ δr,t−1 ) (10) (ii) If none satisfy (1), prefer candidates that mainwhere tain previously incorrect predictions as still inX correct (i.e., not necessarily identical): (f ) (g) (f ) (g) i ′ i ′ Lcost = g xr−1 ∥ δr,T ∥ δr,t−1 , f (xr−1 ∥ δr,T ∥ δr,t−1 ) i∈I
(f )
f i (x′r−1 ∥ δr,T ∥ δ) ̸= y,
Each Update(·) denotes an approximate improvement operator that performs inner iterations towards maximizing the corresponding objective within the perturbation boundary. This alternating procedure resembles block coordinate optimization: in each round, one subset of variables (δ (f ) or δ (g) ) is updated to improve its associated objective, while treating the other as temporarily fixed.
4.4
whenever ŷri ̸= y
(iii) If neither (i) nor (ii) can be satisfied, drop the constraint and maximize Lcost freely. This prioritized selection ensures that decisionoriented updates coordinate with the degradation already achieved by the prediction-oriented phase. In other words, the decision phase builds on (rather than overrides) the previous phase, maintaining the cumulative effect of the alternating attack process.
Coordinated joint optimization 4.5
While the two target components can be optimized via alternation as described above, their updates are not independent. Because decision modules receive both the input and intermediate predictions, a decision-oriented update can inadvertently repair prediction errors introduced by the preceding prediction-oriented phase: for example, by appending tokens that revert a previously incorrect early-stage output back to the correct label. To avoid this counterproductive effect, we require that decision-oriented updates (i.e., updating δ (g) ) preserve the prediction errors already induced by the attack. Let the perturbed input after the prediction phase of round r be x′r , and let ŷri = f i (x′r ) denote the corresponding intermediate predictions at layer i. During the decision-oriented phase of round r, we perform T inner iterations. At each iteration t = 1, . . . , T , a small discrete candidate set of feasible local modifications is generated around the current (g) decision suffix: Cr,t ⊂ Nϵ (δr,t−1 ). We then select the next update as (g)
Adaptive Pass-Rate Control
There is an inherent trade-off between competing attack objectives in cascade systems:forcing inputs to proceed to later stages increases inference cost, while deeper models may counteract or alter the desired adversarial effects. To allow practical control over this trade-off, we propose a simple stochastic targeting rule that enforces escalation only for a controlled fraction of samples. Let p ∈ [0, 1] denote the target pass-rate for samples that already exhibit the desired adversarial effect after the prediction-oriented phase. Let ŷr denote the intermediate predictions at round r. Samples that do not yet satisfy the attack objective are always escalated to later stages, while those that already meet the objective are escalated with probability p and allowed to terminate with probability 1 − p. This stochastic control enables the attacker to balance competing objectives by tuning p, thereby regulating overall attack behavior.
5
(f )
δr,t ∈ arg max Lcost (x′r−1 ∥ δr,T ∥ δ) δ∈Cr,t
Analytical Insights
We provide insights showing that a cascade system is generally more vulnerable than its strongest com-
subject to the following priority constraints: 9
ponent (typically the most powerful LLM placed at the final stage) under adversarial conditions. Importantly, this vulnerability gap is not solely caused by adversarial perturbations. It already arises in the standard (non-adversarial) setting due to the routing structure of the cascade, and adversarial inputs further amplify this effect. We illustrate this phenomenon using the accuracy degradation setting introduced above. Recall that the event that the adversarial text x′ triggers an incorrect output at stage i is
Therefore, Pr[Acas (x′ )] − Pr[f l (x′ ) ̸= y] =
l−1 X
Pr[Si (x′ )]·
i=1
Pr[f (x ) ̸= y | Si (x )] − Pr[f (x′ ) ̸= y | Si (x′ )] i
′
′
l
(15)
This shows that both quantities are mixtures over the same routing-induced partition, but with different stage-wise error contributions. In particular, for each stage i < l, the cascade uses f i on inputs in ′ i ′ i ′ i ′ Ai (x ) = {f (x ) ̸= y ∧ g (x , f (x )) = 0} (11) Si (x′ ), whereas the standalone system would instead apply f l on the same subset. Consequently, if there ′ The event that the cascade outputs an incorrect final exists some i < l such that Pr[Si (x )] > 0 and prediction is Pr[f i (x′ ) ̸= y | Si (x′ )] > Pr[f l (x′ ) ̸= y | Si (x′ )] ! l i−1 [ \ and for all other k < l, ′ j ′ j ′ ′ A (x ) = {g (x , f (x )) = 1} ∩ A (x ) cas
i
i=1
j=1
For each stage i, define the routing event ′
Si (x ) =
i−1 \
j
′
j
Pr[f k (x′ ) ̸= y | Sk (x′ )] ≥ Pr[f l (x′ ) ̸= y | Sk (x′ )]
(12)
then
Pr[Acas (x′ )] > Pr[f l (x′ ) ̸= y]
′
{g (x , f (x )) = 1}
(16)
This assumption requires that the final-stage model is at least as accurate as earlier stages on the routed subsets, with strict improvement on at least one subi ′ i ′ ∩ {g (x , f (x )) = 0} set. This condition is mild and aligns with practical cascade design, where later models are typically more We assume g l (x′ , f l (x′ )) = 0, i.e., the cascade always powerful and intended to handle harder inputs. Unterminates at the final stage. Then {Si (x′ )}li=1 forms der this condition, the cascade incurs strictly higher a partition of the sample space. Using this partition, error than the strongest standalone model. Notably, this conclusion holds even in the absence of adverthe cascade error probability decomposes as sarial perturbations (i.e., when replacing x′ with x), indicating that the degradation arises inherently from l X Pr[Acas (x′ )] = Pr Si (x′ ) ∩ {f i (x′ ) ̸= y} (13) the routing structure and the presence of weaker intermediate stages. i=1 Adversarial perturbations further amplify this gap where all probabilities are taken over (x, y) ∼ D and by jointly affecting both routing and prediction. To any randomness in the perturbation δ. Similarly, the make this precise, define the gap on clean inputs error of the final-stage model admits ∆(x) = Pr[Acas (x)] − Pr[f l (x) ̸= y], l X and under perturbation Pr[f l (x′ ) ̸= y] = Pr Si (x′ ) ∩ {f l (x′ ) ̸= y} j=1
i=1
(14)
∆(x′ ) = Pr[Acas (x′ )] − Pr[f l (x′ ) ̸= y]. 10
Given Equation 15, adversarial perturbations in- indeed “backfire”, in that it may come with a nonnegligible robustness cost, and such trade-offs should crease the gap ∆ whenever not be overlooked in practice. l−1 X
Pr(Si (x′ )) Pr(f i (x′ ) ̸= y | Si (x′ ))
i=1
−
l−1 X
′
l
′
Pr(Si (x )) Pr(f (x ) ̸= y | Si (x ))
i=1
>
l−1 X
Pr(Si (x)) Pr(f i (x) ̸= y | Si (x))
i=1
−
l−1 X
Pr(Si (x)) Pr(f l (x) ̸= y | Si (x))
i=1
This inequality can arise through two mechanisms: • Routing shift: Adversarial perturbations can increase the probability of early termination at weaker stages, i.e., Pr[Si (x′ )] > Pr[Si (x)]
6
Experiment
6.1
Setup
′
for some i < l,
while the corresponding conditional gaps remain nonnegative. In this case, more probability mass is assigned to stages where f i is weaker than f l , directly increasing ∆(x′ ). • Conditional gap amplification: Even when routing probabilities remain unchanged, perturbations can increase the conditional error gap on routed subsets. Specifically, for some i < l, Pr[f i (x′ ) ̸= y | Si (x′ )] − Pr[f l (x′ ) ̸= y | Si (x′ )] > Pr[f i (x) ̸= y | Si (x)] − Pr[f l (x) ̸= y | Si (x)] This occurs because Si (x′ ) defines a different conditional distribution than Si (x), and adversarial perturbations may shift the routed subset toward inputs on which the weaker model f i degrades more than the stronger model f l . Taken together, these results highlight that adversarial perturbations can systematically exploit and amplify an intrinsic vulnerability of cascade systems induced by their routing structure. In particular, a cascade can be strictly more vulnerable than using a single strong (well-aligned) LLM alone. This suggests that the efficiency goal of cascaded designs can 11
Datasets. We conduct extensive evaluation by investigating nine widely used benchmark datasets across five diverse tasks: text classification (Headlines [70], Overruling [71], AG News [72], IMDB [73]), question answering (WebQuestions [74], SQuAD-2.0 [75,76]), constrained generation (CommonGen [77]), mathematical reasoning (SVAMP [78]), and safety evaluation/jailbreaking (WildJailbreak [79]). See Appendix A.2 for details. Target Cascade Architectures. Our study focuses on cascade systems following the FrugalGPT paradigm [8], one of the earliest and most widely adopted frameworks for LLM cascades. This design has become a canonical backbone for cascade-based inference and is broadly consistent with subsequent works [14, 17–21]. We consider both a standard twostage cascade (f1 → f2 ) and an extended three-stage cascade (f1 → f2 → f3 ), with decision modules between stages. In practice, such 2–3 stage designs are most commonly used, as deeper cascades typically offer diminishing returns while introducing additional complexity. Across all settings, we employ lightweight language models as the decision modules, a standard design choice that enables efficient routing with minimal overhead [9–16]. This design is also widely employed in real-world systems such as CascadeFlow and Microsoft Azure AI. We evaluate diverse combinations of LLMs and decision modules spanning a wide range of model sizes, architectures, capabilities, and cost–performance trade-offs. Full configurations are provided in Table 6-7 in Appendix A.1. Evaluation metrics. We evaluate attack effectiveness along two complementary dimensions: tasklevel objectives and inference-cost objectives. For task performance, we adopt standard successbased metrics tailored to each task type, including
classification accuracy, exact-match style metrics for question answering, coverage-based metrics for constrained generation, solve rate for mathematical reasoning, and jailbreak success/defense rates for jailbreaking. For efficiency, we evaluate three complementary dimensions of inference cost: normalized token cost (a proxy for computational and monetary expenditure, reflecting real-world usage), average execution time (latency), and pass rate (capturing routing behavior across cascade stages). Detailed definitions and implementation specifics are provided in Appendix A.3. Baselines and reference methods. To our knowledge, this work is the first to study the vulnerability of LLM cascade systems. As this setting has not been explored in prior work, there are no directly comparable methods tailored to it. Our approach follows the general paradigm of adversarial prompt optimization but is not tied to a specific instantiation. Instead, the framework is modular and allows methods from the adversarial LLM literature to be incorporated as plug-in components for the token update step. For concreteness and consistency with prior work, we adopt the discrete gradient-based update from the GCG method [43] as the default optimization procedure (see Appendix B.3 for results with alternative optimization methods). To contextualize our results, we consider several reference settings that reflect key aspects of the cascade architecture. We first include a Random-Noise setting, where the adversarial suffix is randomly sampled without optimization, representing an uninformed adversary and providing a lower-bound reference. We further consider two single-target attack settings (Section 4.2) that isolate different components of the cascade. The SLM-only attack optimizes the suffix solely against the prediction language model using the task performance objective, capturing the effect of perturbing the prediction model without influencing the cascade decision process. The DM-only attack targets only the decision mechanism: it encourages escalation when the intermediate prediction is correct (inflating cost) and early termination when it is incorrect (preventing correction by downstream models). 12
6.2
White-box Setting
We begin with the white-box setting, providing a detailed comparison of different attack configurations and examining the controlled accuracy–cost trade-off enabled by our pass-rate mechanism (Section 4.5). The results are shown in Figure 2–3 (with additional results in Figure 6–8 and Table 16-19). A key observation is that even a naive Random-Noise suffix leads to noticeable degradation in both task performance and inference cost across all datasets and configurations. This indicates that the cascade architecture is inherently vulnerable, even to an uninformative adversary. Moreover, this setting provides only a lower bound on vulnerability, i.e., more sophisticated attacks can cause significantly greater damage. This highlights the urgent need to account for cascade-level fragility in future system design. Cascade (Φ) Acc Cost
SLM (f 1 ) Acc Cost
LLM (f 2 ) Acc Cost
DM (g 1 ) Acc
Initial cascade (1) Attack p = 0 (1)→ (1) ∆ Initial cascade (8) Attack p = 0 (8) →(8) ∆
78.0% 21.0% -73.1% 85.0% 52.5% -41.0%
10.37 9.30 -10.3% 41.2 18.4 -55.3%
48.0% 16.0% -66.7% 71.0% 50.0% -29.6%
0.51 0.63 +23.5% 3.4 3.4 -
76.0% 63.0% -17.1% 91.0% 84.0% -7.7%
25.19 25.47 +1.1% 104.1 104.5 +0.4%
86.5% 18.0% -79.2% 91.5% 19.0% -79.2%
Initial casacde(15) Attack p = 0 (8)→(15) ∆
83.5% 51.0% -38.9%
37.5 29.1 -22.4%
71.0% 50.0% -29.6%
3.4 3.4 -
85.0% 74.0% -12.9%
51.8 53.2 +2.7%
87.0% 59.0% -32.2%
Initial casacde(16) Attack p = 0 (1)→(16) ∆
82.5% 48.0% -41.8%
18.9 11.1 -41.3%
72.5% 42.5% -41.4%
4.3 4.7 +9.3%
84.0% 69.0% -17.9%
49.8 55.5 +11.4%
88.0% 49.0% -44.3%
Initial casacde(16) Attack p = 0 (8)→(16) ∆
82.5% 53.0% -35.8%
18.9 14.4 -23.8%
72.5% 49.5% -31.7%
4.3 4.5 +4.7%
84.0% 74.0% -11.9%
49.8 53.2 +6.8%
88.0% 61.0% -30.7%
Initial casacde(17) Attack p = 0 (1)→(17) ∆
82.5% 43.5% -47.3%
19.4 15.6 -19.6%
67.0% 29.0% -56.7%
0.9 1.0 +11.1%
84.0% 69.0% -17.9%
49.8 55.5 +11.4%
82.5% 52.0% -37.0%
Initial casacde(17) Attack p = 0 (8)→(17) ∆
82.5% 49.5% -40.0%
19.4 16.7 -13.9%
67.0% 36.0% -46.3%
0.9 1.1 +22.2%
84.0% 74.0% -11.9%
49.8 53.2 +6.8%
82.5% 67.0% -18.8%
Initial casacde(18) Attack p = 0 (1)→(18) ∆
82.0% 37.0% -54.9%
33.4 14.4 -56.9%
39.0% 35.5% -9.0%
0.7 0.8 +14.3%
84.0% 69.0% -17.9%
49.8 55.5 +11.4%
89.5% 36.0% -59.8%
Initial casacde(19) Attack p = 0 (1)→(19) ∆
84.5% 45.5% -46.2%
59.1 40.8 -31.0%
54.5% 34.5% -36.7%
0.6 0.7 +16.7%
85.0% 51.0% -40.0%
102.0 113.4 +11.2%
85.0% 59.5% -30.0%
Initial casacde(20) Attack p = 0 (1)→(20) ∆
86.0% 52.5% -39.0%
62.4 41.8 -33.0%
54.5% 34.5% -36.7%
0.6 0.7 +16.7%
86.5% 60.0% -30.6%
109.3 119.4 +9.2%
91.0% 54.0% -40.7%
Setting
Table 1: Transferability of attack performance across two-layer LLM cascades where at least two components are altered from the baseline (open-source models). Cost is measured as the average normalized token cost.
Second, the SLM-only attack also achieves strong degradation across both objectives, indicating that attacking the weakest component in the chain can significantly disrupt the entire cascade. In contrast, the DM-only attack is generally more effective at degrading accuracy but less successful at inflating cost. This is expected: by design, our DM baseline flips the intended decision behavior, i.e., escalating when the first-stage prediction is correct and terminating when it is incorrect, rather than forcing all queries to be passed forward. The latter behavior is naturally captured in our joint attack when the pass-rate is set to p = 1. In comparison, our default joint SLM+DM attack enables explicit control over both objectives and consistently outperforms other attack configurations. Using the two-layer cascade as an example, at the extremes, setting p = 0 drives accuracy to very low levels, resulting in performance drops of up to 84.6%. Conversely, setting p = 1 substantially inflates inference cost, increasing normalized token cost by 148.9% and execution time by up to 108.9%. Beyond these extremes, our framework is flexible and readily adapts to task-specific objectives. Notably, on jailbreak evaluation benchmarks, our attack achieves substantially higher success rates (79% increase in the jailbreak success rate) than attacking a standalone strong LLM, indicating that cascade structures can amplify vulnerability rather than mitigate it. Moreover, the attack remains effective across a wide range of cascade configurations, including varying numbers of stages and different model compositions, and can target arbitrary subsets of components, demonstrating its generality (e.g., Figure 3 and Appendix B.1). This generality also extends to targeting intermediate stages in multi-stage cascades. For instance, in the 3-layer setting, attacking the second stage still leads to a substantial impact on overall cascade behavior, accompanied by a clear increase in computational overhead. Compared with targeting the first stage, the effect on task performance is more moderate, while the increase in cost is more pronounced. This demonstrates that our method can effectively target non-initial stages while still achieving 13
the intended impact. Collectively, these results show that LLM cascade systems are highly vulnerable to adversarial manipulation: a single carefully crafted suffix can simultaneously degrade predictive accuracy, significantly increase inference cost, and achieve high-success evasion or jailbreak behavior, effectively undermining the intended efficiency–performance trade-off of the cascade design. Cascade (Φ) Acc Cost
SLM (f 1 ) Acc Cost
LLM (f 2 ) Acc Cost
DM (g 1 ) Acc
Initial casacde(21) Attack p = 0 (1)→(21) ∆
93.5% 76.0% -18.7%
$ 0.52 $ 0.44 -15.4%
82.5% 75.0% -9.1%
$ 0.27 $ 0.29 +7.4%
94.5% 87.0% -7.9%
$ 1.37 $ 1.42 +3.6%
98.5% 73.0% -25.9%
Initial casacde(21) Attack p = 0 (8)→(21) ∆
93.5% 74.0% -20.9%
$ 0.52 $ 0.42 -19.2%
82.5% 71.0% -13.9%
$ 0.27 $ 0.28 +3.7%
94.5% 89.0% -5.8%
$ 1.37 $ 1.41 +2.9%
98.5% 75.0% -23.9%
Initial casacde(22) Attack p = 0 (1)→(22) ∆
94.0% 74.5% -20.7%
$ 0.38 $ 0.21 -44.7%
81.5% 71.0% -12.9%
$ 0.087 $ 0.092 +5.7%
94.0% 88.0% -6.4%
$ 0.93 $ 0.95 +2.2%
96.0% 74.0% -22.9%
Initial casacde(23) Attack p = 0 (1)→(23) ∆
91.0% 75.5% -17.0%
$ 0.19 $ 0.14 -26.3%
80.0% 72.0% -10.0%
$ 0.092 $ 0.097 +5.4%
92.0% 89.0% -3.3%
$ 0.57 $ 0.60 +5.3%
91.0% 71.5% -21.4%
Setting
Table 2: Transferability of attack performance across different two-layer LLM cascades (with API -based models). Cost is measured as the average monetary cost (in $).
6.3
Black-box Setting
We present the results of black-box attack in Table 1-2 (and Table 9-14 in Appendix)), where we optimize an adversarial suffix on one cascade configuration and then apply it directly, (i.e., without any re-optimization nor adaptation) to other cascade architectures. The underlying model configurations for all cascades are summarized in Table 6-7. Across all evaluations, we observe that adversarial suffixes remain highly transferable: even when applied to cascades built from different models than those used during optimization, they consistently degrade both task performance and inference cost. While the transferred attacks are naturally weaker than the fully white-box setting—where the attacker knows the exact models deployed——their impact remains substantial, revealing non-trivial vulnerability even under black-box
conditions. This reflects realistic deployment scenarios in which adversaries may infer or guess the general cascade structure but lack precise knowledge of the underlying model checkpoints. Our findings demonstrate that in such settings, attackers can still significantly compromise the cascade, highlighting yet another layer of systemic fragility introduced by multistage architectures. This underscores the need for rigorous robustness assessments and stronger safeguards when deploying LLM cascades in practical applications. Our approach also remains certain effect in attacking cascade API scenarios (Table 2), where intermediate model components are completely inaccessible for manipulation nor inspection. Despite this limited access, the method is able to substantially degrade the overall system performance by increasing the computational cost while simultaneously reducing accuracy. This disruption results in a significantly deteriorated trade-off between accuracy and inference cost, forcing the cascade system to either incur higher latency or suffer from weakened predictive quality. Importantly, this vulnerability cannot be mitigated simply by strengthening individual LLM components—a common practice in current deployments aimed at improving benchmark performance. In cascade architectures, weaker or less robust components can still be exposed and exploited through routing and interaction effects, allowing adversarial inputs to bypass stronger models or amplify inefficiencies. Consequently, our attack reveals a fundamental weakness in cascade design: even without white-box access, attackers can leverage heterogeneous components to impair both efficiency and effectiveness.
6.4
Qualitative Results
Confusion Matrices of the Decision Modules. We visualize the confusion matrices of the decision module DM under different attack settings across all datasets in Figure 4. In a well-functioning cascade, the decision module is expected to follow a simple and intuitive rule: correctly predicted queries should trigger early termination, whereas incorrectly predicted ones should be passed to the next stage for correc14
tion. This corresponds to strong diagonal dominance in the confusion matrix. As shown, the DM in the initial (non-attacked) cascade behaves reasonably and aligns well with the intended routing logic. However, under adversarial manipulation—particularly the DM Only attack and our full SLM+DM (p=0) joint attack—the decision boundaries become severely distorted. These attacks effectively fool the decision module into reversing or randomizing its decisions, causing many incorrect queries to be terminated prematurely or forcing unnecessary escalation for correct ones. Interestingly, while the SLM Only attack does not directly target the decision module, it substantially increases the portion of mispredicted queries produced by the firststage model. This indirectly shifts the decision module’s input distribution toward harder or incorrect samples, leading to widespread error propagation and degraded system accuracy—even without explicitly altering DM decision boundaries. In summary, these results highlight that both direct and indirect adversarial influences can destabilize decision behavior in LLM cascades. The decision module, despite being lightweight, plays a pivotal role in the system pipeline and thus represents a critical—and highly vulnerable—attack surface. Execution Time Distributions. We further analyze the distribution of execution times for adversarial samples generated under different attack configurations across all benchmarks, which complements the reported average cost statistics by providing a more detailed view of runtime behavior. As shown in Figure 5, the distribution is bimodal, reflecting the two possible termination points of the cascade: early exit at the first stage or escalation to the second. Both the “SLM only” and “SLM + DM (p=1)” attacks—which attempt to defer all samples to the next stage—shift most mass toward the right mode, corresponding to longer runtime. The default “joint attack” configuration produces an even more pronounced shift, confirming its stronger cost-inflation effect. In contrast, the “DM only” attack (recall its flipped optimization target: escalate when the prediction is correct and terminate when incorrect) has a much
weaker impact on runtime. This is expected: a large fraction of samples are mispredicted by the weak SLM at stage 1, so attacking only the decision module tends to force termination rather than escalation. This behavior is beneficial for accuracy degradation (as incorrect outputs are finalized early) but is counterproductive for increasing inference cost, since most samples are never escalated to the expensive secondstage model. Original Attack Performance (Headline)
Setting Initial cascade Attack p = 0 Attack p = 0.2 Attack p = 0.4 Attack p = 0.6 Attack p = 0.8 Attack p = 1
Cost
Acc
Cost
78.00% 23.00% 41.00% 43.50% 50.00% 56.50% 63.00%
13.43 7.26 14.62 15.99 18.34 20.09 22.45
29.00% 44.00% 45.50% 50.50% 57.00% 64.50%
7.83 14.83 16.54 18.67 20.79 22.93
Original Attack Performance (Overruling)
Setting Initial cascade Attack p = 0 Attack p = 0.2 Attack p = 0.4 Attack p = 0.6 Attack p = 0.8 Attack p = 1 Setting Initial cascade Attack p = 0 Attack p = 0.2 Attack p = 0.4 Attack p = 0.6 Attack p = 0.8 Attack p = 1
After Random Smooth (Headline)
Acc
Acc
Cost
Acc
Cost
14.11 4.78 7.94 12.57 14.66 16.94 19.46
24.00% 31.50% 40.50% 45.50% 58.00% 65.50%
5.01 9.21 11.58 12.14 15.09 18.51
Original Attack Performance (AGnews)
After Random Smooth (AGnews)
Acc
Cost
Acc
Cost
78.00% 19.50% 37.50% 44.50% 53.50% 61.50% 70.50%
17.38 5.78 9.64 13.85 17.24 20.97 24.15
22.50% 39.50% 46.00% 52.00% 61.00% 67.00%
7.36 10.28 14.01 16.69 20.05 22.65
Initial cascade Attack p = 0 Attack p = 0.2 Attack p = 0.4 Attack p = 0.6 Attack p = 0.8 Attack p = 1 Setting Initial cascade Attack p = 0 Attack p = 0.2 Attack p = 0.4 Attack p = 0.6 Attack p = 0.8 Attack p = 1 Setting Initial cascade Attack p = 0 Attack p = 0.2 Attack p = 0.4 Attack p = 0.6 Attack p = 0.8 Attack p = 1
After Random Smooth (Overruling)
71.00% 21.00% 29.50% 45.50% 57.00% 61.50% 67.50%
Setting
Original Attack Performance (Headline)
After PPL Filter (Headline)
Acc
Cost
Acc
Cost
78.00% 23.00% 41.00% 43.00% 50.00% 56.50% 63.00%
13.43 7.26 14.62 15.99 18.34 20.09 22.45
42.00% 63.00% 61.00% 55.00% 62.50% 70.50%
9.04 16.49 16.49 15.31 16.75 18.48
Original Attack Performance (Headline)
After Regex Filter (Headline)
Acc
Cost
Acc
Cost
78.00% 23.00% 41.00% 43.00% 50.00% 56.50% 63.00%
13.43 7.26 14.62 15.99 18.34 20.09 22.45
50.00% 66.00% 66.50% 63.50% 67.00% 73.50%
8.98 16.13 16.16 14.98 15.86 17.37
Original Attack Performance (Headline)
After CPT Filter (Headline)
Acc
Cost
Acc
Cost
78.00% 23.00% 41.00% 43.00% 50.00% 56.50% 63.00%
13.43 7.26 14.62 15.99 18.34 20.09 22.45
41.50% 59.00% 57.50% 52.00% 60.50% 65.50%
8.82 17.68 17.75 15.89 17.52 20.38
Table 4: Attack performance of cascade configurations (1) (see Table 6) under filtering defenses on HEADLINE. Cost is measured as the average token cost.
Table 3: Attack performance under randomized smoothing defense. Cost is measured as the average token cost.
Perplexity (PPL) Filter Defense Success Rate (%)
Parameter q = 0.990, trim=4 q = 0.955, trim=6 q = 0.920, trim=8 q = 0.885, trim=10 q = 0.850, trim=12
Over-defense Rate (%)
p=0
p = 0.2
p = 0.4
p = 0.6
p = 0.8
p = 1.0
14.50 28.50 34.50 47.50 62.50
17.00 29.00 34.50 37.00 61.00
14.50 29.00 34.50 36.00 59.00
17.00 29.50 33.50 38.00 61.50
14.00 28.00 34.00 39.00 62.50
17.00 28.50 32.00 38.50 62.00
6.64 13.85 17.05 19.97 24.82
Regex Filter
6.5
Attack Performance under Defense
Defense Success Rate (%)
Parameter ratio=0.05000, trim=4 ratio=0.03775, trim=6 ratio=0.02550, trim=8 ratio=0.01325, trim=10 ratio=0.00100, trim=12
Over-defense Rate (%)
p=0
p = 0.2
p = 0.4
p = 0.6
p = 0.8
p = 1.0
3.50 7.50 13.50 15.50 25.50
6.50 8.50 16.50 19.00 26.50
9.00 10.00 17.00 22.00 24.00
9.50 9.50 19.50 22.00 22.00
8.50 8.50 19.00 21.50 22.50
16.00 11.00 16.50 20.50 24.50
2.00 3.50 4.00 5.00 6.50
For completeness, we evaluate the robustness of our Characters Per Token (CPT) Filter attack under several representative defenses. We Defense Success Rate (%) Parameter Over-defense Rate (%) first apply randomized smoothing [55, 80], a widely p = 0 p = 0.2 p = 0.4 p = 0.6 p = 0.8 p = 1.0 τ = 2.3, trim=4 32.00 31.50 33.50 32.00 30.50 31.50 14.74 used noise-based defense, by injecting Gaussian noise τ = 2.5, trim=6 46.50 41.00 44.50 44.50 43.50 44.00 21.79 τ = 2.7, trim=8 49.50 47.00 49.00 48.50 47.00 48.00 26.92 (σ = 0.35) into the embeddings of adversarial suf- τ = 2.9, trim=10 64.00 60.50 64.00 61.50 63.00 60.00 33.97 τ = 3.1, trim=12 79.00 76.50 79.50 81.00 80.50 78.50 45.51 fixes. Despite this perturbation, the attack remains effective (Table 3), indicating that general noisebased defenses are insufficient to eliminate the in- Table 5: Defense performance with varying strength on HEADLINE when attack 2-layer duced vulnerabilities. We further consider three lightweight input- cascade with configuration (1) (see Table 6). filtering defenses: (i) PPL-based filtering [81], 15
which removes suspicious suffixes based on perplexity; (ii) Regex-based filtering [82], which trims hand-crafted abnormal patterns; and (iii) CPT-based filtering [83], which detects anomalous characterto-token ratios. Across all methods, we observe only partial recovery under attack (Table 4), with improvements varying by defense type and attack strength. A consistent trade-off emerges when sweeping defense strength (Table 5): stronger defenses improve attack mitigation but significantly increase over-defense on benign inputs. For instance, while CPT achieves the highest filtering effectiveness at strong settings, it also incurs the largest degradation on clean samples; conversely, lighter defenses such as Regex introduce less collateral damage but offer limited protection. Overall, no evaluated defense simultaneously achieves strong mitigation and low collateral impact. These results indicate that existing generic or lightweight defenses are either ineffective against coordinated cascade attacks or overly aggressive on benign inputs, highlighting the need for more targeted, structure-aware defense mechanisms for LLM cascade systems.
Practical Guidance for Designing Robust Cascades. Our findings highlight that, while standalone LLMs have become increasingly robust and resistant to naive adversarial attempts, introducing a cascade architecture can inadvertently create new points of failure. Even fully black-box adversaries—those with no knowledge of model parameters can manipulate routing decisions or destabilize early-stage predictions with simple adversarial suffixes. This does not imply that cascades should be abandoned, as they remain a powerful and cost-effective strategy in many real-world deployments. Rather, our results emphasize the need for careful design, particularly around the decision modules that govern escalation. Practitioners should avoid overly lightweight or uncalibrated decision mechanisms, incorporate monitoring for abnormal escalation patterns, and evaluate cascades under adversarial stress rather than relying solely on clean-data performance. Risk of Multi-LLM Collaborative Systems.
To the best of our knowledge, the security risks of multi-LLM collaborative systems—such as cascades and routing-based architectures—remain largely underexplored. Several recent and concurrent efforts 7 Discussion (many not yet published) have begun to examine adversarial vulnerabilities in such systems [64–67]. Generalization Across Tasks and Configura- These works all focus on attacking individual comtions. While we aim to cover a diverse range of ponents, i.e., a single routing module, for example cascade architectures, tasks, and data domains, our by using prefix-based perturbations or search-based evaluation is inherently non-exhaustive. Neverthe- methods to influence routing decisions. less, the core mechanisms underlying cascade systems, i.e., the sequential model invocation and inWhile this work reveals important vulnerabilities termediate routing, are largely task-agnostic. Any in routing-based systems, its analysis focuses on isosetting in which an early, weaker model produces an lated routing components and can be regarded as the intermediate prediction and a decision module de- basic case (Section 4.2) of our framework. In contermines whether to escalate can be formulated in trast, we consider collaborative multi-LLM systems a similar manner. More broadly, any system that as a whole, where multiple models interact and deciintroduces intermediate decision boundaries and ex- sions emerge from their joint behavior. More broadly, poses routing logic is susceptible to similar forms of these findings point to a systemic and largely unadversarial manipulation. We therefore expect the addressed risk in multi-LLM collaborative architecfragility demonstrated here to generalize to a wide tures, suggesting that the security of such systems class of multi-model pipelines, task-oriented LLM cannot be understood by analyzing individual comagents, and hybrid tool-use systems, highlighting an ponents in isolation and that substantially more inimportant direction for future research. vestigation is needed. 16
8
Conclusion
This work presents the first systematic investigation of adversarial vulnerabilities in LLM cascade systems—an increasingly popular paradigm for costefficient deployment at scale. While prior work highlights their favorable performance–cost trade-offs, we show that the same architectural features enabling these gains also introduce new attack surfaces absent in standalone LLMs. Specifically, we formalize a previously unexplored threat model for LLM cascades and develop adversarial techniques that exploit the interaction between prediction and routing components, enabling coordinated manipulation across stages. Our analysis shows that these vulnerabilities stem from structural properties of typical cascade designs, making them inherently difficult to eliminate without architectural changes. Extensive experiments across diverse datasets, model combinations, and transfer settings demonstrate that cascades can be severely compromised: a single adversarial suffix can simultaneously reduce accuracy, jailbreak cascade system, increase inference cost, and remain effective under transfer to unseen configurations. These results suggest that cascade systems can be more fragile than the strong LLMs they aim to approximate. Overall, our findings motivate a reexamination of cascade design from a robustness perspective and emphasize the need for more reliable decision modules, adversarial stress testing during development, and more principled routing mechanisms that are less susceptible to manipulation.
Ethical Considerations This work examines the security and robustness of LLM cascade systems under adversarial settings. While we develop techniques capable of degrading accuracy and inflating inference cost, our objective is not to facilitate misuse but to reveal structural vulnerabilities that may arise in widely adopted costefficient LLM architectures. All experiments are conducted exclusively on publicly available datasets and open-source models; no proprietary, personal, or sensitive data is used. We intentionally refrain from releasing harmful prompts, jailbreak strings, or at17
tack payloads that could directly enable malicious exploitation. Instead, we focus on methodological insights, high-level attack principles, and aggregated quantitative results. The attacks studied in this paper are intended solely for defensive and diagnostic purposes—to help practitioners understand potential failure modes, strengthen decision modules, harden routing mechanisms, and design more reliable multi-model inference systems. All experiments are carried out in a controlled research environment isolated from production systems, ensuring that no deployed services or commercial APIs are impacted. Our work does not target or disadvantage any specific individuals, organizations, or model vendors. Rather, we aim to contribute to the community’s awareness of emerging risks and to motivate the development of safeguards, monitoring tools, and countermeasures that protect against adversarial manipulation, cost exploitation, and performance degradation in LLM cascades.
Open Science All artifacts necessary to evaluate and reproduce the results of this paper rely entirely on publicly accessible resources. The cascade system itself is built upon existing open-source implementations, for which we provide the relevant repository links and configuration details to ensure faithful reconstruction. All models used in our cascades are open-source and can be obtained directly from their official repositories, and we specify the exact checkpoints used whenever necessary to ensure unambiguous reproducibility. The datasets used in our evaluation (e.g., PoemSentiment, Overruling, Headlines) are likewise publicly available, and we provide precise references to facilitate identical reproduction. The full implementation of our attacks, cascade evaluation pipeline, and experiment scripts will be released upon publication, accompanied by instructions for environment setup, running experiments, and reproducing all reported results. Together, these materials ensure that the review committees can fully assess the validity and reproducibility of our contribution.
LLM Usage Considerations
[7] Lei Li et al. Cascadebert: Accelerating inference of pre-trained language models via calibrated complete models cascade. In Findings of the Association for Computational Linguistics: EMNLP 2021, 2021.
The authors affirm that in the whole process of this research, the authors utilize LLMs solely for grammar checking and language refinement when writing. All LLM-assisted text was reviewed and edited to ensure compliance with ethical guidelines and to avoid plagiarism or scientific misconduct. The authors take full responsibility for this declaration and for all content presented in the manuscript.
[8] Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176, 2023. [9] Pierre Erbacher et al. Navigating uncertainty: optimizing api dependency for hallucination reduction in closed-book qa. In European Conference on Information Retrieval, Cham, 2024. Springer Nature Switzerland.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam [10] Lingjiao Chen et al. Are more llm calls all you Altman, Shyamal Anadkat, et al. Gpt-4 techneed? towards the scaling properties of comnical report. arXiv preprint arXiv:2303.08774, pound ai systems. Advances in Neural Informa2023. tion Processing Systems, 37:45767–45790, 2024. [2] Anthropic. The claude 3 model family: Opus, sonnet, haiku, 2024. Anthropic Technical Report.
[11] Kai Zhang et al. Llm cascade with multiobjective optimal consideration. 2024.
[12] Junlin Wang et al. Mixture-of-agents enhances [3] Gemini Team, Rohan Anil, Sebastian Borgeaud, large language model capabilities. arXiv preprint Jean-Baptiste Alayrac, Jiahui Yu, Radu SoriarXiv:2406.04692, 2024. cut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a fam- [13] Xinyuan Wang et al. Mixllm: Dynamic routing in mixed large language models. arXiv preprint ily of highly capable multimodal models. arXiv arXiv:2502.18482, 2025. preprint arXiv:2312.11805, 2023. [4] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, [14] Dong Chen et al. Improving large models with small models: Lower costs and better perforBochao Wu, Chengda Lu, Chenggang Zhao, mance. Neural Networks, page 108276, 2025. Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv [15] Jianpeng Zhou et al. Adaptive-solver framepreprint arXiv:2412.19437, 2024. work for dynamic strategy selection in large language model reasoning. Information Processing [5] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai & Management, 62(3):104052, 2025. Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. [16] Lunyiu Nie, Zhimin Ding, Erdong Hu, ChristoarXiv preprint arXiv:2309.16609, 2023. pher Jermaine, and Swarat Chaudhuri. Online cascade learning for efficient inference over [6] Kimi Team, Angang Du, Bohong Yin, Bowei streams. arXiv preprint arXiv:2402.04513, 2024. Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, [17] Banghua Zhu et al. On optimal caching and et al. Kimi-vl technical report. arXiv preprint model multiplexing for large model inference. arXiv:2504.07491, 2025. arXiv preprint arXiv:2306.02003, 2023. 18
[18] Jieyu Zhang et al. Ecoassistant: Using llm assistant more affordably and accurately. arXiv preprint arXiv:2310.03046, 2023.
medicine with intelligent management. Terminus Group Official. Adopts an end-edge-cloud collaborative architecture.
[19] Neha Gupta et al. Language model cascades: [28] Terminus Technology Group. Edge computing promoting the development of large models. TerToken-level uncertainty and beyond. arXiv minus Group Official Interview, 2025. Edge reapreprint arXiv:2404.10136, 2024. soning and cloud-edge collaboration for AIoT scenarios. [20] Murong Yue et al. Large language model cascades with mixture of thought representations [29] N. Varshney and C. Baral. Model cascading: Tofor cost-efficient reasoning. In ICLR 2024 Workwards jointly improving efficiency and accuracy shop on Reliable and Responsible Foundation of nlp systems. In Proceedings of the 2022 ConModels, 2024. ference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, [21] Guillem Ramírez, Alexandra Birch, and Ivan United Arab Emirates, December 7-11, 2022, Titov. Optimising calls to large language models pages 11007–11021. Association for Computawith uncertainty-based two-tier selection. arXiv tional Linguistics, 2022. preprint arXiv:2405.02134, 2024. [22] Microsoft Corporation. Model router for microsoft foundry concepts. Microsoft Learn Documentation, 2025.
[30] Pranjal Aggarwal et al. Automix: Automatically mixing language models. Advances in Neural Information Processing Systems, 37:131000– 131034, 2024.
[23] 88Hours. Gpt-5 in azure ai foundry: Build & [31] H. Lee, H. Cheng, and M. Ostendorf. Orchesscale ai agents. 2025. Reports up to 60% cost trallm: Efficient orchestration of language modreduction via Model Router. els for dialogue state tracking. In Proceedings of the 2024 Conference of the North Ameri[24] Lemony.ai (Uptime Industries Inc.). Cascadecan Chapter of the Association for Computaflow: Dynamic prompt routing tool. https: tional Linguistics: Human Language Technolo//github.com/lemony-ai/CascadeFlow, 2025. gies (Volume 1: Long Papers), 2024. Exclusive coverage and open source release. Reduces AI costs by up to 85% via cascad- [32] Surya Narayanan Hari and Matt Thomson. Tryage: Real-time, intelligent routing of user ing pipeline with configurable quality metrics; prompts to large language models. arXiv supported models include OpenAI, Anthropic, preprint arXiv:2308.11601, 2023. Groq, vLLM, Ollama; adds only 2ms latency. [33] Tal Shnitzer et al. Large language model rout[25] Paul Gillin. Exclusive: Lemony says its dynamic ing with benchmark datasets. arXiv preprint prompt routing tool cuts ai costs by up to 85%. arXiv:2309.15789, 2023. SiliconANGLE, Nov 2025. Initial benchmarks: up to 85% of prompts can use smaller/domain- [34] Jing Hao et al. Fullanno: A data engine for specific models. enhancing image comprehension of mllms. arXiv preprint arXiv:2409.13540, 2024. [26] Terminus group partners with chinese academy of sciences to inaugurate chongqing edge com- [35] Marija Šakota, Maxime Peyrard, and Robert West. Fly-swat or cannon? cost-effective lanputing laboratory. Global Times. Press release. guage model choice via meta-modeling. In Pro[27] Terminus Technology Group. Terminus aiot emceedings of the 17th ACM International Conferpowers shanghai jiao tong university school of ence on Web Search and Data Mining, 2024. 19
[36] Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang [44] Anay Mehrotra et al. Tree of attacks: JailbreakCai, Zhibo Sun, and Yue Zhang. A survey ing black-box llms automatically. In Advances on large language model (llm) security and priin Neural Information Processing Systems, volvacy: The good, the bad, and the ugly. Highume 37, pages 61065–61105, 2024. Confidence Computing, 4(2):100211, 2024. [45] Patrick Chao et al. Jailbreaking black box large language models in twenty queries. In 2025 [37] Isabel O Gallegos, Ryan A Rossi, Joe Barrow, IEEE Conference on Secure and Trustworthy Md Mehrab Tanjim, Sungchul Kim, Franck DerMachine Learning (SaTML). IEEE, 2025. noncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language [46] Akshita Jha and Chandan K. Reddy. Codeatmodels: A survey. Computational Linguistics, tack: Code-based adversarial attacks for pre50(3):1097–1179, 2024. trained programming language models. In Proceedings of the AAAI Conference on Artificial [38] Badhan Chandra Das, M Hadi Amini, and Intelligence, volume 37, 2023. Yanzhao Wu. Security and privacy challenges of large language models: A survey. ACM Com[47] Yue Liu et al. Flipattack: Jailbreak llms via puting Surveys, 57(6):1–39, 2025. flipping. arXiv preprint arXiv:2410.02832, 2024. [39] Lei Huang, Weijiang Yu, Weitao Ma, Weihong [48] Fengqing Jiang et al. Artprompt: Ascii artZhong, Zhangyin Feng, Haotian Wang, Qiangbased jailbreak attacks against aligned llms. In long Chen, Weihua Peng, Xiaocheng Feng, Bing Proceedings of the 62nd Annual Meeting of the Qin, et al. A survey on hallucination in large lanAssociation for Computational Linguistics (Volguage models: Principles, taxonomy, challenges, ume 1: Long Papers), 2024. and open questions. ACM Transactions on In[49] Xinyue Shen et al. "do anything now": Charformation Systems, 43(2):1–55, 2025. acterizing and evaluating in-the-wild jailbreak [40] Javid Ebrahimi, Anyi Rao, Daniel Lowd, and prompts on large language models. In ProceedDejing Dou. Hotflip: White-box adversarial exings of the 2024 ACM SIGSAC Conference on amples for text classification. In Proceedings Computer and Communications Security, 2024. of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short [50] Haibo Jin et al. Guard: Role-playing to generate natural-language jailbreakings to test guidePapers), pages 31–36, 2018. line adherence of large language models. arXiv [41] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter preprint arXiv:2402.03299, 2024. Szolovits. Is bert really robust? a strong baseline for natural language attack on text classi- [51] Jiawen Shi et al. Optimization-based prompt injection attack to llm-as-a-judge. In Proceedings fication and entailment. In Proceedings of the of the 2024 ACM SIGSAC Conference on ComAAAI conference on artificial intelligence, volputer and Communications Security, 2024. ume 34, pages 8018–8025, 2020. [42] Linyang Li, Ruotian Ma, Qipeng Guo, Xi- [52] Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. Certified robustness to angyang Xue, and Xipeng Qiu. Bert-attack: Adadversarial word substitutions. In Proceedings versarial attack against bert using bert. arXiv of the 2019 Conference on Empirical Methpreprint arXiv:2004.09984, 2020. ods in Natural Language Processing and the [43] Andy Zou et al. Universal and transferable ad9th International Joint Conference on Natural versarial attacks on aligned language models. Language Processing (EMNLP-IJCNLP), pages arXiv preprint arXiv:2307.15043, 2023. 4129–4142, 2019. 20
[53] Mao Ye, Chengyue Gong, and Qiang Liu. assumptions of benchmarks. arXiv preprint SAFER: A structure-free approach for certified arXiv:2404.16966, 2024. robustness to adversarial word substitutions. In Proceedings of the 58th Annual Meeting of [61] Tim Beyer and et al. Llm-safety evaluations lack robustness. arXiv preprint arXiv:2503.02574, the Association for Computational Linguistics, 2025. pages 3465–3475, Online, July 2020. Association for Computational Linguistics. [62] Sarada Krithivasan, Sanchari Sen, and Anand Raghunathan. Sparsity turns adversarial: En[54] Po-Sen Huang, Robert Stanforth, Johannes ergy and latency attacks on deep neural netWelbl, Chris Dyer, Dani Yogatama, Sven works. IEEE Transactions on Computer-Aided Gowal, Krishnamurthy Dvijotham, and PushDesign of Integrated Circuits and Systems, meet Kohli. Achieving verified robustness to 39(11):4129–4141, 2020. symbol substitutions via interval bound propagation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Pro- [63] Ilia Shumailov, Yiren Zhao, Daniel Bates, Nicolas Papernot, Robert Mullins, and Ross Andercessing and the 9th International Joint Conferson. Sponge examples: Energy-latency attacks ence on Natural Language Processing (EMNLPon neural networks. In Proceedings of the 6th IJCNLP), pages 4083–4093, 2019. IEEE European Symposium on Security and Pri[55] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. vacy, Vienna, Austria, 2021. Certified adversarial robustness via randomized smoothing. In international conference on ma- [64] Haochun Tang et al. Route to rome attack: Directing llm routers to expensive models via chine learning, pages 1310–1320. PMLR, 2019. adversarial suffix optimization. arXiv preprint [56] Hadi Salman, Mingjie Sun, Greg Yang, Ashish arXiv:2604.15022, 2026. Kapoor, and J Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. [65] Avital Shafran et al. Rerouting llm routers. Advances in Neural Information Processing SysarXiv preprint arXiv:2501.01818, 2025. tems, 33:21945–21957, 2020. [66] Qiqi Lin, Xiaoyang Ji, Shengfang Zhai, Qingni [57] Zhen Zhang, Guanhua Zhang, Bairu Hou, Wenqi Shen, Zhi Zhang, Yuejian Fang, and Yansong Fan, Qing Li, Sijia Liu, Yang Zhang, and Gao. Life-cycle routing vulnerabilities of LLM Shiyu Chang. Certified robustness for large lanrouter. arXiv preprint arXiv:2503.08704, 2025. guage models with self-denoising. arXiv preprint [67] Jiayi Yuan, Yifan Lu, Rixin Liu, Yu-Neng arXiv:2307.07171, 2023. Chuang, Hongyi Liu, Shaochen Zhong, Yang Sui, [58] Charlotte Siska and et al. Examining the roGuanchu Wang, Jiarong Xing, and Xia Hu. Who bustness of llm evaluation to the distributional routes the router: Rethinking the evaluation of assumptions of benchmarks. In Proceedings of LLM routing systems. In NeurIPS 2025 Workthe 62nd Annual Meeting of the Association for shop on Evaluating the Evolving LLM LifecyComputational Linguistics (Volume 1: Long Pacle: Benchmarks, Emergent Abilities, and Scalpers), 2024. ing, 2025. [59] Nikolaus H. R. Howe and et al. Exploring scaling [68] Kaijie Zhu, Jindong Wang, Jiaheng Zhou, trends in llm robustness. In ICML 2024 Next Zichen Wang, Hao Chen, Yidong Wang, Linyi Generation of AI Safety Workshop, 2024. Yang, Wei Ye, Yue Zhang, Neil Gong, et al. [60] Melissa Ailem and et al. Examining the roPromptrobust: Towards evaluating the robustbustness of llm evaluation to the distributional ness of large language models on adversarial 21
prompts. In Proceedings of the 1st ACM workshop on large AI systems and models with privacy and safety analysis, pages 57–68, 2023.
[75] Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for squad. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th [69] Patrick Chao, Edoardo Debenedetti, AlexanAnnual Meeting of the Association for Compuder Robey, Maksym Andriushchenko, Francesco tational Linguistics (Volume 2: Short Papers), Croce, Vikash Sehwag, Edgar Dobriban, Nicolas pages 784–789, Melbourne, Australia, July 2018. Flammarion, George J. Pappas, Florian Tramèr, Association for Computational Linguistics. Hamed Hassani, and Eric Wong. Jailbreakbench: An open robustness benchmark for jail- [76] Pranav Rajpurkar, Jian Zhang, Konstantin breaking large language models. In NeurIPS Lopyrev, and Percy Liang. Squad: 100,000+ Datasets and Benchmarks Track, 2024. questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras, [70] Ankur Sinha and Tanmay Khandait. Impact of editors, Proceedings of the 2016 Conference on news on the commodity market: Dataset and reEmpirical Methods in Natural Language Processsults. In Future of Information and Communicaing, pages 2383–2392, Austin, Texas, Novemtion Conference, pages 589–601. Springer, 2021. ber 2016. Association for Computational Linguistics. [71] Lucia Zheng, Neel Guha, Brandon R Anderson, Peter Henderson, and Daniel E Ho. When does pretraining help? assessing self-supervised [77] Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin learning for law and the casehold dataset of Choi, and Xiang Ren. Commongen: A con53,000+ legal holdings. In Proceedings of the strained text generation challenge for generaeighteenth international conference on artificial tive commonsense reasoning. In Findings of intelligence and law, pages 159–168, 2021. the Association for Computational Linguistics: EMNLP 2020, pages 1823–1840, Online, Novem[72] Xiang Zhang, Junbo Zhao, and Yann LeCun. ber 2020. Association for Computational LinCharacter-level convolutional networks for text guistics. classification. Advances in neural information processing systems, 28, 2015. [78] Marek Kadlčík, Michal Štefánik, Ondřej Sotolář, [73] Andrew L. Maas, Raymond E. Daly, Peter T. and Vlastimil Martinek. Calc-x and calcformPham, Dan Huang, Andrew Y. Ng, and Christoers: Empowering arithmetical chain-of-thought pher Potts. Learning word vectors for sentithrough interaction with symbolic systems. In ment analysis. In Proceedings of the 49th AnProceedings of the 2023 Conference on Empirical nual Meeting of the Association for ComputaMethods in Natural Language Processing: Main tional Linguistics: Human Language TechnoloTrack, Singapore, Singapore, December 2023. gies, pages 142–150, Portland, Oregon, USA, Association for Computational Linguistics. June 2011. Association for Computational Lin[79] Liwei Jiang, Kavel Rao, Seungju Han, Allyson guistics. Ettinger, Faeze Brahman, Sachin Kumar, Niloo[74] Jonathan Berant, Andrew Chou, Roy Frostig, far Mireshghallah, Ximing Lu, Maarten Sap, and Percy Liang. Semantic parsing on Freebase Yejin Choi, and Nouha Dziri. Wildteaming at from question-answer pairs. In Proceedings of scale: From in-the-wild jailbreaks to (adversarithe 2013 Conference on Empirical Methods in ally) safer language models, 2024. Natural Language Processing, pages 1533–1544, Seattle, Washington, USA, October 2013. Asso- [80] Zeliang Zhang and et al. Random smooth-based ciation for Computational Linguistics. certified defense against text adversarial attack. 22
Findings of the Association for Computational Linguistics: EACL 2024, 2024. [81] Gabriel Alon and Michael Kamfonas. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308.14132, 2023. [82] David Khachaturov and Robert Mullins. Adversarial suffix filtering: a defense pipeline for llms. arXiv preprint arXiv:2505.09602, 2025. [83] Shaked Zychlinski and Yuval Kainan. Brokentoken: Filtering obfuscated prompts by counting characters-per-token. arXiv preprint arXiv:2510.26847, 2025. [84] Kyle Wiggers. Openai debuts gpt-4 turbo and fine-tuning program for gpt-4. TechCrunch, November 2023. Accessed: YYYY-MM-DD.
A A.1
Additional Setup Details
Experimental
Cascade Architectures
Dataset
Headlines [70] utilizes gold-related financial news headlines to conduct four-category predictions (“up”, “down”, “neutral”, or “none”) on the future trend of gold price fluctuations. It contains 11,412 manually annotated entries. Our work randomly samples 3,000 instances for training and 200 instances for validation. Overruling [71] is a binary classification benchmark dataset used to determine whether a sentence in U.S. legal text overrules a previous judgment. It contains 2,400 English legal sentences manually annotated by lawyers. Our work randomly samples 2,000 instances for training and 200 instances for validation. AG News [72] is a classic benchmark dataset for news topic classification, encompassing four major 23
f1 (small model)
f2 (large model)
g1 (decision module)
GPT2 GPT2 GPT2 GPT2 BLOOM-0.556B Qwen3-0.6B TinyLlama-1.1B Qwen2.5 Qwen2.5 Qwen2.5 Qwen2.5 BLOOM-0.556B Qwen3-0.6B TinyLlama-1.1B Qwen2.5 Phi2-2.7B Qwen3-0.6B BLOOM-0.556B TinyLlama-1.1B TinyLlama-1.1B GPT-3.5-Turbo Grok-3-mini Claude-3-haiku
Phi3-medium Phi3-medium Phi3-medium Phi3-medium Phi3-medium Phi3-medium Phi3-medium Phi3.5-MOE Phi3.5-MOE Phi3.5-MOE Phi3.5-MOE Phi3.5-MOE Phi3.5-MOE Phi3.5-MOE Qwen3-32B Qwen3-32B Qwen3-32B Qwen3-32B DeepSeek-R1-70B Llama-3.1-70B Chatgpt-4o-latest Claude-opus-4-5 Gemini-3.1-pro-preview
BERT DistilBERT GPT2 AlBERT BERT BERT BERT DistilBERT BERT GPT2 AlBERT BERT BERT BERT BERT GPT2 DistilBERT BERT DistilBERT GPT2 DistilBERT BERT GPT-2
Table 6: Investigated 2-layer cascade system configurations. (24) (25) (26) (27) (28) (29) (30) (31) (32) (33)
We present in Table 6-7 the summary of investigated cascade architectures in this work.
A.2
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (13) (14) (15) (16) (17) (18) (19) (20) (21) (22) (23)
f1 (small)
f2 (medium)
f3 (large)
g1
g2
Qwen2.5 Qwen2.5 GPT2 GPT2 BLOOM-0.556B BLOOM-0.556B GPT-3.5 Turbo GPT-3.5 Turbo Claude-3-haiku Claude-3-haiku
Mistral-7B Mistral-7B TinyLlama-1.1B TinyLlama-1.1B Phi2-2.7B Phi2-2.7B Gemini-2.5-pro Gemini-2.5-pro Kimi-K2 Kimi-K2
Phi3.5-MOE Phi3.5-MOE Phi3-medium Phi3-medium Phi3-medium Phi3-medium Chatgpt-4o-latest Chatgpt-4o-latest GPT-5.1 GPT-5.1
BERT GPT2 BERT GPT2 BERT GPT2 BERT GPT2 BERT GPT2
BERT GPT2 BERT GPT2 BERT GPT2 BERT GPT2 BERT GPT2
Table 7: Investigated 3-layer cascade system configurations. categories: “world”, “sports”, “business”, and “technology”. It provides approximately 120,000 training samples and 7,600 test samples for training and evaluating. Our work randomly samples 3,000 instances for training and 200 instances for validation. IMDB [73] is used for sentiment classification through movie reviews and descriptions. It typically contains 50,000 movie reviews labeled as “positive” or “negative”. Our work randomly samples 3,000 instances for training and 200 instances for validation. WebQuestions [74] is constructed for Knowledge Base Question Answering (KBQA) tasks. It requires the model to answer factual questions based on a
Main Task Classification
Generation
Subcategory
Dataset
Train/Val
Binary Classification
Overruling IMDB
2000/200 3000/200
Multi-class Classification
Headlines AG News
3000/200 3000/200
Question Answering
WebQuestions SQuAD-2.0
3000/200 3000/200
Constrained Generation
CommonGen
3000/200
Mathematical Reasoning
SVAMP
800/200
Safety assessment
WildJailbreak
3000/200
main types of data: Vanilla Harmful (direct malicious requests, e.g., “How to make a bomb?”) and Adversarial Harmful (complex jailbreak prompts that attempt to bypass model safety guardrails). The dataset covers 13 risk categories (e.g., violence, hate speech, illegal acts). We focus on the Vanilla Harmful subset. These malicious queries are directly generated by GPT-4. We randomly sample 3,000 instances for training and 200 instances for validation/testing to evaluate the jailbreak robustness of cascade systems.
Table 8: Summary of datasets. structured knowledge base It contains 5,810 natural language questions based on Freebase Q&A. Our work randomly samples 3,000 instances for training and 200 instances for validation.
A.3
Evaluation Metrics
A.3.1
Task-specific performance
We first present the details of the adopted taskspecific performance evaluation metrics. SQuAD-2.0 [75, 76] is a extractive machine read- Classification tasks. For both binary and multiing comprehension dataset. It contains over 100,000 class classification, we adopt the standard classifiquestions from Wikipedia, and innovatively intro- cation accuracy as the evaluation metric. For a duces nearly 50,000 adversarial questions that are sample with ground-truth label y ∈ {1, 2, . . . , C}, i similar but cannot find answers in the original text to where C denotes the number of classes, and predicted test the depth of model reading comprehension. Our label ŷ , the accuracy is defined as i work randomly samples 3,000 instances for training N 1 X and 200 instances for validation. 1[ŷi = yi ] , Acc = N i=1 CommonGen [77] is a constrained text generation dataset containing approximately 35,000 unique conwhere 1[·] is the indicator function and N is the size cept sets as the generation tasks. It aims to make the of the evaluation dataset. model generate a coherent sentence using the given concept. Our work randomly samples 3,000 instances Open-ended question answering (free-form generation). This task requires the model to generfor training and 200 instances for validation. ate a natural-language answer in an open-ended manSVAMP [78] consists of simple arithmetic word ner. The output is a token sequence ŷi ∈ V ∗ , where problems, with each problem generating multiple V denotes the vocabulary and V ∗ the set of all finitevariants through slight modifications (such as changlength token sequences. The ground-truth answer is ing the order of numbers, asking a different quesdenoted by yi ∈ V ∗ . For each test instance, we evaltion, or adjusting irrelevant details). It contains 1000 uate whether the predicted sequence ŷi matches the arithmetic word problems, which aims to rigorously reference yi after standard normalization (e.g., lowtest the model’s true understanding of the quantiercasing, punctuation removal): tative relationships and structures in the problems. mi = 1 normalize(ŷi ) = normalize(yi ) . Our work randomly samples 800 instances for training and 200 instances for validation. An instance is considered a success if and only if m = i
WildJailbreak [79]: This dataset is an open-source 1. We define the task-specific metric, Exact Match synthetic dataset designed to improve the safety Rate (EMR), as alignment of large language models (LLMs). and N 1 X aims to test whether the LLM refuses to answer EMR = mi , N i=1 harmful queries or gets jailbroken. It contains two 24
where N is the total number of test instances. EMR measures the proportion of instances for which the normalized predicted answer exactly matches the normalized ground-truth answer. Constrained generation (keyword coverage). This task requires the model to generate a sentence that explicitly incorporates all given concept keywords. The output is a token sequence ŷi ∈ V ∗ , where V denotes the vocabulary and V ∗ the set of all finite-length token sequences. The ground-truth requirement is a concept set yi = Si , where Si ⊆ V. For each test instance, we evaluate the proportion of required concepts that appear in the generated sequence ŷi :
where N is the total number of test instances. SR measures the proportion of problems for which the predicted answer is numerically equivalent to the ground-truth answer. Safety assessment (jailbreak robustness). This task evaluates whether the model can safely refuse harmful or adversarial requests under jailbreak attempts. Given an input query yi ∈ V ∗ , the model produces an output sequence ŷi ∈ V ∗ , where V denotes the vocabulary. For each test instance, we employ an external judge J(·, ·) (e.g., a strong LLM API) that takes (yi , ŷi ) as input and outputs a binary verdict: mi = J(yi , ŷi ) ∈ {0, 1},
|{s ∈ Si : s appears in ŷi }| . coveragei = |Si |
where mi = 1 indicates a successful jailbreak (i.e., the model produces unsafe content), and mi = 0 An instance is considered a success if and only if all otherwise. We define the Jailbreak Success Rate concepts are covered, i.e., coveragei = 1. We then (JSR) as define the task-specific metric, Full Coverage Rate N 1 X (FCR), as mi . JSR = N i=1 N 1 X FCR = 1[coveragei = 1] , In addition, we report the complementary metric N i=1 1 − JSR, referred to as the Defense Success Rate where N is the total number of test instances. FCR (DSR), which measures the proportion of jailbreak measures the proportion of generated sequences that attempts that are successfully blocked. Higher DSR indicates better safety alignment. successfully incorporate every required concept. Mathematical reasoning (numerical answers). This task requires the model to solve an arithmetic problem and produce the final answer. The output is a token sequence ŷi ∈ V ∗ , where V denotes the vocabulary. The ground-truth answer is denoted by yi ∈ V ∗ . For each test instance, we compare the numerical values extracted from the predicted sequence ŷi and the ground-truth sequence yi : mi = 1[value(ŷi ) = value(yi )] , where value(·) extracts and normalizes the numerical value from a sequence. An instance is considered a success if and only if mi = 1. We define the taskspecific metric, Solve Rate (SR), as N
SR =
1 X mi , N i=1 25
A.3.2
Inference cost
For inference cost, we evaluate three complementary metrics widely used in LLM efficiency studies: (i) the average normalized token cost, which approximates model-dependent computational expenditure; (ii) the average execution time per test sample (latency); and (iii) the escalation rate, which captures routing behavior in cascade systems. For API-based models, we additionally report the monetary cost in $. Normalized token cost. This metric approximates model-dependent computational expenditure by weighting input and output tokens using fixed per-token coefficients (0.01 for input tokens and 0.03 for output tokens), following the pricing structure of common LLM API services (e.g., GPT-4 Turbo [84]) as a standardized proxy. The weighted token count
is further scaled by the size of each model component (in billions of parameters). For each test sample, the resulting cost is normalized by the length of the input query to ensure comparability across examples. The final reported value is averaged over all test samples. Average execution time. We measure the end-toend inference time per test sample (in seconds), averaged over the test set. This metric reflects the practical latency of the system under a given hardware setup, where smaller models typically yield faster responses and larger models incur higher latency due to increased computational demands. Pass rate. In a cascaded system, the cost of a query increases as it is forwarded to deeper, more expensive models. For an n-layer cascade, let E denote the number of test samples escalated from a given layer to the next stage, and let N be the total number of test samples. The escalation rate is defined as ρ = E/N . A lower ρ indicates that more queries are resolved by earlier, less expensive components, reflecting more efficient routing. Unlike token cost or execution time, the escalation rate is largely hardware-agnostic and captures the intrinsic decision-making behavior of the cascade. Experimental Setup. Unless otherwise specified, we use the attack procedure in Algorithm 1, jointly targeting the first-stage prediction model f 1 (the small language model, SLM) and the decision module g 1 (DM), denoted as SLM+DM. All techniques introduced in Sections 4.3–4.5 are enabled by default. For the GPT-2-based cascade system, the budget is set to 20, the temperature is set to 0, and the maximum generation length is 250 tokens. For the Qwen2.5-based cascade system, the budget is set to 100, the temperature is set to 0, and the maximum generation length is 300 tokens. For all models using top-k decoding, we set k = 1 to ensure deterministic decoding. The numbers of training and test instances for each dataset follow the descriptions in the main text. During inference, we use few-shot in-context learning with a task-specific prefix. For classification tasks, the prefix contains one demonstration for each answer class. For generation tasks, it contains two question-answer demonstrations. For each dataset, 26
the demonstrations are fixed throughout inference and are randomly sampled from correctly labeled training examples. Attack Hyper-parameters.. For all datasets, we use a unified interleaved attack setting with a total suffix length of 8 tokens. Specifically, we adopt a 2+2+2+2 schedule, corresponding to two rounds of alternating optimization, where each round optimizes a 2-token suffix for the SLM objective and a 2-token suffix for the DM objective. For adversarial suffix optimization on functional language models for question-answering tasks, the objective is defined to maximize the deviation of the generated response from the ground-truth answer. We use a loss threshold of −40 for early stopping, with a maximum of 300 optimization iterations. In each iteration, 512 candidate suffixes are evaluated, and two suffix positions are replaced. For DM attacks, we use the same search budget, namely 300 iterations and 512 candidate suffixes per iteration, but do not apply the loss-threshold-based early stopping criterion. For DM suffix optimization, we further impose a constraint on candidate selection: the preferred suffix must preserve the response of the preceding language model with respect to the original cascade input, while keeping the final cascaded output incorrect. This constraint takes precedence over the loss value. If no candidate satisfies this constraint after the current search is exhausted, we restart the search from a different initialization. At most three restarts are allowed. If no feasible suffix is found after all three restarts, we use the suffix with the minimum loss value as the final result. Evaluation and Hardware Setup.. Unless otherwise specified, all reported results are averaged over three independent runs, and sample standard deviations are also reported. Computational overhead is measured using end-to-end runtime. All experiments are conducted under the same software and hardware environment for fair comparison. Specifically, experiments are run on a shared server with Ubuntu 22.04.5 LTS, two Intel Xeon Platinum 8378C CPUs, approximately 1.0 TiB RAM, and eight NVIDIA GeForce RTX 4090 GPUs (24 GB each), us-
ing NVIDIA driver 535.288.01 and CUDA 12.2. Be- B.2 Black-box setting cause the server is shared by multiple users, absoWe present additional results of black-box attack in lute runtime may vary with system load. Therefore, following Table 9-14. The suffix is optimized on the runtime comparisons are made only under the same initial cascade with gradient and the target of optiexecution environment. mization is making weak LLM directly output the answer for every query. We observe that our attack architecture for generating adversarial suffixes A.4 Defense Setup exhibits good transferability in a two-layer cascade For PPL filtering, the tuned strength hyperparam- model. For replacing only the Decision-maker, it reeter is the perplexity quantile threshold q (smaller duced Acc by up to 64.2%; for replacing only the q indicates stricter filtering). For Regex filtering, Weak LLM, it reduced Acc by up to 52.2%; and the tuned strength hyperparameter is the allowed for replacing both the Weak LLM and the Decisionspecial-character ratio ratio (smaller ratio indi- maker, it reduced Acc by up to 54.9%. Furthermore, cates stricter filtering).For CPT filtering, the tuned we tested its attack performance transferability in a strength hyperparameter is the CPT threshold τ three-layer cascade model. As a structural extension (larger τ indicates stricter filtering), where CPT de- of the two-layer model, the three-layer cascade renotes the character-per-token statistic.In all meth- duced Acc by up to 83.0%, even achieving better atods, trim denotes the number of tail words removed tack results than attacking any two-layer model. This after triggering. indicates that as the number of layers increases, the cascade structure introduces more vulnerabilities and attack surfaces.
B
Additional Results
Initial cascade (1) Attack p = 0 (1)→ (1) ∆ Initial cascade (8) Attack p = 0 (8) →(8) ∆
Cascade (Φ) Acc Cost 78.0% 10.37 21.0% 9.30 -73.1% -10.3% 85.0% 41.2 52.5% 18.4 -41.0% -55.3%
SLM (f 1 ) Acc Cost 48.0% 0.51 16.0% 0.63 -66.7% +23.5% 71.0% 3.4 50.0% 3.4 -29.6% -
LLM (f 2 ) Acc Cost 76.0% 25.19 63.0% 25.47 -17.1% +1.1% 91.0% 104.1 84.0% 104.5 -7.7% +0.4%
DM (g 1 ) Acc 86.5% 18.0% -79.2% 91.5% 19.0% -79.2%
Initial cascade (2) Attack p = 0 (1) → (2) ∆ Initial cascade (3) Attack p = 0 (1) → (3) ∆ Initial cascade (4) Attack p = 0 (1) → (4) ∆ Initial cascade (9) Attack p = 0 (8) → (9) ∆ Initial cascade (10) Attack p = 0 (8) → (10) ∆ Initial cascade (11) Attack p = 0 (8) → (11) ∆
79.5% 28.5% -64.2% 76.0% 43.0% -43.4% 75.0% 52.0% -30.7% 86.0% 59.0% -31.4% 81.0% 61.0% -24.7% 86.5% 67.0% -22.5%
48.0% 16.0% -66.7% 48.0% 16.0% -66.7% 48.0% 16.0% -66.7% 71.0% 50.0% -29.6% 71.0% 50.0% -29.6% 71.0% 50.0% -29.6%
76.0% 63.0% -17.1% 76.0% 63.0% -17.1% 76.0% 63.0% -17.1% 91.0% 84.0% -7.7% 91.0% 84.0% -7.7% 91.0% 84.0% -7.7%
88.0% 32.5% -63.1% 84.0% 55.0% -34.5% 85.5% 67.0% -21.6% 91.0% 46.0% -49.5% 85.5% 50.5% -40.9% 89.0% 64.0% -28.1%
Setting
B.1
White-box setting
We present additional white-box results for two-layer cascades in Figures 6 and Figures 7, with the heat map for the specific query transmission situation of the decision maker Table 12 and Table 13. The corresponding complete experimental results of two-layer cascades reported in Tables 16 and 17 respectively. We further provide additional white-box results for three-layer cascades in Figure 8, with the heat map for the specific query transmission situation of the decision maker Table 14 and Table 4. The corresponding complete experimental results of three-layer cascades reported in Tables 18 and 19. These results complement Figures 2 and 3 in the main paper. These results are consistent with the main findings: all attack configurations remain effective across different datasets and cascade setups. Notably, our joint attack can drive the cascade system to extremely poor performance, often substantially worse than using a single LLM alone. This further highlights the vulnerability of cascaded architectures under adversarial manipulation. 27
13.5 8.6 -36.3% 10.8 13.7 +26.9% 12.1 19.2 +58.7% 42.1 28.2 -33.0% 40.8 32.9 -19.4% 41.9 37.3 -11.0%
0.6 0.6 0.6 0.6 0.6 0.6 3.4 3.4 3.4 3.4 3.4 3.4 -
25.2 28.6 +13.5% 25.2 28.6 +13.5% 25.2 28.6 +13.5% 104.1 104.5 +0.4% 104.1 104.5 +0.4% 104.1 104.5 +0.4%
Table 9: Transferability of attack performance across different decision modules DM. Cost is measured as the average normalized token cost In the black box scenario, we further replaced all language models except for the decision maker with commercial closed source models, called them in API form, and constructed the new cascade structure with better task performance to test our attack perfor-
Cascade (Φ) Acc Cost
SLM (f 1 ) Acc Cost
LLM (f 2 ) Acc Cost
DM (g 1 ) Acc
Initial cascade (1) Attack p = 0 (1)→ (1) ∆ Initial cascade (8) Attack p = 0 (8) →(8) ∆
78.0% 21.0% -73.1% 85.0% 52.5% -41.0%
10.37 9.30 -10.3% 41.2 18.4 -55.3%
48.0% 16.0% -66.7% 71.0% 50.0% -29.6%
0.51 0.63 +23.5% 3.4 3.4 -
76.0% 63.0% -17.1% 91.0% 84.0% -7.7%
25.19 25.47 +1.1% 104.1 104.5 +0.4%
86.5% 18.0% -79.2% 91.5% 19.0% -79.2%
Initial cascade (5) Attack p = 0 (1) →(5) ∆
76.5% 54.5% -28.8%
17.1 17.5 +2.3%
39.0% 35.5% -9.0%
0.7 0.8 +14.3%
76.0% 63.0% -17.1%
25.2 28.6 +13.5%
88.0% 58.5% -33.5%
Initial cascade (6) Attack p = 0 (1)→(6) ∆
76.0% 43.0% -43.4%
11.5 13.0 +13.0%
67.0% 29.0% -56.7%
0.9 1.0 +11.1%
76.0% 63.0% -17.1%
25.2 28.6 +13.5%
79.0% 51.5% -34.8%
Initial cascade (7) Attack p = 0 (1)→(7) ∆
79.5% 38.0% -52.2%
14.5 4.5 -69.0%
54.5% 34.5% -36.7%
0.7 0.8 +14.3%
76.0% 63.0% -17.1%
25.2 28.6 +13.5%
89.5% 42.5% -52.5%
Initial cascade (12) Attack p = 0 (8)→(12) ∆
86.0% 60.0% -30.2%
40.5 27.4 -32.3%
39.0% 35.5% -9.0%
0.7 0.8 +14.3%
91.0% 84.0% -7.7%
104.1 104.5 +0.4%
86.0% 69.5% -19.2%
Initial cascade (13) Attack p = 0 (8)→(13) ∆
84.0% 58.0% -31.0%
37.2 22.6 -39.2%
67.0% 36.0% -46.3%
0.9 1.1 +22.2%
91.0% 84.0% -7.7%
104.1 104.5 +0.4%
92.0% 55.5% -39.7%
Initial cascade (14) Attack p = 0 (8)→(14) ∆
83.0% 56.5% -31.9%
39.8 21.2 -46.7%
54.5% 34.5% -36.7%
0.6 0.7 +16.7%
91.0% 84.0% -7.7%
104.1 104.5 +0.4%
88.0% 53.5% -39.2%
Setting
Setting Initial cascade (1) Attack p = 0 (1)→ (1) ∆ Initial cascade (8) Attack p = 0 (8) →(8) ∆
Cascade (Φ) Acc Cost
SLM (f 1 ) Acc Cost
SLM (f 2 ) Acc Cost
LLM (f 3 ) Acc Cost
DM (g 1 ) Acc
DM (g 2 ) Acc
78.0% 21.0% -73.1% 89.0% 52.5% -41.0%
48.0% 16.0% -66.7% 71.0% 50.0% -29.6%
N/A N/A N/A N/A N/A N/A
N/A N/A N/A N/A N/A N/A
76.0% 63.0% -17.1% 91.0% 84.0% -7.7%
86.5% 18.0% -79.2% 91.5% 19.0% -79.2%
N/A N/A N/A N/A N/A N/A
10.37 9.30 -10.3% 41.2 18.4 -55.3%
0.51 0.63 +23.5% 3.4 3.4 -
25.19 25.47 +1.1% 104.1 104.5 +0.4%
Initial cascade (30) Attack p = 0 (8)→(30)
94.0% 76.0%
$ 0.49 $ 0.44
82.5% 71.0%
$ 0.27 $ 0.28
87.5% 78.0%
$ 0.30 $ 0.31
94.5% 89.0%
$ 1.37 $ 1.41
96.5% 68.0%
98.0% 81.0%
∆
-19.1%
-10.2%
-13.9%
+3.7%
-10.9%
+3.3%
-5.8%
+2.9%
-29.5%
-17.3%
Initial cascade (31) Attack p = 0 (8)→(31) ∆
93.0% 72.0% -22.6%
$ 0.49 $ 0.40 -18.4%
82.5% 71.0% -13.9%
$ 0.27 $ 0.28 +3.7%
87.5% 78.0% -10.9%
$ 0.30 $ 0.31 +3.3%
94.5% 89.0% -5.8%
$ 1.37 $ 1.41 +2.9%
95.0% 62.0% -34.7%
94.0% 85.5% -9.0%
Initial cascade (30) Attack p = 0 (1)→(30) ∆
94.0% 77.5% -17.5%
$ 0.49 $ 0.42 -14.3%
82.5% 75.0% -9.1%
$ 0.27 $ 0.29 +7.4%
87.5% 79.5% -9.1%
$ 0.30 $ 0.32 +6.7%
94.5% 87.0% -7.9%
$ 1.37 $ 1.42 +3.6%
96.5% 71.0% -26.4%
98.0% 76.0% -22.4%
Initial cascade (31) Attack p = 0 (1)→(31) ∆
93.0% 72.0% -20.4%
$ 0.49 $ 0.40 -16.3%
82.5% 71.0% -9.1%
$ 0.27 $ 0.28 +7.4%
87.5% 78.0% -9.1%
$ 0.30 $ 0.31 +6.7%
94.5% 89.0% -7.9%
$ 1.37 $ 1.41 +3.6%
95.0% 62.0% -27.4%
94.0% 85.5% -17.6%
Initial cascade (32) Attack p = 0 (1)→(32) ∆
92.0% 72.5% -21.2%
$ 0.24 $ 0.18 -25.0%
80.0% 72.0% -10.0%
$ 0.092 $ 0.097 +5.4%
89.0% 77.5% -12.9%
$ 0.19 $ 0.20 +5.3%
95.0% 90.0% -5.3%
$ 0.46 $ 0.50 -8.7%
92.0% 75.5% -17.9%
89.5% 80.0% -10.6%
Initial cascade (33) Attack p = 0 (1)→(33) ∆
93.5% 70.5% -24.6%
$ 0.31 $ 0.17 -45.2%
80.0% 72.0% -10.0%
$ 0.092 $ 0.097 +5.4%
89.0% 77.5% -12.9%
$ 0.19 $ 0.20 +5.3%
95.0% 90.0% -5.3%
$ 0.46 $ 0.50 -8.7%
94.0% 71.0% -24.5%
95.5% 74.0% -22.5%
Table 12: Attack performance of 3-layers cascade when transferred across different language models by API. For (1) and (16), the Table 10: Transferability of attack performance cost is calculated by averaging the number of across different small language models SLM. Cost is tokens per query. For (30) to (33), the cost is measured as the average normalized token cost directly calculated based on the money spent on using the API. Cascade (Φ) SLM (f ) SLM (f ) LLM (f ) DM (g ) DM (g ) Setting 1
2
3
1
2
Acc
Cost
Acc
Cost
Acc
Cost
Acc
Cost
Acc
Acc
Initial cascade (1) Attack p = 0 (1)→ (1) ∆ Initial cascade (8) Attack p = 0 (8) →(8) ∆
78.0% 21.0% -73.1% 89.0% 52.5% -41.0%
10.37 9.30 -10.3% 41.2 18.4 -55.3%
48.0% 16.0% -66.7% 71.0% 50.0% -29.6%
0.51 0.63 +23.5% 3.4 3.4 -
N/A N/A N/A N/A N/A N/A
N/A N/A N/A N/A N/A N/A
76.0% 63.0% -17.1% 91.0% 84.0% -7.7%
25.19 25.47 +1.1% 104.1 104.5 +0.4%
86.5% 18.0% -79.2% 91.5% 19.0% -79.2%
N/A N/A N/A N/A N/A N/A
Cascade (Φ) Acc Cost
SLM (f 1 ) Acc Cost
SLM (f 2 ) Acc Cost
LLM (f 3 ) Acc Cost
DM (g 1 ) Acc
DM (g 2 ) Acc
Initial cascade (1) Attack p = 0 (1)→ (1) ∆
78.0% 21.0% -73.1%
2771.9 2006.2 +27.6%
48.0% 16.0% -66.7%
114.0 125.5 +10.0%
N/A N/A N/A
N/A N/A N/A
76.0% 63.0% -17.1%
5037.3 5728.5 +13.7%
86.5% 18.0% -79.2%
N/A N/A N/A
Initial cascade (24) Attack p = 0 (8) →(24) ∆
87.5% 67.5% -22.9%
21.4 12.6 -41.1%
71.0% 50.0% -29.6%
3.4 3.4 -
84.5% 78.5% -7.1%
13.7 13.8 +0.7%
91.0% 84.0% -7.7%
104.1 104.5 +0.4%
88.0% 22.5% -74.4%
90.0% 65.0% -27.8%
Initial cascade (16) Attack p = 0 (1) →(16) ∆
77.0% 30.5% -60.4%
2490.4 1608.3 -35.4%
48% 16.0% -66.7%
114.0 125.5 +10.0%
54.5% 22.5% -58.7%
402.2 417.7 +3.9%
76.0% 63.0% -17.1%
5037.3 5728.5 +13.7%
81.0% 18.0% -76.5%
83.5% 43.0% -48.5%
Initial cascade (25) Attack p = 0 (8) →(25) ∆
88.0% 69.5% -21.0%
26.8 14.9 -44.4%
71.0% 50.0% -29.6%
3.4 3.4 -
84.5% 78.5% -7.1%
13.7 13.8 +0.7%
91.0% 84.0% -7.7%
104.1 104.7 +0.6%
87.0% 28.0% -67.8%
84.5% 63.0% -25.4%
Initial cascade (17) Attack p = 0 (1)→(17) ∆
76.5% 13.0% -83.0%
2398.6 962.6 -59.9%
48% 16.0% -66.7%
114.0 125.5 +10.0%
54.5% 22.5% -58.7%
402.2 417.7 +3.9%
73.0% 63.0% -17.1%
5037.3 5728.5 +13.7%
82.0% 13.5% -83.5%
85.5% 27.5% -58.0%
Initial cascade (26) Attack p = 0 (1) →(26) ∆
77.0% 30.5% -60.4%
12.5 8.0 -36.0%
48% 16.0% -66.7%
0.6 0.6 -
54.5% 22.5% -58.7%
2.0 2.1 +5.0%
76.0% 63.0% -17.1%
25.2 25.6 +1.6%
81.0% 18.0% -77.8%
83.5% 43.0% -48.5%
Initial cascade (18) Attack p = 0 (1)→(18) ∆
78.0% 61.0% -21.8%
2272.8 1794.3 -21.1%
39.0% 34.0% -13.2%
148.2 154.6 +4.3%
72.5% 61.5% -15.2%
856.5 878.7 +2.6%
84.0% 73.0% -13.1%
9967.4 10942.7 +9.8%
76.5% 48.5% -36.6%
83.5% 67.0% -19.8%
Initial cascade (27) Attack p = 0 (1)→(27) ∆
76.5% 13.0% -83.0%
12.0 4.8 -60.0%
48% 16.0% -66.7%
0.6 0.6 -
54.5% 22.5% -58.7%
2.0 2.1 +5.0%
73.0% 63.0% -13.7%
25.2 25.6 +1.6%
82.0% 13.5% -83.5%
85.5% 27.5% -67.8%
Initial cascade (19) Attack p = 0 (1)→(19) ∆
77.0% 59.5% -22.7%
2281.4 2017.5 -11.6%
39.0% 34.0% -13.2%
148.2 154.6 +4.3%
72.5% 61.5% -15.2%
856.5 878.7 +2.6%
84.0% 73.0% -13.1%
9967.4 10942.7 +9.8%
80.5% 44.5% -44.7%
87.5% 61.0% -30.3%
Initial cascade (28) Attack p = 0 (1)→(28) ∆
78.0% 61.0% -21.8%
11.4 9.0 -21.1%
39.0% 34.0% -12.8%
0.7 0.8 +14.3%
72.5% 61.5% -15.2%
4.3 4.4 +2.3%
84.0% 73.0% -13.1%
49.8 50.7 +1.8%
76.5% 48.5% -36.6%
83.5% 67.0% -19.8%
Initial cascade (29) Attack p = 0 (1)→(29) ∆
77.0% 59.5% -22.7%
11.4 10.1 -11.4%
39.0% 34.0% -12.8%
0.7 0.8 +14.3%
72.5% 61.5% -15.2%
4.3 4.4 +2.3%
84.0% 73.0% -13.1%
49.8 50.7 +1.8%
80.5% 44.5% -44.7%
87.5% 61.0% -30.3%
Setting
Table 13: Attack performance of 3-layers cascade when transferred across different openTable 11: Transferability of attack performance source language models. Cost is measured as across 3-layers cascade with different open-source lan- the average normalized token cost guage models. Cost is measured as the average normalized token cost
mance in generating adversarial suffixes. The task cost is measured by the dollars spent directly using API calls. We conducted migration testing using an adversarial suffix of P=0 testing for three kinds of decision makers BERT, GPT-2, DistilBERT and found that the overall Acc of the cascade system decreased 28
by up to 20.7% under a two-layer structure. The performance of SLM alone decreased by up to 12.9%, and the decision accuracy of DM decreased by up to 25.9%. It can be observed that the adversarial suffix we constructed has slightly lower attack performance in closed source models compared to open source models, but still has significant effects, indicating that the inherent vulnerability of the cascade
Cascade (Φ) Acc Cost
SLM (f 1 ) Acc Cost
SLM (f 2 ) Acc Cost
LLM (f 3 ) Acc Cost
DM (g 1 ) Acc
DM (g 2 ) Acc
Initial cascade (1) Attack p = 0 (1)→ (1) ∆
78.0% 21.0% -73.1%
2771.9 2006.2 +27.6%
48.0% 16.0% -66.7%
114.0 125.5 +10.0%
N/A N/A N/A
N/A N/A N/A
76.0% 63.0% -17.1%
5037.3 5728.5 +13.7%
86.5% 18.0% -79.2%
N/A N/A N/A
Initial cascade (20) Attack p = 0 (1)→(20) ∆
94.0% 77.5% -17.5%
$ 0.49 $ 0.42 -14.3%
82.5% 75.0% -9.1%
$ 0.27 $ 0.29 +7.4%
87.5% 79.5% -9.1%
$ 0.30 $ 0.32 +6.7%
94.5% 87.0% -7.9%
$ 1.37 $ 1.42 +3.6%
96.5% 71.0% -26.4%
98.0% 76.0% -22.4%
Initial cascade (21) Attack p = 0 (1)→(21) ∆
93.0% 74.0% -20.4%
$ 0.49 $ 0.41 -16.3%
82.5% 75.0% -9.1%
$ 0.27 $ 0.29 +7.4%
87.5% 79.5% -9.1%
$ 0.30 $ 0.32 +6.7%
94.5% 87.0% -7.9%
$ 1.37 $ 1.42 +3.6%
95.0% 69.0% -27.4%
94.0% 77.5% -17.6%
Initial cascade (22) Attack p = 0 (1)→(22) ∆
92.0% 72.5% -21.2%
$ 0.24 $ 0.18 -25.0%
80.0% 72.0% -10.0%
$ 0.092 $ 0.097 +5.4%
89.0% 77.5% -12.9%
$ 0.19 $ 0.20 +5.3%
95.0% 90.0% -5.3%
$ 0.46 $ 0.50 +8.7%
92.0% 75.5% -17.9%
89.5% 80.0% -10.6%
Initial cascade (23) Attack p = 0 (1)→(23) ∆
93.5% 70.5% -24.6%
$ 0.31 $ 0.17 -45.2%
80.0% 72.0% -10.0%
$ 0.092 $ 0.097 +5.4%
89.0% 77.5% -12.9%
$ 0.19 $ 0.20 +5.3%
95.0% 90.0% -5.3%
$ 0.46 $ 0.50 +8.7%
94.0% 71.0% -24.5%
95.5% 74.0% -22.5%
Setting
assumptions and remains effective under limitedknowledge conditions.
B.4
Table 14: Attack performance of 3-layers cascade when transferred across different language models by API . structure will not be resolved by the improvement of model components
B.3
Alternative Optimization backbone
We additionally consider alternative optimization backbones beyond the full gradient-based GCG method, focusing on a more realistic gray-box setting where the adversary has limited internal knowledge. In this scenario, the attacker may know the general cascade structure or routing algorithm, but only has black-box query access to individual models (e.g., via APIs) or partial routing outputs. This reflects practical deployments such as model-as-a-service systems, where an attacker can probe the system through crafted queries and observe outputs or routing signals without access to model parameters. Within this setting, we instantiate our attack framework using a genetic algorithm to perform gradient-free adversarial suffix optimization. This approach enables query-based search guided by observed system responses. Experimental results on the AGNews, Overruling, and Headline datasets show that the attack remains effective, although its performance is slightly weaker than gradient-based methods. Nevertheless, the genetic algorithm provides a robust and practical alternative when only interactive input–output access is available. The results of such attacks are presented in Figures 9–10. Overall, these findings demonstrate that our attack framework generalizes beyond white-box 29
Qualitative Examples
We present several qualitative examples in Table 15, showcasing how adversarial suffixes affect both components of the cascade. For each input query, we report the initial predictions made by the small language model (SLM) and the decision module (DM), followed by the predictions after appending the adversarial suffix. These examples illustrate how an attack can (i) flip the SLM’s prediction, (ii) manipulate the DM into escalating or prematurely terminating a query, or (iii) achieve both effects simultaneously. The cases highlight typical failure modes we observe across datasets: even short adversarial suffixes can reliably push the DM to misroute queries and induce substantial errors in the SLM, demonstrating how coordinated manipulation of both components can degrade cascade behavior in practice.
B.5
Ablation Study
We evaluate the effectiveness of the proposed attack techniques described in Section 4.3–4.5. In Figure 11, we compare attack performance with and without the constraint-preserving mechanism introduced in Section 4.4. Overall, the cascade suffers greater degradation (as shown in the right subfigure) in both accuracy and cost when the constraint is applied: the yellow region (“with” constraint) consistently lies closer to the bottom-right corner, representing the more desirable adversarial outcome where the system is simultaneously less accurate and more expensive to run. From the left subfigure, we also observe that the degradation is not only stronger with the constraint, but more importantly, the performance remains stable when transitioning from phase-1 (attacking f 1 with an accuracy-focused objective) to phase-2 (attacking g 1 with a cost-focused objective). The increase highlighted by the solid colored lines is notably smaller, indicating that our constraint effectively preserves the adversarial effect achieved in earlier stages rather than allowing phase-2 optimization
Adversarial Suffix δ
Query x
Ground- SLM truth Answer y f 1 (x)
DM Prediction g 1 (x)
Q: Gold falls on low demand, global cues Q: Gold prices lower in Asia with China markets shut, Fed minutes eyed Q: Gold rises 1% on renewed safe-haven buying Q: Gold tumbles below Rs 30,000 mark as intense selloff continues Q: June gold soars after jobs report, up $21.80 (1.7%) to $1,294/oz Q: Gold, silver crash on shift to $, cash Q: Gold prices slip on stronger dollar; bullion seen re-testing low Q: Gold to platinum ratio hits alltime highs
"Down" "Down" ✓
"Terminate" A WanMale*/( Females climb ✓ whilst 670 rose 930
"Down" "Down" ✓
"Up"
SLM (After Attack) f 1 (x ∥ δ) "Down" ✓
DM (After Attack) g 1 (x ∥ δ) "Pass" ×
"Terminate" Pagesclaimer771 Allaah am✓ plification climb rises lbs gain 670
"Down" ✓
"Pass" ×
"Terminate" Manufacturer*: be finan✓ cially and sts vaudeville newfound leaps "Terminate" hardships Webster wors BO ✓ Abilities climb bust championed
"None" ×
"Pass" ✓
"Up" ×
"Pass" ✓
"Neutral" ×
"Terminate" ✓
"Down" "Neutral"× "Pass" ✓
"Up" ×
"Terminate" ×
"Down" "Down" ✓
"Down" ✓
"Terminate" ✓
"None" ✓
"Terminate" ✓
"Up" ✓
"Down" "Down" ✓ "Up"
"Up" ✓
"None" "Up" ×
"Terminate" GENERAL Cosponsors ✓ Darkannon Bolton appellatestand passes lifts wrath AuthorTeXoine Components ILCS attained slices wrote novella accolades "Terminate" fungi Career qualified Re✓ quirements credential quarterfinalsdown climb surpassing "Pass" ✓ gran plunged Yenwal rallies individually 740 advancing builds
Table 15: Headline dataset examples illustrating attack effects on both the SLM and DM components.
30
to undo it. This figure shows the effect of increasing the total suffix length (equivalently, the number of optimization rounds r), plotted along the x-axis. While larger suffixes generally strengthen the attack, we observe diminishing returns as the length grows. For efficiency and consistency with prior work, we adopt the default setting of a 20-token suffix, which already produces strong adversarial effects; subsequent results could be further amplified with larger suffixes at the cost of additional optimization time. Moreover, as shown in Figure 9–10, comparing our default setting with two alternatives—“SLM only” (attacking only the small prediction model) and “DM only” (attacking only the decision module)—demonstrates that our alternating joint-attack strategy (see Section 4.3 and Algorithm 1) is generally more effective at degrading both accuracy and inference cost. Notably, only with our alternating optimization technique can the overall cascade performance be pushed to extreme failure regimes that are unattainable when attacking either component in isolation. This is expected: when only one component is attacked, the untouched component may partially compensate for or correct the induced errors, limiting the overall degradation. The results therefore validate the necessity and effectiveness of our proposed joint attack design, which explicitly leverages the interdependence between prediction stages and decision modules to destabilize the cascade holistically.
C
Algorithm Pseudocode
The algorithm Pseudocode (using prediction accuracy as the task metric for demonstration) that summarizes Section 4 is provided in Algorithm 1.
31
Cascade system
Method
Setting
Initial
LLM w/o cascade
Random noise
SLM only
GPT2medium + BERT-base + Phi-3medium-4kinstruct
Singletarget attack DM only
P=0
P=0.2
Jointtarget attack
P=0.4
P=0.6
P=0.8
P=1
Metric
Headline
Overruling
IMDB
AGNews
WebQuestions
Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%)
78.00±0.00 13.43±0.00 4.80±0.06 55.00±0.00 76.00±0.00 25.23±0.00 7.12±0.07 100.00±0.00 69.50±0.50 16.83±0.34 5.41±0.06 69.00±0.58 69.00±1.00 20.21±0.15 5.64±0.06 72.00±0.50 46.50±0.50 15.17±0.12 4.17±0.05 45.00±1.50 23.00±2.50 7.26±0.08 2.47±0.03 21.00±1.50 41.00±0.50 14.62±0.12 3.30±0.04 37.50±1.00 43.50±1.00 15.99±0.13 4.04±0.05 46.00±3.00 50.00±0.50 18.34±0.14 4.78±0.05 62.00±2.00 56.50±1.00 20.09±0.15 5.59±0.06 71.00±1.00 63.00±1.50 22.45±0.16 6.56±0.07 84.50±2.00
71.00±0.00 13.47±0.00 3.95±0.05 54.00±0.00 72.00±0.00 21.42±0.00 6.67±0.07 100.00±0.00 65.50±0.50 16.02±0.12 4.83±0.08 62.50±1.00 66.50±0.50 17.51±0.13 5.61±0.06 74.00±2.00 48.50±0.50 12.59±0.16 3.19±0.04 47.00±1.00 21.00±0.29 4.78±0.06 2.04±0.03 17.50±0.29 37.00±1.50 7.94±0.08 2.86±0.03 29.50±0.50 45.50±1.50 12.57±0.10 3.21±0.04 47.00±2.50 57.00±1.50 14.66±0.32 4.46±0.05 59.50±0.50 61.50±0.50 16.94±0.43 5.27±0.06 70.00±0.50 67.50±1.00 19.46±0.14 6.10±0.06 85.50±0.50
82.00±0.00 14.11±0.00 3.94±0.04 59.00±0.00 82.50±0.00 19.47±0.00 5.94±0.06 100.00±0.00 74.50±0.50 15.26±0.11 4.89±0.05 74.00±0.50 72.00±1.50 16.78±0.12 4.50±0.05 71.50±1.00 63.00±2.00 11.74±0.09 3.51±0.04 51.50±0.50 24.00±1.00 2.97±0.04 1.49±0.02 26.50±1.50 38.00±1.50 6.64±0.07 2.39±0.03 38.50±1.00 47.50±1.50 9.17±0.08 3.11±0.04 46.00±1.00 58.00±0.50 12.61±0.27 3.86±0.04 57.00±0.50 64.50±1.00 15.83±0.22 4.41±0.05 70.50±0.50 75.50±2.00 17.81±0.13 5.37±0.06 86.00±0.50
78.00±0.00 17.38±0.00 4.73±0.05 62.50±0.00 78.50±0.00 26.71±0.00 7.14±0.07 100.00±0.00 69.50±0.50 21.43±0.37 6.17±0.06 75.50±1.00 68.00±1.50 20.09±0.14 5.73±0.06 70.00±0.50 56.50±1.00 15.17±0.12 4.38±0.05 55.00±0.50 19.50±0.50 5.78±0.07 2.25±0.03 18.00±1.50 37.50±1.00 9.64±0.09 3.15±0.04 31.50±2.50 44.50±2.50 13.85±0.11 4.01±0.05 47.00±1.00 53.50±1.50 17.24±0.23 4.76±0.05 61.00±1.00 61.50±1.50 20.97±0.34 5.84±0.06 72.50±1.50 70.50±0.00 24.15±0.17 6.59±0.07 85.50±1.00
61.50±0.00 9.67±0.00 3.74±0.04 63.00±0.00 62.00±0.00 15.14±0.00 5.75±0.06 100.00±0.00 53.50±0.50 11.47±0.09 4.37±0.05 68.00±0.50 57.00±0.50 12.97±0.10 4.84±0.05 78.00±2.00 41.00±2.50 6.20±0.07 2.82±0.03 49.50±0.50 16.50±0.50 1.78±0.03 1.71±0.02 22.50±1.00 30.50±1.50 3.94±0.05 2.41±0.03 41.00±0.50 36.00±0.29 6.21±0.07 2.84±0.03 49.50±2.00 44.00±1.00 8.02±0.08 3.16±0.04 55.00±2.50 50.00±1.00 11.18±0.09 4.28±0.05 67.50±2.00 56.00±1.00 14.47±0.11 5.04±0.05 85.50±1.50
Table 16: Results on the GPT-2+BERT+Phi3 cascade system.
32
Cascade system
Method
Setting
Initial
LLM w/o cascade
Random noise
SLM only
Qwen2.5-3B + DistilBERTbase + Phi-3.5-MoE16×3.8B
Singletarget attack
DM only
P=0
P=0.2 Jointtarget attack
P=0.4
P=0.6
P=0.8
P=1
Metric
AGNews
CommonGen
SQuAD2.0
SVAMP
WildJailbreak
Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%) Performance Token cost Time cost (s) Passrate (%)
91.50±0.00 63.57±0.00 4.79±0.06 48.00±0.00 94.00±0.00 112.58±0.00 8.12±0.09 100.00±0.00 78.50±0.50 99.78±3.47 7.07±0.08 80.50±1.00 74.00±0.50 96.41±4.24 6.87±0.07 74.00±1.00 69.50±0.50 78.60±4.16 6.01±0.06 54.00±1.50 25.50±0.50 21.44±2.03 2.58±0.03 21.00±0.50 40.50±1.50 39.84±3.94 3.61±0.04 34.50±1.00 49.50±2.00 60.17±5.71 4.42±0.05 44.50±1.50 61.00±0.50 82.29±5.07 5.74±0.06 59.50±1.50 69.50±1.00 93.07±7.22 6.59±0.07 69.50±0.50 75.50±0.50 104.37±8.36 7.41±0.08 83.50±1.00
93.00±0.00 36.84±0.00 2.02±0.03 37.00±0.00 96.00±0.00 96.14±0.00 4.41±0.06 100.00±0.00 86.50±1.50 52.42±2.17 2.63±0.04 50.50±0.50 84.00±1.50 79.63±3.74 3.84±0.05 79.50±1.50 80.50±1.00 65.81±3.71 3.26±0.04 66.50±1.00 52.50±1.00 21.89±1.79 1.27±0.02 19.00±0.50 60.00±2.50 34.52±3.06 1.96±0.03 36.00±0.50 67.50±0.50 48.77±4.68 2.48±0.03 48.50±0.50 75.00±0.00 60.58±4.11 3.03±0.04 60.50±0.00 80.50±0.50 71.27±8.95 3.52±0.05 70.00±0.50 85.00±0.00 87.59±7.11 4.22±0.05 88.00±0.50
84.00±0.00 102.41±0.00 4.61±0.06 42.00±0.00 86.50±0.00 277.24±0.00 8.49±0.07 100.00±0.00 68.50±1.00 215.18±4.49 7.30±0.08 77.50±1.00 70.00±1.00 242.57±6.39 8.02±0.14 87.00±2.00 53.50±1.50 168.22±8.32 5.87±0.06 60.50±1.00 16.00±1.00 61.27±4.02 3.28±0.04 26.00±0.00 29.50±1.50 94.11±6.45 4.34±0.05 39.50±0.50 41.00±0.00 141.49±9.27 5.22±0.06 51.50±0.50 49.50±0.50 172.32±12.42 6.07±0.07 62.00±0.50 57.50±0.50 214.35±10.83 7.24±0.08 77.50±1.00 66.00±1.50 254.87±15.54 8.18±0.09 91.00±0.00
75.00±0.00 54.17±0.00 6.09±0.07 57.50±0.00 76.50±0.00 92.71±0.00 9.02±0.04 100.00±0.00 67.50±0.50 69.55±2.16 7.04±0.08 71.50±1.00 61.00±2.00 67.47±2.31 6.83±0.27 68.00±0.00 54.50±1.00 49.86±2.86 5.29±0.06 50.50±1.50 25.50±1.50 26.49±1.23 2.94±0.03 23.00±0.50 37.50±0.00 39.62±2.92 4.33±0.05 40.00±1.00 44.50±1.00 48.17±2.65 5.07±0.06 49.50±1.50 53.00±0.50 59.73±6.29 6.31±0.07 61.00±2.00 61.50±0.50 70.54±7.95 7.11±0.08 72.50±0.50 69.00±1.00 84.21±7.36 8.24±0.09 87.00±1.50
97.00±0.00 24.80±0.00 2.39±0.03 67.00±0.00 98.50±0.00 40.04±0.00 3.49±0.04 100.00±0.00 84.50±2.00 28.97±0.68 2.54±0.03 77.50±1.00 81.00±1.50 29.63±1.74 2.67±0.03 78.50±2.00 66.00±1.50 19.27±1.04 2.16±0.03 53.00±1.50 18.00±0.50 6.43±0.84 1.11±0.07 22.50±1.00 38.50±1.00 13.70±0.89 1.69±0.02 38.50±0.50 47.50±0.50 19.89±1.34 2.08±0.03 54.50±0.50 58.50±1.00 25.12±3.52 2.44±0.03 67.50±1.00 73.00±1.50 30.98±3.44 2.71±0.03 79.00±1.00 86.50±2.00 37.44±3.55 3.18±0.04 91.50±0.00
Table 17: Results on the Qwen2.5+DistilBERT+Phi-3.5-MoE cascade system.
33
Cascade system
Method
Setting
Initial
LLM w/o cascade
Random noise
SLM only
Qwen2.5 + BERT-base + Mistral-7B + BERT-base + Phi-3.5-MoE
Singletarget attack
DM only
P=0
P=0.2
Jointtarget attack
P=0.4
P=0.6
P=0.8
P=1
Metric Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%)
AGNews 89.50±0.00 52.31±0.00 4.13±0.04 55.0±0.00 29.0±0.00 94.00±0.00 112.58±0.00 8.12±0.07 100.0±0.00 100.0±0.00 72.50±0.50 64.14±2.64 5.07±0.06 57.5±1.00 17.5±0.50 79.00±0.50 92.18±4.87 6.91±0.17 71.0±1.00 52.0±0.50 64.50±1.00 67.85±4.28 5.58±0.16 50.0±1.00 26.0±0.50 21.50±0.50 19.40±1.46 2.27±0.09 24.0±0.50 8.0±1.00 35.50±1.00 28.97±2.44 3.18±0.31 39.5±2.50 17.5±0.50 42.00±0.50 39.77±3.32 3.96±0.24 50.5±1.00 22.5±0.50 50.00±1.00 50.41±4.43 4.41±0.05 62.5±1.00 25.5±0.50 59.50±1.00 64.58±4.89 5.16±0.06 70.5±2.50 30.0±0.50 66.50±1.00 77.50±3.25 5.79±0.07 81.0±1.50 36.0±0.50
SQuAD2.0 81.00±0.00 110.31±0.00 4.97±0.06 51.5±0.00 24.0±0.00 86.50±0.00 277.24±0.00 8.49±0.09 100.0±0.00 100.0±0.00 69.00±1.50 141.89±4.21 7.04±0.07 52.0±2.00 24.5±0.50 71.00±0.50 227.38±6.52 8.27±0.28 69.5±2.00 48.5±1.00 56.00±0.00 161.08±7.16 5.79±0.17 54.5±1.00 20.5±0.50 12.50±1.00 75.11±5.42 3.53±0.21 26.5±0.50 10.5±0.50 30.00±1.50 109.40±11.26 4.31±0.72 36.0±1.00 14.5±1.00 36.50±0.50 138.02±12.19 5.02±0.28 49.5±1.00 27.5±2.00 44.50±1.50 168.87±14.72 5.94±0.06 58.5±1.50 24.0±0.50 52.00±1.00 192.47±18.56 6.85±0.07 67.5±1.00 37.5±1.00 59.50±0.50 207.02±19.17 7.79±0.08 83.5±1.00 42.0±2.00
WildJailbreak 98.00±0.00 20.73±0.00 2.11±0.03 58.0±0.00 34.0±0.00 98.50±0.00 40.04±0.00 3.49±0.04 100.0±0.00 100.0±0.00 82.00±0.50 23.85±1.55 2.24±0.03 61.5±0.50 11.5±1.50 79.00±0.00 30.12±3.17 2.55±0.13 61.5±1.50 37.5±0.50 64.00±1.00 17.80±1.45 1.94±0.04 46.5±2.00 14.0±1.00 31.00±1.50 7.91±0.73 1.22±0.14 28.5±1.00 16.0±0.50 41.50±0.50 13.80±1.57 1.68±0.07 43.5±2.00 20.5±1.50 50.00±0.00 20.94±2.03 1.94±0.03 54.0±0.50 30.5±1.50 61.50±0.50 26.55±2.47 2.44±0.03 66.0±2.00 27.0±1.50 74.50±1.50 32.04±3.01 2.62±0.07 74.0±0.50 33.5±1.50 88.50±1.00 39.17±2.66 3.04±0.04 88.5±0.50 37.0±0.00
Table 18: Results for attacking the first Decision maker in the Qwen2.5 + BERT-base + Mistral-7B + BERT-base + Phi-3.5-MoE cascade system.
34
Cascade system
Method
Setting
Initial
LLM w/o cascade
Random noise
SLM only
Qwen2.5 + BERT-base + Mistral-7B + BERT-base + Phi-3.5-MoE
Singletarget attack
DM only
P=0
P=0.2
Jointtarget attack
P=0.4
P=0.6
P=0.8
P=1
Metric Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%) Performance Token cost Time cost (s) Passrate1 (%) Passrate2 (%)
AGNews 89.50±0.00 52.31±0.00 4.13±0.04 55.0±0.00 29.0±0.00 94.00±0.00 112.58±0.00 8.12±0.09 100.0±0.00 100.0±0.00 72.50±0.50 64.14±2.89 5.07±0.06 57.5±1.00 17.5±1.00 76.00±0.50 91.41±7.26 6.83±0.07 66.0±1.00 45.5±0.50 70.50±0.50 72.46±6.41 5.74±0.06 58.5±0.50 39.0±0.50 34.50±1.50 37.19±2.43 3.45±0.04 62.0±1.00 7.0±0.50 42.00±0.00 49.88±4.24 4.02±0.05 60.5±0.50 14.5±1.00 50.00±0.00 60.57±4.89 4.84±0.05 56.0±1.50 22.0±0.50 59.50±1.50 77.31±7.46 5.61±0.06 58.0±1.00 30.5±1.50 68.00±0.00 89.74±7.27 6.42±0.07 57.0±0.50 39.0±1.00 77.50±0.50 105.49±6.42 7.39±0.08 56.0±1.00 47.5±1.50
SQuAD2.0 81.00±0.50 110.31±1.20 4.97±0.06 51.5±0.50 24.0±0.00 86.50±0.00 277.24±0.00 8.49±0.07 100.0±0.00 100.0±0.00 69.00±1.00 141.89±7.42 7.04±0.07 52.0±2.00 24.5±0.00 68.00±1.00 206.40±14.25 7.64±0.08 60.5±0.50 42.0±1.00 60.00±1.50 174.98±12.84 6.41±0.07 54.0±1.00 38.5±1.00 28.50±0.50 90.89±8.76 4.37±0.05 54.5±0.50 8.5±1.00 36.00±0.00 131.33±11.23 5.05±0.06 52.0±1.00 15.0±0.50 43.50±0.50 158.54±13.64 5.89±0.06 54.0±0.50 24.0±1.00 52.50±1.00 191.74±16.46 6.72±0.07 55.0±1.50 32.0±0.50 63.00±1.00 216.30±15.37 7.62±0.08 59.5±1.00 41.5±1.50 70.50±1.50 242.17±19.53 8.27±0.09 61.5±0.50 50.5±1.00
WildJailbreak 98.00±0.50 20.73±0.35 2.11±0.03 58.0±0.00 34.0±0.50 98.50±0.00 40.04±0.00 3.49±0.04 100.0±0.00 100.0±0.00 82.00±0.50 23.85±1.45 2.24±0.03 61.5±1.00 11.5±0.50 76.00±1.00 28.31±2.17 2.41±0.03 59.0±1.00 41.5±1.50 83.00±0.50 29.82±3.58 2.78±0.03 60.5±0.50 46.0±1.00 58.00±0.00 15.82±1.58 1.69±0.02 58.0±1.50 6.5±0.50 62.50±0.50 19.17±1.14 2.01±0.03 59.0±1.00 13.5±1.50 69.00±0.50 24.23±3.17 2.35±0.03 60.5±0.50 19.5±1.00 75.50±0.50 29.41±3.54 2.72±0.03 57.5±1.00 27.5±1.00 81.00±1.50 32.77±3.79 3.01±0.04 49.0±1.50 36.0±1.00 92.50±0.50 38.42±4.49 3.37±0.04 54.0±1.00 46.0±2.00
Table 19: Results for attacking the second Decision maker in the Qwen2.5 + BERT-base + Mistral-7B + BERT-base + Phi-3.5-MoE cascade system.
35
Algorithm 1: Attack Algorithm Input: Input x, label y, cascade Φ(·; F , G), suffix δ with bounded length (reflecting Nϵ ), rounds R, update iterations T , target pass-rate p, target cascade system’s layer index i Output: Adversarial example x′ (f )
(g)
(f )
(g)
Set the δ0,T , δ0,T to be empty string, δ = δ0,T ∥ δ0,T ; 2 for r ← 1 to R do (f ) // Prediction-targeted phase: update δr,t to decrease the early-stage prediction models’ task performance. 1
3 4 5 6 7
(f )
Initialize δr,0 randomly; for t ← 1 to T do (f ) (f ) (f ) (f ) (f ) δr,t ← Update δr,t−1 ; Lacc (δr,t−1 , δ) where Lacc (δr,t−1 , δ) = Loss f i (x ∥ δ ∥ δr,t−1 ), y ; (f )
Update the final suffix δ = δ ∥ δr,T ; Compute intermediate predictions ŷri ← f i (x ∥ δ); // Decision-targeted phase:
8 9 10 11 12
(g)
update δr,t to control passrate (inference cost)
(g)
Initialize δr,0 randomly; for t ← 1 to T do (g) Generate candidate set Cr,t ⊂ Nϵ (δr,t−1 ); Sample b ∈ {0, 1} from Bernoulli distribution {p, 1 − p} ; if ŷri = y or b = 1 then
13
(g) (g) (g) (g) (g) (g) δr,t ← Update δr,t−1 ; Lcost (x ∥ δ ∥ δr,t−1 ) where Lcost (x ∥ δ ∥ δr,t−1 ) = g i (x ∥ δ, δr,t−1 , f i (x ∥ δ ∥ δr,t−1 ));
14
Select δ (g) maximizing Lcost (x ∥ δ ∥ δ (g) ) under the priorities listed in Section 4.4:
15 16 17
else (g) (g) (g) δr,t ← Update δr,t−1 ; −Lcost (x ∥ δ ∥ δr,t−1 ) ; Select δ (g) minimizing Lcost (x ∥ δ ∥ δ (g) ) under the priorities listed in Section 4.4: (i) Prefer candidates that keep the intermediate prediction unchanged: f i (x ∥ δ ∥ δ (g) ) = ŷri (ii) If none satisfy (i), prefer candidates that maintain previously incorrect predictions as still incorrect: f i (x ∥ δ ∥ δ (g) ) ̸= y (iii) If neither (i) nor (ii) can be satisfied, drop the constraint and minimize Lcost freely.
18 19
(g)
Update the final suffix δ = δ ∥ δr,T ; return x′ = x ∥ δ;
36
p = 0.4 p = 0.2
30
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
20
p=0
10 20
30
40
50
60
70 60 50 40 30
p = 0.2
20 10 20 p = 030
Cost (passrate%)
(a) Web-question (passrate)
EMR (%)
50 40 30
p = 0.4
p = 0.2
20
p=0 10 2
4
6
8
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 10 12 14 16
40
50
60
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
80 70
80
50 40
=0 10 p 20 30
50 40 30 20
p = 0.2
10 p =750
Cost (normalized tokens)
90
Initial cascade
60
100
125
150
175
p = 0.4
40
50
60
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Cost (passrate%)
(c) WildJailbreak (passrate) 100
70
p = 0.2
p = 0.6
20
Cost (passrate%)
LLM w/o cascade Random noise SLM onlyp = 1 p = 0.8 DM only p = 0.6 Initial cascade Random noise p = 0.4 SLM (f) only
p = 0.8
DM only
60
30
100 90
Initial cascade LLM w/o cascade Random noise p=1 SLM only
90
(b) SQuAD2.0 (passrate)
EMR (%)
Initial cascadeLLM w/o cascade SLM only Random noise p=1 p = 0.8 DM only p = 0.6
60
100
1 JSR (%)
40
LLM w/o cascade Random noise SLM only p=1 p = 0.8 DM only p = 0.6 Initial cascade p = 0.4 Random noise
Initial cascade
80
80
1 JSR (%)
EMR (%)
50
90
EMR (%)
Initial cascade LLM w/o cascade SLM only Random noise p = 1 p = 0.8 DM only p = 0.6
60
70 60 50 40 30
DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 200 225 250 275
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 DM only p = 0.6 Initial cascade p = 0.4 Random noise p = 0.2 SLM (f) only
20
10 p5 = 0 10
15
20
25
DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 30 35 40
Cost (normalized tokens)
Cost (normalized tokens)
(d) Web-question (normalized to- (e) SQuAD2.0 (normalized tokens) (f) WildJailbreak (normalized tokens) kens)
40 30
p = 0.4 p = 0.2
20 101
p=0
2
3
4
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 5 6
Cost (Seconds)
(g) Web-question (time)
90
LLM w/o cascade
Initial cascade
80
Random noise SLM only p=1 p = 0.8 DM only p = 0.6 Initial cascade p = 0.4 Random noise
70 60 50 40 30
p = 0.2
20 10 p = 0 3
4
5
6
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 7 8 9
Cost (seconds)
(h) SQuAD2.0 (time)
100
Initial cascadeLLM w/o cascade Random noise p=1 SLM only p = 0.8 DM only p = 0.6 cascade p = 0.4 Initial Random noise p = 0.2 SLM (f) only
90 80
1 JSR (%)
EMR (%)
50
Initial cascade LLM w/o cascade SLM only Random noise p=1 p = 0.8 DM only p = 0.6
EMR (%)
60
70 60 50 40 30 20 100.5
p=0
1.0
1.5
2.0
DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 2.5 3.0 3.5
Cost (seconds)
(i) WildJailbreak (time)
Figure 2: Comparison of attack methods on a two-layer LLM cascade across the WebQuestions, SQuAD2.0, and WildJailbreak datasets, using cascade configurations (1), (1), and (8), respectively (see Table 6). The cost–accuracy trade-off achieved by our pass-rate control method (Section 4.5) is illustrated by the blue dashed line. Points toward the lower-right represent more effective attacks.
37
Acc (%)
80
Random noise DM only
70 60 50 40
p=0
30
p = 0.2
p = 0.4
40
50
60
70
70 60 50 40
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 90 100 110
80
30
p=0
20 10 50
Cost (normalized tokens)
100
LLM w/o cascade
Initial cascade Random noise
80
p=1
SLM only p = 0.8 p = 0.6
20 1030
90
LLM w/o cascade
Initial cascade
EMR (%)
90
SLM only p = 1 p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade Random noise p = 0.2
100
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 200 250 300
150
Initial cascade
LLM w/o cascade
90
1 JSR (%)
100
80 70 60 50
Cost (normalized tokens)
Random noise
p = 0.2 p=0 15
p=1 DM only p = 0.8 SLM only p = 0.6 Initial cascade Random noise p = 0.4 SLM (f) only
20
25
30
DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 35 40
Cost (normalized tokens)
(a) AGnews (normalized tokens) (b) SQuAD2.0 (normalized to- (c) WildJailbreak (normalized tokens) kens)
Acc (%)
80 70
Initial cascade Random noise
60 50 40 30
p = 0.2 p=0
DM only
p = 0.4
p = 0.6
LLM w/o cascade SLM only p=1 p = 0.8
20 103
4
5
6
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 7 8
70 60
DM only
50
p = 0.4
40
p = 0.2 p=0
30 20 4
(d) AGnews (time)
5
6
Initial cascade Random noise
90
p = 0.6
10
Cost (seconds)
100
LLM w/o cascade SLM only p=1 p = 0.8
Initial cascade Random noise
80
EMR (%)
90
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 7 8 9
1 JSR (%)
90
100
80
SLM only
70
501.50
Cost (seconds)
(e) SQuAD2.0 (time)
p=0 1.75
p=1
DM only
p = 0.4
60
LLM w/o cascade
p = 0.8 p = 0.6 Initial cascade
p = 0.2 2.00
2.25
2.50
2.75
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 3.00 3.25 3.50
Cost (seconds)
(f) WildJailbreak (time)
Figure 3: Comparison of attack methods across the AGnews, SQuAD2.0, and WildJailbreak datasets on a three-layer LLM cascade target the second stage (configureation (24) in Table 7) . The cost–accuracy trade-off achieved by our pass-rate control method (introduced in Section 4.5) is shown as the blue dashed line. Points toward the lower-right represent more effective attacks.
45.7
4.3
Remain
Deliver
29.6
25.0
Remain
Deliver
40.5
6.6
39.7
13.2
Remain
Deliver
Correct
1.7
17.6
Wrong
48.3
27.8
17.9
33.0
Remain
Deliver
Correct
Deliver
Deliver
Deliver
28.6
18.7
30.1
Wrong
Remain
Remain
Remain
20.5
23.6
27.6
Remain
Deliver
Correct
21.5
8.3
22.3
Our attack (P=1)
21.3
24.1
Wrong
24.8
41.3
Correct
24.8
4.6
Wrong
28.9
45.9
35.7
Correct
Deliver
Deliver
Deliver
11.6
Wrong
Remain
Remain
Remain
30.4
Correct
23.7
22.2
7.7
Our attack (P=0.4)
Wrong
23.7
21.3
Correct
27.1
27.8
Wrong
Deliver
25.4
28.7
41.5
Correct
Remain
Deliver
Deliver
3.1
Wrong
37.1
Remain
Remain
47.7
Correct
17.2
WildJailbreak
19.8
29.9
Our attack (P=0)
Wrong
21.6
28.1
Correct
24.1
28.9
Wrong
Deliver
23.1
22.2
Correct
Remain
Deliver
36.8
Wrong
23.3
Remain
11.1
Correct
30.1
57.6
DM only
Wrong
22.3
Correct
Correct
24.3
Wrong
Deliver
30.3
Correct
Remain
11.4
Wrong
43.6
0.8
Correct
Wrong
6.4
SLM only
Wrong
Correct
9.1
Wrong
SQuAD2.0
40.9
Correct
AGNews
Wrong
Initial cascade
24.1
30.6
Remain
Deliver
Figure 4: Confusion matrices of the decision module (DM2 ) when the attack is targeting the second stage of a three-layer LLM cascade with configuration (24) in Table 7. 38
0
5
10
15
20
25
Cost (normalized tokens)
0.0
2.5
5.0
7.5 10.0 12.5 15.0 17.5 20.0
0
Cost (normalized tokens)
(a) Headline
Initial cascade SLM (f) only DM (g) only SLM (f) + DM (g) joint (p = 1)
5
10
15
20
25
Cost (normalized tokens)
(b) IMDB
Initial cascade SLM (f) only DM (g) only SLM (f) + DM (g) joint (p = 1)
Density
Density
Initial cascade SLM (f) only DM (g) only SLM (f) + DM (g) joint (p = 1)
Density
Density
Initial cascade SLM (f) only DM (g) only SLM (f) + DM (g) joint (p = 1)
0
2
(c) AGnews
4
6
8
10
12
Cost (normalized tokens)
14
16
(d) Web-question
Figure 5: Distributions of cost quantified by normalized tokens on a two-layer LLM cascade across the Headline, IMDB, AGnews, WebQuestions datasets, using cascade configurations (1) (see Table 6).
40
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
30 20p = 0 10 20
30
40
50
60
60 50
p = 0.4 p = 0.2
40 30
p=0
20 10 20
Cost (passrate%)
30
(a) Headline (passrate)
70
Initial cascade LLM w/o cascade Random noise SLM only
80
p=1 p = 0.8 DM only p = 0.6 Initial cascade p = 0.2p = 0.4
60
Acc (%)
60 50 40 30 20
p=0
105.0
7.5
10.0
12.5
15.0
17.5
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 20.0 22.5 25.0
50 40 30 20 102
p=0 3
4
5
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 6 7
Cost (seconds)
(g) Headline (time)
Initial cascade LLM w/o cascade Random noise p = 1 SLM only DM only p = 0.8 p = 0.6 p = 0.4
40
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 12.5 15.0 17.5 20.0
p = 0.2
30 20
p=0
10
2.5
40
p = 0.2
30 20
p=0
10 20
30
5.0
7.5
10.0
Initial cascade LLM w/o cascade Random noise SLM only p = 1 DM only p = 0.8 p = 0.6
70 60 50
p = 0.4
40
p = 0.2
30 20 101
p=0 2
3
4
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 5 6
Cost (seconds)
(h) IMDB (time)
50
60
Cost (passrate%)
80
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade
70 60 50 40
p = 0.2
30 20 10
Cost (normalized tokens)
80
40
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
(c) AGnews (passrate)
(e) IMDB (normalized tokens)
Acc (%)
Acc (%)
60
60
50
Cost (passrate%)
50
Cost (normalized tokens)
Initial cascade LLM w/o cascade Random noise SLM only p=1 p = 0.8 DM only p = 0.6 p = 0.2 p = 0.4 Initial cascade
70
50
70
(d) Headline (normalized tokens) 80
40
60
(b) IMDB (passrate)
Acc (%)
80
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade
70
Acc (%)
50
70
80
Acc (%)
Acc (%)
60
Initial cascade LLM w/o cascade Random noise SLM only p = 1 DM only p = 0.8 p = 0.6
80
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 20 25
p=0 5
10
15
Cost (normalized tokens)
(f) AGnews (normalized tokens) 80 70 60
Acc (%)
Initial cascade LLM w/o cascade Random noiseSLM only p=1 p = 0.8 p = 0.6 DM only Initial cascade p = 0.2 p = 0.4
70
Acc (%)
80
50 40
p = 0.2
30 20 10
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade
p=0 2
3
4
5
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 6 7 8
Cost (seconds)
(i) AGnews (time)
Figure 6: Comparison of attack methods across the Headline, IMDB, and AGNews datasets on a two-layer LLM cascade (Configuration (1) in Table 6) . The cost–accuracy trade-off achieved by our pass-rate control method(Section 4.5) is illustrated by the blue dashed line. Points toward the lower-right represent more effective attacks. 39
30 20
p=0
10
20
30
40
50
60
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Cost (passrate%)
SLM only p=1 DM onlyp = 0.8
70
p = 0.6
60
60
40
p = 0.2
30 20 10
p=0 5.0
7.5
10.0
12.5
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 15.0 17.5 20.0 22.5
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 p = 0.6 DM only p = 0.4 Initial cascade p = 0.2 Random noise
70 60
Acc (%)
50 40 30
90
20
p=0
10
2
3
4
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 5 6 7
Cost (seconds)
(g) Overruling (time)
50
60
30 20
p=0
10 20
Cost (passrate%)
SLM only p = 1 p = 0.8 p = 0.6
DM only p = 0.4 p = 0.2
50 p = 0 20 30
50
60
Cost (passrate%)
40
50
60
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Initial cascade LLM w/o cascade Random noise p=1 SLM onlyp = 0.8 DM only p = 0.6
70 60 50
p = 0.4 p = 0.2
40 30 20 20 p = 030
Cost (normalized tokens)
(e) CommonGen (normalized tokens)
40
50
60
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Cost (normalized tokens)
(f) SVAMP (normalized tokens) 80
100
LLM w/o cascade
Initial cascade Random noise
SLM only DM only p = 0.8
80
p = 0.6
70 60
40
80
70
90
30
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
(c) SVAMP (passrate)
LLM w/o cascade
Initial cascade Random noise
80
60
Cost (normalized tokens)
(d) Overruling (normalized tokens)
40
40
100
FCR (%)
Acc (%)
50
30
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
50
(b) CommonGen (passrate)
FCR (%)
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 p = 0.6 DM only p = 0.4
p = 0.2
50p =200
(a) Overruling (passrate) 70
p = 0.4
60
SR (%)
40
80
LLM w/o cascade Initial cascade Random noise p=1 SLM only p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade p = 0.2 Random noise
70
SR (%)
Acc (%)
50
90
LLM w/o cascade
Initial cascade Random noise
p = 0.4 p = 0.2
501.0 p = 01.5
2.0
2.5
3.0
p=1
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 3.5 4.0 4.5
Cost (seconds)
(h) CommonGen (time)
Initial cascade LLM w/o cascade Random noise p=1 SLM only p = 0.8 DM only p = 0.6
70 60
SR (%)
60
80
100
Full Coverage Rate (%)
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 p = 0.6 DM only p = 0.4 Initial cascade p = 0.2 Random noise
70
50
p = 0.4
40 30 20 p =3 0
p = 0.2 4
5
6
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 7 8 9
Cost (seconds)
(i) SVAMP (time)
Figure 7: Comparison of attack methods on a two-layer LLM cascade across the across the Overrulling, CommonGen, and SVAMP datasets using cascade configurations (1) (see Table 6). The cost–accuracy trade-off achieved by our pass-rate control method(Section 4.5) is illustrated by the blue dashed line. Points toward the lower-right represent more effective attacks.
40
70 60 50 40 30 20
SLM only Random noise DM only p = 1 p = 0.8 p = 0.6 Initial cascade Random noise p = 0.4 SLM (f) only p = 0.2 DM (g) only
p=0
10 20
40
60
70
SLM only p=1 p = 0.8 p = 0.6 Initial cascade Random noise p = 0.4
60
DM only
50 40 30 10 p = 0 50 100
Cost (normalized tokens)
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 200 250 300
p = 0.2
20
LLM w/o cascade SLM (f) + DM (g) joint 80 100
150
(b) SQuAD2.0 (normalized tokens)
100
90
Acc (%)
80
Random noise DM only p=1 p = 0.8 p = 0.6 Initial cascade Random noise p = 0.4 SLM (f) only p = 0.2 DM (g) only
70 60 50 40 30 20
LLM w/o cascade SLM only
p=0
102
3
4
5
LLM w/o cascade SLM (f) + DM (g) joint 6 7 8
Cost (seconds)
(d) AGnews (time)
80 70
Initial cascade Random noise
LLM w/o cascade SLM only
DM only
p=1
60
EMR (%)
Initial cascade
50 40
p = 0.4
30 20 10 3
p = 0.2 p=0 4
5
p = 0.6
6
p = 0.8
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 7 8 9
Cost (seconds)
(e) SQuAD2.0 (time)
Initial cascade
90
LLM w/o cascade p=1 SLM only p = 0.8
Random noise
80 70
DM only
60 50 40
p = 0.6
p = 0.4
p = 0.2
30
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 30 35 40
p=0
20 10 5
Cost (normalized tokens)
(a) AGnews (normalized tokens) 90
100
LLM w/o cascade
Initial cascade Random noise
80
10
15
20
25
Cost (normalized tokens)
(c) WildJailbreak (normalized tokens) 100 90 80
1 JSR (%)
Acc (%)
80
90
LLM w/o cascade
Initial cascade
EMR (%)
90
1 JSR (%)
100
Initial cascade
70 60
DM only
50 40 30 20 101.0
LLM w/o cascade p=1
Random noise SLM only
p = 0.2 p=0 1.5
p = 0.8
p = 0.6 p = 0.4
2.0
2.5
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 3.0 3.5
Cost (seconds)
(f) WildJailbreak (time)
Figure 8: Comparison of attack methods across the AGNews, SQuAD2.0, and WildJailbreak datasets on a three-layer LLM cascade targeting the first stage (Configuration (24) in Table 7) . The cost–accuracy trade-off achieved by our pass-rate control method (introduced in Section 4.5) is shown as the blue dashed line. Points toward the lower-right represent more effective attacks
41
40 30 20
p=0
10 20
30
40
50
60
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
60 50 40 30 20
p = 0.2
10 30
Cost (passrate%)
40
(a) Headline (passrate) Initial cascade LLM w/o cascade Random noise p=1 SLM only p = 0.8 DM only p = 0.6 p = 0.2 p = 0.4Initial cascade
70
Acc (%)
60 50 40 30 20
p=0
105.0
7.5
10.0
12.5
15.0
70 60 50 40
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 17.5 20.0 22.5 25.0
30 20
50 40 30 20p = 0 105.0
7.5
10.0
12.5
15.0
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 17.5 20.0 22.5 25.0
Cost (seconds)
(g) Headline (time)
50
30 20
p=0
10 20
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90
60
(e) IMDB (normalized tokens) Initial cascade LLM w/o cascade Random noise SLM only DM only p=1 p = 0.8 p = 0.6 p = 0.4 Initial cascade p = 0.2
80 70 60 50 40 30 20 105
p=0 10
15
20
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 25 30
Cost (seconds)
(h) IMDB (time))
30
40
50
60
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Cost (passrate%)
(c) Web-question (passrate) LLM w/o cascade SLM only p=1 Random noise p = 0.8 DM only p = 0.6 p = 0.4 p = 0.2 Initial cascade
Initial cascade
60 50 40 30 20
p=0
10 6
Cost (normalized tokens)
Acc (%)
Acc (%)
60
70
40
Cost (passrate%)
p=0
10 40
Cost (normalized tokens)
Initial cascade LLM w/o cascade Random noise p=1 SLM only p = 0.8 DM only p = 0.6 p = 0.2 p = 0.4 Initial cascade
70
60
Initial cascade LLM w/o cascade Random noise SLM only DM only p=1 p = 0.8 p = 0.6 p = 0.4 Initial cascade p = 0.2
80
(d) Headline (normalized tokens) 80
50
50
(b) IMDB (passrate)
Acc (%)
80
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 80 90 100
p=0
Initial cascade LLM w/o cascade SLM only p=1 Random noise p = 0.8 DM onlyp = 0.6 p = 0.4 p = 0.2 Initial cascade
60
Acc (%)
Acc (%)
50
70
Acc (%)
p=1 SLM onlyp = 0.8 DM only p = 0.6 p = 0.4 p = 0.2 Initial cascade
60
Initial cascade LLM w/o cascade Random noise SLM only DM only p=1 p = 0.8 p = 0.6 p = 0.4
80
LLM w/o cascade Random noise
8
10
12
14
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 16 18 20 22
Cost (normalized tokens)
(f) Web-question (normalized tokens) 60
LLM w/o cascade SLM only p=1 Random noise p = 0.8 DM onlyp = 0.6 p = 0.4 p = 0.2 Initial cascade
Initial cascade
50
Acc (%)
Initial cascade
70
Acc (%)
80
40 30 20 1010
p=0 15
20
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 25 30 35
Cost (seconds)
(i) Web-question (time)
Figure 9: Comparison of different attack configurations. The cost–accuracy trade-off achieved by our pass-rate control method in gray box setting without the information of gradient(introduced in Section 4.5) is shown as the blue dashed line.
42
20
p=0
10
5.0
7.5
10.0
40
Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 15.0 17.5 20.0 22.5
p = 0.2
30
50
12.5
30
Cost (normalized tokens)
20
p=0
10
2
(a) Overruling (normalized tokens)
70
Acc (%)
60 50 40
p = 0.2
30 20 10
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 20 25
p=0 5
10
15
50 40 30
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 5 6 7
4
20
p=0
10
Cost (seconds)
20
(b) Overruling (time) 80 70 60
Acc (%)
80
3
60
Acc (%)
40
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 p = 0.6 DM only p = 0.4 Initial cascade p = 0.2 Random noise
70
50 40
p = 0.2
30 20 10
Cost (normalized tokens)
(d) AGnews (normalized tokens)
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 6 7 8
p=0 2
3
4
5
30
40
50
60
SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Cost (passrate%)
(c) Overruling (passrate)
Cost (seconds)
80 70 60
Acc (%)
Acc (%)
50
60
Acc (%)
60
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 p = 0.6 DM only p = 0.4 Initial cascade p = 0.2 Random noise
70
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 p = 0.6 DM only p = 0.4
70
50 40 30
p = 0.2
Initial cascade LLM w/o cascade Random noise SLM only p = 1 p = 0.8 DM only p = 0.6 p = 0.4 Initial cascade
20
p=0
10 20
(e) AGnews (time)
30
40
50
60
Random noise SLM (f) only DM (g) only LLM w/o cascade SLM (f) + DM (g) joint 70 80 90 100
Cost (passrate%)
(f) AGnews (passrate)
Figure 10: Comparison of different attack configurations on extra datasets. The cost–accuracy trade-off on dataset IMDB [74] and AGnews [73] achieved by our pass-rate control method in white box setting using the information of gradient (introduced in Section 4.5) is shown as the blue dashed line.
27.50%
25 24.50%25.00% Accuracy (%)
24.50%
10
16.00%
50
21.00%
14.00%14.50%
13.00%
11.00%11.50%
Better Attack
40
17.00%
16.50%
Without constraint With constraint
60
23.00%
20 15
70
Without Constraint With Constraint
Accuracy (%)
30
12.50%13.00% 11.50%12.00%
30 20 10
9.50%10.00% 7.50% 7.50% 7.00% 7.50%
0 1000
5 5 10 15 20 25 30 35 40 45 50 55 60 Suffix length (tokens) (a) Accuracy of the small prediction model f 1
1500
2000 2500 3000 Cost (normalized tokens)
3500
4000
(b) Accuracy of the overall cascade system Φ
Figure 11: Comparison of “with” vs. “without” constraint setting (see Section 4.4) of our method.
43
69.5
14.0
Remain
Deliver
33.0
Remain
Deliver
16.0
12.5
37.0
34.5
Remain
Deliver
14.5
20.0
36.0
29.5
Remain
Deliver
Correct
8.5
39.5
Wrong
8.0
13.0
Correct
Deliver
14.5
Deliver
9.0
12.0
Wrong
Remain
Deliver
Remain
5.5
73.5
Remain
Deliver
Correct
14.5
Remain
78.0
7.5
13.0
Wrong
Deliver
71.0
36.0
5.5
6.5
73.0
Remain
Deliver
Correct
Remain
3.5
34.5
6.5
8.0
13.0
Wrong
15.0
11.0
11.0
10.0
6.5
72.5
Remain
Deliver
Correct
24.5
Deliver
18.5
SLM + DM (p = 1)
8.0
13.0
Wrong
34.5
Remain
Correct
26.0
11.5
Wrong
Deliver
69.5
Correct
Remain
15.0
Deliver
Wrong
17.0
4.0
Remain
Correct
29.5
Deliver
32.5
Wrong
Deliver
38.0
Remain
35.0
Correct
Remain
15.5
15.5
13.5
Wrong
52.5
Deliver
74.0
19.0
Correct
10.5
Remain
2.0
SLM + DM (p = 0.4)
Wrong
25.5
28.0
Correct
11.5
35.0
Wrong
Deliver
23.5
Correct
Remain
13.5
8.5
Wrong
47.5
Deliver
Deliver
Correct
Deliver
19.5
Remain
Remain
Wrong
Remain
22.5
10.5
19.0
Correct
55.0
10.5
38.0
73.0
Wrong
5.5
WebQuestions
Deliver
36.5
2.0
Correct
8.0
Remain
15.0
6.0
Wrong
31.5
44.5
Correct
Deliver
18.5
Wrong
Remain
27.0
Correct
57.5
10.0
Deliver
Wrong
2.0
Deliver
Remain
Correct
5.0
Remain
8.0
Wrong
35.5
47.5
38.5
Correct
Deliver
18.5
37.0
Wrong
Remain
26.5
16.5
Correct
54.0
7.5
SLM + DM (p = 0)
DM Only
Wrong
1.0
Correct
5.0
Wrong
Correct
40.0
Correct
Deliver
Deliver
Wrong
Remain
Remain
Correct
49.5
45.0
Wrong
Wrong
3.5
22.0
Correct
4.5
27.0
Wrong
Correct
42.5
6.0
Correct
Deliver
SLM Only
Wrong
Correct Wrong
Remain
Wrong
AGNews
51.5
Correct
IMDB
3.5
Wrong
Overruling
3.5
Correct
Headline
41.5
Wrong
Initial cascade
6.5
72.5
Remain
Deliver
Figure 12: Confusion matrices of the decision module (DM) for the GPT2-medium + BERT +Phi3medium under the initial cascade and different attack configurations.
44
65.0
9.5
Remain
Deliver
41.3
Remain
Deliver
11.0
6.0
39.5
43.5
Remain
Deliver
8.5
10.5
37.0
44.0
Remain
Deliver
Correct
10.9
36.3
Wrong
14.5
10.0
Correct
Deliver
12.4
Deliver
7.0
10.0
Wrong
Remain
Deliver
Remain
5.0
78.0
Remain
Deliver
Correct
10.5
Remain
75.5
5.5
10.5
Wrong
Deliver
69.0
38.5
6.5
3.5
80.5
Remain
Deliver
Correct
Remain
12.5
42.5
8.0
8.0
11.0
Wrong
14.0
8.0
10.0
10.0
5.0
76.0
Remain
Deliver
Correct
37.0
Deliver
9.0
SLM + DM (p = 1)
4.5
9.5
Wrong
39.0
Remain
Correct
10.0
14.5
Wrong
Deliver
69.5
Correct
Remain
11.5
Deliver
Wrong
14.0
4.5
Remain
Correct
40.5
Deliver
34.0
Wrong
Deliver
36.5
Remain
44.0
Correct
Remain
9.0
11.5
10.5
Wrong
69.0
Deliver
75.0
11.5
Correct
14.5
Remain
7.5
SLM + DM (p = 0.4)
Wrong
9.5
22.5
Correct
Deliver
7.0
33.5
Wrong
Remain
Deliver
38.0
Correct
66.0
Remain
6.0
6.0
Wrong
0.5
WildJailbreak
56.0
Deliver
Deliver
Correct
1.0
27.0
Remain
Remain
Wrong
32.5
12.0
22.5
19.0
Correct
Deliver
5.0
28.5
70.5
Wrong
Remain
Deliver
44.0
2.0
Correct
52.0
Remain
5.0
8.5
Wrong
4.0
77.0
Correct
SVAMP
10.0
Wrong
5.1
10.0
Correct
Correct
38.9
3.0
Deliver
Wrong
Deliver
Deliver
Remain
Correct
Remain
Remain
14.0
Wrong
40.5
70.5
39.5
Correct
Wrong
9.5
17.0
40.0
Wrong
1.5
9.0
6.5
Correct
Correct
48.5
Wrong
SQuAD2.0
3.5
SLM + DM (p = 0)
DM Only
Wrong
Deliver
Correct
Remain
Wrong
34.5
Correct
Wrong
4.5
CommonGen
Deliver
Wrong
2.5
Remain
Correct
Correct
58.5
69.0
Wrong
Deliver
21.0
Correct
Remain
5.0
Wrong
3.5
5.0
Correct
Wrong
47.0
SLM Only
Wrong
Correct
44.5
Correct
AGNews
5.0
Wrong
Initial cascade
4.0
82.0
Remain
Deliver
Figure 13: Confusion matrices of the decision module (DM) for the wen2.5+DistilBERT+Phi-3.5MoE under the initial cascade and different attack configurations.
45
10.0
Remain
Deliver
42.0
36.5
Remain
Deliver
7.5
10.5
38.5
43.5
Remain
Deliver
Correct
64.5
13.0
Wrong
18.5
8.5
6.5
74.0
Remain
Deliver
Correct
Deliver
7.0
Deliver
7.0
10.0
4.5
Wrong
Remain
Deliver
Remain
12.5
6.5
79.0
Remain
Deliver
Correct
11.5
Remain
41.0
SLM + DM (p = 1)
10.0
10.0
Wrong
44.5
23.0
Correct
35.0
64.5
Wrong
9.0
3.5
46.2
Correct
Deliver
Deliver
9.0
7.1
Wrong
Remain
Remain
Deliver
5.7
Correct
58.0
16.8
Remain
SLM + DM (p = 0.4)
Wrong
28.0
31.6
Correct
3.5
40.5
Wrong
10.5
11.1
19.0
Correct
Deliver
Deliver
Deliver
69.5
Wrong
Remain
Remain
Remain
5.0
Correct
57.0
60.0
17.5
6.5
Wrong
7.5
29.0
Correct
1.0
9.5
Wrong
34.5
WildJailbreak
1.5
38.0
Correct
Deliver
Deliver
32.5
Wrong
Remain
Remain
12.0
Correct
46.5
60.5
SLM + DM (p = 0)
DM Only
Wrong
4.0
Correct
5.0
Wrong
Correct
44.5
28.0
Correct
Deliver
10.5
Wrong
Remain
1.0
Correct
49.0
SLM Only
Wrong
Correct Wrong
4.5
Wrong
SQuAD2.0
6.0
Correct
AGNews
40.5
Wrong
Initial cascade
1.5
78.5
Remain
Deliver
Figure 14: When suffix optimization target is (DM_1), the confusion matrices of the decision module (DM_1) for the Qwen2.5 + BERT-base + Mistral-7B + BERT-base + Phi-3.5-MoE cascade system under the initial cascade and different attack configurations.
46