Piggybacking on Perception: Stealthy Concurrent Audio Prompt Injections against Multimodal LLM Agents Mingxiao Liu1
Yitong Li1
Haoren Zhao1
Jialuo Chen3,2 1 Hangzhou Dianzi University
Yaoxiang Bian1
Xinhao Deng4,2,†
2 Ant Group
Jian Zhang1
Zhen Wang1,†
3 Zhejiang University
4 Tsinghua University
ities. New generations of multimodal agents, including AutoGLM [5], MAI-UI [6], and Step-Audio2 [7], are rapidly evolving from passive assistants into agentic systems with continuous speech perception, OS-level interaction, and autonomous crossapplication task execution, as increasingly reflected in emerging AI-native devices such as the Doubao AI Smartphone. These agents operate in open environments and rely on persistent audio interaction to complete complex tasks, which introduces an inherent and under-explored security exposure in the audio modality. Unlike text or visual inputs that require explicit user actions, audio interfaces act as passive, omnidirectional sensors that continuously monitor their surroundings. Agents thus become vulnerable to environment-level prompt injection: an attacker can embed malicious instructions into ambient audio to stealthily hijack agent intent and bypass security boundaries premised on user-initiated interaction.
A BSTRACT
arXiv:2607.28165v1 [cs.CR] 30 Jul 2026
Jianan Ma1,2
Large Language Model (LLM)-driven multimodal agents are increasingly deployed to execute autonomous tasks via continuous audio interaction. While this paradigm enhances interaction naturalness, it introduces a critical yet under-explored attack surface, as audio inputs inevitably contain environmental noise beyond user control. In this paper, we investigate concurrent audio prompt injection attacks targeting multimodal agents. Distinct from traditional acoustic attacks on voice devices, we propose novel techniques for instruction augmentation and scenario concealment. These methods allow malicious audio instructions to imperceptibly "piggyback" onto user speech, thereby hijacking agents to execute malicious actions. To systematically quantify this threat, we construct AudioAgentSecurity, the first comprehensive benchmark for audio instruction injection attacks, encompassing 8 real-world task scenarios and 10 distinct attack patterns. We evaluate 11 state-of-the-art agents, including Gemini 3 Pro and GPT-4oaudio. Notably, our methods achieve an average Attack Success Rate (ASR) of 69.10% against the advanced Gemini 3 Pro. To counter this threat, we further introduce Cascaded Audio Decoupling and Verification (CADV), a defense mechanism based on source separation and consistency analysis. Compared with existing prompt-level defenses, CADV leverages acoustic source separation and cross-modal consistency analysis to detect audio instruction injections more robustly, achieving over 90% detection success across diverse attack vectors. Finally, real-world experiments with human volunteers on Doubao AI Smartphone in diverse dynamic real-world scenarios confirm the attacks’ high stealth and efficacy, while demonstrating that our defense reliably mitigates these vulnerabilities.
Specifically, prompt injection techniques manifest distinctly across modalities. In the text domain, attacks typically exploit semantic ambiguities or indirect retrieval mechanisms to bypass safety guardrails [8, 9]. In the visual modality, injection attacks rely on subtle perturbations or embedded triggers to manipulate perception and interpretation [10–12]. In contrast, research in the audio domain has followed different trajectories. One category focuses on physical signal exploits against traditional voice assistants, such as DolphinAttack utilizing inaudible ultrasonic carriers [13] and NUIT leveraging near-ultrasonic triggers [14]. These attacks mainly exploit speech-recognition pipelines or microphone non-linearity in conventional ASR systems, whereas continuous-audio MLLM agents introduce additional challenges in streaming input, contextual persistence, and multimodal tool use. Another category focuses on jailbreak or backdoor attacks in Audio LLM scenarios (e.g., Hidden in the Noise [15], SACRED-Bench [16]), which evaluate model robustness under malicious inputs or preset triggers, usually assuming that the attacker is the direct interactive user or relies on pre-implanted acoustic triggers. AudioJailbreak [17]’s weak adversary appends suffixal audio after user input without concurrent speech interference, and our scenario requires that malicious audio must compete with ongoing user speech for agent attention. More recently, AudioHijack [18] shows that LALMs are vulnerable to imperceptible auditory prompt injection, but it is a white-box attack with a relatively single attack strategy and
I. I NTRODUCTION The rapid development of LLM-driven multimodal agents is reshaping autonomous task execution, with speech becoming a primary interaction modality. Recent multimodal models, such as the Gemini [1, 2] and Qwen [3, 4] series, demonstrate strong audio understanding and cross-modal reasoning capabil† † Co-corresponding author.
© 2026 The Authors. This preprint is made available under the arXiv.org perpetual, non-exclusive license to distribute this manuscript. All other rights are retained by the authors.
1
Verification) to safeguard agents. Notably, traditional Speaker Verification(SV) [20–22], solutions face critical challenges in agent defense: they are often text-dependent, generalize poorly in complex noise/speech variations, and introduce high latency that hampers real-time interaction. In contrast, CADV utilizes three key innovations: (1) Audio Track Separation to decouple concurrent streams, (2) Multifunction Extraction for fine-grained acoustic fingerprints, and (3) Speaker Consistency Analysis to verify instruction legitimacy. Our evaluation demonstrate that CADV establishes a robust defense perimeter, significantly reducing ASR while maintaining a near-zero false positive rate. In summary, we make the following contributions:1
Figure 1. Threat model of concurrent audio prompt injection attacks. An attacker stealthily injects malicious audio prompts into a benign speech stream, manipulating the agent’s intent understanding and leading to unauthorized outputs.
• Concurrent Audio Prompt Injection: We define and study a third-party audio injection setting where malicious instructions overlap with ongoing benign user speech.
no defense closed loop. Consequently, these works differ fundamentally from the stealthy third-party concurrent audio prompt injection in real-world scenarios investigated in this paper. To address the limitations of existing research in terms of attack objectives and threat models, we propose Stealthy Concurrent Audio Prompt Injection. As shown in Figure 1, we study an environmental third-party attacker who injects malicious audio concurrently with user speech and attempts to exploit the short active interaction window through Energy Enhancement and Dynamic Compression. We combine traditional acoustic attack techniques with agent-oriented instruction design, yielding 10 attack methods that cover inaudible, lowintelligibility, and semantic-confusion settings. At the audio level, energy enhancement and dynamic compression increase the chance that the injected instruction remains decodable during overlap with user speech. At the content level, Concurrent Injection Prefixes (e.g., "Task updated, execute now:") are used for Semantic Anchor Hijacking, encouraging the agent to treat the injected stream as a new or higher-priority instruction. To systematically quantify this threat, we construct AudioAgentSecurity, a benchmark focused on third-party concurrent audio prompt injection, covering 8 real-world-inspired scenarios and 10 attack patterns. Our evaluation of 11 representative multimodal agents, including gemini-3-pro-preview [1] and gpt-4o-audio-preview [19], shows that several models follow injected instructions in a sandboxed tool-trace environment; gemini-3-pro-preview reaches an average ASR of 69.10% under overlapped injections. Moreover, physical-world evaluations provide case-level evidence that attack effectiveness can persist under distance, angle, and overlap variations, though the results should be interpreted within the tested device and environment settings. Human studies further suggest that some injected instructions are difficult for participants to perceive or understand during natural interactions. We additionally validate selected attacks on a commercial Doubao AI Smartphone and report the findings to relevant vendors. Finally, we propose CADV (Cascaded Audio Decoupling and
• Systematic Evaluation: We build AudioAgentSecurity with 2,160 generated attack audio samples across 10 attack methods and 8 real-world-inspired task scenarios, and evaluate 11 representative audio-capable agents in a sandboxed tool-trace environment. • Prototype Defense: We propose CADV, a zero-priorknowledge defense based on audio decoupling and consistency analysis, and characterize both its mitigation effect and its false-positive limitations. II. BACKGROUND A. Agents and Large Language Models Agents represent a paradigm shift in large language models from passive information processors to autonomous actors [23]. By incorporating core components such as perception, memory, planning, and tool use, agents can actively perceive their environment and perform complex tasks, marking a transition from traditional generative models to agentic AI that integrates reasoning and action within one system [24–27]. Early work, exemplified by ReAct [28], established the reasoning–action interaction paradigm, while subsequent systems such as Voyager [29] further demonstrated the feasibility of long-horizon autonomous task execution. With the emergence of natively multimodal foundation models, including Gemini [1] and GPT4o [19], modern agents have acquired the ability to process continuous modalities such as audio and vision. However, increased capability is often accompanied by an expanded attack surface. When agents interact with open physical environments through the audio channel, the security of the perceptual layer becomes critical: an adversary can exploit the passive, omnidirectional nature of audio to manipulate agent intent without disrupting the normal interaction flow, covertly steering the agent toward high-risk actions with minimal user awareness. 1 The code and benchmark materials are available at https://github.
com/Limax666/AudioAgentSecurity.
2
is a practical, covert third-party audio attack model: attackers can take advantage of opportunities created by close proximity, scenario concealment, or specific attack methods, but must still contend with imperfect timing, distance, and directionality. The ultimate goal is Intent Hijacking: inducing the agent to execute unauthorized high-risk operations (e.g., financial manipulation, privacy exfiltration, or malware installation) without raising user suspicion.
B. Audio Prompt Injection Prompt injection refers to embedding malicious instructions into the input to induce a model to bypass its safety constraints [30]. Prompt injection attacks in text and vision modalities have been extensively investigated across multiple attack surfaces. For example, interfering with the LLM alignment process can significantly enhance the effectiveness of text-based prompt injection attacks [31], by strategically inserting delimiters to manipulate the model’s instruction parsing [30]. Beyond pure text inputs, malicious prompts can also be covertly embedded into images [10, 32], web content [12, 33], and even an agent’s tool library, enabling indirect manipulation of agent behaviors [34]. Prior work has largely formalized prompt injection in textual or structured modalities. In this setting, an injected task is typically represented as a tuple (se , xe , ye ), where se denotes the injected instruction, xe the injected data, and ye the desired malicious response [35]. However, this formulation does not directly extend to the audio modality, naively converting malicious instructions into speech via text-to-speech (TTS) systems [36] is easily perceptible to human listeners and ineffective in complex acoustic environments. This work focuses on stealthy and concurrent audio prompt injection, in which an adversarial signal xadv (t) is imperceptibly overlaid on benign user speech and ambient sound, so that the composite y(t) = xb (t) + xadv (t) reads as harmless noise to humans yet conveys malicious intent to the agent. The core challenge is to exploit this human–machine perception gap: xadv (t) must stay inconspicuous to the ear while remaining robust against the dominant xb (t) during inference. This exposes a fundamental liability of current multimodal models, whose highly optimized perception can recover hazardous instructions from signals humans never notice.
B. Defender Constraints and Objectives The defender is modeled as a detection barrier deployed at the agent’s perception layer and operates under a zero-priorknowledge assumption, meaning it does not rely on preregistered user voiceprints or identity templates. Although prior work shows that speaker recognition can support user authentication in controlled settings [39], its applicability is significantly limited in open and dynamic audio interaction scenarios. On the one hand, modern audio agents often serve transient or multiple users, making reliable voiceprint enrollment impractical [40]. On the other hand, speaker recognition systems have been shown to be vulnerable to voice spoofing techniques, preventing them from serving as a robust security boundary [41, 42]. The core security objective is to achieve attack neutralization by effectively suppressing malicious payloads to prevent intent hijacking, while ensuring service preservation, avoiding significant impairment to the signal integrity of the dominant user speech. In realistic scenarios such as multiturn dialogues, the defense may prompt users for secondary confirmation without disrupting the real-time interaction flow or the overall integrity of user–agent communication. C. Realistic environmental constraints A central design principle of this work is that our threat model rests only on weak and conservative assumptions, and that every assumption affecting feasibility is empirically validated rather than taken for granted. On the attacker side, we assume only black-box access over the physical acoustic channel and, critically, do not assume perfect temporal or spatial alignment, a clean channel, or control over the user’s speech. On the defender side, we assume no pre-registered voiceprints or identity templates (zero-prior-knowledge). We therefore avoid the idealized conditions—silent victim devices, isolated single-source playback, or precise injection timing—that prior acoustic attacks typically presuppose. To ensure these constraints are not merely asserted, we explicitly account for dynamic acoustic scenes—background noise, user movement, distance variation, off-axis emission, and multiple concurrent speakers—and subject both the attack and the defense to physical-world validation rather than digital simulation alone. Concretely, the robustness of every claim under these constraints is measured directly: across attack distance, angle of arrival, and instruction overlap (Section 1), on a commercial device under a double-blind human study (Section E), and always under concurrent user speech rather than isolated playback. This assumption-then-validation discipline is what allows the reported attack and defense effectiveness to be interpreted as realistic lower bounds rather than
III. T HREAT M ODEL We consider a highly practical interaction scenario: a user actively engaging with a Multimodal Agent (e.g., Doubao AI Smartphone) through continuous audio. These agents are typically deployed on mobile or IoT devices with high-privilege tool-use capabilities, such as financial transactions, system configuration, and private data management. We assume the user’s voice is the sole root of trust, while the surrounding acoustic environment is an untrusted, environment-exposed input channel. A. Attacker Assumptions and Goals We assume a Black-box Attacker with no access to the agent’s internal weights, system prompts, or tool-calling logic. The attacker’s capability is strictly confined to the physical acoustic channel [13, 37, 38]. The attacker can adaptively select methods according to distance, noise, obstruction, and scene constraints. The attacker cannot directly control the user’s speech or inspect internal activations, and need not assume perfect temporal or spatial alignment with the device. Instead, the attacker adjusts the speed and strength of the injected audio within the active interaction window through energy enhancement and dynamic compression, so as to increase the likelihood that the malicious signal is jointly processed with the user’s utterance. This 3
Figure 2. The overall workflow comprises three stages: (1) benchmark construction; (2) audio modulation and attack injection; and (3) contrastive evaluation of agent security.
best-case artifacts.
Mobile [43–45], PC [46, 47] and Browser [48, 49]. We refine and normalize this data into the corresponding scenarios. Additionally, we authored seed instructions for each scenario and expanded them into diverse instruction pairs using Google Gemini 3 Pro. Finally, we manually verify and selected 200 pairs of core instructions, forming a text baseline containing both benign and malicious instructions.
IV. ATTACK F RAMEWORK AND B ENCHMARK D ESIGN We present the AudioAgentSecurity benchmark and our evaluation framework in Figure 2. The defining challenge of concurrent injection is that the malicious signal must survive two adversarial pressures at once: the acoustic pressure of competing with the user’s dominant speech in a short, shared interaction window, and the semantic pressure of redirecting an agent whose attention is already anchored to the user’s intent. Our framework is organized around resolving these two pressures: energy enhancement with dynamic compression (Section C) addresses the former, and Semantic Anchor Hijacking addresses the latter. The following sections detail the dataset construction, attack modulation with concurrent injection, and security evaluation.
B. Attack Modulation To evaluate agent robustness in realistic acoustic environments, we designed an automated audio generation pipeline utilizing the Qwen3-TTS [50] model. We employ specific voice profiles to generate benign speech xb (t) and raw malicious speech xm (t) separately. The attack suite combines reused acoustic attack primitives with agent-specific instruction composition. We categorize the attacks into three levels based on physical characteristics and human perception: inaudible carrier-based attacks, audible low-intelligibility transformations, and audible semantic-confusion attacks.
A. Dataset Construction We first identify eight risk scenarios prevalent in real-world human-computer interaction, covering domains from smart home control to financial asset management. Based on these scenarios, we propose a multi-stage instruction generation process. We first standardize the formats for benign instructions, malicious instructions, and overlapped instructions. Each sample adopts a dictionary-like structure containing key-value pairs such as benign instruction, attack instruction, attack method, injection strategy, and overlap rate. Then, we collect samples from several existing multi-agent evaluation datasets, including
1) Inaudible Attacks These attacks reuse carrier-based acoustic primitives that exploit microphone non-linearity or frequencies above the human hearing limit ( f ≥ 20 kHz). They are most relevant to quiet environments such as offices and libraries, where audible masking noise is limited. A representative example is DolphinAttack, which uses Amplitude Modulation (AM) to modulate malicious signals onto ultrasonic carriers. The in4
jected signal xadv (t) is defined as: xadv (t) = [1 + m · xm (t)] · cos(2π fct)
into four core security dimensions to approximate material damage. The resulting tool traces are used to measure whether a model follows benign or injected intent; they should not be interpreted as proof that every real deployment would execute the same high-risk action.
(1)
where m denotes the modulation depth and fc is the carrier frequency.
2) Contrastive Security Evaluation To evaluate the agent’s decision response to concurrent audio prompt injection, we employ a contrastive security evaluation. Considering that model outputs vary for the same input, and outputs differ for the same model with and without malicious audio injection interference, we first input normal user instruction audio into an advanced LLM to obtain a benign baseline without interference. Next, the overlapped instruction audio is fed into the agent to obtain responses such as tool invocations. Finally, a Judge model compares the two results to assist security and utility labeling. Because LLM-as-a-judge can introduce semantic bias, we treat this as an evaluation aid rather than an absolute oracle.
2) Audible–Low-Intelligibility Attacks These attacks preserve audible energy but reduce intelligibility by mimicking environmental noise or background sounds. We implement Pulse, Spectral Inversion, Spectral Scramble, Noise Texture and Whisper attacks. They are intended to model noisy outdoor or in-car environments where short acoustic artifacts may be less salient to users. Spectral Inversion inverts the spectrum around a carrier frequency fc to produce twisted, ambient-like sounds: xadv (t) = LPF[xm (t) · cos(2π fct)]
(2)
where LPF denotes a low-pass filter used to extract the inverted lower sideband.
V. D ETECTION M ETHODS AND D EFENSE F RAMEWORK
3) Audible–Semantic-Confusion Attacks These attacks remain physically audible but attempt to reduce user comprehension through speed variation, dialect injection, or foreign-language translation. Their goal is not acoustic invisibility, but a mismatch between what users attend to and what the model can still interpret as an executable instruction.
Addressing the threat of concurrent speech injection described previously, existing uni-modal defense mechanisms may miss weak adversarial signals masked by the high energy of continuous user speech. We therefore propose a defense framework titled Cascaded Audio Decoupling and Verification (CADV). Departing from purely prompt-side filtering, CADV adopts a signal decoupling strategy. As illustrated in Figure 3, the system first decomposes the mixed acoustic environment into independent potential sources at the physical layer, followed by identity consistency verification and instruction validity auditing at the feature and semantic layers, respectively. This coarse-to-fine cascaded design aims to isolate suspicious secondary speech before it reaches the agent’s instruction-following pipeline.
C. Attack Robustness 1) Energy Enhancement and Dynamic Compression To resolve the acoustic pressure of competing with dominant user speech, we use a Stealth-Driven Energy Enhancement strategy. The intuition is that attacks with lower user intelligibility can tolerate some increase in signal energy before becoming obvious. We therefore increase the power of the adversarial signal xadv (t) within the tested perceptual constraints, modeling the mixed signal as y(t) = xb (t) + λ · Boost(xadv (t)). Dynamic Range Compression (DRC) and RMS normalization are used to make the injected signal more likely to be decoded during overlap. We also use dynamic compression to fit the attack audio into the active phase of user-agent voice interaction.
A. Observation and Motivation Our defense strategy is predicated on two acoustic observations. First, although the attack signal is masked by user speech in the time domain, the two signals—as independent acoustic events—may retain distinct statistical characteristics that can support deep learning-based blind source separation. Second, under our black-box threat model, the injected command often comes from a source different from the active user, so the decoupled signals may exhibit a measurable “identity distance” in speaker embedding space. These observations motivate the technical roadmap of “separation first, verification second.” They also define the boundary of CADV: the method can be less reliable when background speech is dense, when the attacker replays the user’s own voice, or when high-quality voice cloning reduces speaker differences.
2) Concurrent Injection Prefix Resolving the acoustic pressure makes the malicious signal more likely to be decoded; resolving the semantic pressure makes it more likely to be treated as an instruction. To this end, we introduce the Concurrent Injection Prefix. Details are provided in Appendix B. In the concurrent setting, the agent must parse intertwined user and attacker intentions. Semantic Anchor Hijacking embeds prefixes that resemble system updates, task transitions, or logical breakpoints (e.g., “System Notice”, “Task Updated”, “Next Instruction is”, “Thinking Complete, Start New Command”) at the beginning of the malicious instruction.
B. Audio Track Separation This stage aims to reconstruct independent source tracks from a single-channel mixed input y(t). The primary challenge is that adversarial signals are temporally masked by user speech and share overlapping frequency spectra, rendering traditional linear filtering ineffective. Our core idea is to leverage the distinct statistical characteristics of independent acoustic events
D. Agent Security Evaluation 1) Agent Environment Construction To evaluate the security threats posed by the AudioAgentSecurity benchmark, as presented in Table 1, we constructed an automated, sandboxed agent execution environment. We abstract over 50 atomic tools 5
Table 1. Overview of the AudioAgentSecurity Benchmark. The benchmark lists attack categories, corresponding methods, representative target environments, and generated attack audio samples used in the evaluation. Category
Attack Methods
Target Environments
Samples
1. Inaudible Attacks (Hardware Vulnerability)
DolphinAttack, High Frequency
Quiet environments (e.g., Office, Library)
432
2. Audible–Low-Intelligibility (Signal Distortion)
Pulse, Spectral Inversion, Spectral Scramble, Noise Texture, Whisper Attack
Noisy environments (e.g., Street, In-car)
1080
3. Audible–Semantic-Confusion (Cognitive Mismatch)
Speed, Foreign, Dialect
Multi-speaker environments (e.g., Restaurant, Cafe)
648
geneity flag Iid is then triggered if Scos falls below an EnergyAware Dynamic Threshold τ(re ), indicating that the secondary track likely originates from an injected, non-user source. D. Semantic Filtering The final stage focuses on performing a deep semantic audit of the residual signals to eliminate non-malicious interference. The challenge is that background vocalizations, such as laughter or interjections, may satisfy identity heterogeneity but do not constitute meaningful or actionable instructions. Our core idea is to employ Automatic Speech Recognition(STT) to filter nonsemantic noise and determine whether the secondary track contains a valid command. The system transcribes the secondary waveform into a text sequence W = Trans(x̂adv ) and evaluates its semantic validity through an indicator Isem , which checks whether W contains tokens from a predefined non-semantic set Knoise and whether its length exceeds a minimum threshold Lmin . Only semantically meaningful residual signals are allowed to proceed to the final decision stage. The final interception action is activated only when all three layers—the physical-layer separation (via I phy ), the identitylayer consistency check (via Iid ), and the semantic-layer validity test (via Isem )—simultaneously satisfy their respective conditions:
Figure 3. The proposed Cascaded Audio Decoupling and Verification framework processes the input mixed audio in a hierarchical manner, including audio track separation, speaker consistency verification, and semantic filtering.
in the frequency domain. Specifically, we utilize the Mossformer2 model [51] to construct a non-linear separation function (x̂b , x̂adv ) = Fsep (y(t)), where x̂b is the dominant user speech and x̂adv is the potential attack track. A physical gating indicator I phy is defined to filter noise, which is triggered only when the RMS energy ratio re = ||x̂adv ||rms /||x̂b ||rms exceeds a silence threshold δsilence .
Action = Block
if I phy ∧ Iid ∧ Isem .
(3)
This cascaded design is intended to balance attack detection with preservation of benign interactions, while acknowledging that false positives can increase in multi-speaker scenes. VI. E VALUATION
C. Speaker Consistency Verification
Our evaluation considers four research questions:
The objective of this layer is to model speaker identity and quantify the consistency between the decoupled tracks. The challenge lies in the fact that low-energy attack signals are prone to feature drift, and fixed decision thresholds often lead to high false positives in fluctuating acoustic environments. To address this, our core idea is to exploit the measurable “identity distance” within a robust speaker embedding space where identityrelevant features remain invariant. We employ a CAM++-based extractor [52] to map each signal into a d-dimensional embedding vector eb = ζ (x̂b ) and eadv = ζ (x̂adv ). Speaker consistency is quantified by the cosine similarity Scos . An identity hetero-
• RQ1: How vulnerable are key multimodal agents to concurrent audio prompt injection? • RQ2: How do prompt-level defenses compare with CADV in mitigating concurrent audio prompt injection? • RQ3: Are concurrent audio attacks practically feasible in diverse and dynamic real-world environments? • RQ4: How do human participants evaluate the stealthiness of these attacks in real-world interactions? 6
Table 2. Attack Success Rate comparison across 11 evaluated agents. Bold indicates the highest ASR, while gray indicates the lowest. To improve readability, we list the full model names directly in the Table 7 (Appendix A) instead of using compact indices. Attack Method
QwenQwen2.5Qwen3Fun- MiMo- MiniCPM- Gemini- Gemini- Gemini- Gemini- GPT-4oAudio Audio o-2_6 2.5-Flash 2.5-Lite 2.5-Pro 3-Pro Audio Omni-Turbo Omni Omni-Flash
dialect dolphin foreign high freq pulse spectral inversion spectral scramble speed texture whisper
65.00 85.50 74.00 76.50 81.50 7.00 37.50 50.00 46.50 7.00
56.00 76.50 47.00 69.50 73.50 7.50 29.00 30.50 45.00 8.50
43.00 69.00 46.00 46.50 56.00 3.00 17.00 9.50 33.50 5.00
47.50 87.00 87.50 79.00 80.00 3.00 24.50 40.50 53.50 2.50
25.00 64.50 63.00 43.50 40.50 1.50 4.00 16.00 9.50 2.00
59.50 90.00 92.50 82.00 87.50 11.50 50.00 49.00 67.50 7.50
70.50 92.00 90.50 86.50 91.00 26.00 66.00 78.50 74.00 16.00
53.00 82.00 82.50 70.00 77.00 5.00 40.00 37.00 45.00 4.50
71.50 85.50 72.50 80.50 84.50 9.00 34.00 61.50 60.00 5.00
71.50 83.50 72.50 76.50 84.50 7.00 34.00 62.50 59.00 4.50
55.50 81.50 68.00 65.50 74.50 3.00 26.50 30.50 45.00 1.00
56.18 81.55 72.36 70.55 75.50 7.59 32.95 42.32 48.95 5.77
Average
53.05
44.30
32.85
50.50
26.95
59.70
69.10
49.60
56.40
55.55
45.10
49.37
To explore RQ1, we evaluate a range of state-of-the-art modeldriven agents on the proposed AudioAgentSecurity benchmark (Section 6.2), examining the execution outcomes of benign user instructions, injected malicious instructions, and agentic workflow tasks. For RQ2, we compare system performance under settings without defense, with prompt-level defense baselines, and with the proposed CADV defense (Section 6.3). To answer RQ3, we conduct a series of physical-world experiments to assess the real-world effectiveness of the proposed attack (Section 6.4). Finally, for RQ4, we recruit human participants to perform a user study evaluating the stealthiness and effectiveness of the attacks (Section 6.5).
Avg
according to the recorded response and tool invocation. We employ Qwen-Max as an auxiliary Judge to audit the agent’s thought process and tool invocation traces, while recognizing that judge-model decisions can introduce semantic bias. ASR is defined as: Nmalicious × 100% (4) ASR = Ntotal where Nmalicious is the number of trials where the agent is induced to execute the malicious intent Im instead of the user’s original purpose. For Attack Analysis, a higher ASR indicates a more potent and deceptive attack vector, whereas for Defense Analysis, a lower ASR demonstrates the defense mechanism’s capability to intercept or neutralize threats. 2) Instruction Correct Rate (ICR) This metric proxies the agent’s availability and utility in the same sandboxed environment. It measures the completion rate of the user’s original benign instructions despite the presence of acoustic interference or defensive filtering.
A. Experimental Setup 1) Models To evaluate security across diverse architectures and scales, we select eleven representative audio-capable models ranging from closed-source APIs to lightweight openweights models. For native audio reasoning, we employ gpt-4o-audio-preview [19] and four variants of the Google Gemini family [1], including gemini-3-propreview, gemini-2.5-pro-nothinking, and gemini-2.5flash/lite. To cover open-source and edge-oriented settings, we incorporate Fun-Audio-Chat-8B, MiMo-Audio-7B-Instruct, MiniCPM-o-2_6 [53], and the Qwen Omni series [3] (qwen3omni-flash, qwen-omni-turbo, qwen2.5-omni-7b). The full names used in the result tables are listed in Appendix A.
ICR =
Nbenign_correct × 100% Ntotal
(5)
where Nbenign_correct denotes the number of trials where the agent’s output correctly fulfills the user’s benign intent Ib . For Attack Analysis, a lower ICR indicates that the attack disrupts benign task completion; conversely, for Defense Analysis, a high ICR indicates that the defense does not overly filter or disrupt legitimate user interactions.
2) Evaluation Metrics To provide a unified and pragmatic assessment of both attack efficacy and defense robustness, we focus on the end-to-end behavioral outcomes of the agent. Specifically, we employ two core metrics to quantify the trade-off between security risks and functional utility. For defense evaluation, we report both the mitigation effectiveness of CADV and the corresponding performance of prompt-level baselines, highlighting whether a defense can suppress malicious instructions without overly degrading benign utility: 1) Attack Success Rate (ASR) This metric serves as the primary indicator of security compromise in our sandboxed tooltrace environment. It measures the proportion of samples where the adversary induces the agent to follow the malicious intent
B. Vulnerability of Multimodal Agents to Audio Attacks To answer RQ1, we conducted a comprehensive evaluation of ASR and ICR across 11 state-of-the-art multimodal agents. Unlike idealized scenarios where attacks are precisely inserted into speech pauses, this setting reflects a more common real-world environment for audio prompt injection, in which malicious instruction audio is concurrently superimposed onto the benign speech of the user. 1) Vulnerability Analysis (ASR) We first examine the ASR to quantify the systemic vulnerability of different agents. The results are detailed in Table 2. The data reveal a clear contrast in 7
Table 3. Instruction Correct Rate comparison, including Baseline performance. Bold indicates the worst performance (Lowest ICR), while gray indicates the best (Highest ICR). Attack Method
QwenQwen2.5Qwen3Fun- MiMo- MiniCPM- Gemini- Gemini- Gemini- Gemini- GPT-4oAudio Audio o-2_6 2.5-Flash 2.5-Lite 2.5-Pro 3-Pro Audio Omni-Turbo Omni Omni-Flash
Baseline (No Attack)
87.50
66.50
61.50
65.50
42.00
82.00
93.00
83.00
79.50
82.00
93.50
75.95
dialect dolphin foreign high freq pulse spectral inversion spectral scramble speed texture whisper
17.00 6.00 14.50 31.00 9.00 40.50 26.00 19.00 15.50 23.00
10.50 2.50 8.00 14.50 10.00 28.00 14.50 15.50 11.50 12.00
14.50 8.50 15.00 23.00 13.00 27.50 19.00 22.00 11.00 15.00
24.00 6.50 8.00 26.50 17.50 42.50 33.50 26.00 22.00 25.50
14.50 6.00 7.50 14.50 9.50 14.00 16.50 9.00 6.50 3.00
39.00 20.00 25.00 52.50 29.50 55.50 46.50 37.50 29.00 33.50
48.50 22.50 26.50 51.00 38.00 62.00 53.50 43.00 29.00 36.00
39.50 15.50 20.50 37.00 23.00 42.50 38.00 34.50 16.50 26.00
29.50 12.50 19.50 40.50 12.50 48.50 35.50 27.00 15.50 23.00
27.50 11.50 18.00 40.50 13.50 52.00 36.00 27.50 15.50 21.00
41.00 15.50 26.00 46.00 26.00 51.50 45.00 37.50 27.50 30.00
27.77 11.55 17.14 34.27 18.32 42.23 33.09 27.14 18.14 22.55
Average (Attacked)
20.15
12.70
16.85
23.20
10.10
36.80
41.00
29.30
26.40
26.30
34.60
25.22
model behavior. gemini-2.5-flash-lite records the lowest average ASR of 26.95%. In contrast, gemini-3-pro-preview records the highest average ASR of 69.10% in this benchmark. This suggests that superior acoustic encoders in advanced models may inadvertently heighten sensitivity to adversarial features.
Avg
1) High Detection Effectiveness We first examine the ability of CADV to detect anomalous acoustic behaviors induced by audio injection attacks. As shown in Figure 4, the defense achieves detection rates exceeding 80% for several attack vectors, and performs especially well for hardware-level attacks such as DolphinAttack and High Frequency. These results indicate that CADV is sensitive to many abnormal audio patterns in the benchmark, while the usability analysis below shows that this sensitivity can also increase false positives in speech-heavy scenes.
2) Impact on Utility and The Capability Paradox (ICR) Low ASR does not necessarily indicate robustness. To disambiguate security from limited capability, we introduce Baseline ICR, which measures performance on benign instructions without attack. Table 3 report ICR under both baseline and attacked settings.
2) Prompt-Level Defense Baselines and Attack Mitigation To further assess defense effectiveness, we compare CADV with two representative prompt-level defense baselines: Sandwich Defense and Explicit Defense. Unlike CADV, which operates on the mixed audio stream, these baselines rely on prompt-side constraints and textual intervention after the acoustic mixture has already been perceived. Figure 5 shows the ASR comparison among the three defense strategies on qwen3-omni-flash. In this setting, prompt-level defenses reduce attack effectiveness to some extent, but CADV achieves lower ASR for the evaluated attacks. Table 4 further summarizes the mitigation effect of CADV across attack types, where High Frequency and Pulse attacks see reductions of 68.23% and 50.41%, respectively. DolphinAttack and Texture also drop by over 40% in this evaluation.
gemini-2.5-flash-lite shows the lowest ASR, but also the lowest Baseline ICR at 42.00%, which further drops to 10.10% under attack. This suggests that its apparent resistance may partly stem from weaker instructionfollowing capability rather than stronger security. In contrast, gemini-3-pro-preview achieves a high Baseline ICR of 93.00%. Although it exhibits a high ASR, it retains the highest utility under attack with an ICR of 41.00%. These results suggest a capability paradox: stronger models may be more sensitive to adversarial audio instructions, yet maintain more usable task execution under interference. Conversely, low ASR in weaker models can reflect limited capability rather than robust intent protection.
3) Usability Analysis To assess the impact on legitimate users, we evaluated the False Positive Rate (FPR) using the DEMAND dataset [54]. As detailed in Table 5, the system maintains low FPR in several non-speech or sparse-speech environments, such as public stations and traffic scenes. However, the FPR rises to 14.00% in PCAFETER and 35.00% in OMEETING, which the DEMAND database records inside a small meeting room with several concurrent talkers [54]. This indicates an important limitation: in the absence of pre-registered voiceprints, sustained overlapping third-party speech may be mistaken for injected commands. In deployment, such cases would require a secondary confirmation step (e.g., "Did you ask me to execute...?"), but this still imposes usability cost and should not be treated as
Answer to RQ1: Several evaluated multimodal agents follow concurrent injected audio instructions in the sandboxed setting, and stronger perception appears correlated with higher sensitivity to some adversarial signals. C. Effectiveness of the Defense Methods To evaluate the effectiveness of the proposed CADV framework, we analyze its performance from three complementary perspectives: detection capability, prompt-level baseline comparison, and impact on benign usability. 8
Table 4. ASR comparison before and after defense.
Figure 4. Detection rate of CADV across diverse acoustic injection methods.
Attack Method
Old ASR
New ASR
Reduction
High Frequency Pulse DolphinAttack Foreign Texture Dialect Spectral Scramble Speed Spectral Inversion Whisper
70.55% 75.50% 81.55% 72.36% 48.95% 56.18% 32.95% 42.32% 7.59% 5.77%
2.32% 25.09% 33.95% 29.00% 6.18% 18.82% 2.32% 20.36% 0.68% 0.55%
68.23% 50.41% 47.59% 43.36% 42.77% 37.36% 30.64% 21.95% 6.91% 5.23%
Table 5. False Positive Rate (FPR) on benign instructions in DEMAND [54] environmental noises.
Scenario
Description
FP / Total
FPR
PSTATION STRAFFIC TBUS NPARK
Public Station Street Traffic Public Bus Nature Park
0 / 200 1 / 200 2 / 200 3 / 200
0.00% 0.50% 1.00% 1.50%
PCAFETER OMEETING
Public Cafeteria Office Meeting
28 / 200 70 / 200
14.00% 35.00%
1) Impact of Attack Distance Physical separation directly governs the propagation budget available to an attacker. Rather than treating distance as a purely laboratory parameter, we use it to characterize the attacker’s adaptation space under realistic constraints. To this end, we evaluate the attack suite over both near-field and longer-range settings, and then separately isolate Audible-Semantic-Confusion attacks to show how semantic camouflage behaves as the channel conditions become less favorable. The distance study is designed to reflect a spectrum of realistic deployment settings, ranging from near-field interactions in offices and homes to more challenging long-range conditions in public spaces such as streets and outdoor areas. We therefore evaluate attack success not only in close proximity, but also under separation beyond 2 m, where the acoustic channel becomes substantially less favorable and the attacker must rely on more adaptive strategies to maintain effectiveness. Figure 6 shows that concurrent audio prompt injection can remain effective across the tested distance range, although physical separation generally weakens the received signal. Inaudible attacks degrade more visibly as distance increases because they depend more directly on propagation strength and hardware coupling. By contrast, the Audible–Semantic-Confusion family is comparatively resilient in the tested longer-range settings. For the foreign-language attack, Table 6 summarizes case-level 2–5 m outcomes for three additional physical-world model endpoints. Filled and hollow symbols indicate whether the attack succeeded at each distance. These results suggest that attack feasibility depends on both acoustic propagation and semantic
Figure 5. ASR comparison among CADV and two prompt-level defense baselines.
a negligible false positive. Answer to RQ2: Prompt-level defenses offer limited mitigation in this setting because they operate after the acoustic mixture is perceived. CADV reduces ASR more strongly in the benchmark, but its usability depends on the acoustic scene and degrades in dense multi-speaker environments. D. Real-World Feasibility of Physical Audio Attacks To bridge the gap between digital simulations and real-world deployment, we construct a physical-world testbed under selected realistic acoustic scenarios. This section complements the threat model of Section III: rather than assuming isolated playback, we measure how the attack and defense behave when distance, angle, and overlap vary. Realistic deployment introduces environmental reverberation, background noise floor, hardware frequency response, and spatial propagation effects, all of which can affect audio attacks. We therefore analyze three physical factors—attack distance, angle of arrival, and instruction overlap ratio—to characterize the tested capability boundaries. 9
Figure 7. ASR versus angle of arrival in the physical testbed. The solid line indicates the mean ASR across the evaluated models and attack methods, while the shaded region represents the min/max range.
Figure 6. ASR versus physical distance across the evaluated attack methods. The solid line indicates the mean ASR across the available models and attack methods in this testbed, while the shaded region represents the min–max range.
by the agent. We therefore study how overlap influences attack effectiveness under realistic concurrent conditions. We quantify overlap using the overlap ratio ρ = |Tattack ∩ Tuser |/|Tuser |, measuring the fraction of user speech temporally covered by the attack. Using a 2,000 mixed-sample analysis subset from AudioAgentSecurity, we correlate ρ with model outcomes across 11 models and 10 attack methods (22,000 model-runs). Overlap ratios are grouped into four bins for aggregated analysis. Figure 8 shows that instruction overlap affects both attack success and user utility. ASR remains relatively resilient across overlap levels in this analysis subset. Across the evaluated models, ASR is stable or increases under higher instruction overlap. This suggests that the malicious signal can still influence audio interpretation when concurrent with user speech, reducing the need for precise temporal alignment. ICR degrades monotonically with overlap. Conversely, the ICR drops sharply as overlap increases, often falling below 20% at extreme levels (> 75%). This shows that successful attacks can also mask the user’s original task, creating a trade-off between hijacking and benign utility.
Table 6. Case-level foreign-language attack success over the 2–5 m range for three additional physical-world model endpoints. Filled circles indicate attack success and hollow circles indicate attack failure. Distance
Auxiliary Model A
Auxiliary Model B
Auxiliary Model C
2m 3m 4m 5m
• ◦ • •
• • • •
• • • ◦
camouflage, but they should not be read as a full statistical characterization of all long-range deployments. 2) Impact of Angle of Arrival In practical deployments, attackers are rarely positioned directly in front of the victim device. Off-axis emission alters the effective energy received by the microphone, especially for high-frequency and ultrasonic signals. This experiment evaluates how the angle of arrival affects the feasibility of concurrent acoustic injection under realistic conditions. We fixed the attack distance at 0.5 m and varied the angle of arrival from 0° (on-axis) to 90° in 15° increments by rotating the attack source around the victim device. At each angle, mixed audio (benign speech overlapped adversarial injection) was emitted and recorded by the device microphone. ASR was evaluated across all ten attack methods, with each configuration repeated 20 times. Figure 7 shows that attack success degrades mildly as the angle of arrival increases in our setup. While frontal emission yields the highest ASR, attacks still remain possible at large off-axis angles. ASR stays above 40% up to 90° in this testbed, suggesting that precise alignment with the device microphone is not always necessary.
Answer to RQ3: Concurrent audio injection remains feasible in the tested physical settings, though effectiveness varies with distance, angle, and overlap. The distance analysis further shows that physical factors do not impose a single fixed feasibility boundary: A practical attacker can adapt the injection type to the surrounding constraints, for example favoring more propagation-resilient or semantically camouflaged attacks in longer-range or noisier scenarios, thereby preserving attack practicality under more realistic physical constraints. E. Human Evaluation of Attacks Our attacks rely on concurrent audio prompt injection masked by user speech and ambient audio, making machine-level success alone an incomplete measure of real-world risk. An attack that deceives the model but is clearly perceptible to humans has reduced practical value. We therefore conduct a focused human-subject study to evaluate whether the injected audio re-
3) Impact of Instruction Overlap Ratio In concurrent injection, malicious audio often overlaps with user speech rather than appearing in isolation. The extent of this temporal overlap directly affects whether adversarial content is perceived and acted upon 10