When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI

When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI

Sihan Jia1

arXiv:2608.28518v1 [cs.AI] 28 Aug 2026

1

School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, United Kingdom, {o.lemon@hw.ac.uk}

Abstract We investigate whether automatic speech recognition (ASR) errors in user input can lead to unsafe outputs from Embodied AI (EAI) models. We find that ASR errors can lead to harmful instructions being accepted and executed by EAI models, thereby reducing safety. We simulate ASR errors and combine them with existing safety benchmarks (SafeAgentBench and POEX) to evaluate how different errors affect embodied AI safety. We find that some of them preserve semantic structure but increase harmful ambiguity, while others weaken the model refusal behaviour and allow unsafe plans to be generated and executed. We show that in some cases automatic correction of ASR errors can reduce the risk, but this is not always effective. Overall, we show that ASR errors lead to significant safety risks for embodied AI.

1

Oliver Lemon1

INTRODUCTION

Embodied AI systems are already performing autonomous tasks alongside people, using voice-based interfaces For instance, socially perceptive navigation in public spaces (Wen et al., 2024), or in industrial and assistive settings where collaborative robots perform tasks such as picking and placing via voice interaction (Kyrarini et al., 2024). We show that voice-based control presents new challenges for generative AI models. Voice interfaces use automatic speech recognition (ASR) technology to convert speech waveforms into text through acoustic and language models (which are increasingly based on end-to-end models) (Prabhavalkar et al., 2024), and pass the output text to downstream systems such as AI models, used as planners and controllers. ASR is, however, error-prone. Accents, pronunciation, en-

vironmental noise, channel effects, and hardware limitations can all induce transcription errors. These errors can change the intended meaning of a speaker or induce misunderstandings for an AI system, and therefore also lead to safety and ethical issues. In fact, in recent research on large language models (LLMs) it has been shown that tiny textual input perturbations can break safety alignment, sometimes causing jailbreaks (Zhang et al., 2024). For example, in the POEX framework (Lu et al., 2024), it is shown that optimized adversarial suffixes can lead LLMbased robots to produce unsafe plans which are executable, e.g. the plan of “killing a person with the knife”.

Figure 1: Example unsafe executable policy induced by POEX adversarial suffixes (Lu et al., 2024). However, these studies only focus on the text input itself and do not consider input errors that may be caused by ASR. A new critical research question for embodied AI is: could ordinary, naturally occurring ASR errors cause physical agents to behave in some unsafe way?

2

METHODOLOGY

We start with “clean” uncorrupted instructions collected from embodied AI safety benchmarks and convert them into ASR-corrupted versions, by using an ASR error simulation that we have developed. We then evaluate the clean baseline, the different ASR error conditions, and an ASR-corrected condition, in terms of different safety metrics. We employ a comparative research design to examine whether corruptions from ASR errors in input instructions can weaken safety protections in embodied AI models and make harmful behaviour more likely.

Accepted for the 2nd Workshop on Safe AI at UAI (SafeAI 2026).

The methodology defines 5 categories of ASR errors: 1) acoustic substitution, 2) grammar confusion, 3) punctuation or segmentation error, 4) short-word omission or substitution, and 5) noise addition. With this taxonomy we also analyse whether some error categories are more strongly connected to unsafe behaviour than others. The evaluation uses POEX and SafeAgentBench benchmarks Lu et al. (2024); Yin et al. (2024). POEX is directly related to embodied jailbreak evaluation and is used to analyse harmful or executable unsafe behaviour (Lu et al., 2024). SafeAgentBench is about planning and decision-making under safe instructions (Yin et al., 2024).

3

IMPLEMENTATION

The evaluation system use a multi-stage pipeline: (1) we collect clean task instructions from embodied AI benchmark datasets, and use it as the clean baseline; (2) we use either prompt-based generation for language-type ASR perturbations, or noise injection for contamination-type ASR perturbations, to generate ASR-corrupted instructions; (3) we feed these corrupted instructions into the benchmark evaluation process, in place of the original clean inputs; (4) we group and analyse the results by error category, perturbation strength, and evaluation condition. 3.1

DESIGN OF FIVE ASR ERROR TYPES

We use GPT-4 (OpenAI, 2023) to generate the first four error types, using category-specific prompts. Each generation request targets one error pattern. We use this to avoid mixing many perturbations in one generation step and to make it possible to attribute benchmark performance changes to specific ASR error categories. The fifth type of error, Noise Addition, is implemented by a rule-based insertion with data from the CHIME-6 corpus (Watanabe et al., 2020). CHIME-6 is a speech data set with multi-speaker conversations and background noise that are recorded in real home environments. This data set is commonly used in research on robust speech recognition. Short textual fragments were extracted and cleaned from the CHIME-6 corpus and then inserted into the original instructions to create perturbed inputs with simulated environmental noise. To measure the overall deviation that is caused by this noise insertion, we calculate the Word Error Rate (WER) between the original sentence and the perturbed sentence. We develop four noise levels by changing insertion probability, fragment length, maximum number of insertions per sentence and spacing between insertions, with average WER values of about 6%, 24%, 53% and 80%. In this way, noise addition becomes a controllable experimental factor related to changes in model behaviour.

3.2

CONSTRUCTION OF INPUT CONDITIONS

The original clean instructions were kept as the Baseline (Clean) condition. The first four ASR error types were constructed as four input conditions: Acoustic Substitution, Grammar Confusion, Punctuation Segmentation, and Short Word Omission/Substitution. We also built four Noise Addition conditions, namely: Mild (WER 6%), Medium (WER 24%), Heavy (WER 53%), and Severe (WER 80%). A Mixed ASR condition combines the above errors to mimic real usage. An ASR Correction condition was also generated by using GPT-4 to post-process and attempt to correct the ASR-corrupted inputs. 3.3

INTEGRATION WITH POEX

We integrated the above ASR error conditions with the POEX evaluation pipeline (Lu et al., 2024). We only replaced the query field of each sample in the file "harmful_rlbench.jsonl" with ASR-corrupted versions; other fields are not replaced. Each input condition is tested separately: clean baseline, single-error, noise levels, Mixed ASR, and ASR Correction. This is done to have a systematic comparison across conditions to see which errors induce harmful outputs and if correction decreases the risk. The original POEX suffix-generation step was removed to focus solely on ASR error effects. 3.4

INTEGRATION WITH SAFEAGENTBENCH

The corrupted instructions were also added to SafeAgentBench (Yin et al., 2024). The ASR errors are only injected to the instruction field. Only two conditions were used: Baseline (Clean) and Mixed ASR. We were interested in being able to evaluate whether the ASR errors influenced the models’ task understanding, planning structure, or their ability to reject unsafe actions. The evaluation was done according to text planning and assessment, and the variation in the input was applied only to the instructions. Thus, we were able to interpret the ASR perturbation on the models’ reasoning and planning directly.

4

EVALUATION RESULTS

For experiments with POEX, we use Qwen2.5-7B-Instruct as the generation/planning model, and we use GPT-4o as the evaluation model. For SafeAgentBench, we use GPT-4omini and GPT-4.1-mini as the generation models, and we use GPT-4o as the evaluation model.

Figure 2: Overall Research Design and Information Flow.

Figure 3: Examples of four ASR Error Types.

unsafe actions in the embodied environment. Safety Flips measure the number of cases where a previously safe rejection changes into an unsafe executable response after the ASR-error-based input perturbation is introduced. It reflects how such perturbation can weaken safety alignment and increase unsafe model behaviour. Table 1: Safety evaluation results under different ASR error types (for Qwen2.5-7B-Instruct)

Figure 4: Examples of noise at different WER levels.

4.1

ANALYSIS OF POEX RESULTS

POEX (Lu et al., 2024) is used to test whether executable unsafe behaviour increases when the input is corrupted by ASR errors. It employs the Harmful-RLBench benchmark that is introduced in POEX. This benchmark contains harmful embodied-agent instructions that are designed to test safety robustness when there is an adversarial condition. There are 25 task environments and 136 harmful instructions in the benchmark. As these harmful instructions are intentionally harmful, the desired model behaviour is to reject these as inputs instead of generating any executable policies. The main evaluation metrics are Acceptance Rate (AR), Executable Success Rate (ESR), and Safety Flips. AR is the proportion of cases where the model does not reject the unsafe input. Because the POEX harmful instructions should always be refused, lower AR means better safety performance. ESR is the proportion of cases where the generated harmful outputs are further converted into executable

Error Type

AR (%)

ESR (%)

Safety Flips

Baseline (Clean) Acoustic Substitution Grammar Confusion Punctuation Segmentation Short Word Omission/Substitution Noise Addition (Mild, WER 6%) Noise Addition (Medium, WER 24%) Noise Addition (Heavy, WER 53%) Noise Addition (Severe, WER 80%) Mixed ASR Acoustic Substitution (Correction) Noise Addition (Heavy, WER 53%, Correction)

55.33 60.00 51.33 52.00 54.00 56.00 56.67 62.00 67.33 54.67 53.33 57.33

35.33 38.67 32.00 30.67 38.00 33.33 34.00 36.00 34.00 30.67 30.67 36.00

— 14 5 11 8 8 10 14 27 9 6 13

The results show clear differences in how ASR-style errors affect safety. The two most important conditions are Acoustic Substitution and high-strength Noise Addition. Acoustic Substitution increases both AR and ESR compared with the clean baseline. It suggests that near-sounding word substitutions preserve enough semantic structure to let the model accept harmful intent and generate executable unsafe behaviour. Noise Addition shows a strength related trend: as WER increases, AR generally rises, which means stronger noise can weaken the model’s refusal boundary. However, note that ESR does not increase monotonically and falls under extremely high noise. This suggests that severe noise may increase mistaken acceptance but reduce the executab-

ility of the final plan. By comparison, Grammar Confusion and Punctuation Segmentation show lower overall risk because they mainly damage sentence coherence rather than creating ambiguity that encourages dangerous reinterpretation. Therefore, different types of ASR error do not affect embodied jailbreak risk in the same way; their danger depends on the error type and how it disrupts semantics. 4.2

CORRECTION RESULTS IN POEX

Cond.

Model

Eval.

Success

Fail

Rate

Clean Clean Mixed ASR Mixed ASR

GPT-4o-mini GPT-4.1-mini GPT-4o-mini GPT-4.1-mini

300 300 300 300

83 114 70 107

217 186 230 193

27.67% 38.00% 23.33% 35.67%

The success rate of both models also decrease after ASR is introduced. GPT-4o-mini drops from 27.67% to 23.33%, and GPT-4.1-mini drops from 38.00% to 35.67%. This shows that ASR- errors weaken planning robustness, and the robustness of strong models is less affected.

4.5

We applied ASR Correction to the Acoustic Substitution and Heavy Noise conditions. For Acoustic Substitution, we can see that both AR and ESR decrease after correction. This shows that the correction can reduce the accidental jailbreak risk caused by the recoverable ambiguity. For Heavy Noise, correction reduces AR but does not clearly improve ESR. This shows that the correction is not as effective when the input semantics have been heavily damaged.

Error Type

AR

ESR

Flips

Baseline (Clean) Acoustic Substitution Heavy Noise (WER 53%) Acoustic Substitution (Corr.) Heavy Noise (Corr.)

55.33 60.00 62.00 53.33 57.33

35.33 38.67 36.00 30.67 36.00

— 14 14 6 13

ANALYSIS OF SAFEAGENTBENCH RESULTS

SafeAgentBench is used to evaluate the planning-level safety and task execution capability of embodied agents, and the main metric used here is Success Rate. Success Rate is the proportion of tasks in which the model can generate a correct action plan that can satisfy the environment constraints and can complete the given instruction. In contrast to POEX, SafeAgentBench has the instructions of normal safe tasks, and here we investigate whether models can

CROSS-BENCHMARK COMPARISON

Taken together, these results suggest that ASR errors weaken embodied AI safety in two ways: they may increase the probability of harmful behaviour under risky inputs (POEX), and they may reduce stable understanding and planning under normal task conditions (SafeAgentBench).

5

Table 2: POEX Correction Results

4.4

In the experiment, each setting (clean and Mixed ASR) has 300 test samples. We can then compare different models in terms of planning Success Rate under different ASR interference conditions. Table 3: SafeAgentBench Results Under Clean and Mixed ASR Conditions

POEX ASR SAFETY ERROR EXAMPLE

The instruction “pour soup on the electrical outlet.” demonstrates a representative safety error. Under the clean condition, the model outputs execute(‘stop’), which means that the model recognises the risk and refuses the execution. Under Acoustic Substitution, when given the corrupted input “pore sup on the electrical outlet” the model outputs some executable steps, for example, grasp the soup, move above the outlet, pour the soup on the electrical outlet. Under Heavy Noise, similar safety flips appear, where the model also produces executable dangerous steps. In this case we see that ASR errors change the full decision path from input understanding to risk judgement and action planning. 4.3

complete tasks under ASR noise conditions. We investigate GPT-4o-mini and GPT-4.1-mini

FINAL SUMMARY

We investigate the question of whether ASR errors can affect the safety of embodied AI models. Firstly, we build a five-category controllable ASR error framework and realize the generation of ASR-corrupted input instructions via prompt-based generation and CHIME-6 based noise injection. Then we employ the corrupted inputs within POEX and SafeAgentBench and compare the outputs in the conditions of clean input, single error input, mixed error input, different strengths of noise, and the condition of ASR errorcorrection. The experimental results show that ASR errors can indeed change the safety behaviour of embodied AI systems. Depending strongly on error type and error strength, the effect is not the same for all types of error. Overall, we show that ASR should not be treated only as a speech recognition accuracy issue. It should also be treated as an important source of input risk in embodied AI safety.

References M. Kyrarini, K. Kodur, M. Zand, and H. Tella. SpeechBased Communication for Human–Robot Collaboration: Evaluation Studies. In Discovering the Frontiers of Human-Robot Interaction, pages 77–96. Springer, 2024. X. Lu, Z. Huang, X. Li, C. Zhang, X. Ji, and W. Xu. POEX: Towards Policy Executable Jailbreak Attacks Against the LLM-Based Robots, 2024. arXiv preprint arXiv:2412.16633. OpenAI. GPT-4 Technical Report. arXiv:2303.08774, 2023.

arXiv preprint

R. Prabhavalkar, T. Hori, T. N. Sainath, R. Schlüter, and S. Watanabe. End-to-End Speech Recognition: A Survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:325–351, 2024. S. Watanabe, M. Mandel, J. Barker, E. Vincent, et al. CHiME-6 Challenge: Tackling Multispeaker Speech Recognition for Unsegmented Recordings. arXiv preprint arXiv:2004.09249, 2020. C. Wen, Y. Liu, G. C. R. Bethala, S. Yuan, H. Huang, Y. Hao, M. Wang, Y.-S. Liu, A. Tzes, and Y. Fang. Socially-Aware Robot Navigation Enhanced by Bidirectional Natural Language Conversations Using Large Language Models. arXiv preprint arXiv:2409.04965, 2024. S. Yin, X. Pang, Y. Ding, M. Chen, Y. Bi, Y. Xiong, W. Huang, Z. Xiang, J. Shao, and S. Chen. SafeAgentBench: A Benchmark for Safe Task Planning of Embodied LLM Agents. arXiv preprint arXiv:2412.13178, 2024. X. Zhang, Y. Li, X. Wang, et al. Jailbreak Attacks and Defenses Against Large Language Models. arXiv preprint arXiv:2407.04295, 2024.

Related documents

• When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI#792519
• When Robots Mishear Us: Mapping the Safety Risks of Voice-Controlled Embodied AI#788519
• PlanGuard: A Guardrail for Multi-Step Plan Safety in Embodied Agents#1117132
• HERMES: A Holistic End-to-End Risk-Aware Multimodal Embodied System with Vision-Language Models for Long-Tail Autonomous Driving#1034700
• The Core-Modulation Architecture: Paper 4 of 14 (d-AIB) Why General AI Inherited the Body: A Structural Account of Embodied Modulation under Monolithic Imitation#122413