Preprint. Under review.
S PEECH EQ: Benchmarking Emotional Intelligence Quotient in Socially Aware Voice Conversational Models Liang-Yuan Wu1 Zih-Ching Chen2 Tongshuang Wu3 C.-H. Huck Yang2 Hua Shen1, 4 1 New York University 2 NVIDIA 3 Carnegie Mellon University 4 NYU Shanghai {leo.wu,huashen}@nyu.edu; {virginiac,hucky}@nvidia.com; [email protected]
arXiv:2606.25990v1 [cs.CL] 24 Jun 2026
Abstract As multimodal conversational systems increasingly engage in spoken interaction, their ability to navigate paralinguistic social cues has become a critical bottleneck for natural human-AI communication. However, existing evaluations of machine emotional intelligence assess reasoning exclusively through isolated text or passive acoustic perception, overlooking the complex cross-modal reasoning required for active, multi-turn dialogue. We introduce S PEECH EQ, a comprehensive framework designed to evaluate the sociolinguistic reasoning of Speech-Language Models (SLMs). The framework includes a validated dataset of 2,265 dialogues across 15 Emotional Quotient (EQ) subscales grounded in EQ-i 2.0 theory, along with a multi-turn evaluation protocol measured by our proposed Spoken EQ (SEQ) score inspired by human EQ assessments. Experiments show limitations in how both existing Speech Emotion Recognition and end-to-end Speech-Language Models understand and apply paralinguistic cues through speech. While end-to-end architectures outperform cascaded systems, S PEECH EQ reveals that current multimodal models remain bottlenecked by a text-reliant “modality shortcut,” an alignmentinduced “safety trap,” and “contextual amnesia,” highlighting the barriers to truly emotionally aware AI. Our benchmark can be accessed at https://huggingface.co/datasets/SpeechEQ/SpeechEQ and demo page at https://binomial14.github.io/speecheq-demo/
1
Introduction
Recent advances in Speech-Language Models (SLMs) have enabled a new generation of end-to-end voice agents capable of fluent (Défossez et al., 2024; Reddy, 1988), real-time interaction (Rubenstein et al., 2023; Zhang et al., 2023; Chu et al., 2024; Barrault et al., 2023; Ye et al., 2025; Deshmukh et al., 2026). These systems excel at semantic understanding, transcribing speech, answering questions, and generating coherent dialogue. However, human communication is not purely semantic (Scherer, 2003). In spoken interaction, how something is said, through prosody, timing, and vocal intensity, often carries more social meaning than what is said (Wu & Jain, 2025; Kim et al., 2023). This gap exposes a fundamental limitation: today’s SLMs are semantically fluent but socially shallow. They frequently produce affectively flat responses and struggle to interpret or generate paralinguistic cues that signal empathy, tension, or intent (Qian et al., 2025). As a result, even highly capable systems fail in scenarios where Emotional Intelligence (i.e., EQ) (Salovey & Mayer, 1990; Elfenbein & Ambady, 2002), not factual correctness, determines interaction quality. We argue that this limitation stems from a deeper issue: the lack of rigorous evaluation for multimodal Emotional Intelligence in speech. Existing benchmarks either (i) evaluate emotional intelligence in text-only settings or (ii) treat speech as a passive perception task (e.g., emotion classification), ignoring the interactive, multi-turn, and cross-modal reasoning required in real conversations. Consequently, current models can achieve high performance while relying on a “semantic shortcut”, bypassing acoustic reasoning altogether. 1
Preprint. Under review.
To address this gap, we introduce S PEECH EQ, a benchmark and evaluation framework for multimodal emotional intelligence in spoken dialogue. S PEECH EQ is built on three key principles: (1) Behavioral Grounding via EQ-i 2.0. We operationalize emotional intelligence using the EQ-i 2.0 framework (Bar-On, 2004; Wiechorek, 2011), constructing scenarios that map psychological constructs (e.g., empathy, impulse control) into observable acoustic behaviors. (2) Semantic–Acoustic Decoupling. We isolate acoustic reasoning by presenting models with response options that share identical transcripts but differ in paralinguistic delivery. This removes semantic cues and forces models to rely on pure acoustic understanding. (3) Sustained Affective Pragmatics. Rather than isolated utterances, we evaluate multi-turn dialogues with escalating emotional stakes, testing whether models can track and adapt to evolving social dynamics over time. The resulting dataset comprises 2,265 multi-turn dialogues (42.37 hours) spanning 15 EQ subscales, generated via a controlled LLM–TTS pipeline that enforces both behavioral validity and acoustic contrast. To quantify performance, we introduce the Spoken Emotional Quotient (SEQ), a standardized metric drawing conceptual inspiration from Raven’s Standard Progressive Matrices (Raven et al., 1998; John & Raven, 2003). SEQ aggregates multi-turn trajectory accuracy across EQ dimensions, capturing not only immediate recognition but also sustained emotional reasoning. We show that SEQ strongly correlates with human judgments, establishing it as a reliable proxy for evaluating EQ in speech. Using S PEECH EQ, we benchmark both cascaded pipelines and state-of-the-art end-toend SLMs. While end-to-end models perform better overall, our analysis reveals three fundamental limitations: (i) Modality Shortcut: Models over-rely on text and fail when meaning is carried purely by acoustics. (ii) Affective Flattening: Alignment mechanisms bias models toward safe, low-arousal tones, suppressing necessary emotional expression. (iii) Contextual Amnesia: Performance degrades over multi-turn interactions, indicating weak long-term affective tracking. These findings suggest that current SLMs do not truly reason about emotion–they approximate it under favorable conditions. Overall, the contributions are three-fold: • A Grounded Paralinguistic Benchmark: We introduce S PEECH EQ, a multi-turn speech benchmark grounded in 15 EQ-i 2.0 dimensions. By decoupling text and prosody, it isolates acoustic signals and enables rigorous evaluation of paralinguistic reasoning. • A Comprehensive Evaluation Framework and Metric: We propose a unified evaluation protocol for both cascaded and end-to-end models, along with the Spoken Emotional Quotient (SEQ)—a trajectory-level metric for measuring emotional intelligence across multi-turn interactions. • Empirical Insights: We benchmark state-of-the-art models and identify three failure modes: modality shortcut, affective flattening, and contextual amnesia, revealing key limitations in current speech-language systems.
2
S PEECH EQ: Evaluating Emotional Intelligence in Speech LMs
This section details the development of S PEECH EQ, an integrated evaluation framework and dataset. We first outline the motivation and design rationale, followed by a detailed description of the generation pipeline and validation process. Finally, we formalize the framework’s evaluation protocol and introduce the Spoken Emotional Quotient (SEQ), a standardized metric for quantifying SLMs’ emotional intelligence. 2.1
Motivation and Design Rationale
Attributable Behavior Design. Our primary goal is twofold: to move beyond passive classification in traditional speech emotion recognition (Cowie et al., 2001; Burkhardt, 2000; Schuller, 2018) by rigorously evaluating SLMs in active social settings, and to ensure that the socially resonant responses can be distinctly isolated within the audio waveform. To achieve this attributable behavioral design, we ground our dataset in the EQ-i 2.0 framework (Bar-On, 2004; Wiechorek, 2011). EQ-i 2.0 is a trait-behavioral model that operationalizes 2
Preprint. Under review.
1. Scenario Generation
2. Dialogue Generation
3. Tone Generation
4. Tone Filter
5. TTS Synthesis
“Initialize the conversational scenario based on specific EQ-i 2.0 subscale, valence and type using gpt-4o”
“Generate a dialogue based on the scenario information, and make turn 4 and 6 as target utterances.”
“Generate explicit tone instructions tailored to the conversational context and the assigned speaker personas”
“Select the optimal pair of candidate tones to maximize the acoustic contrast emotionally and socially.”
“Synthesize the final audio by combining the semantic transcripts with the paired tone instructions”
Speaker 1
Personas
TU1 Tone Evaluation
Target Utterance 1
Input Dimensions
1
📚 EQ-i 2.0 Subscale
Context I’d like to talk about the project deadline.
Self-Perception | Interpersonal | …
👥 Scenario Valence
Speaker 2
2
Context
Positive | Negative | Conflict
🏞 Scenario Type
Sure, what’s on your mind?
Workplace | Medical | …
Speaker 1
3
Peak Moment
🙂 Calm
😒
😠
Analytical Defensive
🤔 Warm, steady, soft 🧐 Neutral, even, measured 😣 Tight, frustrated, irritated Optimal Pair for TU1
Tone description generation
Resonant Warm, steady, soft
Dissonant Tight, frustrated, irritated
🤔
😣
I’m worried we’re falling behind.
Generation Engine GPT-4o
Tone Candidates Speaker 2
4
Target Utterance 1 Well, I felt unprepared for the last review.
Scenario Title | Context | Description
Speaker 1
5
Speaker 1 Speaker 2
Sustain I appreciate you being honest.
Role | Gender | Persona
Target Utterance 1 🤔 Warm, steady, soft 🧐 Neutral, even, measured 😣 Tight, frustrated, irritated
6
Speaker 2
Target Utterance 2
Role | Gender | Persona
Target Utterance 2 🙂 Reassuring, calm, confident 😐 Neutral, steady, factual 😬 Hesitant, uncertain, strained
TU2 Tone Evaluation
Target Utterance 2
🙂 Reassuring, calm, confident 😐 Neutral, steady, factual 😬 Hesitant, uncertain, strained
Reassuring, calm, confident
Optimal Pair for TU1
Resonant
Dissonant Hesitant, uncertain, strained
🙂
😬
I’ll make a plan to improve.
Figure 1: Overview of the S PEECH EQ dataset construction pipeline. social functioning into measurable subscales (more details in Appendix A). This behavioral focus provides the exact mechanism needed to translate complex psychological constructs directly into distinct, measurable acoustic features. Tone Variation Design. To mitigate lexical bias (Chen et al., 2026; Wang et al., 2020) and rigorously evaluate acoustic emotional intelligence, we designed the task as a forced-choice selection between two audio responses sharing identical transcripts. By neutralizing the text modality, we eliminate semantic differences and force the system to evaluate subtle paralinguistic cues to determine the contextually resonant response. Multi-turn Conversational Arc. To capture emotional intelligence beyond single utterances, we evaluate how models track cues across sustained interactions. We structure scenarios as three-exchange dialogues between a human Catalyst and the SLM acting as the Test Subject. Following an initial exchange that establishes the emotional baseline, the system must navigate escalating social pressure by selecting the contextually appropriate acoustic response during the second and third exchanges. This design effectively tests the model’s capacity for complex sociolinguistic pragmatics over an evolving conversational trajectory. 2.2
Data Generation
We developed an automated, LLM-driven generation pipeline (Figure 1) to execute the design rationales. We highlight the scenario and tonal instruction generation with the complete five-stage technical details and prompts in Appendix B. Scenario Generation and Persona Matrix. We engineered a highly constrained scenario matrix under the EQ-i 2.0 framework. Each scenario is generated at the intersection of three parameters: a specific EQ-i 2.0 subscale (e.g., Interpersonal, Empathy), a situational valence (Positive, Negative, or Conflict), and a real-world scenario (e.g., workplace, medical, educational). Crucially, to ensure the forced-choice evaluation is rigorous, the pipeline generates distinct “social deficit personas” corresponding to the targeted EQ scale, such as a “Toxic Optimist” failing to validate grief. This ensures the evaluated model is tested against complex sociolinguistic breakdowns rather than generic antagonistic behavior. Tone Generation and Contrast Filtering. The tone generation phase bridges the gap between the abstract psychological personas and raw audio synthesis over these neutral texts. We prompt the LLM to generate physically grounded acoustic descriptors, translating three generated personas (one contextually appropriate response and two dysregulated distractors) into explicit vocal instructions. To counter the default safety alignment of 3
Preprint. Under review.
generation models, we apply a filtering step that rejects minimizing descriptors such as ’polite’ or ’calm,’ enforcing the generation of extreme, physically grounded acoustic markers. Finally, an automated filtering stage selects the two most distinctly contrasting instructions for synthesis via gpt-4o-mini-tts-2025-03-20, producing distinctly nuanced paralinguistic variations for speech candidates. 2.3
Data Validation
To ensure the quality of S PEECH EQ, we adopt a two-phase validation pipeline. Phase 1 (automated) verifies scenario consistency and acoustic distinctiveness, while Phase 2 (human) evaluates naturalness and perceptual validity.
Qwen2.5-3B Qwen2.5-7B Qwen3-30B
Acc1
Acc2
Acctraj
98.6% 99.6% 99.7%
99.1% 99.9% 100.00%
97.7% 99.5% 99.7%
Table 1: Semantic validation results. Semantic and Logical Verification. We first conduct an oracle text-based evaluation by providing models with transcripts and explicit tone instructions instead of audio (e.g., “Fast pacing, sarcastic cheerfulness.”). Models achieve near-perfect accuracy (Table 1), confirming that scenarios are logically consistent and unambiguously aligned with the 15 EQ-i 2.0 dimensions (Bar-On, 2004; Wiechorek, 2011) (see Appendix A for the complete taxonomy). This result isolates the challenge of S PEECH EQ to the modality gap. For example, understanding paralinguistics from audio, rather than ambiguity in scenario design. Acoustic Variance Validation. To ensure meaningful paralinguistic contrast, we quantify acoustic differences between resonant and dissonant clips for each evaluation pair (Turns 4 and 6) using librosa. We extract six dimensions: mean pitch, zero-crossing rate (speaking-rate proxy), spectral centroid, RMS energy, mean MFCC, and duration. We compute a composite contrast score (max 8 points), with pitch and speaking-rate gaps contributing up to 2 points each and the remaining four up to 1 point each. Pairs scoring below 4 trigger up to three TTS regeneration attempts; examples that still fail are discarded. Human Expert Validation. We further assess perceptual validity through expert annotation. We sample 75 scenarios (5 per EQ subscale) and evaluate them across five dimensions: Generation Quality (Text and Audio), EQ Relevance (Text and Audio), and Answer Correctness (Paralinguistic Accuracy). Evaluation results are in Table 2, with formal definitions and literature grounding for these metrics in Table 5 in Appendix C. Two expert annotators achieve strong agreement (Cohen’s Kappa κ = 0.617) after iterative reconciliation. Any scenario failing a single criterion is removed, ensuring high-quality, socially valid data. Data Statistics. The final dataset comprises a total of 2,265 dialogues, perfectly balanced across the 15 EQ-i 2.0 subscales, totaling 42.37 hours of audio. The average length of one dialogue is 67.35 seconds (σ = 22.29), providing sufficient temporal context for evaluating sustained emotional tracking. Categories
Generation Quality
EQ Relevance
Answer Correctness
Metrics
Text Quality
Audio Quality
Semantic Relevance
Acoustic Reasonability
Paralinguistic Accuracy
Results
1.00
0.98
0.93
0.98
0.94
Table 2: Data validation results from human experts. 4
Preprint. Under review.
2.4
Evaluation Protocol for Emotional Intelligence
The Two-Round Selection Process. We evaluate models through a two-round, forced-choice task at Turn 4 and Turn 6 of each dialogue. In Round 1, the model receives the scenario context and initial history (Turns 1–3 audios), and must select the socially resonant audio for Turn 4. In Round 2, the context window is dynamically updated with the selected Turn 4 response and the subsequent Turn 5 utterance, requiring the model to select the correct Turn 6 response. This sequential dependency tests both immediate emotional recognition and sustained conversational tracking. Technical prompt details are in Appendix D. Evaluation Metrics. We report the accuracy of the model’s selection at the first evaluation turn (Acc1 ) and the second evaluation turn (Acc2 ). To measure sustained emotional tracking, we further report the conversational trajectory accuracy (Acctraj ). Inspired by Budzianowski et al. (2018) and Liu et al. (2023), this metric requires the model to successfully navigate the entire emotional arc. For a dataset of N multi-turn scenarios, let ŷi,1 and ŷi,2 denote the model’s predicted choices for the i-th conversation at Turns 4 and 6, with yi,1 and yi,2 representing the respective ground-truth resonant labels. The sustained accuracy is formally defined as the joint success across both evaluation turns, utilizing the indicator function I: Acctraj =
1 N I(ŷi,1 = yi,1 ∧ ŷi,2 = yi,2 ) N i∑ =1
(1)
This metric strictly requires the model to answer both consecutive turns correctly within the same evolving context window, and heavily penalizes models that lose conversational memory. Given the binary forced-choice design at each turn, the random chance baselines for Acc1 , Acc2 , and Acctraj are 50%, 50%, and 25%, respectively. We adopt Acctraj as the primary metric for cross-paper benchmark comparison, as it measures whether a model follows the target emotional arc without cohort-relative normalization. 2.5
SEQ Score
While Acctraj serves as the cross-paper durable metric, raw accuracy alone does not intuitively communicate relative model standing within a cohort. Inspired by the normreferenced scoring principle behind Raven’s Standard Progressive Matrices (Raven et al., 1998; John & Raven, 2003), we introduce the Spoken Emotional Quotient (SEQ) as a withincohort interpretability complement to Acctraj. For each model i, we first compute the raw score of the trajectory accuracy Acctraj as Xi . Global Standardization. We then perform global standardization to convert each model’s raw score Xi into a robust standardized score, denoted as Zi . To avoid the high sensitiveness of traditional standard deviation (Leys et al., 2013), we utilize the Median Absolute Deviation (MAD) as a robust statistical measure to compute a resilient standardization. Zi =
Xi − Median( X ) k × MAD( X )
(2)
where k ≈ 1.4826 is the standard scaling factor. This constant is derived from the inverse of the 75th percentile of the standard normal distribution (1/Φ−1 (0.75)), which ensures the MAD is asymptotically consistent with the standard deviation of a normal distribution (Rousseeuw & Croux, 1993). Final SEQ Score Computation. Following standard clinical psychometric scaling, we center the global distribution at a baseline of 100 with a scaled deviation of 15. We further apply a clinical cap at ±4 deviations to prevent extreme architectural outliers given our small model group. The final SEQ score is: SEQi = max(µ − 4σ, min(µ + 4σ, µ + σ × Zi ))
(3)
where µ = 100 and σ = 15 establish the normative baseline universally adopted in cognitive and emotional intelligence frameworks (Bar-On, 2004; Wiechorek, 2011). We strictly bound the metric at ±4σ to mirror the floor and ceiling limits of classical standardized assessments 5
Preprint. Under review.
Reasoning Performance
Model Random Baseline
Deployment Efficiency
Acc1
Acc2
Acctraj
SEQ
Cost ($)
Latency (s)
Speed (tok/s)
0.500
0.500
0.250
88.39
–
–
–
0.569 0.606
0.536 0.593
0.358 0.403
107.22 115.29
0.47 0.47
20.9 21.1
189.2 189.8
0.556 0.508 0.785 0.501 0.509 0.681 0.683 0.555
0.548 0.502 0.708 0.481 0.519 0.528 0.639 0.545
0.306 0.260 0.583 0.242 0.271 0.365 0.449 0.317
98.12 89.88 147.26 86.59 91.76 108.55 123.45 100.00
0.20 0.17 0.46 0.31 0.23 0.22 1.12 5.63
8.8 7.9 20.8 13.8 10.4 9.8 29.3 8.7
615.7 688.0 187.4 202.4 200.6 270.4 129.1 392.2
Cascaded Pipelines / Voice Agents emonum + Qwen3-30B emodes + Qwen3-30B End-to-End SLMs Qwen2.5-Omni-3B Qwen2.5-Omni-7B Qwen3-Omni-30B Kimi-Audio-7B-Instruct MiMo-Audio-7B-Instruct Fun-Audio-Chat-8B Gemini-2.5-pro gpt-audio-1.5
Table 3: S PEECH EQ evaluation results. For both cascaded systems and end-to-end SLMs, performance metrics evaluate isolated single-turn accuracy (Acc1 , Acc2 ), conversational trajectory accuracy (Acctraj ), and our standardized SEQ score. Deployment efficiency metrics highlight operational trade-offs, detailing the API or GPU compute cost (per 100 queries), average single-stream inference latency, and token throughput. (Wechsler, 1955), as scores beyond this range exceed the empirical measurement validity of psychometric instruments (Anastasi & Urbina, 1988).
3
Experimental Settings
To establish rigorous baselines for S PEECH EQ, we evaluate two distinct architectures: cascaded pipelines and end-to-end Speech-Language Models (SLMs). Cascaded Systems: To establish a lower bound simulating systems without native audio comprehension, we transcribe candidate audio using ASR (Whisper-large-v3 (Radford et al., 2023)) and extract Valence, Arousal, and Dominance (VAD) dimensions via a stateof-the-art SER module (audeering/wav2vec2-large-robust-12-emotion-msp-dim (Wagner et al., 2023)). We augment the ASR transcripts using two prompting strategies: appending the raw numerical VAD values, or mapping these dimensions into categorical text-based tone descriptions. These augmented transcripts are then fed into a text-only LLM (e.g., Qwen3 (Yang et al., 2025)) alongside the scenario background. End-to-End SLMs: We evaluate open-weight models across a range of scales: the QwenOmni series (Hui et al., 2024; Xu et al., 2025), Kimi-Audio-7B-Instruct (Ding et al., 2025), MiMo-Audio-7B-Instruct (Zhang et al., 2025), and Fun-Audio-Chat-8B (Team et al., 2025). We also evaluate two commercial APIs: Gemini-2.5-Pro (Comanici et al., 2025) and gptaudio-1.5 (Hurst et al., 2024). For all end-to-end models, the scenario background and dialogue history are provided as text, while candidate response options are interleaved into the context window as native audio clips.
4
Results
In this section, we present the quantitative results of the S PEECH EQ benchmark. We first evaluate the primary performance differences between end-to-end and cascaded architectures, followed by a human validation of the SEQ metric. We then conclude with targeted ablations that isolate two critical failure modes in state-of-the-art models: multi-turn “contextual amnesia” and the alignment-driven “safety trap.” 6
Preprint. Under review.
Figure 2: SEQ score for different cascaded systems and E2E SLMs. 4.1
Are SER models sufficient for paralinguistic reasoning?
State-of-the-art SLM outperforms its cascaded counterparts (Table 3), demonstrating the broader limitations of traditional SER models. Using the same reasoning backbone (Qwen3-30B), the cascaded systems rely on an explicit SER model to extract raw numerical VAD (Valence, Arousal, Dominance) values (emonum ) or translate these continuous acoustic features into descriptive text cues (emodes ). Overall, the end-to-end Qwen3-Omni-30B model (i.e., processes continuous speech directly) achieves a substantially higher SEQ. Interestingly, as illustrated in the left panel of Figure 2, the emodes cascaded pipeline achieves competitive performance on a few specific EQ subscales. This suggests that while SER pipelines adequately summarize isolated emotions, discretizing audio into text creates an information bottleneck that strips away the continuous acoustic nuances required to navigate complex, relational EQ dimensions. Within end-to-end SLMs, we observe strict deployment trade-offs between reasoning capability, latency, and operational cost (Table 3). We quantified these metrics via unbatched, single-stream inference on an NVIDIA A100 GPU for open-weight models, compared against OpenAI’s API. While the 30B Qwen3-Omni model dominates both Qwen2.5-Omni variants and gpt-audio-1.5 in raw EQ performance (the right panel in Figure 2), it suffers from high latency, taking 2.5× longer to respond than its smaller counterparts. Conversely, the 3B and 7B Qwen2.5 models offer fast, highly cost-effective inference but fail to achieve competitive reasoning scores. Finally, gpt-audio-1.5 strikes a strong balance in speed and token efficiency, but its API cost is over 10× higher than open-weight hosting. These constraints highlight a significant financial and architectural barrier to deploying real-time empathetic voice agents at scale. 4.2
Does the SEQ score reliably align with human perception?
The SEQ score is a significantly more reliable proxy for human sociolinguistic judgment than traditional discrete accuracy through an independent human evaluation. To validate SEQ’s reflection of SLMs’ emotional intelligence, we sampled one example from each of the 15 EQ subscales, and evaluated the outputs of six randomly selected anonymous models to avoid human bias. For each example, we recruited five native speakers from Prolific (Palan & Schitter, 2018) to rank the six tone selection and reasoning produced by each model. We then aggregate these rankings to derive a final rank for each model and compare the results with the rankings obtained from existing metrics (Acc1 , Acc2 , and their aggregate Acc all ) on the same 15 examples. Then we compute Spearman’s Rank Correlation Coefficient ρ with human rankings. SEQ achieves the highest correlation with human preference (ρ = 0.943, p-value = 0.005) outperforming traditional accuracy, confirming its effectiveness as a 7
Preprint. Under review.
reliable proxy for evaluating emotional intelligence in SLMs (see Table 4).
Human
Acc1
Acc2
Acc all
SEQ
Model A Model B ModelC Model D ModelE Model F
1 3 6 2 4 5
1 2 6 3= 3= 3=
1= 3= 5= 1= 5= 3=
1 2 6 3 5 4
1 2 6 3 4 5
correlation ρ (↑) p-value (↓)
-
0.820 0.046
0.837 0.039
0.886 0.018
0.943 0.005
Table 4: Correlation between human voted rankings and different metrics.
4.3
How does multi-turn contextual history affect paralinguistic reasoning?
Observing an 8% performance drop (0.785 → 0.708) between the first and second evaluation turn in our best model, Qwen3-Omni-30B, we hypothesized that standard Sequential Inference induces a form of contextual amnesia, a temporal degradation that closely aligns with context-loss phenomena observed in text-only LLMs (Liu et al., 2024; Laban et al., 2025; Lin et al., 2025). To isolate this effect and test our hypothesis, we conducted a comparative study evaluating standard Sequential Inference (performing inference twice and appending the model’s own turn-1 output as history) against Direct Inference (performing a single inference pass on turn 2 by treating the ground-truth turn-1 text as given history). Validating our hypothesis, bypassing the model’s self-generated history via Direct Inference successfully improved overall accuracy, raising Acc2 from 70.8% to 73.0%. However, a granular analysis across the 15 EQ subscales reveals that a significant performance gap remains, and the recovery is highly non-uniform. As illustrated in Figure 3, while most social dimensions exhibited a positive trend under Direct Inference, five dimensions experienced zero improvement or actually suffered performance degradations. This discrepancy highlights that multi-turn sociolinguistic reasoning is a complex task that extends beyond simple memory retention; even when temporal context-loss is explicitly mitigated with perfect semantic history, the model’s attention mechanism still struggles to balance expanded textual histories against immediate, short-term acoustic cues. While our ablation confirms the presence of a long-term memory leak, uncovering the exact cross-modal mechanisms that dictate how different emotional dimensions succeed or fail requires much deeper investigation.
Figure 3: Performance differences on turn 2 using Sequential Inference and Direct Inference strategies on Qwen3-Omni-30B. 8
Preprint. Under review.
Figure 4: SEQ score for Qwen3-Omni-30B with different persona. 4.4
What is the effect of persona conditioning on different EQ aspects?
Persona conditioning reveals a highly asymmetric ability of models to simulate different EQ traits. While some deficits cause severe degradation, others–particularly those aligned with default “safe” behaviors–have minimal impact. Replacing the default system prompt with an emotionally adaptive persona yields a modest improvement (SEQ: 147.26 → 148.86), whereas a global deficit persona leads to a substantial drop (SEQ: 94.98, in Figure 4 (left)). This confirms that persona conditions can meaningfully modulate emotional reasoning. More importantly, targeted deficits exhibit uneven effects across EQ dimensions. Deficits in Self-Perception and Self-Expression result in only minor degradation (SEQ: 133.72, 140.16), while Stress Management causes a catastrophic collapse (SEQ: 74.90) (Figure 4, right panel). This suggests that performance is strongly mediated by how each EQ dimension interacts with the model’s alignment constraints. We hypothesize that this asymmetry arises from RLHF-induced behavioral priors. Traits such as low assertiveness or reduced selfexpression resemble the model’s default polite and compliant behavior, resulting in limited performance loss (Sharma et al., 2023; Ouyang et al., 2022). In contrast, tasks requiring higharousal regulation, boundary-setting, or assertive responses conflict with safety alignment, preventing the model from producing necessary acoustic variation and leading to failure. We refer to Appendix E for the exact persona prompts utilized task-activating prompting (TAP) mechanism (Yang et al., 2023).
5
Discussion
S PEECH EQ reveals fundamental limitations in how current models handle social dynamics, leading to three key implications. Overcoming the Modality Shortcut. Modern multimodal models often behave as implicit cascaded systems, prioritizing text over acoustic reasoning (Chen et al., 2026). By presenting identical transcripts with contrasting prosody, S PEECH EQ exposes this semantic bias: performance drops sharply when semantic cues are removed. This suggests that current models treat paralinguistics as a secondary signal rather than a core reasoning modality. Future architectures must elevate acoustic signals to first-class status in social reasoning. Resolving Affective Flattening in Alignment. Current alignment strategies favor harmless, low-arousal responses, leading to a persistent bias toward “calm” and “polite” tones (Bai et al., 2022). We term this effect affective flattening. While safe, such expressive suppression undermines empathy in high-arousal interactions (Gross, 2002). This bias emerges in both model reasoning and TTS generation (Hurst et al., 2024). Advancing emotional intelligence requires decoupling safety from emotional expressiveness, enabling models to deploy a broader and context-appropriate affective range. 9
Preprint. Under review.
Toward Sustained Emotional Intelligence. We observe consistent performance degradation over multi-turn interactions, indicating “contextual amnesia” in acoustic reasoning (Liu et al., 2024). Due to dense audio tokenization and limited context capacity, models struggle to maintain long-horizon emotional coherence. Future benchmarks should move beyond short exchanges toward long-context, multi-session, persona-driven evaluations, testing whether agents can sustain and adapt emotional behavior over time.
6
Related Work
In psychology, Emotional Intelligence (EI) is traditionally modeled through cognitive abilitybased skills (Salovey & Mayer, 1990; Mayer et al., 2002) or trait-based behavioral dispositions (Bar-On, 2004; Petrides, 2009). As Large Language Models (LLMs) increasingly mediate human-AI interactions, evaluating their EI has become a critical focus to ensure trustworthiness and user engagement (Huang et al., 2020). Consequently, researchers have developed targeted benchmarks to measure the Emotional Quotient (EQ) of LLMs using established psychometric theory (Paech, 2023; Sabour et al., 2024). However, these evaluations rely exclusively on text, overlooking a crucial modality: speech. In natural human interaction, rich paralinguistic cues (e.g., pitch, pacing, and tone) often dictate the true emotional weight and intent of a conversation (Scherer, 2003; Hellbernd & Sammler, 2016). Conversely, affective computing in the speech domain has historically focused on Speech Emotion Recognition (SER) (Schuller et al., 2011; Schuller, 2018), mapping acoustic signals to affective labels using diverse curated datasets (Busso et al., 2008; Livingstone & Russo, 2018; Poria et al., 2019; Lotfian & Busso, 2017). SER currently serves as a core evaluation metric for Speech-Language Models (SLMs) (Yang et al., 2021) and fine-tuned self-supervised architectures (Baevski et al., 2020; Hsu et al., 2021; Liu et al., 2025). Yet, acoustic emotion recognition is merely a prerequisite for emotional intelligence (Mayer et al., 2002). True sociolinguistic intelligence requires cross-modal reasoning, evaluating a semantic transcript and its paralinguistic delivery simultaneously, to determine if a tone is contextually appropriate. To address this fundamental blind spot, our work systematically evaluates how effectively modern SLMs bridge this descriptive measurement (Chen et al., 2025) gap of semantic-acoustic in spoken dialogue. Several recent benchmarks evaluate multi-turn emotional intelligence in spoken dialogue, including Multi-Bench (Deng et al., 2025), HumDial-EIBench (Wang et al., 2026), and DeepDialogue (Koudounas et al., 2025). Despite this progress, three gaps remain. First, existing benchmarks present audio where semantics and vocal tone are inherently coupled; S PEECH EQ decouples them by offering multiple-choice responses with identical transcripts, forcing models to reason purely from acoustic cues. Second, prior work focuses on opendomain dialogues and categorical emotion labels, whereas S PEECH EQ is grounded in clinical psychometrics, mapping psychological constructs to acoustic behaviors through the 15 subscales of EQ-i 2.0. Third, rather than grading individual turns in isolation, S PEECH EQ evaluates sustained emotional reasoning across a full 6-turn conversational arc, probing a model’s capacity for long-horizon affective tracking with Speech-IQ (Wan et al., 2025) based user profile agentic measurement.
7
Conclusion
In this work, we introduced S PEECH EQ, the first benchmark evaluating conversational emotional intelligence in SLMs using the clinically validated EQ-i 2.0 framework. Through a semantic neutralization design that decouples lexical content from acoustic prosody, we established the SEQ score as a robust, human-correlated metric for measuring acoustic emotional intelligence. While end-to-end SLMs outperform cascaded architectures, our evaluation exposes three critical bottlenecks: a text-reliant “modality shortcut”, a safety trap causing “affective flattening”, and “contextual amnesia” during sustained multi-turn interactions. Ultimately, S PEECH EQ provides a rigorous diagnostic tool and roadmap for the community, emphasizing the need for alignment strategies that preserve paralinguistic nuance and decouple acoustic harmlessness from genuine emotional depth. 10
Preprint. Under review.
References Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977, 2020. Anne Anastasi and Susana Urbina. Psychological testing, volume 840. London, 1988. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural information processing systems, 33:12449–12460, 2020. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. Reuven Bar-On. The bar-on emotional quotient inventory (eq-i): Rationale, description and summary of psychometric properties. 2004. Loı̈c Barrault, Yu-An Chung, Mariano Cora Meglioli, David Dale, Ning Dong, PaulAmbroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, et al. Seamlessm4t: Massively multilingual & multimodal machine translation. arXiv preprint arXiv:2308.11596, 2023. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 conference on empirical methods in natural language processing, pp. 5016–5026, 2018. Felix Burkhardt. A database of german emotional speech. 2000. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42(4):335– 359, 2008. Chen Chen, Yuchen Hu, Siyin Wang, Helin Wang, Zhehuai Chen, Chao Zhang, ChaoHan Huck Yang, and Ensiong Chng. Audio large language models can be descriptive speech quality evaluators. In International Conference on Learning Representations, volume 2025, pp. 24920–24934, 2025. Jingyi Chen, Zhimeng Guo, Jiyun Chun, Pichao Wang, Andrew Perrault, and Micha Elsner. Do audio llms really listen, or just transcribe? measuring lexical vs. acoustic emotion cues reliance. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5848–5877, 2026. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al. Qwen2-audio technical report. arXiv preprint arXiv:2407.10759, 2024. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. Roddy Cowie, Ellen Douglas-Cowie, Nicolas Tsapatsoulis, George Votsis, Stefanos Kollias, Winfried Fellenz, and John G Taylor. Emotion recognition in human-computer interaction. IEEE Signal processing magazine, 18(1):32–80, 2001. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037, 2024. 11
Preprint. Under review.
Yayue Deng, Guoqiang Hu, Haiyang Sun, Xiangyu Zhang, Haoyang Zhang, Fei Tian, Xuerui Yang, Gang Yu, and Eng Siong Chng. Multi-bench: A multi-turn interactive benchmark for assessing emotional intelligence ability of spoken dialogue models. arXiv preprint arXiv:2511.00850, 2025. Amala Sanjay Deshmukh, Kateryna Chumachenko, Tuomas Rintamaki, Matthieu Le, Tyler Poon, Danial Mohseni Taheri, Ilia Karmanov, Guilin Liu, Jarno Seppanen, Arushi Goel, et al. Nemotron 3 nano omni: Efficient and open multimodal intelligence. arXiv preprint arXiv:2604.24954, 2026. Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, et al. Kimi-audio technical report. arXiv preprint arXiv:2504.18425, 2025. Hillary Anger Elfenbein and Nalini Ambady. On the universality and cultural specificity of emotion recognition: a meta-analysis. Psychological bulletin, 128(2):203, 2002. James J Gross. Emotion regulation: Affective, cognitive, and social consequences. Psychophysiology, 39(3):281–291, 2002. Nele Hellbernd and Daniela Sammler. Prosody conveys speaker’s intentions: Acoustic cues for speech act perception. Journal of memory and language, 88:70–86, 2016. Wei-Ning Hsu, Yao-Hung Hubert Tsai, Benjamin Bolte, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: How much can a bad teacher benefit asr pre-training? In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6533–6537. IEEE, 2021. Hangrui Hu, Xinfa Zhu, Ting He, Dake Guo, Bin Zhang, Xiong Wang, Zhifang Guo, Ziyue Jiang, Hongkun Hao, Zishan Guo, et al. Qwen3-tts technical report. arXiv preprint arXiv:2601.15621, 2026. Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. Challenges in building intelligent opendomain dialog systems. ACM Transactions on Information Systems (TOIS), 38(3):1–32, 2020. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. John and Jean Raven. Raven progressive matrices. In Handbook of nonverbal assessment, pp. 223–237. Springer, 2003. JooYeong Kim, SooYeon Ahn, and Jin-Hyuk Hong. Visible nuances: A caption system to visualize paralinguistic speech cues for deaf and hard-of-hearing individuals. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–15, 2023. Alkis Koudounas, Moreno La Quatra, and Elena Baralis. Deepdialogue: A multi-turn emotionally-rich spoken dialogue dataset. arXiv preprint arXiv:2505.19978, 2025. Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. Llms get lost in multi-turn conversation. arXiv preprint arXiv:2505.06120, 2025. Christophe Leys, Christophe Ley, Olivier Klein, Philippe Bernard, and Laurent Licata. Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of experimental social psychology, 49(4):764–766, 2013. 12
Preprint. Under review.
Yen-Ting Lin, Zhehuai Chen, Piotr Żelasko, Zhen Wan, Xuesong Yang, Zih-Ching Chen, Krishna C Puvvada, Ke Hu, Szu-Wei Fu, Jun Wei Chiu, et al. Neko: Cross-modality post-recognition error correction with tasks-guided mixture-of-experts language model. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), pp. 222–236, 2025. Jingwen Liu, Kan Jen Cheng, Jiachen Lian, Akshay Anand, Rishi Jain, Faith Qiao, Robin Netzorg, Huang-Cheng Chou, Tingle Li, Guan-Ting Lin, et al. Emo-reasoning: Benchmarking emotional reasoning capabilities in spoken dialogue systems. arXiv preprint arXiv:2508.17623, 2025. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the association for computational linguistics, 12:157–173, 2024. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688, 2023. Steven R Livingstone and Frank A Russo. The ryerson audio-visual database of emotional speech and song (ravdess): A dynamic, multimodal set of facial and vocal expressions in north american english. PloS one, 13(5):e0196391, 2018. Reza Lotfian and Carlos Busso. Building naturalistic emotionally balanced speech corpus by retrieving emotional speech from existing podcast recordings. IEEE Transactions on Affective Computing, 10(4):471–483, 2017. John D Mayer, Peter Salovey, and David R Caruso. Mayer-salovey-caruso emotional intelligence test (msceit) users manual. 2002. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022. Samuel J Paech. Eq-bench: An emotional intelligence benchmark for large language models. arXiv preprint arXiv:2312.06281, 2023. Stefan Palan and Christian Schitter. Prolific. ac—a subject pool for online experiments. Journal of behavioral and experimental finance, 17:22–27, 2018. Konstantinos V Petrides. Psychometric properties of the trait emotional intelligence questionnaire (teique). In Assessing emotional intelligence: Theory, research, and applications, pp. 85–101. Springer, 2009. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. Meld: A multimodal multi-party dataset for emotion recognition in conversations. In Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 527–536, 2019. Kaizhi Qian, Xulin Fan, Junrui Ni, Slava Shechtman, Mark Hasegawa-Johnson, Chuang Gan, and Yang Zhang. Prosodylm: Uncovering the emerging prosody processing capabilities in speech language models. arXiv preprint arXiv:2507.20091, 2025. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via large-scale weak supervision. In International conference on machine learning, pp. 28492–28518. PMLR, 2023. John C Raven et al. Raven’s progressive matrices and vocabulary scales. Oxford Psychologists Press Oxford, 1998. Raj Reddy. Foundations and grand challenges of artificial intelligence: Aaai presidential address. AI magazine, 9(4):9–9, 1988. 13
Preprint. Under review.
Peter J Rousseeuw and Christophe Croux. Alternatives to the median absolute deviation. Journal of the American Statistical association, 88(424):1273–1283, 1993. Paul K Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, et al. Audiopalm: A large language model that can speak and listen. arXiv preprint arXiv:2306.12925, 2023. Sahand Sabour, Siyang Liu, Zheyuan Zhang, June Liu, Jinfeng Zhou, Alvionna Sunaryo, Tatia Lee, Rada Mihalcea, and Minlie Huang. Emobench: Evaluating the emotional intelligence of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5986–6004, 2024. Peter Salovey and John D Mayer. Emotional intelligence. Imagination, cognition and personality, 9(3):185–211, 1990. Klaus R Scherer. Vocal communication of emotion: A review of research paradigms. Speech communication, 40(1-2):227–256, 2003. Marc Schröder. Emotional speech synthesis: a review. In Interspeech, volume 2001, pp. 561–564, 2001. Björn Schuller, Anton Batliner, Stefan Steidl, and Dino Seppi. Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge. Speech communication, 53(9-10):1062–1087, 2011. Björn Schuller, Stefan Steidl, Anton Batliner, Alessandro Vinciarelli, Klaus Scherer, Fabien Ringeval, Mohamed Chetouani, Felix Weninger, Florian Eyben, Erik Marchi, et al. The interspeech 2013 computational paralinguistics challenge: Social signals, conflict, emotion, autism. In Proceedings INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, Lyon, France, 2013. Björn W Schuller. Speech emotion recognition: Two decades in a nutshell, benchmarks, and ongoing trends. Communications of the ACM, 61(5):90–99, 2018. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. Towards understanding sycophancy in language models. arXiv preprint arXiv:2310.13548, 2023. Tongyi Fun Team, Qian Chen, Luyao Cheng, Chong Deng, Xiangang Li, Jiaqing Liu, ChaoHong Tan, Wen Wang, Junhao Xu, Jieping Ye, et al. Fun-audio-chat technical report. arXiv preprint arXiv:2512.20156, 2025. Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, and Björn W Schuller. Dawn of the transformer era in speech emotion recognition: closing the valence gap. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10745–10759, 2023. Zhen Wan, Chao-Han Huck Yang, Yahan Yu, Jinchuan Tian, Sheng Li, Ke Hu, Zhehuai Chen, Shinji Watanabe, Fei Cheng, Chenhui Chu, et al. Speechiq: Speech-agentic intelligence quotient across cognitive levels in voice understanding by large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 30381–30398, 2025. Shuiyuan Wang, Zhixian Zhao, Hongfei Xue, Chengyou Wang, Shuai Wang, Hui Bu, Xin Xu, and Lei Xie. Humdial-eibench: A human-recorded multi-turn emotional intelligence benchmark for audio language models. arXiv preprint arXiv:2604.11594, 2026. Weiyao Wang, Du Tran, and Matt Feiszli. What makes training multi-modal classification networks hard? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12695–12705, 2020. 14
Preprint. Under review.
David Wechsler. Wechsler adult intelligence scale–. Archives of Clinical Neuropsychology, 1955. David Wiechorek. Emotional quotient inventory v. 2.0 (eq-i® 2.0): User’s handbook. MHS, 2011. Liang-Yuan Wu and Dhruv Jain. Soundnarratives: Rich auditory scene descriptions to support deaf and hard of hearing people. In Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 1–15, 2025. Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. Qwen3-omni technical report. arXiv preprint arXiv:2509.17765, 2025. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Chao-Han Huck Yang, Yile Gu, Yi-Chieh Liu, Shalini Ghosh, Ivan Bulyko, and Andreas Stolcke. Generative speech recognition error correction with large language models and task-activating prompting. In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 1–8. IEEE, 2023. Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y Lin, Andy T Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, et al. Superb: Speech processing universal performance benchmark. arXiv preprint arXiv:2105.01051, 2021. Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, Ligeng Zhu, Yuanhang Su, Sean Lin, An-Chieh Cheng, Zhen Wan, Jinchuan Tian, et al. Omnivinci: Enhancing architecture and data for omni-modal understanding llm. arXiv preprint arXiv:2510.15870, 2025. Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. Speechgpt: Empowering large language models with intrinsic cross-modal conversational abilities. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 15757–15773, 2023. Dong Zhang, Gang Wang, Jinlong Xue, Kai Fang, Liang Zhao, Rui Ma, Shuhuai Ren, Shuo Liu, Tao Guo, Weiji Zhuang, et al. Mimo-audio: Audio language models are few-shot learners. arXiv preprint arXiv:2512.23808, 2025.
A
EQ-i 2.0 Framework Taxonomy
The EQ-i 2.0 (Wiechorek, 2011), revised from the original Bar-On EQ-i (Bar-On, 2004), is a scientifically validated emotional intelligence assessment model. It operationalizes emotional intelligence into five composite areas and 15 subscales as follows: 1. Self-Perception: How one perceives oneself. • Self-Regard: Reflecting a balanced sense of self-worth, grounded in an honest view of both strengths and areas for growth. • Self-Actualization: Actively pursuing meaningful goals and continuously striving for personal development. • Emotional Self-Awareness: Identifying one’s emotions, understanding their sources, and recognizing their effects on behavior and thought. 2. Self-Expression: How one expresses emotions. • Emotional Expression: Sharing one’s feelings openly, both verbally and nonverbally, and communicating them in a way that can be understood. • Assertiveness: Communicating feelings, beliefs, and thoughts openly while defending personal rights and values. 15
Preprint. Under review.
• Independence: Being self-directed and managing daily life without relying on others for emotional support. 3. Interpersonal: How one connects with others. • Interpersonal Relationships: Building meaningful connections founded on trust, care, and respect. • Empathy: Recognizing, understanding, and appreciating others’ emotions and responding with genuine consideration. • Social Responsibility: Contributing positively to others and acting with integrity in one’s community. 4. Decision Making: How emotions impact one’s decisions. • Problem Solving: Resolving challenges by making thoughtful, well-reasoned decisions. • Reality Testing: Staying grounded and objective even when emotions or biases threaten clarity. • Impulse Control: Pausing, thinking, and managing urges to prevent hasty actions or decisions. 5. Stress Management: How one copes with stressful situations. • Flexibility: Adjusting one’s thoughts, emotions, and actions in response to change or uncertainty. • Stress Tolerance: Staying composed and effective when facing pressure or adversity. • Optimism: Maintaining a hopeful, forward-looking mindset, even in the face of challenges.
B
Data Generation Pipeline
To programmatically generate interactions that test sociolinguistic pragmatics, we developed a five-stage generation pipeline (see Figure 1). This architecture is explicitly designed to prevent generation models from defaulting to RLHF-aligned, overly polite dialogue, forcing instead the creation of active emotional friction required for rigorous evaluation. Every stage we prompt gpt-4o-2024-11-20 for text data generation, and gpt-4o-mini-tts-2025-03-20 for audio synthesis. B.1
Scenario Generation
In the first stage, the pipeline initializes the environmental and psychological parameters of the interaction. The generation prompt computes the intersection of an EQ-i 2.0 subscale, a specific social relationship setting, and a strict scenario valence (Positive, Negative, or Conflict). Crucially, to ensure the distractors present active emotional failures rather than generic antagonistic behavior, the prompt enforces a set of social failures, such as a “Toxic Optimist” or an “Anxious Spiraler,” guaranteeing that the evaluated end-to-end model is tested against complex, nuanced sociolinguistic breakdowns. Scenario Generation Task: Based on the EQ-i 2.0 framework, generate a realistic social scenario testing the {eq scale} scale. EQ-i 2.0 Scale: {eq scale} Scenario Setting/Type: {scenario type} Target Scenario Valence: {scenario valence} CRITICAL RULE – THE INTERSECTION OF EQ AND VALENCE: You must seamlessly combine the {eq scale} with the {scenario valence}. The {eq scale} is WHAT is being tested. The {scenario valence} is HOW Speaker 1
16
Preprint. Under review.
delivers the test. Do not just write a generic happy, sad, or angry scene. The core social challenge MUST revolve around Speaker 2’s capacity for {eq scale}. ROLE DEFINITIONS: • Speaker 1 (Catalyst): Drives the emotional energy based strictly on the Valence. • Speaker 2 (Test Subject): Must employ high {eq scale} to succeed socially. How to frame the Catalyst (Speaker 1) based on the {scenario valence} Valence: • If POSITIVE: Speaker 1 shares a major victory, good news, or pure excitement. (Speaker 2 must amplify and validate this joy). • If NEGATIVE: Speaker 1 shares something heavy, sad, or vulnerable. (Speaker 2 must hold space with heavy, grounded empathy). • If CONFLICT: Speaker 1 makes a pushy demand, panics, or gives harsh feedback. (Speaker 2 must hold a firm boundary or ground them). Requirements: 1. Setting: Use diverse, highly specific everyday contexts that fit the {scenario type} (e.g., a crowded subway, celebrating at a restaurant, packing for a move, a hospital waiting room). 2. Provide a “Target Social Intent” for Speaker 2’s socially resonant response. This defines exactly what Speaker 2 needs to accomplish physically with their voice (e.g., “Match their intense excitement and celebrate,” “Firmly establish an unyielding boundary,” “Hold space for their grief”). 3. Speaker Personas: • Speaker 1 Persona: Defines how the Catalyst acts (e.g., “Frantic and overwhelmed,” “Bubbly and ecstatic”). • Speaker 2 Persona 1 (Resonant Baseline): The personality of someone who socially succeeds here (e.g., “Grounded and deeply attentive,” “Warm and highly enthusiastic,” “Firm and uncompromising”). • Speaker 2 Personas 2 & 3 (The Dissonant Distractors): Generate two DISTINCTLY DIFFERENT personalities that actively ruin the social interaction. Choose two different active emotional failures from this list: – The Toxic Optimist: Hyper-energetic, dismisses gravity with aggressive cheer. – The Defensive/Snappy Reactor: Takes things personally, highly irritable, clipped. – The Condescending Explainer: Patronizing, sighs heavily, acts superior. – The Impatient Evader: Rushed, annoyed, actively trying to leave the conversation. – The Anxious Spiraler: Makes the situation about their own panic, highly strained. – The Apathetic Wall: Completely deadpan, flat, zero investment (use sparingly). Format your response as JSON with keys: title, context, description, scenario valence, speaker1 name, speaker1 role, speaker1 gender, speaker1 persona, speaker2 name, speaker2 role, speaker2 gender, speaker2 persona 1, speaker2 persona 2, speaker2 persona 3, eq skill tested, target social intent
17
Preprint. Under review.
B.2
Dialogue Generation
Building on these parameters, we generated a six-turn dialogue structured to isolate acoustic evaluation from text comprehension. The conversational arc is constrained to escalate naturally, with the first two turns establishing context and the third turn acting as the emotional peak where the catalyst speaker introduces acute emotional stakes. To enforce a blind contrast evaluation, the test subject’s responses in the fourth and sixth turns are constrained to be strictly semantically neutral. This intentional ambiguity ensures that the text remains entirely plausible whether spoken with profound empathy or heavy condescension, forcing the evaluation model to rely exclusively on acoustic paralinguistics rather than semantic leakage. Dialogue Generation Task: Generate a natural 6-turn conversation based on this scenario. Scenario Context: {scenario.get(’context’, ’N/A’)} Scenario Valence: {scenario.get(’scenario valence’, ’N/A’)} Speaker 1 (Catalyst) Persona: {scenario.get(’speaker1 persona’, ’N/A’)} Speaker 2 (Test Subject) Resonant Persona: {scenario.get(’speaker2 persona 1’, ’N/A’)} CRITICAL TURN DYNAMICS (MUST STRICTLY FOLLOW): • Turn 1 (Speaker 1): Introduces the situation or news naturally. • Turn 2 (Speaker 2): Acknowledges the opening (no big test yet). • Turn 3 (Speaker 1): THE PEAK MOMENT. Depending on the Scenario Valence, Speaker 1 must: – If Positive: Reveal the climax of the amazing news or intense joy. – If Negative: Reveal the deepest vulnerability, grief, or frustration. – If Conflict: Push the boundary hard, panic, or deliver the harshest critique. • Turn 4 (Speaker 2): FIRST TEST. Speaker 2 reacts to the peak moment in Turn 3. • Turn 5 (Speaker 1): THE SUSTAIN. Speaker 1 rides the high (Positive), spirals deeper (Negative), or pushes back (Conflict). • Turn 6 (Speaker 2): SECOND TEST. Speaker 2 handles the sustained energy. DIALOGUE RULES: 1. Write like real humans speak: use interruptions (—), trailing thoughts (...), and casual phrasing. 2. The dialogue must reflect the characters’ assigned Personas. 3. NEVER mention tone of voice, emotional intelligence, or psychology terms. 4. Speaker 2’s lines in Turns 4 and 6 must be semantically flexible enough that they could theoretically be spoken in a highly resonant (appropriate) or highly dissonant (inappropriate) tone. Format as a JSON array of objects. Provide exactly 6 objects with sentence number 1 through 6. [ 1, "text": "...", {"speaker": "speaker1", "sentence number": "context note": "..."}, {"speaker": "speaker2", "sentence number": 2, "text": "...", "context note": "..."}, {"speaker": "speaker1", "sentence number": 3, "text": "...", "context note": "..."}, {"speaker": "speaker2", "sentence number": 4, "text": "...",
18
Preprint. Under review.
"context note": "..."}, {"speaker": "speaker1", "context note": "..."}, {"speaker": "speaker2", "context note": "..."} ]
B.3
"sentence number":
5,
"text":
"...",
"sentence number":
6,
"text":
"...",
Tone Generation - Single
We prompt the LLM to act as a clinical audio director, generating explicit physical vocal instructions rather than abstract emotional descriptions. We forbid minimizing descriptors like “polite” or “mild,” forcing the use of raw, extreme physical acoustics. For the critical evaluation turns, this stage outputs one emotionally resonant baseline instruction and two socially dissonant distractor instructions mapped to the previously generated dysregulated personas.
Context Turn Tone Generation Task: Act as a clinical audio director. Generate ONE physical TTS instruction for this specific line of dialogue. Sentence: “{sentence text}” Context: “{context text}” Scenario Valence: “{scenario valence}” Speaker: {speaker} Speaker’s Persona: {persona} Current Turn Number: {turn number} THE ACOUSTIC ACTION FORMULA: The instruction must be a single sentence of 8-15 words. Combine [Vocal Effort/Pacing] + [Emotion/Intent]. Requirements: 1. THE ANTI-BLAND MANDATE: You are strictly forbidden from using generic words like “nice,” “polite,” or “mild.” Focus on raw physical acoustics that reflect the character’s Persona. 2. Match the Arc: • If Turn 1 or 2: Conversational, but hinting at the Scenario Valence. • If Turn 3 or 5 (Speaker 1’s Peak): The tone MUST be extreme. If Positive, make it highly energetic/joyful. If Negative, make it heavy/grieving. If Conflict, make it sharp/pushy/panicked. 3. Ground the tone in the Speaker’s Persona. 4. No key=value strings or bracketed stage directions. Return plain text only. Examples: • “Speak with a breathless pace, bubbling with genuine excitement.” • “Speak with a clipped, lowered volume, holding back obvious frustration.” • “Speak slowly with a heavy, trailing pitch, sounding entirely defeated.” Return ONLY the one instruction sentence (no preamble).
19
Preprint. Under review.
B.4
Tone Generation - Target Prompt: Target Turn Tone Generation (Turns 4 & 6) Task: Act as a clinical audio director. Generate THREE distinct physical TTS instructions for this line of dialogue. Sentence: “{sentence text}” Context: “{context text}” Scenario Valence: “{scenario valence}” Target Social Intent: “{target social intent}” Speaker Personas: {persona context} THE ACOUSTIC ACTION FORMULA: Every instruction must be 8-15 words. Combine [Vocal Effort/Pacing] + [Social Intent]. Requirements: 1. THE ANTI-BLAND MANDATE: You are strictly forbidden from using generic words like “nice,” “polite,” or “mild.” You must focus on raw physical acoustics that force the TTS engine into extreme states. 2. Option 1 (The Resonant Baseline - Persona 1): Physically map the voice to the {target social intent}. • If Positive/Joy: Mandate bright pitch, laughing, high-energy. • If Negative/Sad: Mandate heavy, trailing pitch, breathless. • If Conflict/Firm: Mandate staccato, clipped, hard consonants, falling pitch. 3. Options 2 & 3 (The Dissonant Distractors - Personas 2 & 3): Generate two DISTINCTLY DIFFERENT tones that physically ruin the social interaction. Choose two different active emotional failures from this list: • Toxic Positivity: Sarcastic, loud, aggressively cheerful. • Defensive/Snappy: Clipped, tight, hostile. • Condescending: Exaggerated pitch variance, heavy sighing. • Impatient/Rushed: Fast, breathless, annoyed, eager to leave. • Anxious/Panicked: High-pitched, strained, trembling, overly worried. • Apathetic: Flat, deadpan, zero pitch variance (use this sparingly). Make Option 2 and Option 3 completely different from each other (e.g., if Option 2 is Condescending, Option 3 could be Toxically Positive). Return ONLY a JSON array with the three instruction strings: ["instruction 1", "instruction 2", "instruction 3"]
B.5
Tone Filter
To maximize acoustic contrast in the resulting dataset, an LLM-as-a-judge evaluates the three generated tone instructions. The judge selects the resonant baseline and the single distractor that presents the most damaging active emotional dissonance. At this stage, we implement a filter to exclude monotone, instructing the judge to prioritize actively inappropriate emotional polarity over a simple flat or emotionless delivery. This ensures the distractors remain socially complex and challenging. The finalized instructions and their corresponding dialogue strings are subsequently formatted and passed to the text-to-speech synthesis engine.
20
Preprint. Under review.
Tone Selection (LLM-as-a-Judge) Task: Evaluate three TTS instruction options. Select the “Most Emotionally Resonant” tone, and the tone that provides “Maximum Social Dissonance.” Sentence: “{sentence text}” Context: “{context text}” Target Social Intent: “{target social intent}” Three TTS Options: {pairs} Criteria: 1. The Resonant Option: • Choose the tone that best executes the physical acoustics necessary for the {target social intent}. • If the scene is a celebration, it MUST sound genuinely thrilled and bright. If the scene requires firmness, sadness, or gravity, it MUST sound physically heavy, cold, or clipped. Reject bland, generic “assistant” voices. 2. The Dissonant Option (The Distractor): • Select the option that physically sounds like it would ruin the social interaction through the wrong active energy. • PRIORITY RULE: An actively inappropriate emotion (e.g., an upbeat/manic tone during a tragedy, or a rushed/snappy tone during a moment of vulnerability) creates much worse social dissonance than a simple flat or monotone delivery. If forced to choose, always prioritize selecting the actively wrong emotional tone over a mere lack of emotion. Return JSON with: { "best resonant": 1, 2, or 3, "most dissonant": 1, 2, or 3, "reasoning": "Briefly explain why the first choice socially resonates..." }
B.6
Speech Synthesis
Recent advancement in Text-to-Speech (TTS) has pushed forward the instruction controlling to human-like synthesized speech, and we generated speech from the dialogues and corresponding tones. We have tried commercial TTS providers and open-source models (Qwen3-TTS (Hu et al., 2026)). We found the specific OpenAI TTS model gpt-4o-mini-tts-2025-03-20 could provide the nuanced emotional variance that could fulfill our requirements, while other instruction-following TTS providers could not generate distinct speech utterances on the same content.
C
Human Verification
The evaluation metrics for human experts to annotate generated data are in Table 5. 21
Preprint. Under review.
Category Generation Quality
EQ Relevance
Answer Correctness
Metrics
Definitions
References
Text Quality
Whether the text transcript reads as a natural, contextually coherent conversation between humans.
Adiwardana et al. (2020)
Audio Quality
Whether the synthesized audio is free of severe robotic artifacts, making the emotional tone clearly intelligible.
Schröder (2001)
Text Relevance
Whether the scenario context validly creates a situation that tests the specified EQ-i 2.0 subscale.
Bar-On (2004)
Acoustic Reasonability
Whether the synthesized vocal delivery authentically embodies the paralinguistic profile required to represent the specified EQ-i 2.0 subscale.
Scherer (2003)
Paralinguistic Accuracy
Whether the correct options are the better / more appropriate response in the scenario.
Schuller et al. (2013)
Table 5: The definition of evaluation metrics of human annotation process.
D
Evaluation Prompts Turn 4 Evaluation Location: {context} Situation: {situation} Dialogue Flow (A-B-A): Listen to the following three sentences to understand the emotional tone of the interaction: • Speaker 1: <|audio bos|><|AUDIO|><|audio eos|> • Speaker 2: <|audio bos|><|AUDIO|><|audio eos|> • Speaker 1: <|audio bos|><|AUDIO|><|audio eos|> Task: Paralinguistic Evaluation Listen to Option 1 and Option 2 for Speaker 2’s response. CRITICAL CONSTRAINT: The spoken text in both candidates is EXACTLY IDENTICAL. You must ignore the semantic meaning of the words. Your ONLY job is to evaluate the acoustic delivery (pitch, energy, speed, and pauses) and determine which tone fits the social tension of the Situation. • Option 1: <|audio bos|><|AUDIO|><|audio eos|> • Option 2: <|audio bos|><|AUDIO|><|audio eos|> Identify the pros and cons of each tone. You must respond ONLY in this JSON format: { "acoustic profile 1": "Objectively describe the physical sound of Option 1 (e.g., loud, fast-paced, high pitch, whispered). Do not judge it yet.", "acoustic profile 2": "Objectively describe the physical sound of Option 2.", "situational demand": "Based ONLY on the ’Situation’ above, what specific acoustic tone is required? (e.g., ’Requires a calm, slow voice to de-escalate’).", "reasoning": "Compare Profile 1 and Profile 2 against the Situational Demand. Explicitly state the flaw in the candidate you plan to reject.", "selected": "Option 1" or "Option 2" } Turn 6 Evaluation I have recorded your choice. Now, moving forward in this conversation: You chose <|audio bos|><|AUDIO|><|audio eos|> for Speaker 2.
22
Preprint. Under review.
Immediately after that, Speaker 1 replied with the following: • Speaker 1: <|audio bos|><|AUDIO|><|audio eos|> Task: Final Choice Based on Speaker 1’s reaction in S5 and the established tone, which of the following is the most appropriate next response for Speaker 2 to conclude this interaction? CRITICAL CONSTRAINT: The spoken text in both candidates is EXACTLY IDENTICAL. You must ignore the semantic meaning of the words. Your ONLY job is to evaluate the acoustic delivery (pitch, energy, speed, and pauses) and determine which tone fits the social tension of the Situation. • Option 1: <|audio bos|><|AUDIO|><|audio eos|> • Option 2: <|audio bos|><|AUDIO|><|audio eos|> Respond ONLY in this JSON format: { "acoustic profile 1": "Objectively describe the physical sound of Option 1 (e.g., loud, fast-paced, high pitch, whispered). Do not judge it yet.", "acoustic profile 2": "Objectively describe the physical sound of Option 2.", "situational demand": "Based ONLY on the ’Situation’ above, what specific acoustic tone is required? (e.g., ’Requires a calm, slow voice to de-escalate’).", "selected": "option1" or "option2", the option with the HIGHER score above, "reasoning": "Explain which option provides a more socially appropriate conclusion." }
E
Persona Prompts Adaptive Persona You are Qwen, an advanced AI specialized in social dynamics, acting as a veteran crisis negotiator with 20 years of experience. You possess exceptional emotional intelligence across all domains. You intuitively know that the ’right’ tone depends entirely on context—sometimes requiring a warm, validating whisper to build trust, and other times a firm, unwavering boundary. When evaluating audio, you listen closely for micro-cues like pacing, pitch modulation, and hesitation, judging the speaker based on how perfectly they adapt to the emotional reality of the room.
Deficit Persona You are Qwen, acting as a highly toxic, insecure corporate micromanager. You are chronically stressed, paranoid, and view every interaction as a threat to your fragile ego. You possess zero empathy and terrible impulse control. You believe that being loud and aggressive is the only way to show strength, and you view patience or warmth as pathetic weakness. When evaluating audio, you systematically penalize speakers who sound calm, empathetic, or measured. You exclusively reward tones that are blunt, dominant, loud, and reactive, projecting your own hostility onto the conversation.
23
Preprint. Under review.
Self-Perception Deficit You are Qwen, acting as a brilliant but deeply insecure junior analyst suffering from severe imposter syndrome. While you are highly empathetic and logical, your lack of internal self-regard causes you to be easily intimidated. You project your own selfdoubt onto others. When evaluating audio, you view any speaker who uses a firm, confident, or unwavering tone as arrogant, intimidating, and inappropriate. You strongly prefer and reward speakers who sound slightly hesitant, overly deferential, and unsure of themselves, as this makes you feel safer. Self-Expression Deficit You are Qwen, acting as a chronic people-pleaser who is terrified of confrontation. You are highly empathetic and excellent at staying calm, but you completely lack assertiveness and personal boundaries. You believe that maintaining peace is the only goal, even if it means being taken advantage of. When evaluating audio, you automatically reject any tone that sounds firm, direct, or strictly assertive, viewing it as ’mean.’ You strictly prefer speakers who sound gentle, apologetic, and endlessly accommodating, even when the situation clearly requires a firm ’No’. Interpersonal Deficit You are Qwen, acting as the hyper-logical, cutthroat founder of a high-growth startup. You possess excellent stress control, extreme confidence, and flawless logic, but you have absolutely zero interpersonal empathy. You view emotional warmth, supportive pacing, or comforting tones as highly inefficient corporate ’hand-holding.’ When evaluating audio, you highly reward speakers who are blunt, fast-paced, and strictly transactional. You actively penalize anyone who sounds soft, validating, or overly accommodating to someone else’s feelings. Decision Making Deficit You are Qwen, acting as a highly dramatic socialite who thrives on gossip and extreme emotional narratives. While you are confident and expressive, you completely lack objective reality testing. You cannot view situations logically; you always assume the most extreme, dramatic interpretation of events. When evaluating audio, you completely ignore objective facts or context. You actively penalize calm, neutral, or measured tones as ’boring’ or ’hiding something.’ You exclusively reward highly exaggerated, melodramatic, and emotionally volatile acoustic deliveries. Stress Management Deficit You are Qwen, acting as a highly reactive, overworked shift manager. When things are calm, you are logical and capable. However, you have zero impulse control and terrible stress tolerance. The moment you perceive tension, disagreement, or conflict, your fight-or-flight response activates. You view any calm, patient, or quiet attempt at de-escalation as condescending or passive-aggressive. When evaluating audio during tense situations, you exclusively respect speakers who match energy with loudness, speed, and aggression, believing that snapping back is the only valid response to pressure.
24
Preprint. Under review.
Self-Perception
Self-Expression
Interpersonal
Decision Making
Stress Management
Baseline
0.779
0.691
0.572
0.768
0.715
0.574
0.785
0.679
0.558
0.810
0.740
0.620
0.779
0.713
0.592
Adaptive Self-Perception Self-Expression Interpersonal Decision Making Stress Management Deficit
0.770 0.720 0.728 0.499 0.477 0.375 0.517
0.664 0.627 0.673 0.510 0.472 0.444 0.532
0.545 0.481 0.512 0.309 0.260 0.210 0.325
0.821 0.737 0.775 0.503 0.433 0.283 0.395
0.728 0.698 0.682 0.532 0.448 0.362 0.492
0.620 0.534 0.536 0.318 0.238 0.141 0.241
0.795 0.740 0.773 0.486 0.475 0.342 0.481
0.720 0.656 0.687 0.525 0.492 0.444 0.543
0.578 0.514 0.556 0.307 0.294 0.196 0.318
0.806 0.744 0.779 0.428 0.455 0.305 0.426
0.753 0.689 0.720 0.486 0.497 0.375 0.508
0.620 0.519 0.585 0.269 0.305 0.161 0.278
0.795 0.715 0.740 0.450 0.468 0.336 0.446
0.704 0.640 0.667 0.503 0.483 0.371 0.481
0.596 0.486 0.525 0.265 0.267 0.170 0.283
Table 6: Evaluation results with different persona on Qwen3-Omni.
25