AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
arXiv:2606.14591v1 [cs.SD] 12 Jun 2026
Hui Geng∗ Yi Su∗ College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China [email protected] [email protected]
Han Yin
Tianjiao Wan
Korea Advanced Institute of Science and Technology (KAIST) Korea [email protected]
College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China [email protected]
Qisheng Xu
Jiaxin Chen
Zijian Gao
College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China [email protected]
College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China [email protected]
College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China [email protected]
Xie Chen
Hengzhu Liu
Kele Xu†
Shanghai Jiaotong University Shanghai, China [email protected]
College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China [email protected]
College of Computer Science and Technology, National University of Defense Technology Changsha, Hunan, China [email protected]
Abstract Large Audio-Language Models (LALMs) have shown strong performance on a wide range of audio understanding tasks, yet they still struggle with complex audio reasoning. A practical way to improve such capabilities is post-training, whose effectiveness critically depends on the quality and diversity of training data. However, existing audio-language datasets often contain substantial redundancy, where many samples are highly similar in acoustic content and thus provide overlapping supervisory signals. Such redundancy not only increases annotation cost, but also limits corpus diversity and reduces the effectiveness of post-training. To address this issue, we propose a redundancy-aware data construction pipeline for building reasoning-oriented supervision for LALMs. Specifically, we first perform acoustic similarity-based deduplication across raw audio datasets to improve corpus diversity. We then integrate existing audio captions and question-answer pairs into a unified multiple-choice format. Based on these unified annotations, we leverage Qwen3-30B to generate chain-of-thought (CoT) rationales for reasoning-oriented supervision. Based on this pipeline, we construct AudioDER, a reasoning-oriented post-training dataset containing approximately 191k samples spanning sound, speech, and music. Each sample consists of an audio clip, a multiple-choice question, four answer candidates, an audio caption, and a CoT rationale. Extensive experiments show that post-training on AudioDER consistently improves the performance of Qwen2-Audio-7B-Instruct ∗ Both authors contributed equally to the paper † Corresponding Authors.
on multiple audio reasoning benchmarks, including MMAU-mini, MMSU, and MMAR. We hope AudioDER can serve as a valuable resource for advancing audio reasoning research and the development of more capable LALMs. Project page: AudioDER.
CCS Concepts • Computing methodologies → Artificial intelligence; Natural language processing; Speech recognition.
Keywords Large Audio-Language Models, Audio Reasoning, Post-Training, Dataset Construction
1
Introduction
Recent advances in Large Language Models (LLMs) [36] and audio foundation models have enabled the development of Large AudioLanguage Models (LALMs) [8, 16, 46], which align continuous audio signals with natural language and support a unified interface for auditory understanding [61]. Benefiting from such a formulation, LALMs have achieved strong performance on a broad range of tasks, including automatic speech recognition (ASR) [2, 4], audio captioning (AC) [3, 52, 79], music captioning (MC) [5, 45], and audio question answering (AQA) [11, 24, 32]. Despite this progress [12], existing LALMs still struggle with complex audio reasoning tasks that require compositional understanding, multi-step training strateies, and fine-grained interpretation of auditory events [55, 73].
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Hui Geng, Yi Su, Han Yin, Tianjiao Wan, Qisheng Xu, Jiaxin Chen, Zijian Gao, Xie Chen, Hengzhu Liu, and Kele Xu
three major domains: sound, speech, and music. Each sample is paired with an audio clip, a caption, a multiple-choice question, candidate answers, and a CoT rationale, providing structured supervision for improving the reasoning ability of LALMs. Extensive experiments demonstrate that post-training on AudioDER consistently improves the performance of Qwen2-Audio7B-Instruct [10], on multiple audio reasoning benchmarks such as MMAU-mini [56], MMSU [67], and MMAR [51]. These results validate the effectiveness of the data construction pipeline and highlight the importance of reducing redundancy in reasoning-oriented audio-language post-training. Our main contributions are summarized as follows: • We identify and analyze dataset redundancy as a critical bottleneck in reasoning-oriented post-training for LALMs, and show that directly aggregating heterogeneous audio sources leads to overlapping supervisory signals and limited reasoning diversity. • We propose a scalable, redundancy-aware data construction pipeline that combines acoustic similarity-based deduplication, caption and Q-A annotation integration, and CoT rationale generation to produce unified reasoning-oriented supervision. • We construct and open-source AudioDER, a 191k-sample reasoning-oriented audio-language dataset covering sound, speech, and music, which provides rich supervision for posttraining LALMs. • We conduct extensive post-training experiments on representative LALM backbone and benchmarks, showing that AudioDER consistently enhances audio reasoning performance and generalizes across different model architectures.
Figure 1: PCA visualization of CLAP audio embeddings from different source datasets. Each point denotes an audio sample, and each star denotes the centroid of a dataset. The overlap among datasets suggests substantial cross-dataset redundancy in the audio embedding space. A practical way to improve such capabilities is to post-train strong existing LALMs with high-quality reasoning-oriented supervision [23]. However, the effectiveness of post-training critically depends on the diversity and quality of the training corpus. In practice, existing audio-language datasets are often collected by simply aggregating samples [7, 26, 37] from multiple sources, with little control over redundancy [61]. As a result, many samples are highly similar in acoustic content and provide overlapping supervisory signals, as illustrated in Figure 1. This redundancy not only increases the cost of constructing large-scale annotated corpora, but also limits the diversity of reasoning patterns during post-training. Therefore, simply scaling up data volume may bring diminishing returns, making data redundancy a key bottleneck for improving LALM reasoning ability [72, 75, 76]. To address this issue, we propose a redundancy-aware data construction pipeline for reasoning-oriented post-training of LALMs. Specifically, we first perform acoustic similarity-based deduplication on raw audio collections to reduce near-duplicate samples and maximize corpus diversity. We then integrate the available captions and question-answer annotations from the source datasets into a unified multiple-choice format. Specifically, existing captions and Q-A pairs are standardized across datasets, and AVQA questions from the training split are adapted by replacing references to “video” with “audio”. After this annotation integration step, we use Qwen3-30B [81] to generate chain-of-thought (CoT) rationales for each unified sample, providing explicit reasoning supervision for post-training. We then leverage Qwen3-30B [81] to automatically generate structured supervision for each audio sample, including caption, multiple-choice question (MCQ), candidate answers, and chainof-thought (CoT) rationale [25, 47, 68]. This process yields rich reasoning signals while maintaining scalability. Based on this pipeline, we construct and open-source AudioDER, a high-quality reasoning-oriented audio-language post-training dataset. AudioDER contains approximately 191k samples spanning
In this paper, we present AudioDER and study its effectiveness for reasoning-oriented post-training of LALMs. We further show that AudioDER consistently enhances the reasoning performance of the Qwen2-Audio-7B-Instruct architecture. The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 describes the construction pipeline and key characteristics of AudioDER. Section 4 presents the experimental setup and results. Section 5 concludes the paper and discusses future directions.
2
Related Work
Large Audio-Language Models. LALMs [27, 34, 44] aim to connect audio perception with language understanding [78, 80]. Existing LALMs can be broadly categorized into audio understanding models and real-time dialogue models [1, 13, 18]. Audio understanding models typically adopt a three-stage architecture composed of an audio encoder, a modality connector, and a large language model, often targeting tasks such as captioning [28], question answering [41], and audio event understanding [6, 70]. Representative examples include LTU [48] and GAMA [29]. More recent models such as LTU-AS [32], SALMONN [63], Qwen2-Audio [10], OpenAIo1 [36], Kimi K1.5 [64], DeepSeekR1 [15], Audio Flamingo [39], Audio Flamingo 2 [30], and Audio Flamingo 3 [31] further move toward unified multitask learning across diverse audio domains [62, 85]. In parallel, real-time dialogue systems [14] focus more on streaming speech interaction and low-latency generation. Since AudioDER is designed for reasoning-oriented post-training on heterogeneous
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
audio understanding tasks, we mainly focus on the first category in this work. Audio Reasoning and Post-Training. Improving reasoning ability has recently become an important direction for LALMs [43, 57, 65]. Early attempts mainly relied on CoT prompting [87] to elicit intermediate reasoning steps without updating model parameters. For example, Audio-CoT [50] is among the first works to explore CoT annotations in audio-language models. While such methods can improve performance on relatively simple tasks, they are often insufficient for more challenging scenarios [22, 54]. To address this limitation, recent studies have increasingly adopted post-training strategies, including SFT [45, 88] and RL [74, 83, 84, 86]. Mellow [17] demonstrates that carefully designed reasoning-oriented training can enable strong performance even with a relatively small model. Audio-Reasoner [77] introduces a multi-stage framework involving planning, captioning, reasoning, and summarization. R1-AQA [42], SARI [69], and Omni-R1 [55] further enhance reasoning through reward-driven optimization and explicit CoT supervision. These efforts collectively show that post-training is a promising route toward effective audio reasoning [59, 60]. However, their effectiveness still depends heavily on the availability of reliable and diverse reasoning data. Audio Reasoning Datasets. The rapid progress of reasoningoriented LALMs [19, 40, 58] has been closely tied to the development of high-quality datasets. Existing datasets have explored various forms of audio reasoning supervision, including adapted question-answer pairs, captions, multiple-choice questions, and CoT annotations. For example, prior work based on AVQA [82] constructed an audio-focused question answering corpus by removing visual dependence from video-based data [69, 73]. CoTA [77] introduced a large-scale dataset with captions and question-answer pairs across multiple audio domains, supporting structured reasoning training. Other works [71] further developed multiple-choice audio question answering resources and reasoning-oriented corpora with explicit CoT supervision. AudioMCQ [35] substantially expanded this line by providing large-scale MCQ data with both structured and unstructured reasoning traces.
3
AudioDER Dataset
Motivated by the redundancy analysis in Figure 1, we aim to build a diverse and scalable post-training dataset for audio understanding and reasoning. Based on the proposed pipeline, we construct AudioDER, a reasoning-oriented dataset containing approximately 191k training samples spanning three major domains: sound, speech, and music, including Clotho [21], CompA-R [29], AVQA, LibriTTSR [38, 53], MusicCaps [20], LP-musiccaps-MTT (MTT) [20], MusicBench [49]. An overview of the source datasets is shown in Table 2, and the overall construction pipeline is illustrated in Figure 2. Each sample is organized into a unified format consisting of an audio input, a multiple-choice question, four answer candidates with one correct answer, an audio caption, and a CoT rationale. Compared with existing post-training datasets for LALMs, AudioDER is designed to provide a more unified and diversity-aware supervision format for reasoning-oriented learning. As summarized in Table 1, AudioDER integrates audio captions, multiple-choice questions, answer candidates, and CoT rationales into a unified
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
training format, while additionally introducing acoustic similaritybased redundancy filtering during dataset construction. Moreover, AudioDER is fully open-source and covers three major domains, making it a practical resource for scalable reasoning-oriented posttraining of LALMs.
3.1
Data Construction
To build a reasoning-oriented post-training dataset, we design a multi-stage data construction pipeline based entirely on opensource models and open-source datasets, as shown in Figure 2. As shown in the Figure 1, the pipeline contains three stages: (1) redundancy analysis and filtering, (2) caption and Q-A annotation integration, and (3) CoT rationale generation. Stage 1: Redundancy Analysis and Filtering. Existing largescale audio-language datasets are typically collected from diverse sources and domains, yet they may still exhibit substantial crossdataset redundancy in the audio embedding space. Such redundancy not only increases the cost of large-scale annotation generation, but also weakens the benefits of reasoning-oriented supervision for LALMs. To better understand this issue, we analyze the redundancy among several general audio datasets from the perspectives of global embedding distribution and pairwise acoustic similarity. We first examine the global distribution of audio content in the embedding space. As shown in Figure 1, principal component analysis (PCA) of CLAP audio embeddings reveals the relative positions of samples from different source datasets. Each point is color-coded by dataset, and the centroid of each dataset is marked with a star. The centroid is computed by averaging the CLAP embeddings of all samples in the corresponding dataset and projecting them into the same two-dimensional space. The visualization shows noticeable overlap among several datasets, suggesting that acoustically similar samples frequently occur across datasets. This finding indicates that simply aggregating multiple general datasets does not necessarily lead to proportionally improved data diversity, while substantially increasing annotation cost. To further quantify this redundancy, we compute pairwise similarities between datasets in the audio embedding space. Specifically, for each audio sample, we extract a CLAP audio embedding and define it as z. Given two audio samples with embeddings z𝑖 and z 𝑗 , we define their cosine similarity as: 𝑠 (z𝑖 , z 𝑗 ) =
< z𝑖 , z 𝑗 > . ∥z𝑖 ∥ · ∥z 𝑗 ∥
(1)
For each dataset D𝑚 , we further compute its centroid in the embedding space as: c𝑚 =
∑︁ 1 z, |D𝑚 | z∈ D
(2)
𝑚
where |D𝑚 | denotes the number of samples in dataset D𝑚 . Based on the centroids, the distance between two datasets D𝑚 and D𝑛 is measured by: 𝑑𝑚,𝑛 = ∥c𝑚 − c𝑛 ∥ 2 .
(3)
We measure cross-dataset redundancy using the proportion of highly similar sample pairs. Given a similarity threshold 𝜏 = 0.99,
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Hui Geng, Yi Su, Han Yin, Tianjiao Wan, Qisheng Xu, Jiaxin Chen, Zijian Gao, Xie Chen, Hengzhu Liu, and Kele Xu
Figure 2: Overview of the AudioDER construction pipeline. Starting from heterogeneous source datasets across sound, speech, and music domains, we first perform acoustic similarity-based redundancy analysis and filtering to improve corpus diversity. We then integrate existing captions and Q-A annotations into a unified multiple-choice format, including the adaptation of AVQA questions by replacing references to “video” with “audio”. Finally, we use Qwen3-30B to generate CoT rationales. The final output is a unified reasoning-oriented training sample for AudioDER. Table 1: Comparison of AudioDER with representative post-training datasets for LALMs. AudioDER is designed as a unified, reasoning-oriented post-training corpus with explicit redundancy filtering and structured supervision.
Dataset
Scale
Caption
MCQ
CoT
Redundancy Filtering
Open-source
Xie et al. [77] Li et al. [42] He et al. [35] Wen et al. [69]
1.2M 40k 571k 42k
✓ – ✓ ✓
– ✓ ✓ ✓
✓ ✓ ✓ ✓
– – – –
✓ – ✓ –
AudioDER (ours)
191k
✓
✓
✓
✓
✓
Table 2: Overview of the source datasets used to construct AudioDER. Dataset
Count
Description
Clotho CompA-R AVQA
3,839 198,648 40,425
Audio captioning Audio question answering Audio question answering
LibriTTS-R
228,944
Speech description
MusicCaps MTT MusicBench
2,649 15,626 20,918
Music captioning Music captioning Music question answering
the redundancy ratio between datasets D𝑚 and D𝑛 is defined as: ∑︁ ∑︁ 1 𝑟𝑚,𝑛 = I 𝑠 (z𝑖 , z 𝑗 ) > 𝜏 , (4) |D𝑚 | |D𝑛 | z ∈ D z ∈ D 𝑖
𝑚
𝑗
𝑛
where I(·) is the indicator function. Based on the above analysis, we explicitly filter out highly similar samples before annotation generation. By performing redundancy reduction at the raw audio datasets, we reduce annotation cost while improving the diversity and coverage of the resulting corpus. Stage 2: Caption and Q-A Annotation Integration. After acoustic redundancy reduction, we convert heterogeneous source datasets into a unified caption-conditioned multiple-choice format. Different from generating captions and questions from scratch, this stage integrates the annotations already provided by the source datasets. For captioning datasets such as Clotho, MusicCaps, MTT, and LibriTTS-R, we retain their dataset-provided captions or textual descriptions as the audio captions. For question-answering datasets such as CompA-R and MusicBench, we standardize the existing Q-A annotations into a unified multiple-choice format. For AVQA,
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
we use the audio-text pairs from the training split and adapt the question text by replacing references to “video” with “audio”. Formally, after annotation integration, each sample is represented as (𝑎𝑖 , 𝑐𝑖 , 𝑞𝑖 , 𝑂𝑖 , 𝑦𝑖 ), where 𝑎𝑖 denotes the audio sample, 𝑐𝑖 denotes the dataset-provided caption or textual description, 𝑞𝑖 is the question, 𝑂𝑖 = {𝑜𝑖1, 𝑜𝑖2, 𝑜𝑖3, 𝑜𝑖4 } is the set of four answer candidates, 𝑦𝑖 is the correct answer. This stage unifies heterogeneous annotations from different source datasets while avoiding unnecessary regeneration of captions and Q-A pairs. Stage 3: CoT Rationale Generation. To provide explicit reasoning supervision, we use Qwen3-30B to generate a CoT rationale Figure 4: Distribution analysis of AudioDER across different for each unified question-answer sample. Each rationale is condisource domains and datasets. tioned on the audio caption, the integrated question, the answer candidates, and the correct option, and serves as structured supervision for intermediate reasoning. Following prior work on structured reasoning [35, 77], we organize the generation process into multiple steps. In the planning stage, the model identifies the core reasoning path implied by the question based on the audio description. In the evidence extraction stage, it highlights the most relevant acoustic or semantic cues from the caption. In the reasoning stage, the model connects the extracted evidence with the question and the answer candidates to derive the correct conclusion. Finally, in the summarization stage, it produces a concise and coherent explanation that justifies the correct answer. After CoT generation, each sample can be represented as (𝑎𝑖 , 𝑐𝑖 , 𝑞𝑖 , 𝑂𝑖 , 𝑦𝑖 , 𝑟𝑖 ), where 𝑟𝑖 denotes the generated CoT rationale. This CoT generation process improves both the interpretability and the training value of the dataset, allowing post-training models to learn not only the final answer but also the reasoning process that leads to it. Figure 5: PCA visualization of CLAP audio embeddings from AudioDER.
Figure 3: The length distribution of CoT rationales in AudioDER.
3.2
Quantitative Analysis of Dataset
We analyze AudioDER from two perspectives: domain coverage and reasoning complexity. AudioDER covers three major audio domains, namely sound, speech, and music, which together provide broad coverage of real-world auditory scenarios. This cross-domain composition exposes models to diverse acoustic patterns and reasoning demands, making AudioDER well suited for general-purpose posttraining of LALMs. We further examine the length distribution of CoT rationales as a proxy for reasoning complexity, as shown in Figure 3. Most CoT rationales fall within a moderate length range,
indicating that the dataset encourages explicit multi-step reasoning without excessive verbosity. More challenging samples tend to produce longer rationales, reflecting the need for deeper evidence aggregation and more elaborate inference, whereas relatively simple samples usually require shorter and more direct reasoning chains. In addition, Figure 4 and Figure 5 show that AudioDER contains samples from diverse source datasets across sound, speech, and music domains, while exhibiting reduced redundancy after the construction process. This balanced yet heterogeneous composition improves both the acoustic coverage and the diversity of reasoning patterns in the final corpus. These statistics suggest that AudioDER provides both broad domain diversity and varied reasoning complexity, which are important properties for training and evaluating reasoning-oriented LALMs.
4 Experiments 4.1 Experimental Setup Implementation Details. We conduct some post-training experiments to evaluate the effectiveness of AudioDER for improving the reasoning ability of LALMs. Following prior work [42, 55], we adopt Qwen2-Audio-7B-Instruct as the backbone model in our experiments. We perform supervised fine-tuning (SFT) on AudioDER using full fine-tuning, with an initial learning rate of 1 × 10−6 for 2
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Hui Geng, Yi Su, Han Yin, Tianjiao Wan, Qisheng Xu, Jiaxin Chen, Zijian Gao, Xie Chen, Hengzhu Liu, and Kele Xu
Table 3: Performance comparison on MMAU-mini, MMSU, and MMAR. For MMAU-mini, we report accuracy on the sound, music, and speech subsets, together with the overall average. “SFT” denotes supervised fine-tuning of Qwen2-Audio-7B-Instruct on AudioDER. Bold indicates the best result in each column among the reported and reproduced open-source methods. MMAU-mini-test Model
Method
LTU LTU-AS Audio Flamingo-Chat SALMONN Qwen-audio-Chat GAMA GAMA-IT Mellow Qwen2-Audio-7B-Instruct Audio-Reasoner SARI R1-AQA Qwen2-Audio-7B-Instruct (ours)
MMAR
sound
music
speech
total
ACC
ACC
Direct Inference Direct Inference Direct Inference Direct Inference Direct Inference Direct Inference Direct Inference Direct Inference Direct Inference SFT SFT SFT
22.52 23.35 23.42 41.14 55.25 41.44 43.24 61.26 67.27 60.06 64.87 60.96
9.69 9.10 15.26 37.13 44.00 32.33 28.44 54.19 56.29 64.30 59.13 49.19
17.71 20.60 11.41 26.43 30.03 18.91 18.91 29.73 55.26 60.70 44.52 45.35
16.89 17.68 16.69 34.90 43.10 30.90 30.20 48.40 59.60 61.71 56.18 51.80
29.76 34.44 35.72 49.20 37.29 35.36
24.80 20.00 32.80 23.50 27.70 22.40 30.00 36.80 -
SFT
71.77
66.77
61.56
66.70
56.49
50.10
epochs. The global batch size is set to 20, and a checkpoint is saved every 100 training steps. Benchmarks. We mainly evaluate the model on multiple-choice audio reasoning benchmarks using accuracy as the metric. MMAUmini [56] is our primary benchmark. It contains challenging audio QA samples that require multi-step reasoning over sound, music, and speech, and we report category-wise accuracy together with the overall performance. To evaluate generalization to broader reasoning settings, we further test on MMSU [67], a multiple-choice benchmark derived from MMLU-Pro and adapted to audio-centered reasoning scenarios. MMSU covers 12 non-mathematical and noncoding knowledge areas and is designed to measure broader reasoning and knowledge transfer. We also evaluate on MMAR [51], which focuses on deep reasoning in realistic audio scenarios involving mixtures of sound, music, and speech. Compared with conventional audio understanding tasks, MMAR places greater emphasis on compositional reasoning and fine-grained evidence aggregation. Baselines. We compare our method with representative large audio-language models, including LTU [33], LTU-AS [32], Audio Flamingo-Chat [66], SALMONN, Qwen-audio-Chat [9], GAMA, GAMA-IT, and Mellow [17]. We also report the performance of the original backbone model before post-training, namely Qwen2Audio-7B-Instruct, as well as Qwen2-Audio-7B-Instruct post-trained with existing reasoning-oriented methods such as Audio-Reasoner, R1-AQA, and SARI. This comparison allows us to evaluate both the absolute effectiveness of AudioDER and its competitiveness against existing post-training approaches.
4.2
MMSU
Main Results
Table 3 reports the main results on MMAU-mini, MMSU, and MMAR. SFT on AudioDER substantially improves the reasoning performance of Qwen2-Audio-7B-Instruct across all evaluated benchmarks. On MMAU-mini, the overall accuracy increases from 59.60% under direct inference to 66.70% after SFT on AudioDER, with
consistent gains on sound (67.27% → 71.77%), music (56.29% → 66.77%), and speech (55.26% → 61.56%). Compared with existing post-training methods built on the same backbone, AudioDER also achieves the best overall performance, outperforming AudioReasoner (61.71%), SARI (56.18%), and R1-AQA (51.80%) on MMAUmini. In addition, our model obtains the best reproduced opensource results on MMSU (56.49%) and MMAR (50.10%), significantly surpassing the direct-inference backbone and prior SFT baselines. These results indicate that AudioDER provides effective reasoningoriented supervision across heterogeneous audio domains and supports strong generalization to broader audio reasoning settings. The consistent gains achieved by simple SFT suggest that better post-training data alone can substantially strengthen the reasoning ability of LALMs.
5
Conclusion
In this paper, we introduce AudioDER, a reasoning-oriented posttraining dataset for LALMs, and propose a scalable redundancyaware pipeline for its construction. By combining acoustic similaritybased deduplication, caption and Q-A annotation integration, and chain-of-thought rationale generation, AudioDER provides unified supervision across sound, speech, and music domains. Experimental results on Qwen2-Audio-7B-Instruct demonstrate that AudioDER consistently improves performance on multiple audio reasoning benchmarks, including MMAU-mini, MMSU, and MMAR, validating the effectiveness of redundancy-aware data construction for reasoning-oriented post-training. These results further suggest that improving the quality and diversity of post-training data is a practical and effective path toward stronger audio reasoning, even without modifying model architectures or introducing more complex training objectives. More broadly, we hope AudioDER can serve as a useful resource for future research on reasoning-oriented learning in LALMs. In future work, we will further expand the diversity and difficulty of AudioDER and investigate its use in stronger
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
post-training frameworks to support more capable audio reasoning systems.
References [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. [2] Harsh Ahlawat, Naveen Aggarwal, and Deepti Gupta. Automatic speech recognition: A survey of deep learning techniques and approaches. International Journal of Cognitive Computing in Engineering, 6:201–237, 2025. [3] Jisheng Bai, Haohe Liu, Mou Wang, Dongyuan Shi, Wenwu Wang, Mark D Plumbley, Woon-Seng Gan, and Jianfeng Chen. Audiosetcaps: An enriched audio-caption dataset using automated generation pipeline with large audio and language models. IEEE Transactions on Audio, Speech and Language Processing, 2025. [4] Mohamed Benzeghiba, Renato De Mori, Olivier Deroo, Stephane Dupont, Teodora Erbes, Denis Jouvet, Luciano Fissore, Pietro Laface, Alfred Mertins, Christophe Ris, et al. Automatic speech recognition and speech variability: A review. Speech communication, 49(10-11):763–786, 2007. [5] Irmak Bukey, Zhepei Wang, Chris Donahue, and Nicholas J Bryan. Rethinking music captioning with music metadata llms. arXiv preprint arXiv:2602.03023, 2026. [6] Umberto Cappellazzo, Minsu Kim, Honglie Chen, Pingchuan Ma, Stavros Petridis, Daniele Falavigna, Alessio Brutti, and Maja Pantic. Large language models are strong audio-visual speech recognition learners. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025. [7] Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: A large-scale audio-visual dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 721–725. IEEE, 2020. [8] Qian Chen, Yafeng Chen, Yanni Chen, Mengzhe Chen, Yingda Chen, Chong Deng, Zhihao Du, Ruize Gao, Changfeng Gao, Zhifu Gao, et al. Minmo: A multimodal large language model for seamless voice interaction. arXiv preprint arXiv:2501.06282, 2025. [9] Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919, 2023. [10] Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, et al. Qwen2-audio technical report. arXiv preprint arXiv:2407.10759, 2024. [11] Yung-Sung Chuang, Chi-Liang Liu, Hung-Yi Lee, and Lin-shan Lee. Speechbert: An audio-and-text jointly learned language model for end-to-end spoken question answering. arXiv preprint arXiv:1910.11559, 2019. [12] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. [13] Nilaksh Das, Saket Dingliwal, Srikanth Ronanki, Rohit Paturi, Zhaocheng Huang, Prashant Mathur, Jie Yuan, Dhanush Bekal, Xing Niu, Sai Muralidhar Jayanthi, et al. Speechverse: A large-scale generalizable audio language model. arXiv preprint arXiv:2405.08295, 2024. [14] Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037, 2024. [15] Zehang Deng, Wanlun Ma, Qing-Long Han, Wei Zhou, Xiaogang Zhu, Sheng Wen, and Yang Xiang. Exploring deepseek: A survey on advances, applications, challenges and future directions. IEEE/CAA Journal of Automatica Sinica, 12(5): 872–893, 2025. [16] Soham Deshmukh, Benjamin Elizalde, Rita Singh, and Huaming Wang. Pengi: An audio language model for audio tasks. Advances in Neural Information Processing Systems, 36:18090–18108, 2023. [17] Soham Deshmukh, Satvik Dixit, Rita Singh, and Bhiksha Raj. Mellow: a small audio language model for reasoning. arXiv preprint arXiv:2503.08540, 2025. [18] Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, et al. Kimi-audio technical report. arXiv preprint arXiv:2504.18425, 2025. [19] Heinrich Dinkel, Zhiyong Yan, Tianzi Wang, Yongqing Wang, Xingwei Sun, Yadong Niu, Jizhong Liu, Gang Li, Junbo Zhang, and Jian Luan. Glap: General contrastive audio-text pretraining across domains and languages. arXiv preprint arXiv:2506.11350, 2025. [20] SeungHeon Doh, Keunwoo Choi, Jongpil Lee, and Juhan Nam. Lp-musiccaps: Llm-based pseudo music captioning. arXiv preprint arXiv:2307.16372, 2023.
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
[21] Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. Clotho: An audio captioning dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 736–740. IEEE, 2020. [22] Logan Engstrom, Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Firdaus Janoos, Larry Rudolph, and Aleksander Madry. Implementation matters in deep policy gradients: A case study on ppo and trpo. arXiv preprint arXiv:2005.12729, 2020. [23] Kaixuan Fan, Kaituo Feng, Haoming Lyu, Dongzhan Zhou, and Xiangyu Yue. Sophiavl-r1: Reinforcing mllms reasoning with thinking reward. arXiv preprint arXiv:2505.17018, 2025. [24] Haytham M Fayek and Justin Johnson. Temporal reasoning via audio question answering. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28:2283–2294, 2020. [25] Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, and Wynne Hsu. Video-of-thought: Step-by-step video reasoning from perception to cognition. arXiv preprint arXiv:2501.03230, 2024. [26] Eduardo Fonseca, Xavier Favory, Jordi Pons, Frederic Font, and Xavier Serra. Fsd50k: an open dataset of human-labeled sound events. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:829–852, 2021. [27] Chaoyou Fu, Haojia Lin, Xiong Wang, Yi-Fan Zhang, Yunhang Shen, Xiaoyu Liu, Haoyu Cao, Zuwei Long, Heting Gao, Ke Li, et al. Vita-1.5: Towards gpt-4o level real-time vision and speech interaction. arXiv preprint arXiv:2501.01957, 2025. [28] Sreyan Ghosh, Sonal Kumar, Chandra Kiran Reddy Evuru, Ramani Duraiswami, and Dinesh Manocha. Recap: Retrieval-augmented audio captioning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1161–1165. IEEE, 2024. [29] Sreyan Ghosh, Sonal Kumar, Ashish Seth, Chandra Kiran Reddy Evuru, Utkarsh Tyagi, S Sakshi, Oriol Nieto, Ramani Duraiswami, and Dinesh Manocha. Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6288–6313, 2024. [30] Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, and Bryan Catanzaro. Audio flamingo 2: An audiolanguage model with long-audio understanding and expert reasoning abilities. arXiv preprint arXiv:2503.03983, 2025. [31] Arushi Goel, Sreyan Ghosh, Jaehyeon Kim, Sonal Kumar, Zhifeng Kong, Sang-gil Lee, Chao-Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, et al. Audio flamingo 3: Advancing audio intelligence with fully open large audio language models. arXiv preprint arXiv:2507.08128, 2025. [32] Yuan Gong, Alexander H Liu, Hongyin Luo, Leonid Karlinsky, and James Glass. Joint audio and speech understanding. In 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pages 1–8. IEEE, 2023. [33] Yuan Gong, Hongyin Luo, Alexander H Liu, Leonid Karlinsky, and James Glass. Listen, think, and understand. arXiv preprint arXiv:2305.10790, 2023. [34] Jiawei Guo, Tianyu Zheng, Yizhi Li, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Graham Neubig, Wenhu Chen, and Xiang Yue. Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13869–13920, 2025. [35] Haolin He, Xingjian Du, Renhe Sun, Zheqi Dai, Yujia Xiao, Mingru Yang, Jiayi Zhou, Xiquan Li, Zhengxi Liu, Zining Liang, et al. Measuring audio’s impact on correctness: Audio-contribution-aware post-training of large audio language models. arXiv preprint arXiv:2509.21060, 2025. [36] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. [37] Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim. Audiocaps: Generating captions for audios in the wild. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 119–132, 2019. [38] Yuma Koizumi, Heiga Zen, Shigeki Karita, Yifan Ding, Kohei Yatabe, Nobuyuki Morioka, Michiel Bacchiani, Yu Zhang, Wei Han, and Ankur Bapna. Libritts-r: A restored multi-speaker text-to-speech corpus. arXiv preprint arXiv:2305.18802, 2023. [39] Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, and Bryan Catanzaro. Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities. arXiv preprint arXiv:2402.01831, 2024. [40] Chun-Yi Kuan, Wei-Ping Huang, and Hung-yi Lee. Understanding sounds, missing the questions: The challenge of object hallucination in large audio-language models. arXiv preprint arXiv:2406.08402, 2024. [41] Jiayi Kuang, Ying Shen, Jingyou Xie, Haohao Luo, Zhe Xu, Ronghao Li, Yinghui Li, Xianfeng Cheng, Xika Lin, and Yu Han. Natural language understanding and inference with mllm in visual question answering: A survey. ACM Computing Surveys, 57(8):1–36, 2025. [42] Gang Li, Jizhong Liu, Heinrich Dinkel, Yadong Niu, Junbo Zhang, and Jian Luan. Reinforcement learning outperforms supervised fine-tuning: A case study on audio question answering. arXiv preprint arXiv:2503.11197, 2025.
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Hui Geng, Yi Su, Han Yin, Tianjiao Wan, Qisheng Xu, Jiaxin Chen, Zijian Gao, Xie Chen, Hengzhu Liu, and Kele Xu
[43] Ming Li, Jike Zhong, Shitian Zhao, Yuxiang Lai, Haoquan Zhang, Wang Bill Zhu, and Kaipeng Zhang. Think or not think: A study of explicit thinking in rule-based visual reinforcement fine-tuning. arXiv preprint arXiv:2503.16188, 2025. [44] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. [45] Shansong Liu, Atin Sakkeer Hussain, Chenshuo Sun, and Ying Shan. Music understanding llama: Advancing text-to-music generation with question answering and captioning. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 286–290. IEEE, 2024. [46] Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu, Chao-Han Huck Yang, Sung-Feng Huang, Chih-Kai Yang, Chee-En Yu, Chun-Wei Chen, Wei-Chih Chen, Chien-yu Huang, et al. Desta2. 5-audio: Toward general-purpose large audio language model with self-generated cross-modal alignment. IEEE Transactions on Audio, Speech and Language Processing, 2026. [47] Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. Faithful chain-of-thought reasoning. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 305–329, 2023. [48] Jie Ma, Min Hu, Pinghui Wang, Wangchun Sun, Lingyun Song, Hongbin Pei, Jun Liu, and Youtian Du. Look, listen, and answer: Overcoming biases for audiovisual question answering. Advances in Neural Information Processing Systems, 37:9507–9531, 2024. [49] Yinghao Ma, Siyou Li, Juntao Yu, Emmanouil Benetos, and Akira Maezawa. Cmibench: A comprehensive benchmark for evaluating music instruction following. arXiv preprint arXiv:2506.12285, 2025. [50] Ziyang Ma, Zhuo Chen, Yuping Wang, Eng Siong Chng, and Xie Chen. Audiocot: Exploring chain-of-thought reasoning in large audio language model. arXiv preprint arXiv:2501.07246, 2025. [51] Ziyang Ma, Yinghao Ma, Yanqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, et al. Mmar: A challenging benchmark for deep reasoning in speech, audio, music, and their mix. arXiv preprint arXiv:2505.13032, 2025. [52] Xinhao Mei, Chutong Meng, Haohe Liu, Qiuqiang Kong, Tom Ko, Chengqi Zhao, Mark D Plumbley, Yuexian Zou, and Wenwu Wang. Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:3339–3354, 2024. [53] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE, 2015. [54] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in neural information processing systems, 36: 53728–53741, 2023. [55] Andrew Rouditchenko, Saurabhchand Bhati, Edson Araujo, Samuel Thomas, Hilde Kuehne, Rogerio Feris, and James Glass. Omni-r1: Do you really need audio to fine-tune your audio llm? arXiv preprint arXiv:2505.09439, 2025. [56] Sakshi Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. Mmau: A massive multi-task audio understanding and reasoning benchmark. arXiv preprint arXiv:2410.19168, 2024. [57] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [58] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. [59] Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. arXiv preprint arXiv:2409.12183, 2024. [60] Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. Chain of thoughtlessness? an analysis of cot in planning. Advances in Neural Information Processing Systems, 37:29106–29141, 2024. [61] Yi Su, Jisheng Bai, Qisheng Xu, Kele Xu, and Yong Dou. Audio-language models for audio-centric tasks: A survey. arXiv preprint arXiv:2501.15177, 2025. [62] Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. Pandagpt: One model to instruction-follow them all. In Proceedings of the 1st Workshop on Taming Large Language Models: Controllability in the era of Interactive Assistants!, pages 11–23, 2023. [63] Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. Salmonn: Towards generic hearing abilities for large language models. arXiv preprint arXiv:2310.13289, 2023. [64] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5:
Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025. [65] Songjun Tu, Jiahao Lin, Qichao Zhang, Xiangyu Tian, Linjing Li, Xiangyuan Lan, and Dongbin Zhao. Learning when to think: Shaping adaptive reasoning in r1-style models via multi-stage rl. arXiv preprint arXiv:2505.10832, 2025. [66] Rafael Valle, Rohan Badlani, Zhifeng Kong, Sang-gil Lee, Arushi Goel, Sungwon Kim, Joao Felipe Santos, Shuqi Dai, Siddharth Gururani, Aya Aljafari, et al. Fugatto 1: Foundational generative audio transformer opus 1. In The Thirteenth International Conference on Learning Representations, 2025. [67] Dingdong Wang, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. Mmsu: A massive multi-task spoken language understanding and reasoning benchmark. arXiv preprint arXiv:2506.04779, 2025. [68] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824–24837, 2022. [69] Cheng Wen, Tingwei Guo, Shuaijiang Zhao, Wei Zou, and Xiangang Li. Sari: Structured audio reasoning via curriculum-guided reinforcement learning. arXiv preprint arXiv:2504.15900, 2025. [70] Gijs Wijngaard, Elia Formisano, Michele Esposito, and Michel Dumontier. Audsemthinker: Enhancing audio-language models through reasoning over semantics of sound. arXiv preprint arXiv:2505.14142, 2025. [71] Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, et al. Step-audio 2 technical report. arXiv preprint arXiv:2507.16632, 2025. [72] Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. Next-gpt: Any-to-any multimodal llm. In Forty-first International Conference on Machine Learning, 2024. [73] Shu Wu, Chenxing Li, Wenfu Wang, Hao Zhang, Hualei Wang, Meng Yu, and Dong Yu. Audio-thinker: Guiding audio language model when and how to think via reinforcement learning. arXiv preprint arXiv:2508.08039, 2025. [74] Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2502.14768, 2025. [75] Zhifei Xie and Changqiao Wu. Mini-omni: Language models can hear, talk while thinking in streaming. arXiv preprint arXiv:2408.16725, 2024. [76] Zhifei Xie and Changqiao Wu. Mini-omni2: Towards open-source gpt-4o with vision, speech and duplex capabilities. arXiv preprint arXiv:2410.11190, 2024. [77] Zhifei Xie, Mingbao Lin, Zihang Liu, Pengcheng Wu, Shuicheng Yan, and Chunyan Miao. Audio-reasoner: Improving reasoning capability in large audio language models. arXiv preprint arXiv:2503.02318, 2025. [78] Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2087–2098, 2025. [79] Xuenan Xu, Zeyu Xie, Mengyue Wu, and Kai Yu. Beyond the status quo: A contemporary survey of advances and challenges in audio captioning. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:95–112, 2023. [80] An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122, 2024. [81] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [82] Pinci Yang, Xin Wang, Xuguang Duan, Hong Chen, Runze Hou, Cong Jin, and Wenwu Zhu. Avqa: A dataset for audio-visual question answering on videos. In Proceedings of the 30th ACM international conference on multimedia, pages 3480–3491, 2022. [83] Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal reasoning through cross-modal formalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2376–2385, 2025. [84] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476, 2025. [85] Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, and Xipeng Qiu. Speechgpt: Empowering large language models with intrinsic crossmodal conversational abilities. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15757–15773, 2023. [86] Yi-Fan Zhang, Xingyu Lu, Xiao Hu, Chaoyou Fu, Bin Wen, Tianke Zhang, Changyi Liu, Kaiyu Jiang, Kaibing Chen, Kaiyu Tang, et al. R1-reward: Training multimodal reward model through stable reinforcement learning. arXiv preprint arXiv:2505.02835, 2025. [87] Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493, 2022. [88] Yuze Zhao, Jintao Huang, Jinghan Hu, Xingjun Wang, Yunlin Mao, Daoze Zhang, Zeyinzi Jiang, Zhikai Wu, Baole Ai, Ang Wang, et al. Swift: a scalable lightweight
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
infrastructure for fine-tuning. In Proceedings of the AAAI Conference on Artificial
Conference acronym ’XX, June 03–05, 2018, Woodstock, NY
Intelligence, volume 39, pages 29733–29735, 2025.