RetroThinker: Enabling Retrospective Thinking in Speech LLMs Yi-Jen Shih∗ , Puyuan Peng† , Abdelrahman Mohamed† , and David Harwath∗ ∗ The University of Texas at Austin
arXiv:2609.11864v1 [eess.AS] 10 Sep 2026
† FAIR, Meta Superintelligence Labs
Abstract—Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech recognition (ASR) and text-based LM architectures. However, they continue to lag behind text-only LLMs on complex reasoning tasks, while real-time spoken interaction imposes strict latency constraints. Although prior works employ Chain-of-Thought (CoT) and concurrent reasoning to enhance reasoning capabilities without inducing prohibitive delays, an inherent accuracy-latency trade-off persists. In this paper, we investigate whether a streaming SpeechLLM can dynamically revise its reasoning traces on the fly. We introduce RetroThinker, a multi-stage post-training framework that equips the Moshi model to self-verify and forward-correct CoT steps during inference. RetroThinker combines supervised fine-tuning (SFT) on curated retrospective thinking data with length-based direct preference optimization (DPO) to optimize retrospective during early reasoning (i.e., reasoning concurrently while the user speaks). Evaluated on the GSM8K benchmark, RetroThinker significantly improves the accuracy-latency trade-off over nonretrospective baselines, achieving an 11% absolute accuracy gain at a comparable latency. Index Terms—Speech LLMs, Reasoning
I. I NTRODUCTION In recent years, AI agents have become pervasive, offering diverse capabilities seamlessly integrated into personal devices like smartphones and smart glasses. Among all modalities, speech serves as the most direct and natural interface for human-agent interaction. Unlike text, speech conveys substantially more information, capturing speaker identity, prosody, and emotional nuances typically lost in transcription. Most contemporary AI agents rely on Large Language Models (LLMs) as their backbone and are trained on massive text corpora. Consequently, enabling them to process speech generally follows one of the two paradigms. The first is a cascaded approach [1]–[3], which concatenates modular components: Automatic Speech Recognition (ASR), a textbased LLM, and Text-to-Speech synthesis (TTS). The second is to pretrain and fine-tune LLMs to directly consume and generate speech tokens. This approach, which we term SpeechLLMs [4]–[6], bypasses the modular constraints and information bottlenecks inherent in cascaded systems. The primary advantages of SpeechLLMs include lower inference latency, simplified deployment, and a superior capacity to process paralinguistic signals. As a result, they are uniquely equipped All data access, experiments, and processing activities were conducted by The University of Texas at Austin.
Fig. 1. Overview of RetroThinker. By combining EarlyReasoning (reasoning starts before the user finishes the question) and Retrospective Thinking (selfverification and forward correction), the model achieves a better accuracylatency trade-off.
to facilitate naturalistic conversations between humans and machines. Despite the inherent advantages of SpeechLLMs, two significant challenges remain. First, recent studies reveal that they often underperform in reasoning and general intelligence when compared to text-based counterparts of similar scale and training data volume [7], [8]. This performance disparity is particularly pronounced in complex reasoning tasks, as opposed to simple factual retrieval. The second challenge is that response latency is far more constrained than in text-based LLMs, as users expect near real-time interactions with voice agents. Motivated by Chain-of-Thought (CoT) prompting [9], recent works [7], [10]–[12] integrate CoT into SpeechLLMs to enhance their conversational reasoning capabilities. Originally introduced in the NLP community, CoT is widely adopted to boost model performance by eliciting step-by-step reasoning. By decomposing complex problems into intermediate logical steps, CoT significantly improves accuracy across diverse benchmarks. However, when applied to SpeechLLMs, generating the additional CoT trace substantially increases latency. To mitigate this, prior work reduces latency by enabling concurrent reasoning: models either initiate the reasoning process while the user is still speaking [7], [10], or simultaneously reason while generating speech output [11], [12]. This paradigm shift allows SpeechLLMs to process audio concurrently with reasoning. While this improves reasoning and reduces response latency, an inherent accuracy-latency Pareto trade-off remains [13]. To push this accuracy-latency Pareto frontier further, we
explore a parallel paradigm: retrospective thinking. In human conversation, when addressing complex inquiries, individuals occasionally commit logical errors during their internal reasoning processes. However, humans possess the cognitive flexibility to reflect on these thoughts and perform realtime revisions during the dialogue [14]. Motivated by this behavior, we introduce the concept of retrospective thinking to SpeechLLMs. Much like their human counterparts, it is inevitable that SpeechLLMs will introduce errors within their reasoning traces during inference, particularly under strict realtime constraints. Consequently, we posit that retrospective thinking can improve reasoning accuracy by allowing the model to identify and rectify such errors dynamically. We propose RetroThinker, a post-training framework that equips SpeechLLMs with real-time retrospective thinking capabilities. Adapting retrospective sampling [15], we apply specialized fine-tuning techniques to SpeechLLMs, enabling them to identify incorrect CoT steps and subsequently revise them. Following [7], we select Moshi [4] as our foundational SpeechLLM. Its native multistream architecture offers the flexibility to inject CoT reasoning into the text stream without disrupting the synchronized audio input and output streams. Furthermore, to mitigate the additional latency introduced by the retrospective thinking process, we investigate its interplay with two latency-reduction techniques from prior work [7]: Early Reasoning (i.e., “thinking while listening”) and Direct Preference Optimization (DPO) [16]. Evaluations on TTSspoken GSM8K demonstrate that combining all three techniques improves the accuracy-latency trade-off in a controlled multi-step reasoning setting. Additionally, we conduct an indepth analysis to elucidate how these techniques influence one another. In summary, our key contributions are as follows: 1) We introduce retrospective thinking for streaming SpeechLLMs, enabling forward-only verification and correction of hidden CoT traces without rolling back generated speech. 2) We provide a concrete post-training recipe for Moshi that combines rule-based initialization, sample-based alignment to model errors, and length-based DPO. 3) On TTS-spoken GSM8K, RetroThinker improves the accuracy-latency frontier, outperforming a comparable non-retrospective baseline by 11 percentage points in accuracy at comparable latency. II. BACKGROUND A. Moshi Moshi [4] is a multi-stream, full-duplex SpeechLLM. During each forward pass, the model concurrently processes three distinct streams: User Audio, System Text (a.k.a., inner monologue), and System Audio. To process these heterogeneous inputs as discrete sequences, Moshi leverages the Mimi codec to discretize continuous speech waveforms into audio tokens at 80 ms intervals using 8 codebooks. The inner monologue stream provides word-level, time-aligned transcripts synchronized with the system’s speech output. Architecturally, Moshi
utilizes a Temporal Transformer coupled with a Depth Transformer. At each timestep, the Temporal Transformer consumes all three input streams to predict the next token in the system text stream. Subsequently, the Depth Transformer takes the Temporal Transformer’s output to predict the 8 audio tokens for the system audio stream. The temporal transformer undergoes initial text-only pre-training before being integrated into Moshi. Finally, the complete model is pre-trained on largescale audio datasets and fine-tuned on multi-stream spoken conversational data. B. CoT Finetuning and Early Reasoning In [7], the authors enhance Moshi’s reasoning capabilities while reducing latency through early reasoning. They achieve this by first relaxing the strict alignment between the system’s audio and text streams. Specifically, the model is trained to generate a CoT trace in the text stream after user audio question and before producing the spoken response. To reduce the latency added by CoT generation, they introduce a “thinkingwhile-listening” framework that initiates reasoning before the user finishes speaking. To determine the optimal moment to start reasoning, they propose the Question Completeness (QC) metric, which evaluates partial question lengths. QC ranges from 0 (empty) to 1 (complete). By defining a threshold θ ∈ [0, 1], training data is curated to trigger reasoning once sufficient information is gathered. On inference, the model naturally initiates early reasoning by emitting a special token, functioning independently of the QC metric. Thus, tuning θ enables explicit control over the accuracy-latency trade-off. Furthermore, they proposed LengthDPO to reduce latency under the Early Reasoning scenario. Specifically, they construct preference pairs where the positive samples are both accurate and concise, while the negative samples are incorrect, thereby minimizing latency without compromising performance. In our work, we define Standard Reasoning as the setting where the CoT trace is generated only after the full user query is completed. Conversely, we define Early Reasoning as the setting where the model begins its reasoning process concurrently while the user is still speaking. C. Retrospective Sampling In REVERSE [15], the authors mitigate hallucinations in Vision-Language Models (VLMs) on image captioning task by training them with retrospective sampling abilities. Within their framework, captions are first partitioned into multiple segments. Subsequently, a combination of rule-based methods and LLM prompting is employed to synthesize error caption segments. These error segments are then randomly inserted before the corresponding ground-truth segments. At the end of each segment, a specialized self-validation token is appended to indicate the accuracy of the preceding segment. During the training phase, the next-token prediction loss is masked for the injected error segments to prevent the model from learning incorrect generation patterns. During inference, the authors introduce a mechanism termed “Retrospective Sampling.” If the self-verification token identifies a segment as incorrect,
the system rolls back the generation to a specific point and employs various strategies to regenerate the segment, ensuring factual consistency. III. R ETROT HINKER We propose RetroThinker, a multi-stage framework (See Fig. 2) to enable retrospective thinking in SpeechLLMs. Built upon the architecture in [7], our framework modifies the reasoning trace while maintaining the integrity of the remaining streams. Consequently, our method is compatible with both standard and early reasoning, and can be combined with other concurrent-reasoning methods [10]–[12]. Stage 1: Rule-based Retrospective Training. We first segment each GSM8K ground-truth rationale into ordered CoT steps using sentence and equation boundaries, keeping the final answer as the spoken response target. For each selected step, we synthesize an erroneous variant by applying rulebased perturbations, such as modifying numerical values or substituting arithmetic operators. The training sequence then contains the erroneous step, a negative self-verification marker, the corrected step, and a positive marker before continuing to the next step. The verification markers are rendered as [✘] and [✔] in our qualitative examples. Unlike REVERSE [15], which performs rollback or terminates the sequence after an incorrect segment, our framework requires the model to continue the generation process. In streaming speech applications, the system cannot perform a temporal “rollback” without disrupting the user experience. Therefore, we structure our data such that the model learns to identify, revise, and proceed with subsequent CoT steps even when interleaved with previous errors. We perform Supervised Fine-Tuning (SFT) on Moshi using this data, masking the loss on the synthesized erroneous CoT tokens while retaining loss on the verification markers and corrected continuation, similarly to [15]. Stage 2: Sample-based Distribution Alignment. While Stage 1 establishes the structural format for retrospective thinking, rule-based errors often fail to reflect the actual distribution of model failures during inference. To mitigate this distribution mismatch, we proposed a second stage of samplebased fine-tuning. We generate reasoning traces from the Stage 1 checkpoint and use an LLM-as-a-judge to evaluate each step given the question and preceding reasoning context. Whenever the judge identifies an incorrect step, it returns a corrected continuation from that point to the final answer. We then construct retrospective training examples by using the model-generated error as the masked negative step and the continuation provided by the judge as the positive correction. This keeps the same loss-masking strategy as Stage 1 while aligning the correction behavior to authentic model failures. Stage 3: Length-based DPO Fine-tuning. While retrospective thinking significantly enhances accuracy, the inclusion of revision steps non-trivially increases the total thinking length, thereby escalating response latency. To address this, we employ LengthDPO [7] to compress the reasoning trace without sacrificing accuracy. Specifically, we sample outputs from the Stage 2 model and construct preference pairs where
preferred examples are correct and concise, while dispreferred examples are incorrect. Beyond latency reduction, DPO finetuning serves as a critical mechanism for reducing the distribution mismatch inherent in early reasoning [7]. In Stage 2, the reasoning traces are judged and revised by an LLM without exposure to the constraint that the user’s query may not yet be fully received. By applying DPO in the early reasoning setting, the model learns to adapt its retrospective capabilities to the uncertainties of incomplete user queries. Inference Strategy. Departing from the rollback mechanism in [15], our inference procedure adheres to the streaming constraints of SpeechLLMs. Our model performs “forward-only” self-correction: it continues generating subsequent CoT steps and the final response while preserving the entire reasoning history, including both incorrect and corrected steps, in the text stream. If the model emits a negative verification marker, correction is produced as ordinary subsequent text. This setup matches the way we finetune the model. IV. E XPERIMENTS A. Training and Inference Details Training Details. To minimize computational cost, we employ LoRA [17] rather than full fine-tuning [7] of the Moshi model. We select GSM8K [18] as our controlled training and evaluation dataset because it provides high-quality multistep ground-truth reasoning steps and objective final answers. These properties makes it suitable for us to evaluate the effectiveness of our approaches. To adapt this text-based dataset for the speech domain, we synthesize the questions and answers into audio using Gemini-Flash-TTS. We emphasize that this setup evaluates spoken delivery of multi-step math reasoning rather than natural open-domain dialogue; broader speech benchmarks such as VoiceBench [19] contain reasoning examples but mix them with many factual or short-form tasks that do not require retrospective CoT correction. For all SFT stages (Stage 1 and Stage 2), we search learning rates in the range 2e-5 to 2e-6, selecting checkpoints based on the minimum validation loss. For the DPO phase, we search learning rates between 1e-7 and 1e-6 with β = 0.1. Additionally, to stabilize preference learning, we add an additional Negative Log-Likelihood (NLL) loss (weighted by 0.1) on the positive examples [20], normalize the probability distribution by length [21], and use only the text stream to calculate the policy distribution [22]. The final checkpoints are selected based on the highest reward-based accuracy on the validation set. Evaluation Details. We adopt the same evaluation pipeline as [7]. During inference, we stream the user’s audio query as input into Moshi and capture the model’s response waveform. All reported settings share the same synthesis, decoding, transcription, and judging pipeline. Latency Metrics. To quantify response latency, we apply Voice Activity Detection (VAD) to the output waveform. Latency is defined as the number of seconds between the end of the user’s speech and the onset of Moshi’s audio response.
Stage 1 Rule-Based Retrospective Tuning Rule-based synthetic error
Ground-truth reasoning trace
1 egg is $2, 12 eggs: $2 x 12 = $24
perturb arithmetic
Synthetic Retrospective SFT data
1 egg is $2, 12 eggs: $2 x 10 = $20 [✗] $2 x 12 = $24 [✓]
SFT
⚙
Stage1 model
From Moshi
Stage 2 Sample-Based Retrospective Tuning Generate samples from Stage1 Model
1 egg is $2, 12 eggs: $4 x 12 = $48
LLM Judge & Rewrite
Aligned Retrospective SFT data
1 egg is $2, 12 eggs: $4 x 12 = $48 [✗] -> $2 x 12 = $24[✓]
append correction
SFT
⚙
Init from Stage1
⇄
Init from Stage2
Stage2 model
Stage 3 Length-DPO Tuning (from [7]) Preferred Generate samples from Stage2 Model
A: ... answer is $24. B: ... therefore $30.
LLM Judge
Correct & Short
DPO pairs dataset
correctness
DPO
Stage3 model
Rejected
Incorrect
Fig. 2. Illustration of the three-stage RetroThinker pipeline. Stage 1 employs a rule-based method to inject synthetic errors, teaching the model proper self-validation and self-correction formats. Stage 2 utilizes an external LLM judge to annotate the authentic errors generated by sampling the Stage 1 model. Finally, in Stage 3, LengthDPO is applied to mitigate the distribution mismatch caused by the external judge and to condense unnecessarily long retrospective reasoning traces.
B. Baselines and Experiment Settings To investigate the interplay between RetroThinker, early reasoning, and LengthDPO, we establish a comprehensive experimental matrix across two primary scenarios: Standard Reasoning and Early Reasoning. For early reasoning, we run experiments with 4 different QC thresholds: θ ∈ {65%, 75%, 85%, 95%}. For both scenarios, we conduct comparative experiments with and without RetroThinker. Finally, we apply LengthDPO to the early-reasoning models to examine the effect of reducing the reasoning trace on overall system latency. We regard all settings without RetroThinker as our baselines. C. Main Results The accuracy and latency performance for each setting is illustrated in Figure 3. Our analysis yields several key insights. RetroThinker generally improves accuracy at the cost of latency. As indicated by the gray arrows, integrating RetroThinker consistently improves accuracy across the evaluated settings. However, these improvements come at the cost of increased latency before applying LengthDPO, averaging approximately two seconds. RetroThinker improves accuracy for early reasoning with the help of DPO. Furthermore, the observed improvement in
StandardReasoning [7] EarlyReasoning [7] EarlyReasoning + DPO [7]
StandardReasoning + Retro EarlyReasoning + Retro EarlyReasoning + Retro + DPO
35
Accuracy (%)
Accuracy Calculation. Following [7], we transcribe the generated audio using Whisper [23] for accuracy evaluation. We evaluate the generated waveform rather than Moshi’s internal text stream because the waveform is the user-facing output. The resulting transcripts are then evaluated using an LLM-asa-Judge framework to determine logical correctness. Across all experiments, we utilize Qwen3-235B-Instruct [24] as our LLM.
30 25 20 15
4
5
6
7
8
Latency (sec)
9
10
Fig. 3. Accuracy-latency trade-off for different configurations. For Early Reasoning settings, varying Question Completeness (QC) thresholds θ ∈ {65%, 75%, 85%, 95%} are represented by different marker sizes; an increase in size denotes a higher threshold. The 95% threshold is highlighted. Gray dotted arrows indicate the performance shift from adding RetroThinker. The baselines are he settings without Retro.
EarlyReasoning is limited when compared to StandardReasoning. This aligns with our hypothesis in Section III (Stage 3), which posits a mismatch when using an external LLM to evaluate and rewrite model-generated samples. By applying DPO, we largely alleviate this discrepancy, yielding accuracy and latency trade-offs comparable to standard reasoning. Combining RetroThinker with EarlyReasoning and LengthDPO gives the best trade-off. Our best setting, EarlyReasoning + Retro + DPO with the 95% QC threshold, achieves an 11 percentage-point absolute accuracy improvement while keeping nearly the same latency compared to Standard Reasoning. It also obtains about a 30% latency reduction compared to StandardReasoning + Retro with comparable
TABLE I E XAMPLES OF DIFFERENT SELF - CORRECTION BEHAVIORS . T HE BOLD QUESTION TEXT IS THE CONTENT SPOKEN BY THE USER BEFORE
E ARLY R EASONING BEGINS . T HE SYMBOLS [✔] AND [✘] INDICATE THE MODEL’ S SELF - VERIFICATION RESULT. Revision is based on new information after the start of reasoning Question: Reasoning:
The rug is 5 feet wider than the chair. The couch is 2 feet longer than twice the width of the rug. If the chair is 3 feet wide. How many feet long is the couch? The rug is 5 feet wide [✘] The rug is 5 feet wider than the chair, so it is 3 + 5 = 8 feet wide. [✔] The couch is 2 feet longer than twice the width of the rug, so it is 2 * 8 + 2 = 18 feet long. [✔]
Revision is purely based on prior step’s logical mistake Question: Reasoning:
Two girls each got 1/6 of the 24 liters of water. Then a boy got 6 liters of water. How many liters of water were left? The two girls got 1/6 * 24 = 4 liters of water. [✘] The two girls each got 1/6 * 24 = 4 liters of water. So, they got 4 + 4 = 8 liters of water. [✔] The boy got 6 liters, so the total amount of water taken is 8 + 6 = 14 liters. [✔] The remaining water is 24 - 14 = 10 liters.
TABLE III C OT LENGTHS AND R ETROT HINKING R ATIO BEFORE AND AFTER L ENGTH DPO FOR THE E ARLY R EASONING + R ETRO SETTING . Settings
CoT Length (# of tokens)
RetroThinking Ratio (%)
θ(%)
65
75
85
95
65
75
85
95
Before DPO After DPO
158.6 134.8
157.7 125.5
138.9 111.2
133.9 99.8
75.7 59.5
80.3 58.2
69.5 30.5
65.1 7.2
automated classification on the EarlyReasoning (θ = 65%) + Retro + DPO setup, we find that 23% of revisions are Type 1 and 32% are Type 2. This suggests the model often refines its reasoning based on incoming user context. We provide one example for each type in Table I. However, in 42% of instances, the revision remains logically identical to the original step, revealing that incorrect verification still causes redundant retrospective thinking. Addressing this inefficiency remains a clear direction for future work. B. Rule-based vs. Sample-based RetroThinker
TABLE II A BLATION RESULTS FOR S TAGE 1 ( RULE - BASED ) AND S TAGE 2 ( SAMPLE - BASED ) SFT ACROSS DIFFERENT QC THRESHOLDS (θ). Settings
Accuracy (%)
Latency (seconds)
θ(%)
65
75
85
95
65
75
85
95
EarlyReasoning + Stage 1 + Stage 1&2
15 13 21
17 16 22
20 19 28
23 23 24
6.1 7.7 9.1
6.2 8.0 10.0
6.1 6.6 9.6
7.0 8.4 8.8
accuracy. In contrast, removing the RetroThinker component (leaving only EarlyReasoning + DPO, as in [7]) results in a substantial accuracy drop. These results indicate that RetroThinker is a critical driver of accuracy, and its combination with Early Reasoning and DPO yields the most favorable accuracy-latency Pareto frontier. Latency control of the QC metric is diminished with RetroThinker. Despite the improved Pareto frontier achieved with RetroThinker, we observe that the QC metric loses its precise control over latency compared to models without retrospective thinking. In analyzing this phenomenon, we identify two primary factors affecting the final latency: how early the model initiates the reasoning trace and the total length of the reasoning trace. We find that while the onset of reasoning is strictly dictated by the QC threshold (i.e., a higher threshold delays initiation), the resulting CoT lengths remain highly variable. We leave explicit control of CoT lengths under retrospective thinking to future work. V. D ETAILED A NALYSIS A. Understanding the Accuracy Boost of RetroThinker To understand how retrospective thinking dynamically improves accuracy, we categorize model revisions into two types: (1) correcting previous logical errors, and (2) integrating new information from the ongoing user query. Utilizing an LLM for
RetroThinker conducts SFT on retrospective thinking in two stages (Section III). As shown in Table II, accuracy remains stagnant after Stage 1 (rule-based). This indicates that while Stage 1 establishes the structural format of retrospective tokens, it fails to impart strong self-correction abilities. In contrast, Stage 2 (sample-based) leverages authentic model-generated errors, yielding significant accuracy improvements—notably a 9% absolute gain at the θ = 0.85 threshold. This highlights the necessity of sample-based training to bridge the gap between synthetic perturbations and actual inference errors. C. Identifying the Accuracy Bottleneck Effective retrospective thinking requires both selfverification of CoT steps and subsequent self-correction. To disentangle these, we replace the model’s internal verification tokens with an external LLM as an “oracle verifier.” During inference, whenever a verification token is sampled, the oracle evaluates the preceding reasoning and replaces the model’s token with its judgment. To establish an upper bound, we apply this to our most accurate setting (Standard Reasoning + Retro). The oracle improves accuracy from 35% to 42%. This 7% gain demonstrates that while some errors stem from internal verification failures, the remaining 58% error rate indicates the primary bottleneck is the error revision process itself. We leave the refinement for future research. D. Impact of LengthDPO on Retrospective Thinking Beyond mitigating distribution mismatch (Section III), we analyze how LengthDPO alters generation by examining CoT length and the Retrospective Thinking Ratio (the proportion of test instances with at least one CoT revision). As shown in Table III, average CoT length and the Retrospective Thinking Ratio decrease consistently across all QC thresholds. The ratio’s reduction is more pronounced at higher θ. Intuitively, limited initial information (lower θ) causes more
Accuracy
100%
EarlyReasoning [7] EarlyReasoning w/ RetroThinking & DPO
Answer accuracy
80% 60%
n=357 n=364
40%
n=290
20%
n=138 n=87
0% 2
3
4
5
6+
Complexity (measured by # of Ground Truth Reasoning Steps)
VI. C ONCLUSION
Latency
Response latency (s)
12
EarlyReasoning [7] EarlyReasoning w/ RetroThinking & DPO
n=87 n=138
10
n=290
8 6
n=364 n=357
4 2 0 2
3
4
problems of moderate difficulty rather than the hardest longhorizon questions, where both systems remain weak. Response latency increases monotonically with the number of reasoning steps for both systems, consistent with longer reasoning chains requiring more generation before an answer is produced. RetroThinking with DPO incurs a modest latency overhead that widens with complexity, from a negligible difference at 2–3 steps to roughly +1.2 s at 4 steps and +1.4 s at the 6+ bin. We attribute this to the additional retrospective reasoning the model performs on harder problems, which is precisely where the accuracy gains are concentrated.
5
6+
Complexity (measured by # of Ground Truth Reasoning Steps)
Fig. 4. Accuracy (top) and latency (bottom) grouped by the question complexity (number of ground-truth reasoning steps). Error bars denote ±1 standard error of the mean.
reasoning errors, whereas more complete input (higher θ) enables robust initial hypotheses, requiring fewer revisions. Coupled with the observed accuracy gains, these findings suggest that DPO finetuning makes the model more selective. By retaining only essential retrospective steps, this optimization reduces response latency without sacrificing accuracy. E. Analysis by Reasoning Complexity To understand how performance scales with problem difficulty, we group the GSM8K test questions by its complexity, measured by the number of intermediate steps in the reference solution. We focus on multi-step problems (2–6+ steps) and merge the sparse tail (≥ 6 steps) into a single “6+” bin; the single-step questions are excluded as their reference annotations are noisy. Figure 4 reports answer accuracy and response latency per group for the baseline (EarlyReasoning [7]) and the proposed EarlyReasoning w/ RetroThinking & DPO. Both systems exhibit the expected decay in accuracy as reasoning complexity grows, dropping from the 2-step regime to near-zero performance at 6+ steps. Across every complexity bin, RetroThinking with DPO improves over the baseline. The gains are largest and most reliable in the mid-complexity regime. At 3 steps accuracy rises from 23.1% to 37.9% (+14.8 points), and at 4 steps from 12.8% to 25.2% (+12.4 points), with non-overlapping error bars in both cases. Improvements at 5 and 6+ steps are positive but smaller and fall within the margin of error, indicating that the method primarily recovers
In this work, we introduced RetroThinker, a novel posttraining framework that equips streaming SpeechLLMs with forward-only retrospective reasoning capabilities. Our experiments on the spoken GSM8K benchmark demonstrate that models such as Moshi can be effectively fine-tuned to selfverify Chain-of-Thought (CoT) steps and generate corrective continuations without requiring trace rollback. Crucially, while retrospective training improves accuracy at the cost of increased latency, we successfully mitigate this overhead by integrating Early Reasoning and LengthDPO. Furthermore, our comprehensive ablation studies validate the efficacy of our three-stage pipeline and substantiate our core hypotheses. By analyzing current failure modes, we highlight promising avenues for future research, ultimately advancing the development of retrospective, low-latency reasoning in spoken language models. VII. L IMITATIONS Our evaluation is limited to a TTS version of GSM8K. This benchmark is useful because it provides step-level rationales and objective answers, but it does not capture the acoustic variability, interruptions, disfluencies, or task diversity of natural spoken dialogue. Therefore, our claims should be interpreted as evidence for streaming speech math reasoning rather than broad conversational generalization. Our accuracy metric also depends on Whisper transcription followed by an LLM judge. We use this pipeline to evaluate the actual user-facing waveform and to remain comparable with prior work, but ASR or judge errors can still affect absolute accuracy. Because every system is evaluated with the same pipeline, the relative comparisons are more reliable than the absolute scores. Finally, while RetroThinker significantly improves performance, it does not entirely eliminate the latency cost of reasoning. Before DPO, retrospective corrections add noticeable delay, and even after DPO, the QC threshold no longer controls latency as precisely as in non-retrospective early reasoning. Furthermore, resolving highly complex, multi-step problems remains an open challenge for our model, reflecting a broader difficulty in streaming speech reasoning. Future work will focus on testing natural and noisy speech benchmarks, as well as developing methods to explicitly control the number and length of retrospective corrections.
VIII. AI-G ENERATED C ONTENT D ISCLOSURE Generative AI tools were utilized exclusively for figure formatting, language editing and refinement of this manuscript. The authors take full responsibility for all scientific content, originality, and the final submitted work. IX. ACKNOWLEDGMENTS We thank Wei Zhou for helpful discussions during the early stage of this project. R EFERENCES [1] T. Likhomanenko, L. Carlson, R. H. Bai, Z. Gu, H. Tran, Z. Aldeneh, Y. Zhang, R. Zhang, H. Zheng, and N. Jaitly, “Chipchat: Low-latency cascaded conversational agent in mlx,” in ASRU, 2025. [Online]. Available: https://arxiv.org/abs/2509.00078 [2] S. Arora, Y. Peng, J. Shi, J. Tian, W. Chen, S. Bharadwaj, H. Futami, Y. Kashiwagi, E. Tsunoo, S. Shimizu, V. Srivastav, and S. Watanabe, “ESPnet-SDS: Unified toolkit and demo for spoken dialogue systems,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (System Demonstrations), N. Dziri, S. X. Ren, and S. Diao, Eds. Albuquerque, New Mexico: Association for Computational Linguistics, Apr. 2025, pp. 248–259. [Online]. Available: https://aclanthology.org/2025.naacl-demo.21/ [3] J. Chen et al., “Fireredchat: A pluggable, full-duplex voice interaction system with cascaded and semi-cascaded implementations,” 2025. [Online]. Available: https://arxiv.org/abs/2509.06502 [4] A. Défossez et al., “Moshi: a speech-text foundation model for real-time dialogue,” 2024. [Online]. Available: https://arxiv.org/abs/2410.00037 [5] A. Zeng, Z. Du, M. Liu, K. Wang, S. Jiang, L. Zhao, Y. Dong, and J. Tang, “Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot,” 2024. [Online]. Available: https://arxiv.org/abs/2412.02612 [6] R. Roy, J. Raiman, S. gil Lee, T.-D. Ene, R. Kirby, S. Kim, J. Kim, and B. Catanzaro, “Personaplex: Voice and role control for full duplex conversational speech models,” 2026. [Online]. Available: https://arxiv.org/abs/2602.06053 [7] Y.-J. Shih, D. Raj, C. Wu, W. Zhou, S. Bong, Y. Gaur, J. Mahadeokar, O. Kalinli, and M. Seltzer, “Can speech llms think while listening?” in ICLR, 2026. [8] B. Xiang, S. Zhao, T. Guo, and W. Zou, “Understanding the modality gap: An empirical study on the speech-text alignment mechanism of large speech language models,” in EMNLP. Suzhou, China: Association for Computational Linguistics, Nov. 2025, pp. 5187–5202. [Online]. Available: https://aclanthology.org/2025.emnlp-main.262/ [9] J. Wei, X. Wang, D. Schuurmans, M. Bosma, brian ichter, F. Xia, E. H. Chi, Q. V. Le, and D. Zhou, “Chain of thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022. [10] C.-H. Chiang, X. Wang, L. Li, C.-C. Lin, K. Lin, S. Liu, Z. Wang, Z. Yang, H. yi Lee, and L. Wang, “Shanks: Simultaneous hearing and thinking for spoken language models,” 2025. [Online]. Available: https://arxiv.org/abs/2510.06917 [11] C.-H. Chiang et al., “STITCH: Simultaneous thinking and talking with chunked reasoning for spoken language models,” in ICLR, 2026. [12] Z. Xie et al., “Mini-omni-reasoner: Token-level thinking-inspeaking in large speech models,” 2025. [Online]. Available: https://arxiv.org/abs/2508.15827 [13] Y. Lin, Z. Hu, Q. Wang, Y. Liu, H. Zhang, J. Subramanian, N. Vlassis, H. H. Li, and Y. Chen, “Voice evaluation of reasoning ability: Diagnosing the modality-induced performance gap,” 2025. [Online]. Available: https://arxiv.org/abs/2509.26542 [14] J. E. Hoffman, B. Nelson, and M. R. Houck, “The role of attentional resources in automatic detection,” Cognitive Psychology, vol. 15, no. 3, pp. 379–410, 1983. [Online]. Available: https://www.sciencedirect.com/science/article/pii/0010028583900130
[15] T.-H. Wu, H. Lee, J. Ge, J. E. Gonzalez, T. Darrell, and D. M. Chan, “Generate, but verify: Reducing hallucination in vision-language models with retrospective resampling,” in The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. [Online]. Available: https://openreview.net/forum?id=3Xmr8WTAld [16] R. Rafailov et al., “Direct preference optimization: Your language model is secretly a reward model,” in Thirty-seventh Conference on Neural Information Processing Systems, 2023. [Online]. Available: https://openreview.net/forum?id=HPuSIXJaa9 [17] E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9 [18] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman, “Training verifiers to solve math word problems,” arXiv preprint arXiv:2110.14168, 2021. [19] Y. Chen, X. Yue, C. Zhang, X. Gao, R. T. Tan, and H. Li, “Voicebench: Benchmarking llm-based voice assistants,” arXiv preprint arXiv:2410.17196, 2024. [20] H. Xu et al., “Contrastive preference optimization: Pushing the boundaries of llm performance in machine translation,” in ICML, 2024. [Online]. Available: https://openreview.net/forum?id=51iwkioZpn [21] Y. Meng, M. Xia, and D. Chen, “SimPO: Simple preference optimization with a reference-free reward,” in The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. [Online]. Available: https://openreview.net/forum?id=3Tzcot1LKb [22] A. Wu, L. Mazaré, N. Zeghidour, and A. Défossez, “Aligning spoken dialogue models from user interactions,” ArXiv, vol. abs/2506.21463, 2025. [Online]. Available: https://api.semanticscholar.org/CorpusID:280012148 [23] A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in Proceedings of the 40th International Conference on Machine Learning, ser. ICML’23. JMLR.org, 2023. [24] Q. Team, “Qwen3 technical report,” 2025. [Online]. Available: https://arxiv.org/abs/2505.09388