Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
arXiv:2605.15102v1 [cs.CL] 14 May 2026
Renning Pang, Tian Lan, Leyuan Liu , Xiaoming Huang, Piao Tong, and Xiaosong Zhang University of Electronic Science and Technology of China [email protected], [email protected], [email protected], [email protected], [email protected]
Abstract. Large language model (LLM) based multi-turn dialogue systems often struggle to track dependencies across non-adjacent turns, undermining both consistency and scalability. As conversations lengthen, essential information becomes sparse and is buried in irrelevant context, while processing the entire dialogue history incurs severe efficiency bottlenecks. Existing solutions either rely on high latency external memory or lose fine-grained details through iterative summarization. In this paper, we propose Self-Recall Thinking (SRT), a framework designed to address long-range contextual dependency and sparse informative signals in multi-turn dialogue. SRT identifies helpful historical turns and uses them to generate contextually appropriate responses, enabling the model to selectively recall and reason over context during inference. This process yields an endogenous reasoning process that integrates interpretable recall steps without external modules. SRT incorporates: (1) Dependency Construction: Generating and converting it into self-recall chains; (2)Capability Initialization: Training to enable reasoning chains with recall tokens capability; (3)Reasoning Improvement: Refining accuracy via verifiable rewards to optimize recall and reasoning for correct answers. Experiments on multiple datasets demonstrate that SRT improves F1 score by 4.7% and reduces end-to-end latency by 14.7% over prior methods, achieving a balance between reasoning latency and accuracy, and outperforming state-of-the-art baselines. Keywords: Relation context · Large Language Models · Chain-of-Thought.
1
Introduction
Multi-turn dialogue is central to real-world assistants in customer service and instant messaging, where sustained, context-aware exchanges drive user satisfaction and operational efficiency [12]. However, this requirement creates a fundamental tension [42] as conversations lengthen, the volume of historical context grows, yet the specific knowledge needed for a coherent response becomes increasingly sparse. Simply processing this entire, ever-growing history is not a viable solution, as it leads to significant performance bottlenecks, including attention dilution and prohibitive computational costs [40]. This challenge forces
2
R. Pang et al.
reliance on imperfect strategies and highlights a critical research gap regarding the need for a mechanism that can leverage long-term dialogue history with both precision and efficiency [2]. As shown in Fig. 1, the utilization of history as the engine of such multi-turn dialogues is typically manifested in two dominant paradigms. One major line of work focuses on structuring memory externally, using components like graph databases or explicit episodic logs to organize dialogue history. However, this approach introduces significant system latency, complex workflow design, and high costs for multi-agent deployment [6]. Another popular approach involves learning an adaptive summary policy, often using alignment tuning to decide state evolution. These methods offer flexibility, but they typically operate by updating a condensed or abstractive memory state, where crucial latent details are progressively lost during compressive update [25, 21]. A common thread across these approaches is their reliance on external architectures, complex preprocessing, or non-trivial learning processes, which introduce latency and loss of detailed knowledge.
User query
User query
User query
External cache Memory Edit
State Evolute KG
Rule
Retrieve / Recall
Update / Fusion Traj
Answer
In-context
Internal parameter
FAQ
Answer
Self Recall Abstract
Role
Intent
Embed
Thinking History Dialogue
Answer
Fig. 1: Comparison of SRT with other multi-turn dialogue generation approaches To bridge these limitations, we propose the Self-Recall Thinking (SRT) mechanism and design an adaptive endogenous explicit recall strategy to balance precision and efficiency, enabling the model to mine and recall historical knowledge rather than passively retrieve. Instead of summarizing the entire dialogue context at a coarse level, we argue that during the reasoning process, models should proactively analyze and recall only the key historical turns directly relevant to the current query. SRT represents a transition from externally-oriented refinement to endogenous recall of fine-grained dependent content. During reasoning, the model can analyze the current query to recall relevant historical turns, generating a predefined signal to replay in thinking on-demand. Specifically, SRT follows an analyze → recall → cite → reason → answer cycle within each turn. For example, in a customer service dialogue, when asked whether a membership discount applies to a specific booking at the Uptown branch, SRT opens <HIS> tag to cite the exact turns that contain the discount rule that applies above one hundred fifty dollars, the service price of one hundred eighty dollars, and the Uptown location: <HIS> Q1: I need to check membership status. I’m under Apex Tier. A5: The Chakra Balancing service is $180. A6: Lena only offers that service at our Uptown branch.</HIS>, then continues think-
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
3
ing to conclude that the discount applies. We also propose a novel adaptive endogenous explicit recall strategy SRT-P, which can be directly applied to dialogue agent invocation scenarios. Empirically, the experimental results confirm that SRT effectively recalls and exploits dialogue history for multi-turn reasoning, surpassing prior state-of-the-art systems with an average F1 increase of 3.4% and an average latency reduction of 13.1% across QA benchmarks, each benchmark gains range from 1.1% to 4.7% in F1 and from 11.1% to 14.7% in end-to-end latency. Additionally, the pluggable strategy abstracted from SRT mechanism can be applied to closed-source commercial models to gain revenue. The key contributions of this work are summarized as follows: – We introduce SRT, a novel reasoning strategy for multi-turn dialogues. SRT enables the model to proactively recall and selectively attend to dependent content from the dialogue history, thereby enhancing reasoning accuracy with fine-grained details. – We constructed instruction fine-tuning datasets and trained the model via fine tuning alignment to equip the model with self-recall capabilities. Furthermore, we designed a verifiable reward mechanism to guide the optimization of its recall behavior, ensuring that the model effectively leverages historical context. – We conducted extensive experiments on several multi-turn dialogue benchmarks. The results demonstrate that our proposed model significantly outperforms existing baseline approaches on key downstream tasks, achieving a superior trade-off between computational cost and reasoning accuracy.
2
Related Work
2.1
External Cache Retrieval
Early retrieval-augmented systems for dialogue agents stored raw or summarized conversation history and relied primarily on vector similarity search [5]. Recent efforts have introduced varying degrees of structural organization into memory. Knowledge graph based methods [33, 13, 7] build explicit relational structures among entities or events, often augmented with temporal edges to support multi-hop reasoning across sessions in practical deployments. A-MEM [32] adopts agentic organization, dynamically indexing and linking memory units as new evidence arrives. DH-RAG [38] employs a hierarchical design that clusters content into multi-level schemas to route queries across abstraction levels. ComoRAG [28] triggers targeted probes into memory, integrates retrieved evidence into a working state, and resumes inference. Extensions such as MemoRAG [22] relax assumptions about query clarity and knowledge structure, improving longcontext understanding at the cost of increased latency and controller complexity. EM-LLM [8] segments conversational streams into coherent episodes and retrieves based on both semantic relevance and temporal contiguity, enabling human-like memory access patterns. SGMem [31] extends this idea to dialogue
4
R. Pang et al.
by linking turns, rounds, and sessions, while mixing raw history with generated summaries to supply coherent multi-granular context. TiM [18] stores and reuses thoughts rather than raw text, pairing pre-answer recall with post-answer updates, and employs locality-sensitive hashing for scalable retrieval. 2.2
Latent State Evolution
A complementary research direction internalizes memory within the LLMs, learning to maintain and update a compact latent state that co-evolves with the reasoning process. This paradigm can be organized into several interrelated strands. Memory-R1 [35] trains a Memory Manager and an Answer Agent using outcome driven reinforcement learning, turning memory management into a learned policy rather than a fixed pipeline. MemAgent [37] learns an overwrite policy via RL to enable efficient long-text processing with constant size memory. Memento [43] further supports continual adaptation through online RL schemes that update case-selection policies and rewrite episodic memory. ReasoningBank [20] complements these by distilling reusable reasoning strategies into a memory bank that is retrieved and scaled at test time. Another strand targets learning the latent memory state itself. MEM1 [45] maintains a constant size internal state that is jointly updated for memory and reasoning each turn, discarding redundancies while integrating new observations for better long-horizon generalization. Mem-α [29] employs RL to learn not only what information to extract and how to structure it across core, episodic, semantic components, and when to update it. MELODI [11] introduces a hierarchical compression mechanism operating on short-term and long-term memory compresses and aggregates information at a mid-layer. Furthermore, MemGen [39] incorporates a memory trigger to decide when to recall and a memory weaver that synthesizes latent tokens within the reasoning process, yielding emergent planning and procedural memories.
3
Method
As shown in Fig. 2, our approach is realized through a three-stage framework. Stage 0: Dependency Construction, which leverages Claude 3.7 Sonnet to convert raw data into a structured dialogue history, complete with corresponding dependencies. Stage 1: Capability Initialization, which involves Supervised FineTuning to train the model to generate <HIS> tag during the thinking process encapsulating recall content. Stage 2: Reasoning Improvement, which utilizes Group Relative Policy Optimization (GRPO) [24] to treat the recalling and answering in each context-dependent turn as a joint optimization objective, using a verifiable outcome reward. 3.1
Preliminaries
We model self-recall as an internal pointer policy πθ that operates over the dialogue history Ct = {u1 , ..., ut−1 }. At turn t, given the query Qt , the model
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
Q7 ?
Q1: I need to check membership status. I'm under Apex Tier.
SRT generates a special token <HIS> to synergize pertinent Step Thinking and Self Recall at every timestep.
A1: Welcome back. I confirm your Apex Tier status. As a reminder, this gives you a 20% discount on any service over $150. Q2: Does that apply to all locations?
<think> 1. Analyze Query: The user is asking to confirm if their Apex Tier discount will apply to a specific booking at the Uptown branch...
A2: Yes, it's valid at both our Downtown and Uptown branches. Q3: Okay. Separately, what is the Chakra Balancing service? A3: That is a 90-minute session combining hot stones and aromatherapy. Q4: Is that new? Who performs that?
A4: Our senior therapist, Lena, the specialist for that service. A5: The Chakra Balancing service is $180. Q6: And which branch does Lena perform that at? A6: Lena only offers that service at our Uptown branch.
History Session
History Token Query Token
Recall Token Answer Token
Q7: Great. So, for that specific booking at the Uptown branch,
Prompt
Text Token
2. Recall History: <HIS>Q1: I need to check membership status. I'm under Apex Tier. A5: The Chakra Balancing service is $180. A6: Lena only offers that service at our Uptown" branch.</HIS> 3. Reasoning:The discount rule recalled from A1 requires the service price to be over $150.The price for the service... Therefore, the discount applies. </think>
Q5: What is the full price for that session?
will my Apex Tier discount apply?
5
Final answer: Let me check. The service is $180 , and your Apex Tier status is valid at the Uptown branch. Yes, the 20% discount will apply.
Query
...
Recall Signal Recall Turns <think>
LLM ... Stage 0：Dependency Construction Raw Data
Dialogue History
Non-continuous Dialogues
Turn 1 Turn 2
RL Alignment SFT warm-up
<HIS>Q1 A2</HIS> <HIS></HIS>
...
Most Relative Turns
Prompt
Rollout
Turn n
Input Query
<HIS>A5</HIS>
Answer
A1
KL
A2
Answer
<HIS>Q1 A5 A6</HIS>
Answer
A3
...
LLM
Normalization and Clip Update
...
Continuous Dialogues
Stage 2：Reasoning Improvement
Stage 1：Capability Initialization
Answer
An
Dependencies Historical Conversation
Recall reward
Current Query
Answer reward
Fig. 2: SRT Framework. In Stage 0, we construct historical dependency structures from continuous/discontinuous dialogues. In Stage 1, we train the model to learn self-recall capability. In Stage 2, we refine the model’s self-recall reasoning capability through RL using verifiable rewards.
generates a CoT reasoning. During this process, the model can take N recall actions. Each action, ok (for k = 1...N ), consists of emitting a <HIS> tag that verbatim-copies a selected historical utterance uik from Ct . The set of recalled information Hk is built recursively: Hk = Hk−1 ∪ {ok }, with H0 = ∅. The model implicitly terminates this recall phase (determining the final count N ) by proceeding to generate the answer At , which is conditioned on the final recall set HN . This mechanism treats the dialogue state update as a deterministic transition. The joint probability of generating N recall actions (o1:N ) and the final answer At is factorized as:
P (At , o1:N |Qt , Ct ) =
"N Y k=1
# πθ (ok |Qt , Ct , Hk−1 ) · πθ (At |Qt , Ct , HN )
(1)
6
R. Pang et al.
This formulation models the probability of each recall action ok as dependent on the query, context, and all previously recalled turns Hk−1 . The final answer At is then conditioned on the complete set of recalled evidence HN . 3.2
Dialogue Dependency Structure Construction
To curate a dataset focused on multi-turn context recall, we first implement a multi-stage dialogue filtering pipeline [9]. We begin by performing redundancy reduction and deduplication, applying clustering on intent features and turnlevel feature hashing to merge or remove semantically identical dialogues. We then retain only dialogues with turn counts ranging from 8 to 32. The core of our filtering is a dependency-aware selection process. A dialogue is selected for annotation only if it contains at least one turn Qt (the current query at turn t) and a corresponding historical utterance ui (at turn i) that satisfy a dualdimensional scoring function: Score(ui , Qt ) = 0.6 · Simsem (ui , Qt ) + 0.4 · e−0.15·|t−i| > 0.6
(2)
Simsem denotes semantic similarity, ensuring the historical turn ui is relevant. The second term is an exponential decay factor as 0.15 is an empirically optimal value that reduces the weight of historical utterances from 10 rounds prior to 0.22. This selection process ensures that the retained dialogues contain quantifiable, non-adjacent dependencies, forcing the model to retrieve information from distant history and directly aligning with SRT’s recall-oriented goal. To generate reliable supervision signals for recall and reasoning, we design a hybrid annotation pipeline. For each turn (Qt , Ct ) in the filtered dialogues, we use Claude 3.7 Sonnet [16] as a teacher model, guided by recall-targeted signals. The model is prompted endogenously to produce the minimal recall set Ht∗ as the smallest set of historical utterances required to answer Qt , formatted as Turn X: ’content’ to enable turn-level recall. Then we get the step-by-step reasoning from recall to concise answer, which is strictly grounded in the content of Ht∗ , designed to facilitate alignment with reward designs. Moreover, we apply a datadriven verification filter to the generated triplets (Ht∗ , Zt , A∗t ). We ensure logical consistency between the thinking Zt and the recall set Ht∗ , and verify strict answer–evidence alignment by confirming no hallucination via string matching between A∗t and Ht∗ . 3.3
Supervised Fine-Tuning for Capability Warm-up
Despite the availability of explicit recall annotations, directly training language models on raw multi-turn dialogues often fails to induce stable CoT behaviors, as models tend to overfit surface patterns or generate ungrounded rationales when processing lengthy, heterogeneous contexts. To address this, we develop a twostage training approach that first bootstraps recall capabilities using dependency pruned histories, then transfers these capabilities to full contextual reasoning.
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
7
Bootstrapping with Dependency-Pruned Histories For each dialogue turn t, we construct simplified training instances by pruning the complete history Ct to a compact subset C̃t using the dependency scoring mechanism from Section 2. From the teacher-generated annotations, we preserve the minimal recall set Ht∗ ⊂ C̃t , the structured reasoning chain Zt = {r1 , . . . , rM } with explicit <HIS> tagging, and the evidence-grounded answer A∗t . This pruning strategy reduces contextual distractions while maintaining logical dependencies, providing clean training signals for learning recall timing, target selection, and information utilization. Reasoning Transfer to Full Contexts The distilled recall reasonings subsequently guide rationale generation on complete dialogue histories. Given a prompt template P(·) and target outputs (Zt , A∗t ), we optimize the negative log-likelihood objective:
LSFT = −
M X
log πθ (ri | P(Qt , Ct ), r<i ) −
i=1
L X
log πθ (aj | P(Qt , Ct ), Zt , a<j )
j=1
(3) This staged training approach effectively activates the model’s inherent reasoning capabilities for the self-recall task, establishing a robust foundation for subsequent alignment enhancement. 3.4
Recall-Reasoning Alignment
Verifiable Composite Reward Design. To refine the model’s reasoning capability and prevent reward hacking behaviors, we design a composite reward function R(τ ) that comprehensively evaluates the entire generated reasoning τ = (Zt , At ). This function is fully verifiable, as all components are computed by comparing the model’s outputs against explicit rule-based criteria and groundtruth annotations, eliminating dependence on learned reward models that may introduce bias or instability. Given a reasoning τ comprising the reasoning chain Zt and final answer At , we first extract the predicted recall set Ĥt by aligning each <HIS> tag in Zt to its corresponding historical utterance ui ∈ Ct through maximal normalized overlap matching. The comprehensive reward function is then defined as a weighted combination of three core components: R(Zt , At ) = format(Zt ) + recall(Ĥt , Ht∗ ) + answer(At , A∗t )
(4)
The format reward ensures the structural integrity of the recall mechanism and prevents malformed outputs that would compromise auditability. It is defined as a binary function that outputs 1 only if the reasoning trace Zt demonstrates complete syntactic validity, meaning all <HIS> tags are properly formatted, correctly paired, and parseable within established reasoning steps:
8
R. Pang et al.
(5)
Rformat (Zt ) = I(is_valid(Zt ))
The Recall Reward (Rrecall ) provides a fine-grained evaluation of recall accuracy, creating balanced incentives for the model to identify all necessary dependent turns while avoiding extraneous recalls. It operates within the range [−1.25, 1.25] and is calculated as a scaled Jaccard Index (Intersection over Union): Rrecall (Ĥt , Ht∗ |) = 2.5 ·
|Ĥt ∩ Ht∗ | |Ĥt ∪ Ht∗ |
! − 1.25
(6)
where Ht∗ represents the ground-truth recall set, and Ĥt denotes the predicted set. This formulation inherently penalizes both Missing Recall (which increases the union |Ĥt ∪ Ht∗ | via Ht∗ ) and Over-Recall, which also increases the union via Ĥt . By rewarding the maximization of the intersection relative to the union, this design promotes both completeness and precision in the recall process. The answer reward measures semantic quality of the final output, ensuring the model is rewarded for producing answers that are semantically equivalent to the reference even when using different phrasing or expression. It operates within the range [−1, 1] and is computed as: Ranswer (At , A∗t ) = CosineSimilarity(v At , v A∗t )
(7)
This reward provides a robust similarity measure beyond surface-level token matching, which ensures the model is optimized not only for the quality of the final output, but also for the fidelity and accuracy of the internal reasoning process that produces it, thereby promoting both performance and interpretability in self-recall thinking while maintaining training stability through verifiable, rule-based signal computation. Policy Optimization with RL. We adopt GRPO for policy optimization due to its memory efficiency and elimination of value network training requirements. This approach simplifies alignment by leveraging group-relative advantages to normalize learning signals across reasonings. For each prompt (Qt , Ct ), we sample K reasonings {τ (k) }K k=1 from the behavior policy πθold and compute their respective rewards R(k) = R(τ (k) ). We then calculate group-relative advantages to reduce variance across reasonings: Ã(k) =
R(k) − R̄ σR + ϵ
ri,t (θ) =
πθ (oi,t | q, oi,<t ) πθold (oi,t | q, oi,<t )
(8)
where R̄ represents the group mean reward, σR denotes the group standard deviation, and ϵ = 10−8 prevents division by zero. The complete GRPO optimization objective is formalized as:
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
9
|oi | G 1 X 1 X JGRPO (θ) = E(q,a)∼D,{oi }G min ri,t (θ)Âi,t , i=1 ∼πθold (·|q) G i=1 |oi | t=1
(9)
i clip ri,t (θ), 1 − ε, 1 + ε Âi,t − βDKL (πθ ∥πref ) where πθ is optimized by maximizing a clipped surrogate objective, augmented with a token-level KL divergence term that anchors to the supervised fine-tuned policy πref = πSFT to prevent catastrophic drift from Stage 1 capabilities.
4
Experiment
4.1
Implementation Details
We adopt Qwen2.5-7B [26] as the backbone model to ensure a strong and consistent baseline. Consistent with the two stage training framework outlined in Section 3, we implement SFT and RL alignment for model training, with all experiments conducted on 8 NVIDIA A100 (80 GB) GPUs to support efficient training of the 7B-scale model. For Stage 1, the model is trained with a learning rate of 5 × 10−6 . For Stage 2, we reduce the learning rate to 3 × 10−7 to stabilize RL updates, leveraging GRPO algorithm for RL and incorporating a KL divergence regularization term. We employ the AdamW optimizer with a constant learning rate schedule supplemented by linear warm-up. We use a rollout size of 8, and set the ratio of the sample batch size to the backpropagation batch size to 4. 4.2
Datasets
To evaluate SRT, we use a diverse set of benchmarks. First, we created SRQA, a unified dataset of 5,000 samples (8-32 turns) emphasizing long-range dependencies. It was curated from MG-ShopDial [3], MultiWOZ [4], DailyDialog [15], ReDial [44] and LoCoMo [19] using the processing method described in Section 3.2. Moreover, we include two standard open-domain QA benchmarks to assess general reasoning: SimpleQA [30] contains 4,326 short, factual questions, where each question has only one clear answer. CoQA [23] is a simulate real-life dialogue question answering dataset with over 127,000 question-answer pairs from 8,000 conversations. 4.3
Baselines
We compare our method against 6 representative baselines from three paradigms, built upon the Qwen2.5-7B foundation model for a fair comparison. We employed single-model inference in our tests rather than multi-agent workflow mechanisms.
10
R. Pang et al.
For the extra augmented paradigm, RQ-RAG [5] for ambiguous or complex queries, first performs query rewriting and decomposition to eliminate ambiguity before searching. QRMeM [27] maintains a dual-pool memory that fuses static text with structured graph information and navigates retrieval via reflective trial and error. LD-Agent [14] employs separate long and short term memory banks with topic-aware retrieval and dynamic persona modeling. For the latent reasoning paradigm, Coconut [10] feeds the LLM’s final hidden states back as continuous thoughts into the input embedding space, enabling parallel exploration of reasoning paths. OPRO [41] uses the language model itself as an optimizer, framing the search for an optimal reasoning instruction as a formal optimization task. SoftCoT [34] employs a frozen lightweight assistant to generate soft thought tokens projected into the backbone model’s representation.
Table 1: Main results on three benchmarks. Metrics are F1 and average end-toend latency (s). Category
Method
Our
RQ-RAG QRMeM LD-Agent OPRO Coconut SoftCoT SRT
4.4
Overall Performance
Extend Cache Latent Parameter
F1 72.9 70.6 73.1 71.2 75.0 70.4 78.4
SRQA Latency 14.7 12.5 11.8 12.0 10.5 13.3 9.1
F1 81.0 75.2 77.6 79.8 83.1 78.6 84.0
CoQA Latency 13.6 12.1 11.5 12.3 10.2 9.9 8.8
SimpleQA F1 Latency 53.6 14.8 50.3 12.9 51.7 12.1 49.2 14.6 52.5 10.8 51.8 10.2 56.1 8.7
Table 1 shows the main results across three question-answering benchmarks. Table 1 compares the F1 scores and end-to-end latency of SRT with baselines across three QA benchmarks. SRT achieves the optimal accuracy–efficiency trade-off universally: 56.1 F1/8.7 s on SimpleQA, 84.0 F1/8.8 s on CoQA, and 78.4 F1/9.1 s on SRQA. Across datasets, SRT outperforms baselines in both accuracy and efficiency: on SimpleQA, it surpasses RQ-RAG 53.6 F1 with 2.5% improvement and Coconut 52.5 F1 with 3.6% improvement, while being 2.1s–6.1s faster; on CoQA, it edges Coconut and RQ-RAG with the lowest latency; on SRQA, it outperforms Coconut by 3.4% and LD-Agent by 5.3%. Furthermore, with a latency of 9.1s, SRT is between 1.4s and 5.6s faster than the baselines, whose latencies range from 10.5s to 14.7s. We further compare SRT against two baseline paradigms. External Augmented Memory methods, which rely on external modules, introduce significant latency, with measured times ranging from 11.5s to 14.8s across the benchmarks. For instance, even the top performing method in this category, RQ-RAG exhibits a latency of at least 13.6s. While LatentMemory methods generally offer lower latency than their external counterparts,
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
11
they still fall short of the efficiency achieved by our approach. In contrast, SRT internalizes retrieval via <HIS> tags, eliminating the need for external modules and providing token-level auditable grounding, which explains its simultaneous accuracy and speed advantages. SRT excels most in conversation-heavy and longcontext scenarios: on SRQA, it outperforms LD-Agent with 5.3% improvement and is 1.4s faster. On CoQA, SRT avoids over/missing-recall errors to maintain 84.0 F1 and 8.8s. 4.5
Model design and optimization strategy
We removed core components of the SRT from the integrated dialogue corpus. As shown in Table 2, removing the reinforcement learning stage component resulted in the most significant performance degradation, a 7.2% drop in historical recall and a 4.8% decrease in answer accuracy. This confirms that the joint reward mechanism for recall and reasoning significantly influences the model’s recall motivation. Disabling the CoT module also caused a noticeable performance drop, with accuracy decreasing by 3.4%, confirming the effectiveness of explicit recall representations. The baseline using only prompts without fine-tuning performed worst, failing to effectively follow instructions within limited model capability.
Table 2: Ablation study results of SRT and its variants. Variant SRT – w/o RL – w/o CoT – w/o SFT
Recall (%) 92.5 85.3 89.1 81.0
Acc. (%) 78.9 74.1 76.0 72.2
(a) Non-recall guided atten- (b) Self-Recall guided at- (c) Attention residual Map tion map tention map
Fig. 3: Attention allocation visualization on the synthetic long-dialogue set.
12
4.6
R. Pang et al.
Attention Allocation Comparison Analysis
We analyze whether decoder attention concentrates on the turns retrieved by our self-recall mechanism during answer generation as shown in Fig. 3. We compute attention on reasoning, the proportion of attention mass from answer tokens to the retrieved tokens, averaged over layers and normalized per row. We compare two settings: Non-Recall Fig. 3(a) and Self-Recall guided Fig. 3(b). As dialogue length increases, the attention scores assigned to any single historical turn inevitably decrease. Critical information from early turns becomes lost in the middle, receiving negligible attention. To better distinguish the gap, Fig. 3(c) visualizes differences in token-level attention distribution. SRT introduces an active, explicit recall mechanism that fundamentally alters the reasoning behavior. By generating <HIS> tag, the model performs a discrete, high-salience action. It is no longer passively searching through a diluted context, which is actively pulling a critical piece of history out of the noise and placing it directly into its immediate working memory. This verbatim copying action forces the attention mechanism to focus on the recalled fact, bypassing the issues of dilution and interference for that piece of information. 4.7
Incorporating failure analysis
We analyze the proportion of each error type in the reasoning process among all bad cases. As shown in Table 3, the contribution of Missing and Wrong-Recall errors increases with dialogue length, while the share of errors from Normal/NoRecall Needed, where the model fails despite correct recall, declines yet remains the single largest category. This trend can be interpreted through the lens of multi-turn customer service dialogues.
Table 3: Error type distribution over different conversation lengths. t (turns) 8 16 24 32
Missing Recall 23.5% 28.2% 31.9% 35.9%
Over Recall 5.3% 5.6% 5.7% 6.1%
Wrong Recall 15.0% 16.7% 18.1% 19.2%
Failure Answer 56.1% 49.5% 44.3% 38.8%
The rise in Missing Recall occurs because crucial information such as order ID or account number is often provided in early turns and queried dozens of turns later, target clue becomes lost within para-service intervening turns like small talk or product inquiries etc. Using <HIS> as a recall trigger mitigates this by performing relative turns recalled in CoT. The increase in wrong recall stems from the prevalence of same entities in long logs, where greater context and noise elevate the risk of false matches. In contrast, Over-Recall remains low
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
13
and stable, slightly increasing as longer histories tempt the model to cite more segments. 4.8
External evaluation of strategy effectiveness
To verify the generalization of the self-recall mechanism beyond our fine-tuned SRT model, we distill it into a pluggable strategy SRT-P, that can be directly invoked via production APIs. We evaluate three closed-source large language models Claude 3.5 Sonnet [1], DeepSeek-V3 [17], and Qwen-Max [36] on a customer service style multi-turn workload. We report two key metrics answer accuracy and end-to-end latency across dialogue turn buckets k ∈ {8, 12, 16, 20, 24, 28, 32}.
(a) DeepSeek-V3
(b) Qwen-Max
(c) Claude 3.5 Sonnet
Fig. 4: Efficiency of the SRT-P to closed-source LLMs.
As shown in Fig. 4, SRT-P achieves consistent accuracy improvements across all three closed-source LLms, with pronounced advantages in multi-turn buckets (k ≥ 24) , where 2.7% for DeepSeek-V3, 2.3% for Qwen-Max, and 1.7% for Claude 3.5 Sonnet improvement. The effectiveness stems from the <HIS>, which explicitly facilitates the selection and citation of dialogue-turn knowledge within black-box reasoning mechanisms, optimizing attention allocation and consistency preservation, which shares consistent latent bias with the minimal sufficient history and correct answering objective of alignment in SRT.
5
Conclusion
In this paper, we introduced SRT, a novel framework designed to enhance the consistency and accuracy of multi-turn dialogue systems. This is achieved through a multi-stage process that combines fine-tuning to build foundational recall paradigm with alignment phase that refines reasoning using verifiable rewards. This results in an efficient and interpretable selection and utilization of historical knowledge mechanism that operates without external retrievers or indexes. Extensive experiments demonstrate that SRT achieves state-of-the-art performance across multiple dialogue benchmarks, offering a superior trade-off between accuracy and latency. Ablation studies further confirm that self-recall mechanism optimizes attention allocation and that reward strategy effectively
14
R. Pang et al.
guides the model’s behavior. In dialogue agent applications, the current selfrecall paradigm maintains multi-turn dialogue consistency well, while balancing compression costs and latency, and can be cost-effectively transferred to workflows.
6
Acknowledgments
This work was substantially supported by the National Natural Science Foundation of China (Grant No. U2336204, 62472075) and the Chengdu Industrial Chain Collaborative Innovation Project (Grant No. 2025-XT00-00017-GX).
References 1. Anthropic, A.: The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card 1(1), 4 (2024) 2. AWS: Building smarter ai agents with long-term memory - agentcore deep dive. https://aws.amazon.com/cn/blogs/machine-learning/building-smarterai-agents-agentcore-long-term-memory-deep-dive/ (2025) 3. Bernard, N., Balog, K.: Mg-shopdial: A multi-goal conversational dataset for ecommerce. In: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 2775–2785 (2023) 4. Budzianowski, P., Wen, T.H., Tseng, B.H., Casanueva, I., Ultes, S., Ramadan, O., Gašić, M.: Multi-user MultiWOZ: Task-oriented dialogues among multiple users (2018) 5. Chan, C.M., Xu, C., Yuan, R., Luo, H., Xue, W., Guo, Y., Fu, J.: Rq-rag: Learning to refine queries for retrieval augmented generation. arXiv preprint arXiv:2404.00610 (2024) 6. Chen, N., Li, H., Chang, J., Huang, J., Wang, B., Li, J.: Compress to impress: Unleashing the potential of compressive memory in real-world long-term conversations. In: Proceedings of the 31st International Conference on Computational Linguistics. pp. 755–773 (2025) 7. Dammu, P.P.S., Naidu, H., Shah, C.: Dynamic-kgqa: A scalable framework for generating adaptive question answering datasets. In: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. pp. 3498–3508 (2025) 8. Fountas, Z., Benfeghoul, M.A., Oomerjee, A., Christopoulou, F., Lampouras, G., Bou-Ammar, H., Wang, J.: Human-like episodic memory for infinite context llms. arXiv preprint arXiv:2407.09450 (2024) 9. Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al.: Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature 645(8081), 633–638 (2025) 10. Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., Tian, Y.: Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769 (2024) 11. Husom, E.J., Goknil, A., Shar, L.K., Sen, S.: The price of prompting: Profiling energy use in large language models inference. arXiv preprint arXiv:2407.16893 (2024)
Improving Multi-turn Dialogue Consistency with Self-Recall Thinking
15
12. IBM: Customer service and the generative ai advantage. White paper, Armonk, NY, USA (2025) 13. Jiang, J., Zhou, K., Zhao, W.X., Song, Y., Zhu, C., Zhu, H., Wen, J.R.: Kg-agent: An efficient autonomous agent framework for complex reasoning over knowledge graph. arXiv preprint arXiv:2402.11163 (2024) 14. Li, H., Yang, C., Zhang, A., Deng, Y., Wang, X., Chua, T.S.: Hello again! llmpowered personalized agent for long-term dialogue. In: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). pp. 5259–5276 (2025) 15. Li, Y., Su, H., Shen, X., Li, W., Cao, Z., Niu, S.: Dailydialog: A manually labelled multi-turn dialogue dataset. In: Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (2017) 16. Lior, G., Habba, E., Levy, S., Caciularu, A., Stanovsky, G.: Reliableeval: A recipe for stochastic llm evaluation via method of moments. arXiv preprint arXiv:2505.22169 (2025) 17. Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al.: Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024) 18. Liu, L., Yang, X., Shen, Y., Hu, B., Zhang, Z., Gu, J., Zhang, G.: Think-in-memory: Recalling and post-thinking enable llms with long-term memory. arXiv preprint arXiv:2311.08719 (2023) 19. Maharana, A., Lee, D.H., Tulyakov, S., Bansal, M., Barbieri, F., Fang, Y.: Evaluating very long-term conversational memory of llm agents. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 13851–13870 (2024) 20. Ouyang, S., Yan, J., Hsu, I., Chen, Y., Jiang, K., Wang, Z., Han, R., Le, L.T., Daruki, S., Tang, X., et al.: Reasoningbank: Scaling agent self-evolving with reasoning memory. arXiv preprint arXiv:2509.25140 (2025) 21. Pan, Z., Wu, Q., Jiang, H., Luo, X., Cheng, H., Li, D., Yang, Y., Lin, C.Y., Zhao, H.V., Qiu, L., et al.: On memory construction and retrieval for personalized conversational agents. arXiv preprint arXiv:2502.05589 (2025) 22. Qian, H., Liu, Z., Zhang, P., Mao, K., Lian, D., Dou, Z., Huang, T.: Memorag: Boosting long context processing with global memory-enhanced retrieval augmentation. In: Proceedings of the ACM on Web Conference 2025. pp. 2366–2377 (2025) 23. Reddy, S., Chen, D., Manning, C.D.: Coqa: A conversational question answering challenge. Transactions of the Association for Computational Linguistics 7, 249– 266 (2019) 24. Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al.: Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024) 25. Tan, Z., Yan, J., Hsu, I., Han, R., Wang, Z., Le, L.T., Song, Y., Chen, Y., Palangi, H., Lee, G., et al.: In prospect and retrospect: Reflective memory management for long-term personalized dialogue agents. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics. pp. 8416–8439 (2025) 26. Team, Q., et al.: Qwen2 technical report. arXiv preprint arXiv:2407.10671 2(3) (2024) 27. Wang, B., Huang, H.Y., Cao, Y., Ying, J., Tang, W., Feng, C.: Qrmem: Unleash the length limitation through question then reflection memory mechanism. In: Findings of the Association for Computational Linguistics: EMNLP 2024. pp. 4837–4851 (2024)
16
R. Pang et al.
28. Wang, J., Zhao, R., Wei, W., Wang, Y., Yu, M., Zhou, J., Xu, J., Xu, L.: Comorag: A cognitive-inspired memory-organized rag for stateful long narrative reasoning. arXiv preprint arXiv:2508.10419 (2025) 29. Wang, Y., Takanobu, R., Liang, Z., Mao, Y., Hu, Y., McAuley, J., Wu, X.: Mem-{\alpha}: Learning memory construction via reinforcement learning. arXiv preprint arXiv:2509.25911 (2025) 30. Wei, J., Karina, N., Chung, H.W., Jiao, Y.J., Papay, S., Glaese, A., Schulman, J., Fedus, W.: Measuring short-form factuality in large language models. arXiv preprint arXiv:2411.04368 (2024) 31. Wu, Y., Zhang, Y., Liang, S., Liu, Y.: Sgmem: Sentence graph memory for longterm conversational agents. arXiv preprint arXiv:2509.21212 (2025) 32. Xu, W., Mei, K., Gao, H., Tan, J., Liang, Z., Zhang, Y.: A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110 (2025) 33. Xu, Y., He, S., Chen, J., Wang, Z., Song, Y., Tong, H., Liu, G., Liu, K., Zhao, J.: Generate-on-graph: Treat llm as both agent and kg in incomplete knowledge graph question answering. arXiv preprint arXiv:2404.14741 (2024) 34. Xu, Y., Guo, X., Zeng, Z., Miao, C.: Softcot: Soft chain-of-thought for efficient reasoning with llms. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). p. 23336–23351 (2025) 35. Yan, S., Yang, X., Huang, Z., Nie, E., Ding, Z., Li, Z., Ma, X., Kersting, K., Pan, J.Z., Schütze, H., et al.: Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning. arXiv preprint arXiv:2508.19828 (2025) 36. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al.: Qwen3 technical report. arXiv preprint arXiv:2505.09388 (2025) 37. Yu, H., Chen, T., Feng, J., Chen, J., Dai, W., Yu, Q., Zhang, Y.Q., Ma, W.Y., Liu, J., Wang, M., et al.: Memagent: Reshaping long-context llm with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02259 (2025) 38. Zhang, F., Zhu, D., Ming, J., Jin, Y., Chai, D., Yang, L., Tian, H., Fan, Z., Chen, K.: Dh-rag: A dynamic historical context-powered retrieval-augmented generation method for multi-turn dialogue. arXiv preprint arXiv:2502.13847 (2025) 39. Zhang, G., Fu, M., Yan, S.: Memgen: Weaving generative latent memory for selfevolving agents. arXiv preprint arXiv:2509.24704 (2025) 40. Zhang, Q., Naradowsky, J., Miyao, Y.: Mind the gap between conversations for improved long-term dialogue generation. In: Findings of the Association for Computational Linguistics: EMNLP 2023. pp. 10735–10762 (2023) 41. Zhang, T., Yuan, J., Avestimehr, S.: Revisiting opro: The limitations of small-scale llms as optimizers. In: Findings of the Association for Computational Linguistics ACL 2024. pp. 1727–1735 (2024) 42. Zhang, Y., Lau, R.Y., David Xu, J., Rao, Y., Li, Y.: Business chatbots with deep learning technologies: state-of-the-art, taxonomies, and future research directions. Artificial Intelligence Review 57(5), 113 (2024) 43. Zhou, H., Chen, Y., Guo, S., Yan, X., Lee, K.H., Wang, Z., Lee, K.Y., Zhang, G., Shao, K., Yang, L., et al.: Memento: Fine-tuning llm agents without fine-tuning llms. arXiv preprint arXiv:2508.16153 (2025) 44. Zhou, K., Zhou, Y., Zhao, W.X., Wang, X., Wen, J.R.: Towards topic-guided conversational recommender system (2020) 45. Zhou, Z., Qu, A., Wu, Z., Kim, S., Prakash, A., Rus, D., Zhao, J., Low, B.K.H., Liang, P.P.: Mem1: Learning to synergize memory and reasoning for efficient longhorizon agents. arXiv preprint arXiv:2506.15841 (2025)