arXiv CSopen access

Loong: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selection

L OONG: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selection

1

Introduction

Large language models (LLMs, Zhao et al., 2026) have exhibited exceptional performance across various natural language processing tasks (Li et al., 2025c; Chen et al., 2025b; Guan et al., 2025). Recently, machine translation (MT) based on prompting LLMs has shown significant advancements in sentence and short-segment translation (Peng et al., 2023; He et al., 2024; Wang et al., 2024; He et al., 2025; Wang et al., 2025b). However, in longdocument translation scenarios—such as speech * Xuebo Liu is the corresponding author.

OONG

Sentence Segment

72

D ELTA

Context capacity exceeded

69

LLM-as-a-Judge

80 Average Score

arXiv:2605.30274v1 [cs.CL] 28 May 2026

Document-level translation remains one of the most challenging tasks for large language models, which are constrained by limited context windows that impede global cohesion, while simultaneously suffering from redundant contextual information that degrades translation quality. To address this, we propose a humanlike long document translation agent called L OONG, which leverages a 3E memory module (Essence-Exemplar-Entity) to store summaries, sentence pairs, and entity records as historical context. Instead of passively attending to all history, L OONG performs deep reasoning to adaptively identify the optimal context for translation guidance. L OONG optimizes its context policy through reinforcement learning, utilizing preference data derived from its own sampled observe-and-act reasoning trajectories. Empirical evaluations demonstrate that L OONG achieves substantial translation quality improvements in English ⇔ Chinese, German, and French directions, with average gains of up to 13.0 points across the three evaluation metrics. Furthermore, L OONG exhibits strong generalization across domains and robustness against contextual noise, while maintaining remarkable stability in ultra-long document translation. Our code is released at https:// github.com/YutongWang1216/LoongDocMT.

Average Score

Yutong Wang1 Xuebo Liu1 * Derek F. Wong2 Zhilin Li1 Rongqing Jiang1 Min Zhang3 Shimin Tao3 Daimeng Wei3 Min Zhang1 1 Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen 2 NLP2 CT Lab, Department of Computer and Information Science, University of Macau 3 Huawei Translation Services Center {wangyutong,lizhilin,jiangtongqing}@stu.hit.edu.cn {liuxuebo,zhangmin2021}@hit.edu.cn [email protected] {zhangmin186,taoshimin,weidaimeng}@huawei.com sCOMET Abstract 75 L Doc2Doc

70 60 0

100

200 300 Translated Document Lines

400

500

Figure 1: Cumulative average sCOMET and LLM-as-aJudge scores of L OONG and baseline methods on ultralong document translation (Chinese ⇒ Portuguese). While standard chunking methods (Sentence, Segment), full-history variants (Doc2Doc, Wang et al., 2023a), and unfiltered memory agents (D ELTA, Wang et al., 2025c) exhibit continuous degradation or even failure due to context length limits, L OONG successfully distinguishes useful information from retrieved memory to sustain stable, high-quality translations. See §4.4 for more experimental details.

transcripts, technical manuals, and literary textscritical issues arise, including the mismanagement of terminology and inconsistencies in stylistic elements due to their limited contextual capacity (Kim et al., 2019; Maruf et al., 2021). Document-level machine translation (DocMT) by LLMs has recently attracted increasing attention to address these challenges. Some studies exploit the extended context windows and long-text processing capabilities of LLMs to generate target documents directly (Wang et al., 2023a; Wu and Hu, 2023; Wu et al., 2024). Other approaches introduce long chain-of-thought (CoT) reasoning or complex multi-step inference procedures to further unlock the DocMT potential of LLMs (Dong et al., 2025; Hu et al., 2025; Ramos et al., 2025). Agentbased workflows (Wang et al., 2025d; Tian et al., 2025; Wang et al., 2026a,b) extract key contextual information to guide the DocMT process (Wang

et al., 2025c; Guo et al., 2025b; Li et al., 2025b; Liu et al., 2025a). While these methods improve contextual utilization, many of them treat contextual signals indiscriminately, often incorporating extraneous or redundant noise, which ultimately leads to suboptimal performance (Karpinska and Iyyer, 2023). To address these challenges, we introduce L OONG, a human-like long-document translation agent with observe-and-act adaptive context selection. Mimicking human experts’ cognitive processes, L OONG maintains a “3E” memory module (Essence-Exemplar-Entity) to store and retrieve segment summaries, bilingual sentence pairs, and entity records as contextual information. Simultaneously, it employs an observe-and-act reasoning mechanism to distinguish useful context from the retrieved pool, retaining beneficial signals while filtering out redundancy. We optimize this capability via reinforcement learning (RL) derived from sampled trajectories. Empirical results demonstrate that L OONG achieves remarkable improvements across diverse backbones (e.g., Qwen, Llama) and language pairs (e.g., English ⇔ Chinese, German, and French), with average gains of up to 13.0 points on sentence- and document-level metrics. Our main contributions are as follows: • We develop L OONG, a human-like DocMT agent that retrieves and adaptively identifies useful contextual information by deep reasoning for document translation. • We design a sequential exploration strategy to identify effective context via sampling observe-and-act trajectories, constructing preference data for agent optimization. • We propose an alignment-enforced translation algorithm for document-level methods that ensures the alignment between source and generated target documents. • As shown in Figure 1, L OONG achieves superior translation quality for ultra-long documents while exhibiting strong generalization across diverse models and domains, as well as robustness against noisy context.

2

Related Works

Document-level MT The technical approaches in DocMT can be broadly categorized into two

types: Document-to-Sentence (Doc2Sent) and Document-to-Document (Doc2Doc). Doc2Sent approaches focus on encoding neighboring context to extract discourse features and generate translations for individual sentences. Advanced neural network designs (Wang et al., 2017; Tan et al., 2021; Lyu et al., 2021) have enhanced the modeling of context, enabling more effective sentence-level decoding. However, Doc2Sent methods still suffer from limited utilization of the target-side context (Sun et al., 2022; Li et al., 2023). In contrast, Doc2Doc methods translate documents segment-by-segment. Traditional approaches optimize attention allocation in transformers (Bao et al., 2021; Li et al., 2023), while LLMbased methods leverage prompting or instruction tuning for high-quality translation (Wang et al., 2023a; Wu et al., 2024; Liu et al., 2025a). However, these methods often fail to maintain strict sentence-level alignment with the source, complicating evaluation and limiting their application in rigorous scenarios (Dong et al., 2025; Wang et al., 2025c). To address this, L OONG proposes an LLMbased Doc2Doc agentic workflow, upon which we further optimize the context selection strategies. We harness the contextual capabilities of LLMs and introduce a specialized algorithm to ensure strict sentence-level alignment between the source and target documents, which combines the strengths of both Doc2Sent and Doc2Doc methods. Deep Reasoning of MT-LLMs In recent years, deep reasoning LLMs such as OpenAI o1 (Jaech et al., 2024) and DeepSeek-R1 (Guo et al., 2025a) have achieved impressive results on challenging problems such as code generation and question answering (Zhang et al., 2024; Guan et al., 2025). To further enhance the reasoning and tool-use capabilities of these models, numerous studies focus on sampling their CoT trajectories and providing stepwise rewards to facilitate fine-grained RL (Pang et al., 2024; Li et al., 2025a; Chen et al., 2025c). Recent research successfully extends deep reasoning LLMs to MT, demonstrating substantial performance gains (Liu et al., 2025b; Chen et al., 2025a; Li et al., 2025b) and tackling open-ended tasks like slang translation (Zhao et al., 2024). Subsequent works further optimize LLM reasoning for MT, employing RL to develop a “think-andtranslate” paradigm (He et al., 2025; Wang et al., 2025b) or targeted training for literary nuances (Wang et al., 2025a). However, few of these ad-

§3.1 Human-like Memory Module Source Segment 𝒔𝝉+𝟏

Split Source Document

§3.2 Observe-and-Act Data Construction Observation 𝑶𝟏

Essence Candidates

Embeddings

Essence of s

Essence of s

Selection

Translation 𝒕෤𝟏

···

Essence 1

Helpful



Essence 2

Unrelated



···

Segment Summaries

Reasoning

×M Translation Samples × N

Action 𝓐𝟏.𝟏 Candidates

Essence

Source Segment 𝒔𝝉+𝟏

Action Samples × M

Essence 3

Helpful



Translation 𝒕෤𝑵

Translation 𝒕෤𝟐 Average Score

Essence of s

Observation 𝑶𝟐

History 𝑶𝟏 → 𝑨𝟏.𝟏

Action Samples × M

Exemplar Sentence Pairs

Exemplar Candidates

×M

Src Embeddings

Translation Samples × N

Candidates

Reasoning

···

Exemplar 1

Unrelated



Src n

Tgt n

Exemplar 2

Helpful



···

Exemplar 3

Helpful



Translation 𝒕෤𝑵

···

Tgt 1 Tgt 2

···

Action 𝑨𝟐.𝟏

Src 1 Src 2

Entity Type

Information Fields

Character

Role Description Relation…

Organization

Type Purpose Members…

···

···

Object

Type Appearance Owner…

Entity Candidates

Translation 𝒕෤𝟏 Translation 𝒕෤𝟐

Reasoning

×M Translation Samples × N

Action 𝑨𝟑.𝟏 Candidates

Average Score

History 𝑶𝟏 → 𝑨𝟏.𝟏 → 𝑶𝟐 → 𝑨𝟐.𝟐

Action Samples × M

Selection

Entity 1

Unrelated



Entity 2

Helpful



Entity 3

Unrelated



Translation 𝒕෤𝑵

O1 ↓ 𝑨𝟏.𝟏 𝒔 𝝉+𝟏 ↓ 𝑪 𝟏.𝟏 𝑶𝟐 ↓ ↓ 𝒕෤𝟐 𝑨𝟐.𝟐 / / 𝒕෤𝟐 𝑨𝟐.𝟏

𝑶𝟏 ↓ 𝑨𝟏.𝟏 ↓ 𝑶𝟐 ↓ 𝒔𝝉+𝟏 A2.2 𝑪𝟏.𝟏 ↓ ↓ 𝒕෤𝟐 𝑶 𝟑

/ ↓ 𝒕෤𝟐 𝑨𝟑.𝟏

Translation 𝒕෤𝟏

···

Update

Target Segment 𝒕𝝉+𝟏

Observation 𝓞𝟑

Selection

𝑶𝟏 𝒔𝝉+𝟏 ↓ 𝑪𝟏.𝟏 𝑨𝟏.𝟏 ↓ / 𝒕෤𝟐 𝑨𝟏.𝟑 / 𝒕෤𝟐

Translation 𝒕෤𝟐

/

Average Score

𝑨𝟑.𝟐

Figure 2: The human-like memory module and observe-and-act dateset construction procedure of L OONG.

vancements extend to DocMT. To overcome the longstanding bottlenecks of document-level issues such as cohesion and coherence, L OONG bridges this gap by introducing an RL-optimized “observeand-act” reasoning process that adaptively filters and integrates multi-granularity context.

3

Methodology

We introduce L OONG, a human-like long document translation agent with observe-and-act adaptive context selection. Drawing inspiration from professional human translators, we design a multigranularity contextual memory module that records and retrieves historical information during translation (§3.1). To filter out redundant information from the retrieved, we implement a deep reasoning mechanism to identify relevant items, and explore optimal context strategies by sampling diverse reasoning trajectories (§3.2). Based on this, we apply RL to optimize the agent’s policy, and deploy it alongside an alignment-enforced translation algorithm, yielding high-quality and strictly aligned target documents (§3.3). 3.1

Human-like Translation Memory Module

Human translators typically follow an integrated reading–translation workflow when handling long documents: they translate the text segment by seg-

ment while continuously memorizing critical contextual information to maintain coherence and cohesion. To emulate this cognitive process, we partition the source document into a sequence of L segments {s1 , . . . , sL }, where each segment contains l sentences. Similar to Wang et al. (2025c), we design a multi-granularity memory architecture, designated as the 3E Module, which is updated after completing each segment. As shown in the left part of Figure 2, the module synergizes three distinct components to capture information at different granularities: Essence (Global), Exemplars (Pattern), and Entities (Specific). Essence To capture semantic progression, we (τ −1) maintain a memory bank Es storing LLMgenerated summaries of the preceding τ − 1 segments. Before translating segment sτ , we use a sentence encoder to extract embeddings and retrieve (τ −1) a candidate subset Ẽs containing the top-Ks summaries with the highest cosine similarity to sτ for the agent’s further selection. Exemplars To maintain stylistic consistency, an (τ −1) exemplar memory Ex stores all previously processed source-target sentence pairs as in-context demonstrations. Similar to Essence, we retrieve a (τ −1) subset Ẽx of the top-Kx exemplars based on the source-sentence embedding similarity to sτ , pro-

viding stylistic references to guide the translation. Entities To enforce terminological precision, we (τ −1) maintain a structured knowledge base En , where each entry (esrc , etgt , I) records an entity’s bilingual names and domain-specific attributes. After translating each segment, the agent extracts entities, classifies them, and updates their attributes I (see Appendix A.1 for more details). For sτ , we retrieve all mentioned entities and generate their context-aware descriptions di conditioned on (τ ) sτ , forming the candidate knowledge set: Ẽn = tgt src src {(ei , ei , di ) | ei ∈ sτ }. 3.2

Observe-and-Act Data Construction

To enable the agent to accurately identify and effectively utilize beneficial context for translation, we focus on exploring diverse context reasoning strategies and collecting preference data. However, performing a joint search over all memory types results in an intractable combinatorial space. Consequently, we decompose the exploration process into three sequential steps, where each step exclusively investigates the utility of one specific context type. This sequential design is driven by two critical advantages. Functionally, it transforms the Q search K space from a combinatorial P K product (O( 2 )) to an additive sum (O( 2 )), rendering the trajectory exploration feasible. Strategically, it allows us to obtain fine-grained feedback on the specific contribution of each context type, facilitating precise adjustments to the agent’s strategy. Observe-and-Act Reasoning Process Based on this decoupled framework, we aim to construct preference datasets targeting two complementary capabilities: context selection (determining what to select) and context utilization (determining how to translate given the context). As shown in the right part of Figure 2, we implement this framework via a multi-step observe-and-act workflow. For a specific segment sτ , we organize the candidate contextual information retrieved from the previous τ − 1 segments (§3.1) into an ordered sequence (τ −1) (τ −1) (τ −1) E = ⟨Ẽs , Ẽx , Ẽn ⟩. The agent then executes a 3-step reasoning ⟨O1 , A1 , O2 , A2 , O3 , A3 ⟩, where the first step (O1 , A1 ) is dedicated to observing and selecting from the Essence candidates, followed sequentially by the processing of Exemplars (Step 2) and Entities (Step 3). At step k, the observation Ok = {Hk−1 , E[k]} encapsulates the history of all previous reasoning steps Hk−1 and

the specific set of retrieved candidates E[k]. The corresponding action Ak is structured as a composite output ⟨rk , Ck ⟩: a reasoning thought rk that explicitly analyzes the semantic relevance of each item in E[k] with respect to sτ , followed by the final selection Ck , a subset of items deemed most beneficial for the translation process. Context Selection Strategy Sampling To explore diverse context selection strategies, we perform parallel sampling on Ak for M times:  M Gk = Aik i=1 , where Aik ∼ πθ (· | Ok ).

(1)

Based on each candidate action, the agent translates sτ using the selected context items as additional guidance. We sample N translations given each candidate action Aik = ⟨rki , Cki ⟩:  N Tki = t̂j j=1 , t̂j ∼ πθ (· | sτ , Cki ).

(2)

We calculate the average quality score of Tki as the utility proxy for Aik : U (Aik ) =

1 X µ(sτ , t̂, t∗ ). N i

(3)

t̂∈Tk

where µ(·) denotes the evaluation metric and t∗ is the reference translation. Accordingly, we identify the action yielding the highest utility as the preferred strategy A+ k , and the lowest as the dispreferred A− : k − A+ k = argmax U (A), Ak = argmin U (A). (4) A∈Gk

A∈Gk

− The triplet (Ok , A+ k , Ak ) constitutes a sample in the context selection strategy dataset Dsel .

Context Utilization Strategy Sampling To further explore the diverse context utilization strategies to enhance the translation process, we select the highest- and lowest-scoring translations in Tki : t+ = arg max µ(sτ , t̂, t∗ ),

(5)

t̂∈Tki

t− = arg min µ(sτ , t̂, t∗ ).

(6)

t̂∈Tki

We then combine the input ⟨sτ , Cki ⟩ with this pair to form a triplet (⟨sτ , Cki ⟩, t+ , t− ) in the context utilization strategy dataset Dutil . The optimal action A+ k are incorporated into the observation for the next step Ok+1 , and the optimal

translation t+ derived in the last step is established as the final translation for sτ , which is then used to update the memory module following the procedure in §3.1. Finally, we merge the datasets into the overall training set D = Dsel ∪ Dutil . The pseudo code of the preference data construction is shown in Appendix A.2. 3.3

Dataset

980 637

Out-of-Domain News Commentary (Unseen Langs) IWSLT2017 (Cettolo et al., 2017)3 GuofengV1 (Wang et al., 2023b)4 Journey to the West

980 1,939 1,445 51,854

Agent Tuning & Inference

where θ denotes the trainable parameters of the original LLM. Next, we apply direct preference optimization (DPO) to further refine the LLM’s context-selection and translation strategies using the full preference dataset. The DPO objective is (8)

where θ and θSFT denote the parameters being updated during DPO and those obtained after SFT, respectively. The advantage difference ∆(θ, θSFT ) between the preferred and dispreferred outputs is defined as follows: ∆ = log

Domain News News, etc. News TED Talks Webnovels Ultra-Long Novel

Table 1: Summary of the evaluation datasets.

Preference Tuning We utilize the constructed preference data to optimize the agent’s context strategies. To equip the LLM with the initial ability to perform deep reasoning and output results, we first perform a cold-start supervised finetuning (SFT), using only the preferred instances from D. The stage maximizes the following objective: 
 E(x,y+ )∼D log πθ y + | x , (7)

E(x,y+ ,y− )∼D [log σ (β∆(θ, θSFT ))] ,

Words / Doc.

In-Domain News Commentary V18.11 WMT24++ (Deutsch et al., 2025)2

πθ (y + | x) πθ (y − | x) −log . (9) πθSFT (y + | x) πθSFT (y − | x)

To reduce GPU memory consumption and improve training efficiency, we employ Low-Rank Adaptation (LoRA, Hu et al., 2022) during DPO. Alignment-Enforced Inference Procedure During inference, the trained agent samples only once for the selection of each type of contextual information. No intermediate translations or quality assessments are generated at this stage. After all three reasoning steps are executed, the agent produces a final translation of the current segment guided by the selected contextual items, and this translation is subsequently used to update the memory module following the same process as §3.1. Crucially, to ensure this generation maintains strict sentence-level alignment with the source segments for accurate evaluation and memory updates, we employ a recursive alignment enforcement algorithm. For a segment ui:j to be translated, we

inject sentence indices and separators, which the LLM is told to preserve. Once the output failed to align with the source sentences, the segment is recursively split into halves (ui:k and uk+1:j , where k = i − 1 + ⌊ j−i+1 2 ⌋) until alignment is achieved or the segment reduces to a single sentence: ( LLM(ui:j ), if aligned or i = j T(ui:j ) = T(ui:k ) ⊕ T(pk+1:j ), otherwise (10) This guarantees an aligned translation. Algorithm details are shown in Appendix A.2. For a complete case of L OONG, please refer to Appendix A.3

4

Experiments

4.1

Setup

Datasets To align with our focus on longdocument translation, we construct the training set by selecting documents comprising more than 50 lines from the News Commentary V18.15 dataset. From this filtered pool, we randomly sample approximately 500 documents for each language pair of English (En) ⇔ Chinese (Zh), German (De), and French (Fr). For evaluation in both the main experiments and subsequent analyses, we assess our approach across a diverse set of in-domain and out-of-domain benchmarks. Table 1 summarizes the datasets utilized for evaluation, detailing their average document lengths and respective domains. The in-domain News Commentary test set is drawn exclusively from the held-out portion of the data not utilized during training. Similar to the training data, we select longer subsets for both the News Commentary and WMT24++ test sets. Detailed information of these datasets and data preprocess1

https://data.statmt.org/news-commentary/v18.1 https://huggingface.co/datasets/google/wmt24pp 3 https://huggingface.co/datasets/IWSLT/iwslt2017 4 https://github.com/longyuewangdcu/GuoFengWebnovel 5 https://data.statmt.org/news-commentary/v18.1 2

News Commentary V18.1 Model System

En ⇒ Xx

WMT24++

Xx ⇒ En

En ⇒ Xx

Xx ⇒ En

Qwen2.5-7B

Sentence Segment Doc2Doc D ELTA L OONG

84.4 84.9 84.5 85.1 85.8

79.1 79.6 79.1 79.8 80.5

56.2 64.0 63.1 63.4 66.7

73.2 76.2 75.6 76.1 77.7

86.7 86.5 85.7 87.4 88.0

82.1 80.2 78.3 83.1 83.7

78.4 79.6 74.3 87.6 90.7

82.4 82.1 79.5 86.0 87.5

74.3 79.6 80.1 80.0 81.1

71.1 77.0 77.4 77.5 78.7

44.9 62.7 64.9 65.2 65.0

63.4 73.1 74.1 74.3 74.9

77.8 82.7 83.0 83.9 84.4

74.6 78.2 78.8 81.4 82.1

57.3 73.8 74.8 80.3 82.0

69.9 78.2 78.8 81.9 82.9

Qwen3-8B

Sentence Segment Doc2Doc D ELTA L OONG

86.9 86.5 86.4 86.7 87.3

81.6 80.9 80.7 81.4 82.1

78.9 78.3 79.3 78.8 81.7

82.5 81.9 82.1 82.3 83.7

87.6 87.2 87.6 87.7 88.1

83.1 81.8 82.6 83.4 83.7

87.8 82.6 86.2 89.1 90.2

86.2 83.9 85.4 86.7 87.3

81.9 81.1 82.0 81.8 82.8

79.4 77.4 79.0 79.5 80.4

72.5 70.9 74.4 75.1 76.8

77.9 76.5 78.5 78.8 80.0

83.5 83.5 83.8 84.2 84.8

80.9 79.3 80.4 81.9 82.5

75.8 74.2 78.6 81.1 83.5

80.0 79.0 80.9 82.4 83.6

Qwen3-14B

Sentence Segment Doc2Doc D ELTA L OONG

87.4 87.3 87.4 87.5 87.7

82.2 82.2 82.2 82.3 82.6

84.4 83.5 84.8 85.0 85.6

84.7 84.4 84.8 84.9 85.3

87.9 88.0 88.0 88.0 88.2

83.5 83.6 83.8 83.7 84.0

88.2 89.1 90.0 90.4 91.0

86.5 86.9 87.3 87.4 87.7

82.8 82.9 83.2 82.9 83.4

80.5 79.8 80.5 80.5 80.8

79.2 78.8 80.5 80.7 80.5

80.8 80.5 81.4 81.4 81.6

84.3 84.8 84.9 84.7 85.0

81.8 82.5 82.6 82.4 82.7

80.4 82.6 85.1 84.3 85.3

82.2 83.3 84.2 83.8 84.4

Llama3.1-8B

sCMT dCMT LLM Avg sCMT dCMT LLM Avg sCMT dCMT LLM Avg sCMT dCMT LLM Avg

Sentence Segment Doc2Doc D ELTA L OONG

84.8 84.3 84.6 85.5 86.3

78.6 78.3 79.1 79.8 81.3

65.4 61.1 64.8 71.2 73.0

76.3 74.6 76.2 78.8 80.2

86.8 86.9 86.9 86.8 87.8

82.3 82.4 82.2 82.3 83.5

81.9 82.0 82.2 84.8 87.6

83.7 83.8 83.7 84.7 86.3

76.7 77.9 78.9 78.5 80.4

73.7 73.9 77.0 76.3 77.9

54.7 55.7 59.9 62.1 64.4

68.4 69.2 71.9 72.3 74.3

80.5 83.6 83.6 79.7 84.1

77.7 80.8 81.0 76.7 81.7

69.6 74.6 76.7 75.2 82.3

75.9 79.6 80.4 77.2 82.7

Table 2: Main results of L OONG compared to the baseline methods. “sCMT” and “dCMT” denote sCOMET and dCOMET, respectively. The best results are marked in bold font. All models are instruction-tuned versions.

ing methods (i.e., length filtering and sampling) is provided in Appendix A.4. Metrics We adopt sCOMET (Rei et al., 2022)6 as the evaluation metric µ during data construction in §3.2, and employ sCOMET and dCOMET (Vernikos et al., 2022)7 as the sentence- and document-level translation quality metrics, respectively, both instantiated with the wmt22-comet-da model. Additionally, we use GPT-4.1 to evaluate the translated documents across five dimensions: General Quality, Cohesion, Coherence, Style Consistency, and Terminology Consistency. Each of these five dimensions is scored on a scale from 1 to 100, with the final meta-score being the average of the individual scores. The metric design prompt is provided in Appendix A.5. Models We employ all-distilroberta-v18 as the sentence encoder, and adopt four open-source LLMs as the backbone models: Qwen2.5-7BInstruct (Team, 2024)9 , Qwen3-8B (Team, 2025), Qwen3-14B, and Llama3.1-8B-Instruct (Grattafiori 6

https://github.com/Unbabel/COMET https://github.com/amazon-science/doc-mt-metrics 8 https://huggingface.co/sentence-transformers/alldistilroberta-v1 9 https://huggingface.co/Qwen/Qwen2.5-7B-Instruct 7

et al., 2024)10 . For each model, we train two variants: an En ⇒ Xx model using training data from En ⇒ Zh, De, Fr directions, and an Xx ⇒ En model using data from the opposite directions. Each model is trained on 4 A800 GPUs, consuming around 20 hours for the whole training process. Hyperparameters We set the segment length l to 5 sentences. In the memory module, both the summary and exemplar retrieval sizes, Ks and Kx , are set to 4. For preference data construction, we set the action sampling count M to 7, and the translation sampling count N to 5. During SFT, we train the models for 1 epoch with a batch size of 64 and a learning rate of 1e-5, using DeepSpeed ZeROStage 3 optimization (Rajbhandari et al., 2020). During DPO, we train the models for 1 epoch with a batch size of 32 and a learning rate of 5e-6. We set β = 0.1 and LoRA rank to 8. The maximum text length for both stages is 2,560. For both preference data generation and test-time inference, we use a temperature of 0.7 and a top-p value of 1.0. Baselines We compare L OONG against the following baseline approaches: a) Sentence: A vanilla sentence-level baseline that translates documents in 10

https://huggingface.co/meta-llama/Llama-3.1-8BInstruct

Method

sCMT

dCMT

LLM

Avg

1 2 3 4 5 6 7

L OONG w/o Context w/o Translation w/o Tuning w/o Essence w/o Exemplar w/o Entity

86.4 85.9 79.3 84.6 86.2 86.3 86.2

81.3 80.6 65.4 79.0 81.1 81.1 81.0

73.0 65.6 46.1 62.5 69.7 70.6 72.0

80.2 77.4 63.6 75.4 79.0 79.3 79.7

Table 3: Results of the ablation study. LLama3.1-8BInstruct is adopted as the backbone model. “sCMT” and “dCMT” denote sCOMET and dCOMET, respectively. We adopt LLama3.1-8B-Instruct trained in En ⇒ Xx directions for the ablation experiments.

an isolated, sentence-by-sentence manner. b) Segment: A simple document-level baseline that translates documents segment by segment without leveraging across-segment context. c) Doc2Doc (Wang et al., 2023a): A DocMT approach in which the LLM translates one segment per conversation turn while storing all previously translated segments in the conversation history. d) D ELTA (Wang et al., 2025c): A DocMT agent equipped with a multi-granularity memory module that stores and retrieves key contextual information to support context-aware translation without further filtering. 4.2

Main Results

As shown in Table 2, L OONG consistently achieves state-of-the-art overall performance across both the News Commentary V18.1 and WMT24++ test sets. Regardless of the backbone model employed or the translation direction, our method attains the highest average scores. Detailed results for individual language pairs are provided in Appendix A.6. Sentence-Level Quality Improvement Incorporating a broad context risks introducing noise that degrades sentence-level accuracy. For example, using the Qwen3-8B backbone on News Commentary (En ⇒ Xx), document-level baselines (D ELTA and Doc2Doc) yield sCOMET scores (86.7 and 86.4) that actually fall behind the naive Sentence baseline (86.9). Overcoming this, L OONG effectively filters irrelevant memory, achieving the highest score of 87.3. L OONG’s consistent top performance across all datasets confirms the robust efficacy of our deep reasoning mechanism across varying benchmarks. Document-Level Quality Improvement L OONG consistently dominates context-aware dCOMET and multi-dimensional LLM metrics. Notably, using Llama3.1-8B-Instruct on WMT24++ (En ⇒ Xx), it attains a dCOMET

80

Score

Id

Ablation Study Results L OONG

w/o Essence

w/o Exemplar

w/o Entity

75 70 65

Gen. Quality Cohesion

Coherence Style Cons. Term. Cons.

Figure 3: Performance changes across individual LLM evaluation dimensions after removing each memory module component.

of 77.9, surpassing the Sentence baseline by 4.2 points. More impressively, in the LLM-as-a-Judge evaluation assessing overall discourse fluency and consistency, L OONG achieves an 82.3 in Xx ⇒ En, outperforming the state-of-the-art D ELTA (75.2) by a remarkable 7.1 points. This substantial gap highlights our agent’s superior capability in modeling long-range dependencies and resolving discourse coherence issues. Robustness Across Model Families and Sizes L OONG consistently performs well across diverse backbones (Qwen2.5/3, Llama3.1) and sizes ranging from 7B to 14B. Notably, it achieves a peak average LLM score of 91.0 on Qwen3-14B. This wide-ranging consistency validates our agent as a robust, model-agnostic framework regardless of the underlying LLM’s architecture or scale. 4.3

Ablation Study

Training Configuration Ablation To disentangle the contribution of the optimized context strategy from simple exposure to translation targets, we train a variant exclusively on translation data, bypassing the observe-and-act reasoning process (Line 2 in Table 3). The resulting performance drop confirms that translation supervision alone is insufficient, and our gains heavily rely on the learned context strategies. We further evaluate variants trained solely on context selection preferences (Line 3) or without any tuning (Line 4). Consistent degradation across all these settings demonstrates that the synergy of context selection and utilization data is indispensable for the agent to master effective context strategies for high-quality translation. Memory Component Ablation To validate the necessity of each memory type, we individually exclude Essence, Exemplars, and Entities information from the training trajectories. Results in Lines 5–7 of Table 3 and Figure 3 reveal consistent degradation upon removing any component, indicating their collective importance. Notably, eliminating

sCOMET

86.5

L OONG

D ELTA

Doc2Doc

80.5

85.5

80.0

85.0

79.5

84.5

79.0

84.0

78.5

0

30 40 50 Pseudo Context Length

dCOMET

81.0

86.0

LLM

68

sCOMET

Es-En

It-En

dCOMET

Es-En

It-En

LLM

Es-En

66 64

Ja-En

62 0

30 40 50 Pseudo Context Length

60

0

30 40 50 Pseudo Context Length

Figure 4: Translation performance on test sets augmented with pseudo-context. A pseudo-context length of 0 indicates the original results without disturbance.

Essence causes the most severe drop. We hypothesize that global summaries provide critical content and genre signals, making them paramount for maintaining coherence across long documents. 4.4

It-En

85

86

87

88

89

Sentence Segment Doc2Doc D ELTA L OONG

Ru-En

90

91

92

Cs-En

Pt-En

Ja-En

76

78

80

Ru-En

82

84

86

88

Cs-En

Pt-En

Ja-En

75

80

90

85

95

Cs-En

Ru-En

Pt-En

Figure 5: Test results on the unseen language pairs. System

Guofeng (Literary)

IWSLT (Speech)

sCMT

dCMT

LLM

sCMT

dCMT

LLM

73.2 74.6 74.9 74.2 76.2

69.8 71.6 71.5 70.5 73.1

52.0 53.3 60.8 63.0 67.3

83.3 83.7 83.8 83.4 84.7

77.9 78.7 79.0 78.4 79.8

68.6 69.0 72.5 71.0 76.0

Sentence Segment Doc2Doc D ELTA L OONG

Table 4: Test results on the OOD test sets.

Analysis

Robustness on Context To evaluate L OONG’s robustness against context noises, we inject distracting pseudo-context (30-50 sentences sampled from other documents) into the source. We compare our results achieved by Qwen2.5 against the selectionfree baselines in En ⇒ Xx directions. As shown in Figure 4, two baselines suffer consistent degradation simultaneously on sCOMET, dCOMET, and LLM scores as noise increases, whereas L OONG remains largely stable, indicating its robustness against context disturbance. Interestingly, L OONG sometimes achieves marginal gains, likely by selectively exploiting domain signals within the pseudocontext while successfully filtering out noise. Transferability between Languages We assess the cross-lingual transferability of L OONG by evaluating its translation performance of the XX ⇒ En version with Qwen3-8B on the unseen Czech (Cs), Spanish (Es), Italian (It), Japanese (Ja), Russian (Ru), and Portuguese (Pt) ⇒ English test sets sampled from News Commentary. As shown in Figure 5, L OONG still achieves consistent performance improvements on all these language pairs, suggesting that the context selection and utilization strategy learned in some languages can be generalized to other language pairs, rendering L OONG applicable to languages with scarce training data. Generalization across Domains To explore the generalization capability of L OONG on out-ofdomain (OOD) data, we evaluate the En ⇒ Xx model with Llama3.1-8B-Instruct as backbone on the GuofengV1 and IWSLT2017 datasets. As reported in Table 4, L OONG consistently outperforms all baselines in the unseen literary and speech domains. These findings suggest that the opti-

mized context selection and utilization strategies are domain-agnostic rather than overfitting to the training data. This enables L OONG to be deployed directly in diverse domains without additional tuning, underscoring its exceptional robustness and cross-domain generalization potential. Stability on Long Document Translation To evaluate L OONG’s adaptability to ultra-long texts, we use the En ⇒ Xx Qwen3-8B to translate the first 12 chapters of the Chinese classical novel Journey to the West into Portuguese. Considering the length, we set the retrieval sizes for the Essence and Exemplar components, Ksum and Kx , to 8 and 6, respectively. As shown in Figure 1, Doc2Doc’s accumulated history rapidly exceeds the model’s context limit by Lines 156–160, causing translation failure. In contrast, L OONG’s structured memory and selective retrieval significantly improve context utilization efficiency. Consequently, it seamlessly completes the entire document without interruption and consistently yields higher-quality translations than all baselines, demonstrating exceptional robustness in ultra-long scenarios.

Conclusion In this paper, we introduce L OONG, a human-like long-document translation agent with reasoningdriven adaptive context selection. By leveraging a “3E” multi-granularity memory model, the agent captures and retrieves key contextual information in real time during sequential segment processing. To mitigate interference from redundancy, L OONG employs an observe-and-act reasoning process to dynamically filter context candidates. These strategies are further refined via RL based on sampled

reasoning trajectories. Empirical results demonstrate that L OONG achieves superior document translation quality across diverse language pairs and backbone models, and exhibits strong generalization ability to unseen languages and domains.

Limitations First, we currently adopt a fixed segmentation strategy (ten sentences per segment). While effective, this static heuristic lacks the flexibility to align with natural discourse boundaries, which may limit the model’s ability to maintain optimal coherence. Future work could incorporate dynamic segmentation mechanisms to better capture discourse structures. Second, the proposed agentic workflow, driven by deep reasoning, introduces additional computational overhead compared to standard one-pass generation. We plan to address this efficiency bottleneck in future iterations through optimization techniques such as distillation or speculative decoding. Finally, while we utilize COMET as the reward model for reinforcement learning, it may be an imperfect proxy for human preference in document-level translation tasks. We look forward to adopting more robust and nuanced evaluation metrics as they become available to better align the agent’s optimization with human judgments.

References Guangsheng Bao, Yue Zhang, Zhiyang Teng, Boxing Chen, and Weihua Luo. 2021. G-transformer for document-level machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3442–3455, Online. Association for Computational Linguistics. Mauro Cettolo, Marcello Federico, Luisa Bentivogli, Jan Niehues, Sebastian Stüker, Katsuhito Sudoh, Koichiro Yoshino, and Christian Federmann. 2017. Overview of the IWSLT 2017 evaluation campaign. In Proceedings of the 14th International Conference on Spoken Language Translation, pages 2–14, Tokyo, Japan. International Workshop on Spoken Language Translation.

reasoning large language models. ArXiv preprint, abs/2503.09567. Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, and Maosong Sun. 2025c. Optima: Optimizing effectiveness and efficiency for llm-based multi-agent system. In Findings of the Association for Computational Linguistics: ACL 2025, pages 11534–11557. Daniel Deutsch, Eleftheria Briakou, Isaac Caswell, Mara Finkelstein, Rebecca Galor, Juraj Juraska, Geza Kovacs, Alison Lui, Ricardo Rei, Jason Riesa, Shruti Rijhwani, Parker Riley, Elizabeth Salesky, Firas Trabelsi, Stephanie Winkler, Biao Zhang, and Markus Freitag. 2025. WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects. Preprint, arXiv:2502.12404. Yichen Dong, Xinglin Lyu, Junhui Li, Daimeng Wei, Min Zhang, Shimin Tao, and Hao Yang. 2025. Two intermediate translations are better than one: Finetuning LLMs for document-level translation refinement. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14917–14933, Vienna, Austria. Association for Computational Linguistics. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AlDahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Xinyan Guan, Jiali Zeng, Fandong Meng, Chunlei Xin, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, and Jie Zhou. 2025. Deeprag: Thinking to retrieve step by step for large language models. ArXiv preprint, abs/2502.01142. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025a. Deepseekr1: Incentivizing reasoning capability in llms via reinforcement learning. ArXiv preprint, abs/2501.12948. Jiaxin Guo, Yuanchang Luo, Daimeng Wei, Ling Zhang, Zongyao Li, Hengchao Shang, Zhiqiang Rao, Shaojun Li, Jinlong Yang, Zhanglin Wu, and 1 others. 2025b. Doc-guided sent2sent++: A sent2sent++ agent with doc-guided memory for document-level machine translation. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 228–240. Springer.

Andong Chen, Yuchen Song, Wenxin Zhu, Kehai Chen, Muyun Yang, Tiejun Zhao, and Min zhang. 2025a. Evaluating o1-like llms: Unlocking reasoning for translation through comprehensive analysis.

Minggui He, Yilun Liu, Shimin Tao, Yuanchang Luo, Hongyong Zeng, Chang Su, Li Zhang, Hongxia Ma, Daimeng Wei, Weibin Meng, and 1 others. 2025. R1-t1: Fully incentivizing translation capability in llms via reasoning learning. ArXiv preprint, abs/2502.19735.

Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. 2025b. Towards reasoning era: A survey of long chain-of-thought for

Zhiwei He, Tian Liang, Wenxiang Jiao, Zhuosheng Zhang, Yujiu Yang, Rui Wang, Zhaopeng Tu, Shuming Shi, and Xing Wang. 2024. Exploring humanlike translation strategy with large language models.

Transactions of the Association for Computational Linguistics, 12:229–246. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan AllenZhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations. Hanxu Hu, Jannis Vamvas, and Rico Sennrich. 2025. Source-primed multi-turn conversation helps large language models translate documents. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23702–23712, Suzhou, China. Association for Computational Linguistics. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. Openai o1 system card. ArXiv preprint, abs/2412.16720. Marzena Karpinska and Mohit Iyyer. 2023. Large language models effectively leverage document-level context for literary translation, but critical errors persist. In Proceedings of the Eighth Conference on Machine Translation, pages 419–451, Singapore. Association for Computational Linguistics. Yunsu Kim, Duc Thanh Tran, and Hermann Ney. 2019. When and why is document-level context useful in neural machine translation? In Proceedings of the Fourth Workshop on Discourse in Machine Translation (DiscoMT 2019), pages 24–34, Hong Kong, China. Association for Computational Linguistics. Pengxiang Li, Zhi Gao, Bofei Zhang, Yapeng Mi, Xiaojian Ma, Chenrui Shi, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, and Qing Li. 2025a. Iterative tool usage exploration for multimodal agents via stepwise preference tuning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. Weiya Li, Junjie Chen, Bei Li, Boyang Liu, Zichen Wen, Nuanqiao Shan, Xiaoqian Liu, Anping Liu, Huajie Liu, Youyan Wang, and 1 others. 2025b. Tactic: Translation agents with cognitive-theoretic interactive collaboration. ArXiv preprint, abs/2506.08403. Yachao Li, Junhui Li, Jing Jiang, Shimin Tao, Hao Yang, and Min Zhang. 2023. P-transformer: Towards better document-to-document neural machine translation. IEEE/ACM Transactions on Audio, Speech, and Language Processing. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, and 1 others. 2025c. From system 1 to system 2: A survey of reasoning large language models. ArXiv preprint, abs/2502.17419.

Bin Liu, Xinglin Lyu, Junhui Li, Daimeng Wei, Min Zhang, Shimin Tao, and Hao Yang. 2025a. Improving llm-based document-level machine translation with multi-knowledge fusion. ArXiv preprint, abs/2503.12152. Sinuo Liu, Chenyang Lyu, Minghao Wu, Longyue Wang, Weihua Luo, Kaifu Zhang, and Zifu Shang. 2025b. New trends for modern machine translation with large reasoning models. Xinglin Lyu, Junhui Li, Zhengxian Gong, and Min Zhang. 2021. Encouraging lexical translation consistency for document-level neural machine translation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3265–3277, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. Sameen Maruf, Fahimeh Saleh, and Gholamreza Haffari. 2021. A survey on document-level neural machine translation: Methods and evaluation. ACM Computing Surveys (CSUR), 54(2):1–36. Richard Yuanzhe Pang, Weizhe Yuan, He He, Kyunghyun Cho, Sainbayar Sukhbaatar, and Jason Weston. 2024. Iterative reasoning preference optimization. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024. Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2023. Towards making the most of ChatGPT for machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5622–5633, Singapore. Association for Computational Linguistics. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: international conference for high performance computing, networking, storage and analysis, pages 1–16. IEEE. Miguel Moura Ramos, Patrick Fernandes, Sweta Agrawal, and André FT Martins. 2025. Multilingual contextualization of large language models for document-level machine translation. ArXiv preprint, abs/2504.12140. Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 578–585, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. Zewei Sun, Mingxuan Wang, Hao Zhou, Chengqi Zhao, Shujian Huang, Jiajun Chen, and Lei Li. 2022. Rethinking document-level neural machine translation. In Findings of the Association for Computational

Linguistics: ACL 2022, pages 3537–3548, Dublin, Ireland. Association for Computational Linguistics. Xin Tan, Longyin Zhang, and Guodong Zhou. 2021. Coupling context modeling with zero pronoun recovering for document-level natural language generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2530–2540, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. Qwen Team. 2024. Qwen2.5: A party of foundation models. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. Chunhao Tian, Yutong Wang, Xuebo Liu, Zhexuan Wang, Liang Ding, Miao Zhang, and Min Zhang. 2025. AgentInit: Initializing LLM-based multi-agent systems via diversity and expertise orchestration for effective and efficient collaboration. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 11870–11902, Suzhou, China. Association for Computational Linguistics. Giorgos Vernikos, Brian Thompson, Prashant Mathur, and Marcello Federico. 2022. Embarrassingly easy document-level MT metrics: How to convert any pretrained metric into a document-level metric. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 118–128, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. Jiaan Wang, Fandong Meng, Yunlong Liang, and Jie Zhou. 2025a. DRT: Deep reasoning translation via long chain-of-thought. In Findings of the Association for Computational Linguistics: ACL 2025, pages 6770–6782, Vienna, Austria. Association for Computational Linguistics. Jiaan Wang, Fandong Meng, and Jie Zhou. 2025b. Deep reasoning translation via reinforcement learning. ArXiv preprint, abs/2504.10187. Longyue Wang, Chenyang Lyu, Tianbo Ji, Zhirui Zhang, Dian Yu, Shuming Shi, and Zhaopeng Tu. 2023a. Document-level machine translation with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16646–16661, Singapore. Association for Computational Linguistics. Longyue Wang, Zhaopeng Tu, Yan Gu, Siyou Liu, Dian Yu, Qingsong Ma, Chenyang Lyu, Liting Zhou, ChaoHong Liu, Yufeng Ma, and 1 others. 2023b. Findings of the wmt 2023 shared task on discourse-level literary translation: A fresh orb in the cosmos of llms. In Proceedings of the Eighth Conference on Machine Translation, pages 55–67. Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. Exploiting cross-sentence context for neural machine translation. In Proceedings of the 2017

Conference on Empirical Methods in Natural Language Processing, pages 2826–2831, Copenhagen, Denmark. Association for Computational Linguistics. Yutong Wang, Siyuan Xiong, Xuebo Liu, Wenkang Zhou, Liang Ding, Miao Zhang, and Min Zhang. 2026a. Agentdropoutv2: Optimizing information flow in multi-agent systems via test-time rectify-orreject pruning. Preprint, arXiv:2602.23258. Yutong Wang, Jiali Zeng, Xuebo Liu, Fandong Meng, Jie Zhou, and Min Zhang. 2024. TasTe: Teaching large language models to translate through selfreflection. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6144–6158, Bangkok, Thailand. Association for Computational Linguistics. Yutong Wang, Jiali Zeng, Xuebo Liu, Derek F. Wong, Fandong Meng, Jie Zhou, and Min Zhang. 2025c. DelTA: An online document-level translation agent based on multi-level memory. In The Thirteenth International Conference on Learning Representations. Zhexuan Wang, Xuebo Liu, Li Wang, Zifei Shan, Yutong Wang, Zhenxi Song, and Min Zhang. 2026b. Maspo: Joint prompt optimization for llm-based multi-agent systems. Preprint, arXiv:2605.06623. Zhexuan Wang, Yutong Wang, Xuebo Liu, Liang Ding, Miao Zhang, Jie Liu, and Min Zhang. 2025d. AgentDropout: Dynamic agent elimination for tokenefficient and high-performance LLM-based multiagent collaboration. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 24013– 24035, Vienna, Austria. Association for Computational Linguistics. Minghao Wu, Thuy-Trang Vu, Lizhen Qu, George Foster, and Gholamreza Haffari. 2024. Adapting large language models for document-level machine translation. ArXiv preprint, abs/2401.06468. Yangjian Wu and Gang Hu. 2023. Exploring prompt engineering with GPT language models for documentlevel machine translation: Insights and findings. In Proceedings of the Eighth Conference on Machine Translation, pages 166–169, Singapore. Association for Computational Linguistics. Yuxiang Zhang, Shangxi Wu, Yuqi Yang, Jiangming Shu, Jinlin Xiao, Chao Kong, and Jitao Sang. 2024. o1-coder: an o1 replication for coding. ArXiv preprint, abs/2412.00154. Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, and 3 others. 2026. A survey of large language models. Preprint, arXiv:2303.18223.

Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. 2024. Marco-o1: Towards open reasoning models for open-ended solutions.

A

Appendix

A.1

Detailed Entity Attribute Fields

All entity categories and their corresponding attribute fields in the memory module are shown in Table 5. The extracted entities are classified into six different categories: Character, Organization, Location, Event, Object, and Other. Each category has its unique attribute fields, reflecting different information focuses. A.2

Pseudo Codes

The pseudo code of the context-aware deep reasoning translation is shown in Algorithm 1. During translation, the agent first roughly retrieves candidate information from the memory module (Lines 7-11), and then it conducts a deep reasoning selection to filter out the noisy items and keep the relevant ones as guidance for translating the current segment (Lines 12-19). After the translation is done, the memory module is updated using the acquired results (Lines 20-24). The pseudo code of the reasoning strategy optimization is shown in Algorithm 2. We first conduct parallel sampling on each reasoning step and the translation process to construct preference data (Lines 1-17), and then optimize the agent on it via reinforcement learning (Lines 18-21). The sentence-level aligned translation algorithm is shown in Algorithm 3. We insert boundary markers “<s>” and “</s>” around each sentence, and add sequential numbers “#i” at the beginning (Lines 3-4). The LLMs are told to keep these markers during translation (the corresponding prompt is shown in Figure 10). Then, we ask the agent to generate a target segment and check whether it is strictly aligned with the source at the sentence level (Lines 5-6). If so, the results are output; otherwise, the source segment will be divided into two halves. This procedure is executed recursively until alignment is achieved, or there’s only one sentence left in the segment part (Lines 7-15). A.3

Case Study

We present a qualitative case study comparing the translation performance of L OONG against the sentence-level baseline utilizing Qwen3-8B as the base model. Table 12 displays the source segment,

the reference translation, and the baseline output. Table 13 details L OONG’s translation workflow, illustrating its reasoning-driven context selection procedure and the final generation. The introduction of Summary 4, which establishes the immediate scene of soldiers being awakened and equipped for action, significantly enhanced the contextual continuity of the translation. By recognizing that the current passage extends this preparation scene, the model maintained the narrative thread of “coordination and readiness” and avoided the disjointed framing seen in the Base version, which treated each line in isolation and consequently invented spurious military ranks (“Lieutenant Xiao”, “Colonel Kolen”, “Admiral Nimitz”). Similarly, the incorporation of Sentence Pairs 3 and 4 directly supplies the canonical translations for the core characters and their actual ranks. Pair 4 in particular pins down “科伦” as a lieutenant (not a Colonel) and “博拉丁·肖上尉” as Captain Borlatin Xiao. With these in-context exemplars, the model produced “Korren” consistently across the entire window—in stark contrast to the baseline, which drifted through “Cole”, “Kolen”, and “Korm” within five sentences, and even mis-spelled the captain’s name as “Borlatan”. Furthermore, the entity-level selection of records 5 and 9 demonstrates active filtering of redundant information: ten entities are pruned to two (i.e., the captain and the supporting armored squad). The model further detects that record 10 is redundant with record 5 and discards it, exhibiting an explicit anti-overfetching behavior. This focused entity grounding is what enables L OONG to render “Captain Borlatin Xiao” with the correct spelling and to anchor “Nemic” as a co-protagonist rather than fabricate the inflated “Admiral Nimitz” that the baseline produces. A.4

Detailed Dataset Information

The detailed information of the constructed training and test sets is shown in Table 6 and 7, respectively. To align with the application scenario of our longdocument translation agent, we utilize the News Commentary V18.1 dataset (News domain) and first filter out documents with fewer than 50 lines. Subsequently, we randomly sample 500 documents for each of the language pairs (English ⇔ Chinese, German, and French) to construct the training set. From the remaining data, we randomly sample 25 documents per language pair to form the test set. For the unseen language pairs employed in §4.4,

we apply the same filtering strategy and randomly sample 25 documents for testing. The WMT24++ dataset encompasses four domains: News, Social, Speech, and Literary. We filter the dataset to retain documents with ≥ 15 lines and then randomly sample 16 documents per language as test data. For the IWSLT2017 (TED talks, speech domain) and GuofengV1 (webnovel, literary domain) datasets, we utilize their standard test sets without any additional filtering or sampling. Finally, to evaluate ultra-long literary document translation, we use Journey to the West, one of the Four Great Classical Novels of Chinese literature, which falls under the ultra-long Literary domain. We utilize a Chinese-Portuguese parallel corpus of this novel and extract the first twelve chapters as our test data. A.5

Prompt Design

For the Essence memory component, the prompt for generating summaries is shown in Figure 6. For the Entity memory component, the prompts of entity classification, attribute fields filling, and attribute fields update are shown in Figure 7, Figure 8, and Figure 9, respectively. The prompt for the final alignment-enforced document translation process of L OONG is shown in Figure 10. The prompt of the LLM evaluator is illustrated in Figure 11. A.6

Detailed Results

The detailed results of L OONG and the baselines in En ⇒ Xx and Xx ⇒ En tested on the News Commentary V18.1 data are shown in Table 8 and Table 9, respectively. Similarly, the corresponding results evaluated on the WMT24++ test set are presented in Table 10 and Table 11. Across nearly all evaluation metrics and backbone models, L OONG consistently achieves the best performance, demonstrating its effectiveness across different evaluation dimensions as well as its robustness across languages and backbone models.

Prompt for Generating Summaries Please provide a summary of the given paragraph, preserving key information as much as possible. Note that the length of the summary should not exceed 50 words. <Paragraph> {text}

Figure 6: The prompt template for generating summaries.

Prompt for Entity Classification Given a text passage and a specified entity, classify the entity into one of these categories: Character, Organization, Location, Event, Object, or Other. Only output the category name. <Text passage> {text} <Entity> {entity}

Figure 7: The prompt template for entity classification.

Prompt for Filling Entity Attribution Fields Given a text passage and a specified entity, summarize the relevant information about the entity including the following items: {info_items} <Text passage> {text} <Entity> {entity} The output should be a Markdown code snippet formatted in the following schema, including the leading and trailing "“‘json" and "“‘", and without any comments: “‘json { "Role": string // a string "Description": string // a string "Relationships": string // a string "Motivation/Goals": string // a string "Development": string // a string } “‘ If an entry has no corresponding content, just fill in "N/A".

Figure 8: The prompt template for filling entity attribution fields.

Prompt for Updating Entity Attribution Fields Given a text passage and a specified entity, update the existing information about this entity including the following items: {info_items} <Text passage> {text} <Entity> {entity} <Existing Information> {exist_info} “‘json { "Role": string // a string "Description": string // a string "Relationships": string // a string "Motivation/Goals": string // a string "Development": string // a string } “‘ If an entry has no corresponding content, just fill in "N/A".

Figure 9: The prompt template for updating entity attribution fields.

Prompt for Document Translation by L OONG Given some auxiliary information, translate the current page of source text from {src_lang} to {tgt_lang}. <Summaries of previous pages> {summaries} <Original texts of previous pages> {exemplars} <Entity Records> {Entities} Now please translate the given English text into Chinese. Make sure to obay the TRANSLATION TASK RULES. <TRANSLATION TASK RULES> 1. Each sentence in the text is marked with “#i” to indicate its order. 2. The beginning and end of an independent sentences are marked by “<s>” and “</s>”, respectively. 3. Output MUST: - Preserve ALL sequence, beginning and end marks (“#i”, “<s>” and “</s>”) - Maintain EXACT 1:1 sentence correspondence - NEVER merge/split/reorder/omit sentences <{src_lang} source text> {src_content}

Figure 10: The prompt template for document translation by L OONG.

Prompt for LLM Evaluator You are an expert linguist and translation quality evaluator. Your task is to evaluate the quality of a document-level translation from {src_language} to {tgt_language} based solely on the Source Document, the Reference Document (Gold Standard), and the Hypothesis Document (Model Output). Please assess the [Hypothesis] text as a whole against the [Source] and [Reference]. Provide a holistic score from 0 to 100 for the following three specific dimensions, where 0 represents a complete failure and 100 represents a perfect, native-level professional translation. [Source]: {src_doc} [Reference]: {ref_doc} [Hypothesis]: {hyp_doc} [Evaluation Dimensions]: 1. **General Quality**: - Focuses on accuracy (faithfulness to the source meaning) and fluency (grammatical correctness and natural flow). - A high score means the translation is precise, preserves the original meaning without omission or hallucination, and reads naturally in the target language. 2. **Cohesion**: - Focuses on the explicit linking words and grammatical connections between sentences and clauses (e.g., correct use of pronouns, conjunctions, substitution, and ellipsis). - A high score means the text is syntactically well-connected, and references (anaphora/cataphora) are clear and unambiguous throughout the document. 3. **Coherence**: - Focuses on the logical arrangement and semantic relationships of ideas. It assesses whether the text "makes sense" as a whole narrative or argument. - A high score means the discourse flows logically, follows the thought patterns/conventions of the target culture, and is easy for a reader to understand without referring to the source. 4. **Style Consistency**: - Focuses on the maintenance of tone, register (formal/informal), and voice throughout the document. - A high score means the translation maintains a unified style that matches the source text’s intent (e.g., not switching between academic and slang phrasing). 5. **Terminology Consistency**: - Focuses on the consistent translation of specific terms, entities, and keywords across the entire document. - A high score means the same concept is translated using the same term throughout, avoiding confusion caused by using multiple synonyms for the same specific entity. [Output Requirement]: For each dimension, provide a score (0-100) and a brief justification based on the whole document. Your response must strictly follow this format: ### Evaluation Report **1. General Quality** Score: [0-100] Rationale: ... **2. Cohesion** Score: [0-100] Rationale: ... **3. Coherence** Score: [0-100] Rationale: ... **4. Style Consistency** Score: [0-100] Rationale: ... **5. Terminology Consistency** Score: [0-100] Rationale: ...

Figure 11: The prompt template of the LLM evaluator.

Category

Attribute Fields

Character Organization Location Event Object Other

Role, Description, Relationships, Motivation/Goals, Development Type, Purpose, Members, Location, Significance Type, Description, Inhabitants, Events, Symbolism Title, Description, Participants, Location, Consequences, Timeline Type, Appearance, Purpose, Owner/Creator, Significance Label, Type, Description, Significance, Interaction, Impact

Table 5: All entity categories and their corresponding attribute fields.

Model Qwen2.5-7B-Instruct Qwen3-8B Qwen3-14B Llama3.1-8B-Instruct

Language

Size

En ⇒ Xx Xx ⇒ En En ⇒ Xx Xx ⇒ En En ⇒ Xx Xx ⇒ En En ⇒ Xx Xx ⇒ En

86,429 78,074 82,324 75,360 78,832 75,052 74,192 79,210

Table 6: Statistics of the constructed training set for each model.

Dataset

News Commentary V18.1

Domain

News

WMT24++

News, Social, Speech, Literary

IWSLT2017

Speech (TED talks)

GuoFengV1 Journey to the West

Literary (Webnovels) Literary (Ultra-Long novel)

Language

# Words

# Docs

Words / Doc.

English ⇔ Chinese English ⇔ German English ⇔ French English ⇔ Czech English ⇔ Spanish English ⇔ Italian English ⇔ Japanese English ⇔ Portuguese English ⇔ Russian English ⇔ Chinese English ⇔ German English ⇔ French English ⇔ Chinese English ⇔ German English ⇔ French Chinese ⇒ English Chinese ⇒ Portuguese

24,284 24,369 24,813 23,961 24,537 27,237 20,054 26,775 24,472 10,186 10,186 10,186 23,863 18,162 23,917 17,335 51,854

25 25 25 25 25 25 25 25 25 16 16 16 12 10 12 12 1

971 975 993 958 982 1,090 802 1,071 979 637 637 637 1,989 1,816 1,993 1,445 51,854

Table 7: Statistics of the adopted test sets.

Model

System

En ⇒ Zh

En ⇒ De

En ⇒ Fr

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

Qwen2.5 7B

Sentence Segment Doc2Doc D ELTA L OONG

87.6 88.0 87.5 87.9 88.2

83.0 83.6 82.8 83.3 83.7

82.9 88.9 85.3 82.5 85.1

84.5 86.8 85.2 84.6 85.7

82.9 82.7 82.7 83.5 84.6

77.3 77.1 77.1 77.8 78.7

40.8 43.0 43.7 44.5 49.4

67.0 67.6 67.8 68.6 70.9

82.8 83.8 83.3 83.9 84.8

77.0 78.1 77.5 78.3 79.1

45.0 60.2 60.3 63.3 65.7

68.3 74.0 73.7 75.2 76.5

Qwen3 8B

Sentence Segment Doc2Doc D ELTA L OONG

88.5 88.7 88.7 88.3 88.9

84.0 84.3 84.2 83.8 84.5

92.4 93.4 93.6 91.1 93.1

88.3 88.8 88.8 87.7 88.8

86.5 85.6 85.7 86.3 86.9

80.9 79.0 79.3 80.6 81.2

67.8 64.9 66.0 66.3 69.6

78.4 76.5 77.0 77.7 79.2

85.5 85.3 85.0 85.5 86.1

80.0 79.4 78.7 79.9 80.5

76.5 76.5 78.1 78.9 82.5

80.7 80.4 80.6 81.4 83.1

Qwen3 14B

Sentence Segment Doc2Doc D ELTA L OONG

88.8 88.9 88.9 88.8 89.0

84.3 84.6 84.5 84.3 84.6

93.1 94.5 94.1 94.2 92.0

88.7 89.3 89.2 89.1 88.5

87.4 87.2 87.3 87.4 87.9

81.8 81.5 81.8 81.9 82.3

76.3 72.9 75.7 75.6 79.2

81.8 80.5 81.6 81.6 83.1

86.2 85.9 85.9 86.1 86.4

80.6 80.5 80.3 80.6 80.9

83.8 83.3 84.5 85.2 85.5

83.5 83.2 83.6 84.0 84.3

Llama3.1 8B

Sentence Segment Doc2Doc D ELTA L OONG

85.0 84.1 85.0 85.8 87.0

77.6 77.2 79.5 79.8 82.6

65.8 61.3 66.5 75.0 76.5

76.1 74.2 77.0 80.2 82.0

85.2 84.8 85.1 85.9 86.6

79.5 79.3 79.5 80.3 81.2

60.9 58.2 62.1 66.4 69.8

75.2 74.1 75.6 77.5 79.2

84.3 84.0 83.7 84.8 85.5

78.8 78.4 78.2 79.2 80.1

69.4 63.9 65.8 72.1 72.8

77.5 75.4 75.9 78.7 79.4

Table 8: Detailed results of L OONG compared to the baseline methods in En ⇒ Xx directions on the News Commentary V18.1 test set. The best results are marked in bold font.

Model

System

Zh ⇒ En

De ⇒ En

Fr ⇒ En

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

Qwen2.5 7B

Sentence Segment Doc2Doc D ELTA L OONG

86.2 86.3 85.7 86.7 87.4

82.3 81.9 81.0 83.0 83.7

81.0 83.5 80.1 87.1 91.5

83.2 83.9 82.3 85.6 87.5

87.4 86.3 86.4 88.3 88.9

82.8 77.7 78.1 83.9 84.4

78.8 71.2 71.8 87.3 90.3

83.0 78.4 78.8 86.5 87.9

86.5 87.0 85.1 87.4 87.9

81.1 81.2 75.9 82.3 83.0

75.4 84.0 70.9 88.3 90.4

81.0 84.0 77.3 86.0 87.1

Qwen3 8B

Sentence Segment Doc2Doc D ELTA L OONG

86.8 87.0 87.2 87.1 87.3

83.0 83.2 83.5 83.3 83.7

86.4 87.2 88.0 88.6 90.2

85.4 85.8 86.2 86.3 87.1

88.5 87.8 88.3 88.7 89.0

84.1 81.7 82.7 84.3 84.5

88.8 79.9 85.0 89.3 90.8

87.1 83.1 85.3 87.4 88.1

87.4 86.9 87.2 87.5 87.8

82.3 80.6 81.5 82.5 82.8

88.1 80.7 85.5 89.3 89.7

86.0 82.7 84.7 86.4 86.8

Qwen3 14B

Sentence Segment Doc2Doc D ELTA L OONG

87.2 87.3 87.4 87.4 87.7

83.4 83.7 83.9 83.8 84.1

88.0 89.7 90.5 90.6 91.1

86.2 86.9 87.3 87.3 87.6

88.7 88.9 89.0 88.9 89.2

84.3 84.5 84.6 84.5 85.0

89.1 89.5 90.1 90.5 91.1

87.4 87.6 87.9 87.9 88.5

87.7 87.6 87.7 87.8 88.0

82.8 82.7 82.9 82.9 83.1

87.6 88.1 89.6 90.1 91.1

86.0 86.1 86.7 87.0 87.4

Llama3.1 8B

Sentence Segment Doc2Doc D ELTA L OONG

85.7 85.8 85.8 85.7 86.8

81.7 82.0 81.9 81.9 83.1

78.1 79.0 79.1 83.1 84.9

81.8 82.2 82.3 83.6 85.0

88.0 88.1 87.9 87.9 88.9

83.6 83.6 82.9 83.5 84.5

84.3 84.9 83.2 86.3 88.6

85.3 85.5 84.7 85.9 87.3

86.8 86.9 86.9 86.8 87.7

81.7 81.6 81.8 81.6 82.8

83.2 82.1 84.2 85.2 89.3

83.9 83.5 84.3 84.5 86.6

Table 9: Detailed results of L OONG compared to the baseline methods in Xx ⇒ En directions on the News Commentary V18.1 test set. The best results are marked in bold font.

Model

System

En ⇒ Zh

En ⇒ De

En ⇒ Fr

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

Qwen2.5 7B

Sentence Segment Doc2Doc D ELTA L OONG

80.3 83.6 83.5 83.8 84.6

78.2 81.7 81.2 82.0 82.8

62.6 79.0 80.1 77.7 77.5

73.7 81.4 81.6 81.2 81.6

71.9 77.1 78.9 77.2 79.5

68.3 74.0 75.4 74.7 76.9

37.9 48.6 52.3 52.2 53.0

59.4 66.6 68.9 68.0 69.8

70.8 78.2 78.0 79.1 79.2

66.8 75.3 75.6 75.9 76.4

34.3 60.4 62.3 65.8 64.4

57.3 71.3 72.0 73.6 73.3

Qwen3 8B

Sentence Segment Doc2Doc D ELTA L OONG

84.4 84.3 84.9 84.6 85.7

82.6 81.3 82.9 83.1 83.8

80.7 80.1 84.8 85.2 84.8

82.6 81.9 84.2 84.3 84.8

80.9 78.8 80.7 80.6 81.7

78.0 74.7 77.6 78.2 78.9

66.1 62.7 68.9 67.4 69.5

75.0 72.0 75.7 75.4 76.7

80.2 80.3 80.3 80.3 81.0

77.5 76.4 76.6 77.3 78.4

70.7 69.9 69.5 72.7 76.0

76.1 75.5 75.5 76.7 78.5

Qwen3 14B

Sentence Segment Doc2Doc D ELTA L OONG

85.1 85.0 85.1 85.2 85.6

83.7 82.6 83.2 83.7 83.6

85.6 84.6 88.3 87.8 88.3

84.8 84.0 85.5 85.5 85.8

81.9 82.3 82.6 82.0 82.7

79.7 79.4 79.8 79.4 80.4

73.6 72.8 74.4 74.3 74.2

78.4 78.1 78.9 78.5 79.1

81.3 81.5 81.8 81.5 81.8

78.2 77.5 78.4 78.6 78.4

78.5 79.0 78.9 79.9 79.1

79.3 79.3 79.7 80.0 79.8

Llama3.1 8B

Sentence Segment Doc2Doc D ELTA L OONG

78.0 76.9 80.3 79.7 81.6

74.0 70.4 78.7 77.9 79.8

55.3 54.3 61.4 63.6 63.0

69.1 67.2 73.5 73.7 74.8

76.4 79.1 78.9 78.5 80.2

73.7 76.2 76.4 75.9 77.2

53.3 56.6 57.2 59.9 63.5

67.8 70.6 70.9 71.4 73.6

75.8 77.8 77.6 77.5 79.6

73.4 75.1 75.9 75.2 76.8

55.5 56.3 61.1 62.7 66.7

68.2 69.7 71.5 71.8 74.3

Table 10: Detailed results of L OONG compared to the baseline methods in En ⇒ Xx directions on the WMT24++ test set. The best results are marked in bold font.

Model

System

Zh ⇒ En

De ⇒ En

Fr ⇒ En

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

sCMT

dCMT

LLM

Avg

Qwen2.5 7B

Sentence Segment Doc2Doc D ELTA L OONG

78.3 82.6 82.8 82.9 83.3

75.3 79.8 80.2 80.4 81.0

56.4 75.2 76.9 76.9 79.8

70.0 79.2 80.0 80.1 81.4

78.0 82.2 82.7 84.7 85.5

74.9 75.6 77.2 82.2 83.3

59.3 69.1 71.4 82.5 83.0

70.7 75.6 77.1 83.1 84.0

77.2 83.5 83.4 84.1 84.4

73.5 79.2 78.9 81.7 82.0

56.4 77.1 76.1 81.7 83.3

69.0 79.9 79.5 82.5 83.2

Qwen3 8B

Sentence Segment Doc2Doc D ELTA L OONG

82.8 83.7 83.6 83.4 84.2

80.0 81.2 80.9 80.9 81.8

74.3 77.0 80.6 78.6 80.8

79.0 80.6 81.7 81.0 82.2

84.2 83.7 84.4 84.8 85.5

81.4 78.3 81.0 82.5 83.1

77.4 72.2 78.7 82.0 85.4

81.0 78.1 81.4 83.1 84.7

83.4 83.1 83.6 84.4 84.8

81.3 78.2 79.4 82.3 82.5

75.6 73.6 76.4 82.8 84.5

80.1 78.3 79.8 83.1 83.9

Qwen3 14B

Sentence Segment Doc2Doc D ELTA L OONG

83.4 84.1 84.0 83.9 84.2

80.8 81.8 81.9 81.8 81.9

76.5 79.8 83.9 83.7 82.3

80.2 81.9 83.3 83.1 82.8

85.1 85.6 85.6 85.5 85.7

82.6 83.3 83.3 82.9 83.4

83.2 83.9 85.3 84.3 87.4

83.6 84.3 84.7 84.2 85.5

84.5 84.9 85.1 84.9 85.2

82.1 82.5 82.7 82.5 82.9

81.6 84.1 86.2 85.1 86.3

82.7 83.8 84.6 84.2 84.8

Llama3.1 8B

Sentence Segment Doc2Doc D ELTA L OONG

79.8 82.5 82.4 78.1 83.0

76.9 79.8 80.1 74.7 80.3

65.9 72.3 75.2 70.6 78.5

74.2 78.2 79.2 74.5 80.6

81.3 84.3 84.7 80.7 85.1

78.3 81.5 81.6 77.9 82.8

73.0 77.8 78.3 78.5 84.9

77.5 81.2 81.5 79.0 84.3

80.2 83.9 83.7 80.2 84.3

77.8 81.2 81.2 77.5 81.9

69.9 73.5 76.6 76.7 83.6

76.0 79.5 80.5 78.1 83.3

Table 11: Detailed results of L OONG compared to the baseline methods in Xx ⇒ En directions on the WMT24++ test set. The best results are marked in bold font.

Algorithm 1: Context-Aware Deep Reasoning Translation with Memory (L OONG) Input :Source Document S, Segment size l, LLM M Output :Target Document T /* Initialize Memory Components (0) 1 Es ←∅; (0) 2 Ex ←∅; (0) 3 En ←∅;

*/ // Essence // Exemplars // Entities

/* Segmentation 4 Split S into L segments {s1 , s2 , . . . , sL }, where |sτ | = l; 5 T ← ∅; for t ← 1 to N do /* Step 1: Memory Retrieval 7 vτ ← Embed(sτ ); 8 Ẽs ← TopKa∈E (τ −1) Sim(Embed(a), vτ , Ks );

*/

6

*/

s

9

Ẽx ← TopK(u,·)∈E (τ −1) Sim(Embed(u), vτ , Kx );

10

(τ −1) src Ẽn ← {(esrc , etgt , LLM(Desc, I, sτ )) | (esrc , etgt , I) ∈ En , e ∈ sτ };

11

Define Candidate Set E ← [E˜s , E˜x , E˜n ];

12 13

14

15 16 17

c

/* Step 2: Deep Reasoning Translation */ H0 ← ∅ ; // Reasoning history for k ← 1 to 3 do /* Observe: History, Specific Context */ Ok ← {Hk−1 , E[k]}; /* Action Sub-steps: Analyze, Choose, Translate */ rk ← LLM(Analyze, Ok ) ; // Assess usefulness Ck ← LLM(Choose, rk ) ; // Select optimal context Hk ← Hk−1 ∪ {(Ok , ⟨rk , Ck )⟩};

19

tτ ← LLM(Translate, sτ , {Ck }3k=1 ); Append tτ to T ;

20

/* Step 3: Memory Update anew ← Summarize(sτ );

21

Es

18

22 23 24 25

(τ )

(τ −1)

← Es

∪ {anew }; (τ ) (τ −1) Ex ← Ex ∪ {(u, v) | u ∈ sτ , v ∈ tτ }; new En ← ExtractEntities(sτ , tτ ); (τ ) (τ −1) En ← UpdateRecords(En , Ennew );

return T

*/

Algorithm 2: Reasoning Strategy Optimization Input :Original Training Set Draw , Pretrained LLM πθ Output :Optimized LLM πθ∗ /* Phase 1: Observe-and-Act Data Construction 1 Initialize datasets Dsel ← ∅, Dutil ← ∅; ∗ 2 for each pair (s, t ) in Draw do 3 for reasoning step k ← 1 to 3 do /* Sample Actions Parallelly for M times 4 Gk ← Sample(πθ , Ok , M ); 5 for each action Aik = ⟨rki , Cki ⟩ ∈ Gk do /* Sample Translations for N times & Evaluate 6 Tki ← Sample(Translate, s, Cki , N ); P 7 U (Aik ) ← N1 t̂∈T i COMET(s, t̂, t∗ );

*/

*/

*/

k

8

/* Construct Translation Preference Data t+ ← arg maxt∈T i COMET(s, t̂, t∗ );

*/

k

9 10

t− ← arg mint∈T i COMET(s, t̂, t∗ ); k Dutil ← Dutil ∪ {(⟨s, Cki ⟩, t+ , t− )}; /* Construct Context-Strategic Preference Data A+ k ← Gk [arg max(U (A))] ; A− k ← Gk [arg min(U (A))] ; − Dsel ← Dsel ∪ {(Ok , A+ k , Ak )}; /* Update State for Next Step Identify t+ associated with A+ k; Construct Ok+1 using A+ ; i Update Memory using t+ ;

*/ // Preferred Action // Dispreferred Action

/* Phase 2: Training (SFT + DPO) 17 Merge datasets: D ← Dsel ∪ Dutil ; // Stage 1: Cold-start SFT + + − 18 Extract preferred samples: DSFT ← {(x, y ) | (x, y , y ) ∈ D}; 19 πSFT ← TrainSFT(πθ , DSFT ); // Stage 2: DPO with LoRA 20 πθ ∗ ← TrainDPO(πSFT , D, use_lora = True); 21 return πθ ∗

*/

11 12 13

14 15 16

*/

Algorithm 3: Sentence-Level Alignment-Enforced Translation Input: Source document sentences Ssource = [s1 , s2 , . . . , sn ] Output: Translated target sentences Tfinal 1 Function RecursiveTranslate(S): // 1. Construct Prompt with Indices and Separators 2 P ← EmptyString; 3 foreach si ∈ S with index i do 4 P ← P + “#i <s> + si + </s>”; 5

6 7 8 9

10 11 12 13 14 15

// 2. Perform Translation via LLM Rraw ← LLM(P ); // 3. Parse Output and Check Alignment T ← ExtractSentences(Rraw ); if Aligned(T, S) or |S| == 1 then return T ; else // 4. Handle Misalignment via Recursion m ← ⌊|S|/2⌋; Stop ← S[1 . . . m]; Sbottom ← S[m + 1 . . . |S|]; Ttop ← RecursiveTranslate(Stop ); Tbottom ← RecursiveTranslate(Sbottom ); return Ttop + Tbottom ;

Main: Tfinal ← RecursiveTranslate(Ssource ); 17 return Tfinal

16

Data & Baseline Translation “肖上尉让我们搭车？”尼米奇有些惊讶地问道。“是的。说实话，他没我想象的那么糟糕。我的意 思是，他虽然不怎么样，但也不至于让人讨厌。”科伦抬起拇指和食指，按着喉部的麦克风，同时 将背包移到背上。“Exodus连所有单位，完成任务后到西区车辆段集合。我们将在05:00与第一装甲 排一起出发。”每隔几秒钟，他就会收到班长们的回复。他们刚来的时候，哈里斯、克雷斯和拉维 克都是级别较低的军士。自从在地球上进行基础训练以来，他们四人就一直是好朋友。科伦觉得认 Source 识他们仿佛是很久以前的事了，尽管这才过去不到两年。他们都是班长，但自从一周前，前排长 被Daybreak狙击手击中，因伤退役后，克雷斯就开始领导第二排了。走到车辆段时，科伦和尼米奇 的目光落在了大约八辆克罗诺斯主战坦克和四辆史赛克装甲车上。机组人员四处奔走，确保他们的 战争机器一切正常。科伦很快就在人群中找到了博拉丁·肖上尉，他正在和装甲排的指挥官谈话。他 们走上前去时，肖上尉看了他们一眼，也迎了上来，他那张惯常冷漠的脸上看不出任何情绪。 “Shaw got us a ride?” Nemic asked, a little surprised. “Yup. Honestly, he isn’t as bad as I thought he was. I mean, he’s not great, but he isn’t unbearable” Cohren lifted his thumb and index finger to his throat mic, shifting his pack on his back while he did. “All units in Exodus Company, meet at Motor Pool West when you’re finished. We are heading out with the 1st Armored Platoon at 0500” Every few seconds he got a response from the squad leaders. Haris, Craith, and Ravik had all been low ranking NCO when they all got there. All four of them had been good friends since basic back on Earth. It felt so long ago now since Cohren had met Reference them, even though it couldn’t have been two years ago. They were squad leaders, But Craith was now leading 2nd Platoon as of a week ago, after the previous leader had gotten medically discharged after catching a bullet from a Daybreak sniper. Walking up to the motor pool, Cohren and Nemic’s eye fell on a group of around eight Kronos Main Battle Tanks and four Strykers. Their crew milled about, making sure their war machines were ready to go. Cohren’s gaze soon found Captain Boradin Shaw, who was talking with the armored platoon’s commander. As they walked up, Shaw looked over and walked up, his usually cold face masking all emotions. “Did Lieutenant Xiao let us ride with him?” Nimitz asked in surprise. “Yes. To be honest, he’s not as bad as I imagined. I mean, he’s not great, but he’s not讨厌 either.” Cole raised his thumb and index finger, pressing against the microphone on his throat, while shifting his backpack onto his back. “All units of Exodus, assemble at the west district vehicle depot after completing the mission. We will depart with the first armor platoon at 05:00.” Every few seconds, he would receive replies from the squad leaders. When they first arrived, Harris, Cress, and Ravek were all junior non-commissioned officers. Since their basic training on Earth, the four Baseline of them have been close friends. Cole felt like he had known them for a long time, even though it had been Translation less than two years. They were all squad leaders, but since a week ago, when the front sergeant was shot by sCMT: 81.1 Daybreak snipers and had to retire due to injuries, Cress began leading the second squad. As they approached the vehicle depot, Colonel Kolen and Admiral Nimitz’s eyes fell upon roughly eight Cronos main battle tanks and four Stryker armored vehicles. Crew members were bustling around, ensuring their war machines were all functioning properly. Kolen quickly spotted Captain Borlatan Xiao in the crowd, who was engaged in conversation with the commander of the armored platoon. As they approached, Captain Xiao glanced at them and came forward, his usual indifferent expression revealing no emotion.

Table 12: A case of source and reference segment with the translation results of the baseline method. “sCMT” denotes the sCOMET score.

L OONG Translation Process

Essence

Analysis

Selection

[1] 科伦和尼米奇被派往伊安洛斯城外支援地面进攻，采用“野鼬鼠”战术对抗敌方轰炸机，前线战事 紧张，他们将与第一装甲排一同行动。 [2] 科伦作为中尉，带领Exodus连准备行动，成员们迅速整理装备，15分钟内完成检查，准备出发。 [3] 科伦回应问题，门突然打开，哈里斯、克雷斯等人已穿戴整齐，准备行动，克雷斯询问是否被什 么吵醒。 [4] 科伦按响警报叫醒士兵，拉维克抱怨早餐糟糕，新兵们虽困倦但已穿戴整齐准备就绪。 [1] This summary provides context about the mission and the tactical approach (“wild goose” tactic), but it is not directly relevant to the current passage, which focuses on the coordination and preparation for the mission rather than the overall strategy. Therefore, it should not be selected. [2] This summary mentions the preparation of the Exodus Company and the time frame for gear checks, but the current passage is more about the communication and the relationship dynamics among the soldiers rather than the timeline of preparation. Therefore, it should not be selected. [3] This summary refers to the soldiers being awakened and the interaction with Krass, but the current passage does not involve waking up or direct dialogue with Krass in the same context. Therefore, it should not be selected. [4] This summary includes details about the soldiers being awakened, the mention of Lavik complaining about breakfast, and the overall readiness of the troops. These details are relevant to the current passage, which includes the soldiers being called to action and their state of readiness. Therefore, it should be selected. 4

Exemplar

Analysis

Selection

Entity

Analysis

[1] [Source] 伊塔尼亚时间，凌晨 4:30。科伦讨厌这么早起床。主要是因为他要负责叫醒连队的其他 人。他默默地走下床，穿上作战服，套上突击背心，从储物柜里收拾好东西，小心翼翼地拿起一个 特别的小盒子。 // [Target] Itania time, 4:30 a.m. Korren hated waking up so early. Mainly because he was responsible for waking up the rest of the squad. He quietly got out of bed, put on his combat gear, strapped on his assault vest, packed his belongings from the storage cabinet, and carefully picked up a small special box. [2] [Source] 他从夹克口袋里掏出打火机，又从另一个口袋里掏出一支烟。点燃香烟后，他轻轻地吸 了一口，然后打开了门。 // [Target] He took out a lighter from his jacket pocket and a cigarette from another pocket. After lighting the cigarette, he took a gentle puff and then opened the door. [3] [Source] “伊安洛斯城外的某个前线基地。我们被派去支援地面进攻。”科伦说着，吸了一口烟。 他有点儿责怪克雷斯让他染上了烟瘾。 // [Target] “Some frontline base outside of Ianlos. We’ve been sent to support the ground assault.” Korren said, taking a puff of his cigarette. He somewhat blamed Krass for getting him addicted to smoking. [4] [Source] “带上你们的私人物品，”科伦继续说道。作为一名中尉和连队的副指挥官，他有责任 协助连长博拉丁·肖上尉监督和领导连队。“我们不会回来了。各位军士和少尉，给你们和你们的队 员15分钟时间做最后的检查，并带上今天所需的口粮。” // [Target] “Pack your personal belongings,” Korren continued. As a lieutenant and deputy commander of the squad, he was responsible for assisting Captain Borlatin Xiao in supervising and leading the squad. “We won’t be coming back. Sergeants and lieutenants, you and your squads have 15 minutes to do your final checks and bring along the rations needed for today.” [1] This sentence pair provides detailed description of Korren’s morning routine and preparation for the mission. However, the current passage is focused on the coordination and communication during the pre-mission setup, not on the individual preparation process. Therefore, it should not be selected. [2] This sentence pair includes details about Korren’s method of waking up his subordinates and his frustration with the alarm system. While it offers insight into his leadership style, the current passage is about the coordination and readiness of the squad rather than the process of waking up. Therefore, it should not be selected. [3] This sentence pair provides context about the mission objective (“support the ground assault”) and includes a dialogue about smoking, which is relevant to understanding the character dynamics and the mission context. The mention of “smoking” also helps in understanding the tone and relationship between Korren and Krass. Therefore, it should be selected. [4] This sentence pair includes key information about the mission structure, Korren’s role as a deputy commander, and the timeline for final checks. It directly relates to the current passage, which discusses the coordination and instructions given to the squad before the mission. Therefore, it should be selected. 3, 4 [1] 科伦 / Korren: A lieutenant who is responsible for assisting the company commander Captain Borlatin Xiao in leading the squad, wearing combat gear and carrying weapons, demonstrating a sense of responsibility for the mission. [2] 哈里斯 / Harris: Harris is a soldier in Korren’s unit and is one of the troops Korren is responsible for waking up, showing preparedness and readiness during the early morning routine. [3] 克雷斯 / Krass: Krass is Korren’s comrade, a soldier already dressed and ready, displaying strong self-discipline and a sense of responsibility, capable of quickly preparing equipment. [4] 拉维克 / Lavik: Lavik is a soldier who dislikes early mornings and strict training but still follows orders, interacting with other soldiers like Korren, Harris, and Krass. [5] 博拉丁·肖上尉 / Captain Borlatin Xiao: Captain Borlatin Xiao is the company commander of Korren’s unit, responsible for leading the Exodus Squad to complete missions, maintaining a direct superior-subordinate relationship with Korren. [6] Exodus连 / Exodus Squad: Exodus Squad is an elite unit affiliated with the Second Armored Division, responsible for combat operations and ground assault support, using Daybreak tactics. [7] 尼米奇 / Nemychev: Nemychev is Korren’s superior lieutenant, with a pessimistic personality and dissatisfaction with war, but still performs his duties and acts alongside Korren cooperatively. [8] Daybreak / Daybreak: Daybreak is a specialized and elite military unit within the Second Army’s Second Assault Division, known for its advanced “wild boar” tactics and combat readiness. [9] 第一装甲排 / First armored squad: First armored squad is the supporting unit of Korren’s company, which will act together with them and provide mobile support, with Captain Xiao appearing at the assembly point. [10] 肖上尉 / Captain Xiao: Captain Xiao is Korren’s superior, the company commander of the Exodus Squad, responsible for leading the entire company and ensuring the smooth completion of missions. [1] This record provides a general description of Korren’s role and habits, but it is too broad and does not offer specific contextual details relevant to the current passage. Therefore, it should not be selected. [2] This record gives information about Harris’s role in the unit and his preparedness, but the current passage does not directly mention Harris or focus on his actions. Therefore, it should not be selected. [3] This record describes Krass’s preparedness and discipline, which is relevant to the context of the squad being ready, but the current passage does not focus on Krass or his interactions. Therefore, it should not be selected. [4] This record provides information about Lavik’s personality and role, but he is a background character and not directly involved in the current scene. Therefore, it should not be selected. [5] This record explains Captain Borlatin Xiao’s role as the leader of the Exodus Squad and his relationship with Korren. This is directly relevant to the current passage, where they are coordinating and preparing for the mission. Therefore, it should be selected.

Selection

[6] This record provides general information about the Exodus Squad, but the current passage focuses on specific interactions and instructions rather than the broader unit structure. Therefore, it should not be selected. [7] This record describes Nemychev’s role and personality, but he is not a central character in the current scene. Therefore, it should not be selected. [8] This record gives background on the Daybreak unit and its tactics, but the current passage does not mention Daybreak or its tactics. Therefore, it should not be selected. [9] This record explains the role of the First Armored Squad, which is mentioned in the current passage as being part of the mission. It helps in understanding the coordination and structure of the operation. Therefore, it should be selected. [10] This record is essentially a duplicate of Record 5, as it refers to the same person (Captain Xiao) and his role. Since Record 5 is already selected, this record is redundant. Therefore, it should not be selected. 5, 9

“Captain Xiao let us ride?” Nemic asked, surprised. “Yes. To be honest, he’s not as bad as I imagined. I mean, he’s not great, but he’s not讨厌 either.” Korren raised his thumb and index finger, pressing against his throat’s microphone, while he moved his backpack onto his back. “All units of Exodus Squad, assemble at the West Zone Vehicle Yard after mission completion. We will depart with the First Armored Squad at 05:00.” Every few seconds, he received replies from the squad leaders. When they first came, Harris, Krass, and Ravek were L OONG all lower-ranking sergeants. Since basic training on Earth, the four of them have been close friends. Korren Translation felt like he had known them for a long time, even though it had been less than two years. They were all squad sCMT: 82.4 leaders, but since a week ago, when the previous squad leader was shot by Daybreak snipers and retired due to injury, Krass started leading the Second Squad. When they arrived at the vehicle yard, Korren and Nemic’s eyes fell on about eight Cronos main battle tanks and four Stryker armored vehicles. Crew members were running around, ensuring their war machines were in working order. Korren quickly found Captain Borlatin Xiao in the crowd, who was talking to the commander of the armored squad. As they approached, Xiao looked at them and came forward, his usual indifferent face showing no emotion.

Table 13: A case of the translation process of L OONG. “sCMT” denotes the sCOMET score.