arXiv CSopen access

DMax: Aggressive Parallel Decoding for dLLMs

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

machine learning, deep learning, neural networks

arXiv:2604.08302v1 [cs.LG] 9 Apr 2026

Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, Xinchao Wang∗ National University of Singapore [email protected], [email protected]

Abstract We present DMax, a new paradigm for efficient diffusion language models (dLLMs). It mitigates error accumulation in parallel decoding, enabling aggressive decoding parallelism while preserving generation quality. Unlike conventional masked dLLMs that decode through a binary mask-to-token transition, DMax reformulates decoding as a progressive self-refinement from mask embeddings to token embeddings. At the core of our approach is On-Policy Uniform Training, a novel training strategy that efficiently unifies masked and uniform dLLMs, equipping the model to recover clean tokens from both masked inputs and its own erroneous predictions. Building on this foundation, we further propose Soft Parallel Decoding. We represent each intermediate decoding state as an interpolation between the predicted token embedding and the mask embedding, enabling iterative self-revising in embedding space. Extensive experiments across a variety of benchmarks demonstrate the effectiveness of DMax. Compared with the original LLaDA-2.0-mini, our method improves TPF on GSM8K from 2.04 to 5.47 while preserving accuracy. On MBPP, it increases TPF from 2.71 to 5.86 while maintaining comparable performance. On two H200 GPUs, our model achieves an average of 1,338 TPS at batch size 1. Code is available at: https://github.com/czg1225/DMax LLaDA-2.0-mini

GSM8K Acc 92.6% TPF: 2.04

DMax (Ours)

GSM8K Acc 92.1% TPF: 5.48

Xt Step 0

[Mask]

Step 1

Diffusion

[Mask]

Models

[Mask]

Step 2

Diffusion

[Mask]

Models

[Mask]

Parallel

Step 3

Diffusion

Language

Models

[Mask]

Parallel

X0 Step 4

Diffusion

Language

Models

Decode

Parallel

Diffusion

Language

Models

Decode

Parallel

Legends:

Mask

Token

Mask --> Token (Binary)

Hybrid Embedding

Mask --> Hybrid-Embedding --> Token (Self-Refine)

Figure 1: Comparison between the original LLaDA-2.0-mini and our proposed DMax. Unlike the original binary mask-to-token decoding process, DMax introduces a self-revising mask-to-hybridembedding-to-token process, enabling highly parallel decoding without accuracy dropping.

Introduction

Recently, Diffusion Language Models (dLLMs) [93, 95, 98, 42, 56, 108] have emerged as a compelling alternative to the long-standing dominance of Autoregressive Language Models (AR-LLM) [1, 6, 25] in text generation. The primary allure of dLLMs lies in their capacity for parallel decoding, which holds great promise for improving inference efficiency ∗

Correspoding Author

Preprint. Under review.

Despite this promise, the practical decoding parallelism of existing dLLMs [58, 109, 92, 17, 10, 90] remains limited, as their performance drops sharply under aggressive parallel decoding. Some prior work has attempted to improve this trade-off through improved decoding [37, 81, 41, 27, 8, 32, 34, 89] or distillation strategies [16, 62, 101, 38]. Nevertheless, these methods do not address the fundamental bottleneck underlying parallel decoding in current dLLM paradigm: error accumulation. In current mask-based dLLMs, decoding is a binary, one-way mask-to-token process. Once a masked position is decoded into a token, that token is fixed and propagated as context to subsequent decoding steps, with no opportunity for revision. Under highly parallel decoding, erroneous predictions are inevitable. Once such errors are committed, they contaminate future predictions and trigger cascading error accumulation, ultimately leading to semantic collapse. Unlike speculative decoding [39, 11, 43], dLLMs lack a mechanism to recover from incorrect predictions, which fundamentally restricts their performance under highly parallel decoding. Addressing this challenge requires a new dLLM paradigm with an intrinsic capability to revise its own predictions during decoding. Building on this insight, we propose DMax, a novel paradigm that reformulates the binary maskto-token decoding process into a self-revising transformation in the embedding space. Central to our approach is On-Policy Uniform Training (OPUT), a training recipe that efficiently extends a pretrained masked diffusion language model into a self-corrective uniform diffusion language model while preserving its original mask denoising capability. Unlike conventional uniform diffusion training that constructs noisy sequences by randomly sampling tokens from the vocabulary, OPUT samples noisy inputs on-policy from the model’s own predictive distribution. This substantially bridges the train-inference gap and enables the model to effectively learn to correct its own potential prediction errors. Building upon OPUT, we further present Soft Parallel Decoding (SPD) for inference. Instead of treating decoded tokens as discrete and irrevocable commitments, SPD represents each intermediate decoding state as a hybrid soft embedding, formed by interpolating between the predicted token embedding and the mask embedding according to the model’s prediction confidence. This simple design provides the model with confidence priors from previous steps, enabling more robust self-correction. Using LLaDA-2.0-mini [10], a state-of-the-art open-source dLLM, as the base model, we validate the effectiveness of our method across multiple widely used benchmarks. On the mathematical reasoning benchmark GSM8K [20], our method increases tokens per forward (TPF) from 2.04 to 5.48 with only minimal accuracy degradation relative to the original model. On the code generation benchmark MBPP [5], it improves TPF from 2.71 to 5.86 while maintaining comparable performance. In summary, we propose DMax, a novel paradigm that enables highly parallel decoding for dLLMs while preserving strong performance. Our central idea is to mitigate the error accumulation issue caused by the conventional one-way mask-to-token decoding. To realize this, we introduce two key designs: on-policy uniform training and soft parallel decoding. Extensive experiments demonstrate the effectiveness and superiority of our approach. This work establishes a new strong baseline for future research on parallel decoding in dLLMs.

Preliminaries

We begin by briefly reviewing the diffusion language modeling paradigms, and then highlight the central challenge for highly parallel decoding and introduce our key motivation. Masked Diffusion Language Models (MDLMs). MDLMs [70, 4, 66, 105, 51] formulate text generation as a discrete denoising process over token sequences, where clean tokens are progressively L replaced by a special [MASK] symbol during corruption. Let x0 = (x10 , . . . , xL 0 ) ∈ V denote a clean sequence of length L, where V is the vocabulary. Given a corrupted sequence xt at noise level t ∈ [0, 1], the denoising model is trained to recover the original tokens only at masked positions. The standard MDLM objective is " L # 1X i i LMDLM (θ) = −Ex0 , t, xt 1(xt = [MASK]) log pθ (x0 | xt ) . (1) t i=1 At inference time, MDLMs start from a fully masked sequence and iteratively decode masked positions in parallel, with an optional remasking step to enable further refinement. 2

Fowrd w/ Grad

The

Grey

Mask

Cat Is

Mask Is

CSE Loss

P5 On-Policy Rollout w/o Grad

Grey

Diffusion LM

Mask

Fowrd w/ Grad

The

Diffusion LM

Cute

ℒ푚푎

Bat Is

Loss Funcrion

The

Diffusion LM

Noisy Data

Add Noise

Clean Data

P1 P2 P3 P4

ℒ푝푟

Labels The Grey Cat Is Cute

Cat

Figure 2: Overview of the proposed On-Policy Uniform Training. Uniform Diffusion Language Models (UDLMs). UDLMs [68, 67, 69] generalize the corruption process by replacing tokens with uniformly sampled vocabulary tokens rather than a dedicated [MASK] symbol. As a result, the model is trained to recover clean tokens from arbitrary noisy token inputs, instead of only from masked positions. A standard UDLM training objective is " L # X i LUDLM (θ) = −Ex0 , t, xt log pθ (x0 | xt ) . (2) i=1

During inference, UDLMs typically start from a fully noisy sequence sampled uniformly from the vocabulary and iteratively update all positions. Error accumulation in MDLMs. Existing dLLMs based on the MDLM paradigm degrade sharply under highly parallel decoding, which limits the practical speedup. The main reason behind it is error accumulation. MDLM decoding follows a binary mask-to-token process: each position is either a mask token or a committed token. Once a masked position is decoded, its prediction is treated as fixed context for subsequent steps. Early mistakes cannot be revised, and instead propagate through later denoising steps as erroneous context. UDLMs as a Promising Solution. In contrast, UDLMs are trained to denoise from arbitrary vocabulary tokens rather than only from [MASK], so all positions can be re-evaluated at every decoding step. This token-to-token denoising mechanism naturally enables self-correction and improves robustness to prediction errors. However, UDLM decoding typically starts from a fully random sequence, which makes denoising harder and leads to very unstable generation. Unify the Strengths of MDLMs and UDLMs. Motivated by this trade-off, we propose to unify the strengths of both paradigms. Specifically, we retain a fully masked sequence as the initialization of UDLM decoding to preserve stability, while continuing to re-predict all tokens that have been decoded from [MASK] at every subsequent step. This design combines the stable initialization with the self-revising capability, enabling a more robust parallel decoding process.

Methodology

3.1

On-Policy Uniform Training

A practical way to achieve this goal is to extend a pretrained MDLM into a UDLM. Accordingly, our first objective is to endow a pretrained MDLM with the self-revision capability of UDLMs while preserving its original mask denoising ability. Extending MDLM toward UDLM is Nontrivial. This is nontrivial because the training objective of UDLMs differs substantially from that of MDLMs and is considerably harder to optimize. In the standard UDLM training paradigm, a clean sequence is first corrupted by randomly selecting a subset 3

of positions and replacing the selected tokens with tokens sampled uniformly from the vocabulary. The resulting noisy sequence is then used as model input, and the model is trained to recover the original clean sequence. However, this training strategy is often unstable in practice and tends to yield suboptimal performance. A key reason is that uniformly sampled tokens lie far outside the natural language manifold, producing highly unnatural corrupted inputs. As a result, the model must spend substantial capacity merely learning to map these corrupted sequences back toward plausible language, rather than directly acquiring effective language modeling and self-correction behaviors. More importantly, this corruption process introduces a severe train–inference mismatch. Unlike conventional UDLMs, our paradigm first predicts tokens from masked positions in parallel and then iteratively refines its own predictions. Consequently, the noisy sequences encountered at inference time are sampled from the model’s own output distribution rather than from a uniform vocabulary distribution. This mismatch hinders self-correction and leads to ineffective training. On-Policy Uniform Training. To address these issues, we propose On-Policy Uniform Training (OPUT), a simple yet effective method for equipping MDLMs with self-corrective denoising capability. The core idea is to construct training inputs using noisy sequences sampled on-policy from the model’s own predictive distribution, rather than from a uniform vocabulary distribution, thereby bridging the train–inference gap. The overview of the training procedure is shown in Figure 2. Training Procedure. Let Mθ denote a pretrained diffusion language model built on the MDLM paradigm, parameterized by θ. We further adapt Mθ on a training dataset D of clean sequences x0 = (x10 , . . . , xL 0 ). At each training iteration, we first sample a corruption level t ∼ Uniform(tl , th ), where tl and th denote the lower and upper bounds of the noise level, respectively. Given a clean (m) sequence x0 ∼ D, we construct a masked noisy sequence xt by independently replacing each token with [MASK] with probability t. (m)

We feed xt into Mθ and predict all masked positions in parallel. By sampling from the model’s (p) predictive distribution at masked positions, we obtain a predicted noisy sequence xt , defined as ( (m),i (m),i xt , if xt ̸= [MASK], (p),i xt = (3) (m) (m),i i i x̂ , x̂ ∼ pθ (· | xt ), if xt = [MASK]. (p)

Importantly, xt is sampled using the current model parameters at each iteration, making this a strictly on-policy rollout process. (m)

Next, we perform two forward passes, using the masked noisy sequence xt (p) sequence xt as inputs, respectively: (m)

pθ

(m)

(· | xt

(m)

) = Mθ (xt

(p)

and the predicted noisy

(p)

pθ (· | xt ) = Mθ (xt ).

(4)

We then supervise both outputs against the original clean sequence x0 using cross-entropy loss over all token positions, regardless of whether a position is masked: Lmask = −

L X

(m)

log pθ

(m)

(xi0 | xt

Lpred = −

i=1

L X

(p)

log pθ (xi0 | xt ).

(5)

i=1

The final training objective is Lon-policy = Lmask + Lpred , (6) By reducing the train–inference mismatch, the proposed OPUT strategy enables a pretrained MDLM to efficiently learn self-correction through limited post-training, while retaining its original mask denoising ability. As a result, the model can correct self-generated errors and effectively mitigate error accumulation under highly parallel decoding. On LLaDA-2.0-mini, our method improves GSM8K accuracy from 78% to 90% under confidence-threshold decoding with a threshold of 0.5, while also delivering faster decoding. 3.2

Soft Parallel Decoding

Although OPUT substantially mitigates error accumulation, it still struggles when many erroneous predictions arise simultaneously within a block. When many positions are decoded in parallel, 4

[Mask]

0.39

0.94

0.98

[Mask]

0.45

0.59

0.77 0.87 0.55

0.98

Diffusion

0.91

Language

0.93 0.92

Sample T=0

0.33

Diffusion LM

0.33

0.65

Diffusion LM

0.43

Diffusion LM

[Mask]

Diffusion LM

[Mask]

Top-1 Converge

Models Decode

[Mask]

0.17

0.52

0.49

0.87

[Mask]

0.09

0.33

0.81

0.92

Parallel

Decoding Begins

��

Renorm

0.67 ×

ℎ�

0.33 ×

Decoding Ends

��

Figure 3: Overview of the Soft Parallel Decoding procedure in DMax. correlated errors can appear at once, making them difficult to fully correct through iterative refinement. For example, for OPUT-trained LLaDA-2.0-mini, if we decode all masked positions in a block at once using a confidence threshold of 0, and then iteratively refine them, the accuracy on GSM8K drops to only 68%. Soft Parallel Decoding. To further enhance self-revising in iterative refinement, we propose soft parallel decoding. The central idea is to preserve predictive uncertainty from earlier iterations and explicitly propagate it to later refinement steps. Concretely, instead of treating intermediate decoding states as discrete tokens, we represent each decoded token as a soft embedding interpolated between the predicted token embedding and the mask embedding. Because the mask embedding naturally encodes maximal uncertainty, this interpolation serves as an explicit carrier of uncertainty across iterations. This enables the model to better distinguish confident predictions from unreliable ones, allowing it to focus on refining low-confidence tokens while avoiding interference from noisy signals. Decoding Procedure. An overview of the decoding process is shown in Figure 3. It follows a block-wise semi-autoregressive process. For each block, we partition its positions into two sets: mask positions and token positions. At initialization, all positions in the block are mask positions. At each decoding step, we use an aggressive confidence threshold τdec to promote some mask positions into token positions. Specifically, we scan the masked region from left to right and promote only its longest contiguous prefix whose confidence exceeds τdec . Once the first mask position with confidence below τdec is encountered, all mask positions to its right remain masked. If no mask position satisfies this criterion, we still promote the leftmost mask position to ensure decoding progress. This design keeps the masked region contiguous and prevents unreliable future tokens on the right from interfering with mask predictions on the left. At decoding step t, every mask position uses the mask embedding as model input: (t)

j ∈ M(t) .

hj = emask ,

(7)

where M(t) denotes the set of mask positions at step t. For each token position j ∈ T (t) , where T (t) is the set of token positions, we construct a hybrid (t−1) embedding from the top-1 prediction at the previous step t − 1 as the model input. Let yj denote (t−1)

the top-1 predicted token at position j, and let πj remaining probability mass to the mask embedding: (t−1)

be its predicted probability. We assign the (t−1)

πj,mask = 1 − πj

The unnormalized hybrid embedding is then (t) (t−1) (t−1) (t−1) h̃j = πj e yj + πj,mask emask ,

(8)

j ∈ T (t) .

(9)

Directly adding high-dimensional embeddings may distort their magnitude and lead to norm collapse. To avoid this issue, we renormalize the hybrid embedding so that its norm matches the probability5

Algorithm 1 Soft Parallel Decoding (Block-Wise) Require: Block positions B, decoding threshold τdec , acceptance threshold τacc 1: M ← B, T ← ∅, hj ← emask ∀j ∈ B ▷ initialize with fully masked block 2: repeat 3: pj (·) ← pθ (· | {hj }j∈B ), ∀j ∈ B 4: ŷj ← arg maxy pj (y), cj ← pj (ŷj ), ∀j ∈ B 5: P ← longest contiguous prefix in M such that cj > τdec for all j ∈ P 6: if P = ∅ then 7: P ← {leftmost position in M} 8: end if 9: T ← T ∪ P, M←B\T 10: hj ← emask , ∀j ∈ M ▷ Eq. (7) 11: hj ← HybridEmb(ŷj , cj , emask ), ∀j ∈ T ▷ Eqs. (8)–(10) (t) (t−1) 12: until ŷj = ŷj , ∀j ∈ B or minj∈B cj > τacc ▷ block converges 13: return ŷj , ∀j ∈ B ▷ commit the block

weighted sum of the component norms: (t)

(t)

hj =

h̃j

(t−1) (t−1) πj e yj

(t)

h̃j

(t−1) + πj,mask ∥emask ∥2 .

(10)

This hybrid embedding serves as a soft intermediate state between decoding steps, explicitly carrying forward the uncertainty of previous predictions. We regard a block as having converged to a stable state if either of the following conditions holds: (1) the top-1 predictions at all positions remain unchanged for two consecutive decoding steps, or (2) the confidence of every position in the block exceeds a high acceptance threshold τacc . Once a block converges, we commit all token positions in the block according to the final predictions and move on to the next block. By interpolating prediction-mask embeddings as intermediate states, the model receives an explicit uncertainty prior before every forward pass, leading to substantially more robust parallel decoding. On OPUT-trained LLaDA-2.0-mini, under the highly aggressive setting of τdec = 0, soft parallel decoding improves GSM8K accuracy from 68% to 90% while achieving a higher speedup. OPUT as a Prerequisite. Notably, soft parallel decoding must be used together with OPUT-trained models. OPUT trains the model to recover the correct target not only from masked inputs, but also from its own sampled predictions. As a result, the model learns a consistent mapping from both mask embeddings and self-predicted token embeddings toward the correct output, which makes interpolation between them meaningful. In contrast, applying soft parallel decoding to a standard diffusion language model without OPUT leads to catastrophic performance collapse.

Experiments

4.1

Experimental Setup

Implementation Details. We build our method on LLaDA-2.0-mini [10], a state-of-the-art opensource diffusion language model. During training, we use OPUT with a fixed mask ratio of 0.75. We perform full-parameter fine-tuning for 2 epochs with a batch size of 8, an initial learning rate of 2 × 10−6 , and a cosine learning rate schedule. Training follows the block-diffusion setting with a block size of 32. To avoid extra memory overhead, the masked noisy sequence and the predicted noisy sequence are optimized in separate iterations within the same epoch, rather than jointly in a single iteration. Under this setup, we train two models: DMax-Math for mathematical reasoning and DMax-Coder for code generation tasks. All training runs are conducted on 8 H200 GPUs. At inference time, we adopt the proposed SPD decoding strategy under semi-autoregressive block diffusion with a block size of 32. The acceptance threshold for determining whether a block has converged to a stable state is set to τacc = 0.9. 6

Table 1: Comparison with the original model and different baselines. For our DMax-Math model, we set the decoding threshold to 0.5; for the DMax-Coder model, we set it to 0.65. In addition to TPF, TPS, and accuracy, we also report the AUP score to provide a more comprehensive evaluation of parallel decoding performance. All evaluations are under zero-shot and a batch size of 1. Benchmark

TPF ↑

Method

TPS ↑

Acc. ↑

AUP Score ↑

Math & Reasoning Benchmarks

GSM8K

LLaDA-2.0-mini Hierarchical Decoding dParallel SFT Uniform Diffusion Training DMax-Math

2.04 2.44 2.79 2.26 5.48

512 577 721 493 1258

92.6% 91.6% 92.3% 68.7% 92.1%

340 357 395 0 557

MATH500

LLaDA-2.0-mini Hierarchical Decoding dParallel SFT Uniform Diffusion Training DMax-Math

2.58 3.01 3.42 2.43 5.94

626 669 823 530 1286

75.8% 73.0% 75.8% 33.6% 75.4%

257 268 310 0 507

Minerva-Algebra

LLaDA-2.0-mini Hierarchical Decoding dParallel SFT Uniform Diffusion Training DMax-Math

3.01 3.40 3.91 2.55 7.03

755 787 943 551 1492

91.4% 90.6% 91.4% 42.7% 91.5%

363 382 430 0 658

ASDIV

LLaDA-2.0-mini Hierarchical Decoding dParallel SFT Uniform Diffusion Training DMax-Math

2.03 2.43 2.72 2.51 5.62

512 528 663 515 1172

92.8% 92.5% 93.0% 80.8% 92.5%

354 366 459 0 556

Code Generation Benchmarks

HumanEval-Instruct

LLaDA-2.0-mini Hierarchical Decoding dParallel SFT Uniform Diffusion Training DMax-Coder

4.38 4.67 5.12 2.93 7.36

1044 1014 1229 628 1557

84.2% 81.1% 76.8% 15.2% 83.5%

369 379 394 0 637

MBPP-Instruct

LLaDA-2.0-mini Hierarchical Decoding dParallel SFT Uniform Diffusion Training DMax-Coder

2.71 2.88 3.66 2.84 5.86

662 685 880 608 1264

80.6% 76.6% 74.7% 23.4% 79.2%

276 241 273 0 482

Training Data. We construct all training data through self-distillation. Specifically, we take prompts from public datasets and use LLaDA-2.0-mini to generate responses as training targets. For math, prompts are collected from GSM8K trainset [20], PRM12K [44], a subset of Numina-Math [40], and a subset of OpenThoughts [26]. For code, prompts are drawn from a subset of OpenCodeInstruct [2]. Responses are generated with a confidence threshold of 0.95, a block size of 32, and a maximum generation length of 2048 tokens. We discard incomplete generations that do not finish within the length budget. This yields 0.7M math samples and 1.0M code samples. Notably, we do not use any external high-quality responses; all supervision is obtained from the model’s own generations. Evaluation Details. We evaluate our method on multiple benchmarks. For mathematical reasoning, we use GSM8K [20], MATH500 [44], Minerva-Algebra [29], and ASDIV [55], and prompt the model to produce chain-of-thought [80] reasoning. For code generation, we use the instruction versions of HumanEval [13] and MBPP [5]. All evaluations are conducted with the dInFer [54] framework on 2 H200 GPUs using tensor parallelism. Besides TPF, TPS, and accuracy, we also report AUP Score [62] to measure parallel decoding performance. The generation length for all benchmarks is 2048. Baselines. We compare our method against four baselines in terms of both decoding efficiency and generation accuracy: (1) LLaDA-2.0-mini, the base model, evaluated with its default confidencethreshold-based parallel decoding strategy using a threshold of 0.95; (2) Hierarchical Decoding, an advanced inference strategy that improves parallel decoding via a divide-and-conquer procedure [61]. The low threshold is set as 0.2; (3) dParallel-SFT, for which we use the LLaDA-2.0-mini-CAP model [10], where the certainty-forcing loss proposed in dParallel [16] is incorporated into large-scale 7

(a) Acc-TPF Trade-off on GSM8K

(b) Acc-TPF Trade-off on MATH500

(d) Acc-TPF Trade-off on MBPP

Figure 4: Comparison of accuracy-TPF trade-off curves between original LLaDA-2.0-mini model and our method. We present curves on GSM8K, MATH500, HumanEval and MBPP benchmarks. Table 2: Our proposed new paradigm also improves the model’s accuracy at low parallelism. Benchmarks

LLaDA-2.0-mini

DMax

TPF ↑

Acc. ↑

TPF ↑

Acc. ↑

GSM8K MATH500 Minerva-Algebra ASDIV

2.04 2.58 3.01 2.03

92.6% 75.8% 91.4% 92.8%

3.54 (+1.50) 3.45 (+0.87) 4.96 (+1.95) 3.10 (+1.07)

93.4% (+0.8%) 78.0% (+2.2%) 93.6% (+2.2%) 93.5% (+0.7%)

HumanEval-Instruct MBPP-Instruct

4.38 2.71

84.2% 80.6%

4.58 (+0.20) 3.58 (+0.87)

87.2% (+3.0%) 83.4% (+2.8%)

supervised fine-tuning to improve decoding parallelism; and (4) Uniform Diffusion Training, which continues training the base model using the conventional UDLM objective. In addition to masked noisy sequences, this baseline also replaces tokens with random vocabulary samples to construct uniformly corrupted noisy sequences, while keeping all other training settings identical to those of DMax. During inference, it updates all tokens within a block at every step until convergence. 4.2

Experimental Results

Aggressive Parallelism While Preserving Accuracy. As shown in Table 1, compared with the original LLaDA-2.0-mini, our method substantially increases decoding parallelism, improving the average TPF from 2.8 to 6.2 while preserving the original accuracy. In contrast, the other baselines provide only limited gains in parallel decoding. This advantage is further reflected in the AUP Score, where DMax consistently outperforms both the original model and all baselines by a large margin. These results demonstrate that our paradigm enables a much stronger parallel decoding capability than conventional MDLMs. Moreover, on two H200 GPUs, our model achieves a practical inference throughput of over 1000 tokens per second. On-Policy Training as the Cornerstone. Table 1 also compares our method with conventional uniform diffusion training. The latter neither improves decoding speed nor preserves model quality, instead causing a noticeable performance drop. We find that this failure stems from the large mismatch between the randomly sampled noisy sequences used in training and the model’s actual decoding trajectories at inference time. Consequently, the model struggles to revise erroneous predictions while unnecessarily perturbing correct ones, resulting in unstable oscillations within each block. By contrast, our on-policy training samples noisy sequences from the model’s own outputs, effectively bridging this train–inference gap and substantially improving self-revision under parallel decoding. Superior Efficiency–Performance Trade-off. Figure 4 compares the accuracy–TPF trade-off curves of our method and the original model on GSM8K, MATH500, HumanEval, and MBPP. As TPF increases, the original model suffers a sharp accuracy drop, whereas our method maintains stable performance. For instance, on MATH500, at around 6.5 TPF, our method still retains over 71.6% accuracy, while the original model falls to 15.2%. The gap is even larger on code benchmarks: on MBPP, at a similar TPF, our method achieves 79.2%, whereas the original model drops to only 2.3%. This superior trade-off stems from the self-revision capability of our paradigm, which effectively mitigates error accumulation under aggressive parallel decoding. 8

Table 3: Ablation on different training and inference strategies with different decoding parallelism. Train On-Policy Rollout

Inference Contiguous Prefix

Hybrid Embedding

✓

✓ ✓ ✓ ✓

✓ ✓

τdec = 0.95

τdec = 0.50

τdec = 0.0

TPF ↑

Acc. ↑

TPF ↑

Acc. ↑

TPF ↑

Acc. ↑

2.04 1.04 2.95 2.85 3.25 3.00

92.6% 0.0% 92.6% 93.0% 92.8% 93.3%

4.47 1.73 5.14 5.28 5.64 5.48

78.0% 0.0% 90.1% 91.3% 91.4% 92.1%

7.86 5.39 5.89 5.98 6.01 6.01

0.9% 0.0% 68.2% 69.6% 90.4% 90.4%

Improved Performance at Low Parallelism. By enabling dLLMs to revise their own predictions, our method not only mitigates error accumulation under aggressive parallel decoding, but also improves performance in the low-parallelism regime. Through iterative re-evaluation of earlier predictions, the model can recover from reasoning errors that would otherwise remain on the original decoding path. As shown in Table 2, our method consistently improves accuracy by 0.8%–3.0% across multiple benchmarks at low parallelism. Importantly, these gains are obtained using only the model’s own generated responses as training data, without introducing any external supervision.

Ablation Study

Ablation Study on Training and Inference Strategies. Table 3 presents a comprehensive ablation study of both our training and inference designs. We compare different combinations of training and decoding strategies on GSM8K under three decoding thresholds, τdec ∈ {0.95, 0.5, 0.0}. On-policy rollout is the core of our training method. Even with OPUT alone, the model acquires the ability to revise its own errors, yielding substantial accuracy gains over the original model at τdec = 0.5 and 0.0. Our proposed SPD further improves robustness when many erroneous predictions emerge simultaneously, allowing the model to remain stable under highly parallel decoding and to preserve strong performance even in the extreme case of τdec = 0.0. The key ingredient of SPD is to use soft embeddings, rather than discrete tokens, as intermediate decoding states. Maintaining the non-masked region as a contiguous prefix further improves performance. Another important result is that OPUT is a prerequisite for SPD. As shown in Table 3, directly applying SPD to the original model causes generation to collapse. This is because OPUT trains the model to recover clean tokens from both mask tokens and predicted tokens, making interpolation between their embeddings a meaningful and effective input for denoising. Ablation Study on Convergence Table 4: Ablation study on block-level convergence criteria. Criteria. We further study in TaThe decoding threshold is set to 0.5. ble 4 how different block-level convergence criteria affect the efficiency– Convergence GSM8K MBPP performance trade-off. We consider Consistency Confidence TPF ↑ Acc. ↑ TPF ↑ Acc. ↑ two criteria: (1) consistency, where ✓ 5.13 92.1% 5.16 79.9% decoding is considered converged if ✓ 2.28 92.2% 3.36 80.1% the model produces the same top-1 ✓ ✓ 5.48 92.1% 5.86 79.2% prediction for the block in two consecutive steps; and (2) confidence, where decoding is considered converged if the confidence of every token in the block exceeds 0.9. As shown in Table 4, consistency serves as the primary convergence signal, with most blocks terminating once this condition is met. Adding the confidence criterion can further improve TPF by allowing decoding to stop before two consecutive identical predictions are observed, thereby saving the final forward pass. Importantly, neither criterion affects the accuracy.

Related Work

Diffusion Language Models. Diffusion models [31, 71] have become dominant in visual generation [64, 60, 65, 99], and recent work has explored their application to text generation. Among existing paradigms, masked diffusion language models (MDLMs) [70, 4, 66, 105, 51] have emerged as a promising alternative to AR-LLMs by modeling language in discrete space through masked token 9

prediction. Building on this formulation, LLaDA [58] and Dream [92] scale MDLMs to the billionparameter regime with large-scale pretraining, demonstrating their practical potential. LLaDA-2.0 [10] and LLaDA-MoE [110] further show that MDLMs can be effectively scaled with mixture-ofexperts architectures. Beyond these developments, dLLMs are also attracting increasing attention in reasoning [109, 59, 86, 63, 74, 57, 103], multimodal tasks [94, 96, 90, 91, 48, 82, 97, 18], code generation [87, 24, 21], long-context modeling [47, 28, 106], and agent [104, 102]. Accelerating Diffusion Language Models. dLLMs are viewed as promising due to their potential for low-cost inference, yet their efficiency remains largely underexplored. Existing efforts improve efficiency from several perspectives. Some methods reduce the cost of each decoding step through techniques including KV caching [53, 49, 84, 35, 45], token dropping [15, 36, 72, 85], and sparse attention [79, 19]. Others design more effective decoding strategies [37, 81, 41, 27, 8, 32, 34, 89, 50, 12, 77, 61, 22] to improve generation efficiency. A separate line of work [73, 62, 101, 7, 14, 33] learns better decoding trajectories so that fewer decoding steps are required. dParallel [16] employs certaintyforcing distillation to accelerate confidence convergence and enable higher parallel decoding. Other methods [83, 17, 78, 3, 52, 75, 46, 23] interpolate between diffusion and autoregressive language models to better balance speed and accuracy. [9, 76, 100] implement uniform training, which trains the model to recover clean tokens from random noisy tokens, thereby enabling token correction during generation. SM [30] and EvoToken [107] introduce soft embeddings into the decoding process, but neither method translates this design into improved decoding efficiency. Further efforts [88] leverage compression techniques to construct lightweight dLLMs.

Conclusion

In this paper, we present DMax, a novel paradigm for efficient diffusion language models that mitigates error accumulation for parallel decoding. DMax enables aggressive decoding parallelism while preserving the accuracy of the original model. We introduce two key components of our approach, namely On-Policy Uniform Training and Soft Parallel Decoding, and demonstrate their effectiveness through extensive experiments on diverse benchmarks. Our results establish a strong new baseline for parallel decoding in dLLMs and suggest a promising new direction for dLLMs.

References [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. [2] Wasi Uddin Ahmad, Aleksander Ficek, Mehrzad Samadi, Jocelyn Huang, Vahid Noroozi, Somshubra Majumdar, and Boris Ginsburg. Opencodeinstruct: A large-scale instruction tuning dataset for code llms. arXiv preprint arXiv:2504.04030, 2025. [3] Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov. Block diffusion: Interpolating between autoregressive and diffusion language models. arXiv preprint arXiv:2503.09573, 2025. [4] Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. Structured denoising diffusion models in discrete state-spaces. Advances in neural information processing systems, 34:17981–17993, 2021. [5] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. [6] Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. [7] Wenrui Bao, Zhiben Chen, Dan Xu, and Yuzhang Shang. Learning to parallel: Accelerating diffusion large language models via adaptive parallel decoding. In The Fourteenth International Conference on Learning Representations, 2025. [8] Heli Ben-Hamu, Itai Gat, Daniel Severo, Niklas Nolte, and Brian Karrer. Accelerated sampling from masked diffusion models via entropy bounded unmasking. arXiv preprint arXiv:2505.24857, 2025. [9] Tiwei Bie, Maosong Cao, Xiang Cao, Bingsen Chen, Fuyuan Chen, Kun Chen, Lun Du, Daozhuo Feng, Haibo Feng, Mingliang Gong, et al. Llada2. 1: Speeding up text diffusion via token editing. arXiv preprint arXiv:2602.08676, 2026. [10] Tiwei Bie, Maosong Cao, Kun Chen, Lun Du, Mingliang Gong, Zhuochen Gong, Yanmei Gu, Jiaqi Hu, Zenan Huang, Zhenzhong Lan, et al. Llada2. 0: Scaling up diffusion language models to 100b. arXiv preprint arXiv:2512.15745, 2025. [11] Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. Medusa: Simple llm inference acceleration framework with multiple decoding heads. arXiv preprint arXiv:2401.10774, 2024. [12] Jian Chen, Yesheng Liang, and Zhijian Liu. Dflash: Block diffusion for flash speculative decoding. arXiv preprint arXiv:2602.06036, 2026. [13] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. 2021. [14] Shirui Chen, Jiantao Jiao, Lillian J Ratliff, and Banghua Zhu. dultra: Ultra-fast diffusion language models via reinforcement learning. arXiv preprint arXiv:2512.21446, 2025. [15] Xinhua Chen, Sitao Huang, Cong Guo, Chiyue Wei, Yintao He, Jianyi Zhang, Hai Li, Yiran Chen, et al. Dpad: Efficient diffusion language models with suffix dropout. arXiv preprint arXiv:2508.14148, 2025. [16] Zigeng Chen, Gongfan Fang, Xinyin Ma, Ruonan Yu, and Xinchao Wang. dparallel: Learnable parallel decoding for dllms. arXiv preprint arXiv:2509.26488, 2025.

[17] Shuang Cheng, Yihan Bian, Dawei Liu, Linfeng Zhang, Qian Yao, Zhongbo Tian, Wenhai Wang, Qipeng Guo, Kai Chen, Biqing Qi, et al. Sdar: A synergistic diffusion-autoregression paradigm for scalable sequence generation. arXiv preprint arXiv:2510.06303, 2025. [18] Shuang Cheng, Yuhua Jiang, Zineng Zhou, Dawei Liu, Wang Tao, Linfeng Zhang, Biqing Qi, and Bowen Zhou. Sdar-vl: Stable and efficient block-wise diffusion for vision-language understanding. arXiv preprint arXiv:2512.14068, 2025. [19] Alexandros Christoforos and Chadbourne Davis. Moe-diffuseq: Enhancing long-document diffusion models with sparse attention and mixture of experts. arXiv preprint arXiv:2512.20604, 2025. [20] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [21] Chenghao Fan, Wen Heng, Bo Li, Sichen Liu, Yuxuan Song, Jing Su, Xiaoye Qu, Kai Shen, and Wei Wei. Stable-diffcoder: Pushing the frontier of code diffusion large language model. arXiv preprint arXiv:2601.15892, 2026. [22] Sicheng Feng, Zigeng Chen, Xinyin Ma, Gongfan Fang, and Xinchao Wang. dvoting: Fast voting for dllms. arXiv preprint arXiv:2602.12153, 2026. [23] Yonggan Fu, Lexington Whalen, Zhifan Ye, Xin Dong, Shizhe Diao, Jingyu Liu, Chengyue Wu, Hao Zhang, Enze Xie, Song Han, et al. Efficient-dlm: From autoregressive to diffusion language models, and beyond in speed. arXiv preprint arXiv:2512.14067, 2025. [24] Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. Diffucoder: Understanding and improving masked diffusion models for code generation. arXiv preprint arXiv:2506.20639, 2025. [25] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [26] Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, et al. Openthoughts: Data recipes for reasoning models. arXiv preprint arXiv:2506.04178, 2025. [27] Daehoon Gwak, Minseo Jung, Junwoo Park, Minho Park, ChaeHun Park, Junha Hyung, and Jaegul Choo. Reward-weighted sampling: Enhancing non-autoregressive characteristics in masked diffusion llms. arXiv preprint arXiv:2509.00707, 2025. [28] Guangxin He, Shen Nie, Fengqi Zhu, Yuankang Zhao, Tianyi Bai, Ran Yan, Jie Fu, Chongxuan Li, and Binhang Yuan. Ultrallada: Scaling the context length to 128k for diffusion large language models. arXiv preprint arXiv:2510.10481, 2025. [29] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. [30] Michael Hersche, Samuel Moor-Smith, Thomas Hofmann, and Abbas Rahimi. Soft-masked diffusion language models. arXiv preprint arXiv:2510.17206, 2025. [31] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. [32] Feng Hong, Geng Yu, Yushi Ye, Haicheng Huang, Huangjie Zheng, Ya Zhang, Yanfeng Wang, and Jiangchao Yao. Wide-in, narrow-out: Revokable decoding for efficient and effective dllms. arXiv preprint arXiv:2507.18578, 2025. [33] Yanzhe Hu, Yijie Jin, Pengfei Liu, Kai Yu, and Zhijie Deng. Lightningrl: Breaking the accuracyparallelism trade-off of block-wise dllms via reinforcement learning. arXiv preprint arXiv:2603.13319, 2026. [34] Yuezhou Hu, Harman Singh, Monishwaran Maheswaran, Haocheng Xi, Coleman Hooper, Jintao Zhang, Aditya Tomar, Michael W Mahoney, Sewon Min, Mehrdad Farajtabar, et al. Residual context diffusion language models. arXiv preprint arXiv:2601.22954, 2026.

[35] Zhanqiu Hu, Jian Meng, Yash Akhauri, Mohamed S Abdelfattah, Jae-sun Seo, Zhiru Zhang, and Udit Gupta. Accelerating diffusion language model inference via efficient kv caching and guided diffusion. arXiv preprint arXiv:2505.21467, 2025. [36] Jianuo Huang, Yaojie Zhang, Yicun Yang, Benhao Huang, Biqing Qi, Dongrui Liu, and Linfeng Zhang. Mask tokens as prophet: Fine-grained cache eviction for efficient dllm inference. arXiv preprint arXiv:2510.09309, 2025. [37] Daniel Israel, Guy Van den Broeck, and Aditya Grover. Accelerating diffusion llms via adaptive parallel decoding. arXiv preprint arXiv:2506.00413, 2025. [38] Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman Singh, Ben Athiwaratkun, Ce Zhang, Kurt Keutzer, and Amir Gholami. Cdlm: Consistency diffusion language models for faster sampling. arXiv preprint arXiv:2511.19269, 2025. [39] Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pages 19274–19286. PMLR, 2023. [40] Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9):9, 2024. [41] Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, and Dahua Lin. Beyond fixed: Variable-length denoising for diffusion large language models. arXiv e-prints, pages arXiv–2508, 2025. [42] Tianyi Li, Mingda Chen, Bowei Guo, and Zhiqiang Shen. A survey on diffusion language models. arXiv preprint arXiv:2508.10875, 2025. [43] Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle: Speculative sampling requires rethinking feature uncertainty. arXiv preprint arXiv:2401.15077, 2024. [44] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, 2023. [45] Aiwei Liu, Minghua He, Shaoxun Zeng, Sijun Zhang, Linhao Zhang, Chuhan Wu, Wei Jia, Yuan Liu, Xiao Zhou, and Jie Zhou. Wedlm: Reconciling diffusion language models with standard causal attention for fast inference. arXiv preprint arXiv:2512.22737, 2025. [46] Jingyu Liu, Xin Dong, Zhifan Ye, Rishabh Mehta, Yonggan Fu, Vartika Singh, Jan Kautz, Ce Zhang, and Pavlo Molchanov. Tidar: Think in diffusion, talk in autoregression. arXiv preprint arXiv:2511.08923, 2025. [47] Xiaoran Liu, Yuerong Song, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, and Xipeng Qiu. Longllada: Unlocking long context capabilities in diffusion llms. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 32186–32194, 2026. [48] Yang Liu, Pengxiang Ding, Tengyue Jiang, Xudong Wang, Wenxuan Song, Minghui Lin, Han Zhao, Hongyin Zhang, Zifeng Zhuang, Wei Zhao, et al. Mmada-vla: Large diffusion vision-language-action model with unified multi-modal instruction and generation. arXiv preprint arXiv:2603.25406, 2026. [49] Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint arXiv:2506.06295, 2025. [50] Lingkun Long, Yushi Huang, Shihao Bai, Ruihao Gong, Jun Zhang, Ao Zhou, and Jianlei Yang. Focusdllm: Accelerating long-context diffusion llm inference via confidence-guided context focusing. arXiv preprint arXiv:2602.02159, 2026. [51] Aaron Lou, Chenlin Meng, and Stefano Ermon. Discrete diffusion language modeling by estimating the ratios of the data distribution. 2023. [52] Linrui Ma, Yufei Cui, Kai Han, and Yunhe Wang. Diffusion in diffusion: Breaking the autoregressive bottleneck in block diffusion models. arXiv preprint arXiv:2601.13599, 2026. [53] Xinyin Ma, Runpeng Yu, Gongfan Fang, and Xinchao Wang. dkv-cache: The cache for diffusion language models. arXiv preprint arXiv:2505.15781, 2025.

[54] Yuxin Ma, Lun Du, Lanning Wei, Kun Chen, Qian Xu, Kangyu Wang, Guofeng Feng, Guoshan Lu, Lin Liu, Xiaojing Qi, et al. dinfer: An efficient inference framework for diffusion language models. arXiv preprint arXiv:2510.08666, 2025. [55] Shen-Yun Miao, Chao-Chun Liang, and Keh-Yih Su. A diverse corpus for evaluating and developing english math word problem solvers, 2021. [56] Jinjie Ni, Qian Liu, Longxu Dou, Chao Du, Zili Wang, Hang Yan, Tianyu Pang, and Michael Qizhe Shieh. Diffusion language models are super data learners. arXiv preprint arXiv:2511.03276, 2025. [57] Zanlin Ni, Shenzhi Wang, Yang Yue, Tianyu Yu, Weilin Zhao, Yeguo Hua, Tianyi Chen, Jun Song, Cheng Yu, Bo Zheng, et al. The flexibility trap: Why arbitrary order limits reasoning potential in diffusion language models. arXiv preprint arXiv:2601.15165, 2026. [58] Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. Large language diffusion models. arXiv preprint arXiv:2502.09992, 2025. [59] Leyi Pan, Shuchang Tao, Yunpeng Zhai, Zheyu Fu, Liancheng Fang, Minghua He, Lingzhe Zhang, Zhaoyang Liu, Bolin Ding, Aiwei Liu, et al. d-treerpo: Towards more reliable policy optimization for diffusion language models. arXiv preprint arXiv:2512.09675, 2025. [60] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. [61] Xiaojing Qi, Lun Du, Xinyuan Zhang, Lanning Wei, Tao Jin, and Da Zheng. Hierarchy decoding: A training-free parallel decoding strategy for diffusion large language models. In The Fourteenth International Conference on Learning Representations. [62] Yu-Yang Qian, Junda Su, Lanxiang Hu, Peiyuan Zhang, Zhijie Deng, Peng Zhao, and Hao Zhang. d3llm: Ultra-fast diffusion llm using pseudo-trajectory distillation. arXiv preprint arXiv:2601.07568, 2026. [63] Kevin Rojas, Jiahe Lin, Kashif Rasul, Anderson Schneider, Yuriy Nevmyvaka, Molei Tao, and Wei Deng. Improving reasoning for diffusion language models via group diffusion policy optimization. arXiv preprint arXiv:2510.08554, 2025. [64] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. [65] Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22500–22510, 2023. [66] Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. Simple and effective masked diffusion language models. Advances in Neural Information Processing Systems, 37:130136–130184, 2024. [67] Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin Chiu, and Volodymyr Kuleshov. The diffusion duality. arXiv preprint arXiv:2506.10892, 2025. [68] Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Alexander Rush, Volodymyr Kuleshov, Hugo Dalla-Torre, Sam Boshar, Bernardo P de Almeida, and Thomas Pierrot. Simple guidance mechanisms for discrete diffusion models. In ... International Conference on Learning Representations, volume 2025, page 44153, 2025. [69] Subham Sekhar Sahoo, Jean-Marie Lemercier, Zhihan Yang, Justin Deschenaux, Jingyu Liu, John Thickstun, and Ante Jukic. Scaling beyond masked diffusion language models. arXiv e-prints, pages arXiv–2602, 2026. [70] Jiaxin Shi, Kehang Han, Zhe Wang, Arnaud Doucet, and Michalis Titsias. Simplified and generalized masked diffusion for discrete data. Advances in neural information processing systems, 37:103131– 103167, 2024. [71] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. [72] Yuerong Song, Xiaoran Liu, Ruixiao Li, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, and Xipeng Qiu. Sparse-dllm: Accelerating diffusion llms with dynamic cache eviction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 33038–33046, 2026.

[73] Yuxuan Song, Zheng Zhang, Cheng Luo, Pengyang Gao, Fan Xia, Hao Luo, Zheng Li, Yuehang Yang, Hongli Yu, Xingwei Qu, et al. Seed diffusion: A large-scale diffusion language model with high-speed inference. arXiv preprint arXiv:2508.02193, 2025. [74] Xiaohang Tang, Rares Dolga, Sangwoong Yoon, and Ilija Bogunovic. wd1: Weighted policy optimization for reasoning in diffusion language models. arXiv preprint arXiv:2507.08838, 2025. [75] Yuchuan Tian, Yuchen Liang, Shuo Zhang, Yingte Shu, Guangwen Yang, Wei He, Sibo Fang, Tianyu Guo, Kai Han, Chao Xu, et al. From next-token to next-block: A principled adaptation path for diffusion llms. arXiv preprint arXiv:2512.06776, 2025. [76] Dimitri Von Rütte, Janis Fluri, Yuhui Ding, Antonio Orvieto, Bernhard Schölkopf, and Thomas Hofmann. Generalized interpolating discrete diffusion. arXiv preprint arXiv:2503.04482, 2025. [77] Kangyu Wang, Zhiyun Jiang, Haibo Feng, Weijia Zhao, Lin Liu, Jianguo Li, Zhenzhong Lan, and Weiyao Lin. Creditdecoding: Accelerating parallel decoding in diffusion large language models with trace credits. arXiv preprint arXiv:2510.06133, 2025. [78] Xu Wang, Chenkai Xu, Yijie Jin, Jiachun Jin, Hao Zhang, and Zhijie Deng. Diffusion llms can do faster-than-ar inference via discrete diffusion forcing. arXiv preprint arXiv:2508.09192, 2025. [79] Zeqing Wang, Gongfan Fang, Xinyin Ma, Xingyi Yang, and Xinchao Wang. Sparsed: Sparse attention for diffusion language models. In The Fourteenth International Conference on Learning Representations, 2026. [80] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. [81] Qingyan Wei, Yaojie Zhang, Zhiyuan Liu, Dongrui Liu, and Linfeng Zhang. Accelerating diffusion large language models with slowfast: The three golden principles. arXiv preprint arXiv:2506.10848, 2025. [82] Yuqing Wen, Hebei Li, Kefan Gu, Yucheng Zhao, Tiancai Wang, and Xiaoyan Sun. Llada-vla: Vision language diffusion action models. arXiv preprint arXiv:2509.06932, 2025. [83] Chengyue Wu, Hao Zhang, Shuchen Xue, Shizhe Diao, Yonggan Fu, Zhijian Liu, Pavlo Molchanov, Ping Luo, Song Han, and Enze Xie. Fast-dllm v2: Efficient block-diffusion llm. arXiv preprint arXiv:2509.26328, 2025. [84] Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. arXiv preprint arXiv:2505.22618, 2025. [85] Zhongyu Xiao, Zhiwei Hao, Jianyuan Guo, Yong Luo, Jia Liu, Jie Xu, and Han Hu. Streaming-dllm: Accelerating diffusion llms via suffix pruning and dynamic decoding. arXiv e-prints, pages arXiv–2601, 2026. [86] Shaoan Xie, Lingjing Kong, Xiangchen Song, Xinshuai Dong, Guangyi Chen, Eric P Xing, and Kun Zhang. Step-aware policy optimization for reasoning in diffusion large language models. arXiv preprint arXiv:2510.01544, 2025. [87] Zhihui Xie, Jiacheng Ye, Lin Zheng, Jiahui Gao, Jingwei Dong, Zirui Wu, Xueliang Zhao, Shansan Gong, Xin Jiang, Zhenguo Li, et al. Dream-coder 7b: An open diffusion language model for code. arXiv preprint arXiv:2509.01142, 2025. [88] Chen Xu and Dawei Yang. Dllmquant: Quantizing diffusion-based large language models. arXiv preprint arXiv:2508.14090, 2025. [89] Chenkai Xu, Yijie Jin, Jiajun Li, Yi Tu, Guoping Long, Dandan Tu, Mingcong Song, Hongjie Si, Tianqi Hou, Junchi Yan, et al. Lopa: Scaling dllm inference via lookahead parallel decoding. arXiv preprint arXiv:2512.16229, 2025. [90] Ling Yang, Ye Tian, Bowen Li, Xinchen Zhang, Ke Shen, Yunhai Tong, and Mengdi Wang. Mmada: Multimodal large diffusion language models. arXiv preprint arXiv:2505.15809, 2025. [91] Jiacheng Ye, Shansan Gong, Jiahui Gao, Junming Fan, Shuang Wu, Wei Bi, Haoli Bai, Lifeng Shang, and Lingpeng Kong. Dream-vl & dream-vla: Open vision-language and vision-language-action models with diffusion language model backbone. arXiv preprint arXiv:2512.22615, 2025.

[92] Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508.15487, 2025. [93] Qiuhua Yi, Xiangfan Chen, Chenwei Zhang, Zehai Zhou, Linan Zhu, and Xiangjie Kong. Diffusion models in text generation: a survey. PeerJ Computer Science, 10:e1905, 2024. [94] Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Zhiwu Lu, Ji-Rong Wen, and Chongxuan Li. Llada-v: Large language diffusion models with visual instruction tuning. arXiv preprint arXiv:2505.16933, 2025. [95] Runpeng Yu, Qi Li, and Xinchao Wang. Discrete diffusion in large language and multimodal models: A survey. arXiv preprint arXiv:2506.13759, 2025. [96] Runpeng Yu, Xinyin Ma, and Xinchao Wang. Dimple: Discrete diffusion multimodal large language model with parallel decoding. arXiv preprint arXiv:2505.16990, 2025. [97] Lunbin Zeng, Jingfeng Yao, Bencheng Liao, Hongyuan Tao, Wenyu Liu, and Xinggang Wang. Diffusionvl: Translating any autoregressive models into diffusion vision language models. arXiv preprint arXiv:2512.15713, 2025. [98] Lingzhe Zhang, Liancheng Fang, Chiming Duan, Minghua He, Leyi Pan, Pei Xiao, Shiyu Huang, Yunpeng Zhai, Xuming Hu, Philip S Yu, et al. A survey on parallel text generation: From parallel decoding to diffusion language models. arXiv preprint arXiv:2508.08712, 2025. [99] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. [100] Shuibai Zhang, Fred Zhangzhi Peng, Yiheng Zhang, Jin Pan, and Grigorios G Chrysos. Corrective diffusion language models. arXiv preprint arXiv:2512.15596, 2025. [101] Tunyu Zhang, Xinxi Zhang, Ligong Han, Haizhou Shi, Xiaoxiao He, Zhuowei Li, Hao Wang, Kai Xu, Akash Srivastava, Vladimir Pavlovic, et al. T3d: Few-step diffusion language models via trajectory self-distillation with direct discriminative optimization. arXiv preprint arXiv:2602.12262, 2026. [102] Jiahao Zhao, Shaoxuan Xu, Zhongxiang Sun, Fengqi Zhu, Jingyang Ou, Yuling Shi, Chongxuan Li, Xiao Zhang, and Jun Xu. Dllm-searcher: Adapting diffusion large language model for search agents. arXiv preprint arXiv:2602.07035, 2026. [103] Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. d1: Scaling reasoning in diffusion large language models via reinforcement learning. arXiv preprint arXiv:2504.12216, 2025. [104] Huiling Zhen, Weizhe Lin, Renxi Liu, Kai Han, Yiming Li, Yuchuan Tian, Hanting Chen, Xiaoguang Li, Xiaosong Li, Chen Chen, et al. Dllm agent: See farther, run faster. arXiv preprint arXiv:2602.07451, 2026. [105] Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling. arXiv preprint arXiv:2409.02908, 2024. [106] Liang Zheng, Bowen Shi, Yitao Hu, Jiawei Zhang, Ruofan Li, Sheng Chen, Wenxin Li, and Keqiu Li. Mosaic: Unlocking long-context inference for diffusion llms via global memory planning and dynamic peak taming. arXiv preprint arXiv:2601.06562, 2026. [107] Linhao Zhong, Linyu Wu, Bozhen Fang, Tianjian Feng, Chenchen Jing, Wen Wang, Jiaheng Zhang, Hao Chen, and Chunhua Shen. Beyond hard masks: Progressive token evolution for diffusion language models. arXiv preprint arXiv:2601.07351, 2026. [108] Zhanhui Zhou, Lingjie Chen, Hanghang Tong, and Dawn Song. dllm: Simple diffusion language modeling. arXiv preprint arXiv:2602.22661, 2026. [109] Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, et al. Llada 1.5: Variance-reduced preference optimization for large language diffusion models. arXiv preprint arXiv:2505.19223, 2025. [110] Fengqi Zhu, Zebin You, Yipeng Xing, Zenan Huang, Lin Liu, Yihong Zhuang, Guoshan Lu, Kangyu Wang, Xudong Wang, Lanning Wei, et al. Llada-moe: A sparse moe diffusion language model. arXiv preprint arXiv:2509.24389, 2025.

Record · ID 2609 · SHA-256 3a80552c15cd33a5
Conceptio Open Knowledge Archive — every document is proof-bundled with source, license, and retrieval metadata.