arXiv CS open access

Measured Coupling for Parallel Decoding in Diffusion Language Models

Tal Oved et al. · arxiv_cs

arXiv CS · Papers

cloud distributed-computing parallel-computing

Diffusion language models can write many tokens per step, but must choose which tokens are safe to commit together. Mean-field decoding penalizes pairs of positions whose predictions look alike. We replace this proxy with the coupling itself: how much committing one token changes the model's confide

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Measured Coupling for Parallel Decoding in Diffusion Language Models Tal Oved Tel Aviv University

Lior Wolf Tel Aviv University

arXiv:2611.08307v1 [cs.CL] 5 Nov 2026

Abstract Diffusion language models can write many tokens per step, but must choose which tokens are safe to commit together. Mean-field decoding penalizes pairs of positions whose predictions look alike. We replace this proxy with the coupling itself: how much committing one token changes the model’s confidence at another. Cheap probes inside the main forward pass measure it, with no training or extra model, and it plugs into the same mean-field rule. On three models and four benchmarks, our method needs far fewer steps than mean-field decoding and is faster in most settings, also with a key-value cache, at an average accuracy cost of 1.1 points. Shuffling the couplings or replacing them with a uniform bonus loses this gain.

Introduction

Large language models (Achiam et al., 2023; Bai et al., 2023; Comanici et al., 2025; Liu et al., 2024) write text one token at a time. Each new token needs a full forward pass, so the time to answer grows with the length of the answer. Discrete diffusion language models (Austin et al., 2021a; Li et al., 2022; Nie et al., 2026; Zhu et al., 2026; Ye et al., 2025) offer a parallel alternative. They treat generation as iterative denoising: the answer starts fully masked, and every step can fill in several positions at once. Recent models scale this idea to strong, large systems (Bie et al., 2025; 2026), which makes fast parallel decoding a practical goal. The hard part is deciding which tokens to commit together. A diffusion model predicts every masked position independently, from the same context. Two tokens can each look certain on their own and still form a wrong pair, because neither was predicted with the other in view (Ringel et al., 2026; Luo et al., 2026; Zhang et al., 2026). Confidence thresholds commit only positions whose top prediction is very likely (Wu et al., 2026). Dependency-aware decoders estimate which positions interact, from attention maps (Luo et al., 2026; Kim et al., 2026) or with a small trained predictor (Ringel et al., 2026). Mean-field parallel decoding (Zoabi et al., 2026) takes a training-free view based on an energy model. Each masked position gets a commit score, a pairwise term penalizes committing two positions whose predictive distributions are similar, measured by the Jensen-Shannon divergence, and a few mean-field updates decide which positions to commit. In this work, we tackle the question of what pairwise quantity should this energy use. The quantity that matters is the effect of committing token j on the confidence of position i. We call it the coupling ∆ij (Figure 1). It can be positive, when j makes i less certain, or negative, when j supports i. Two positions with similar distributions may not affect each other at all, and two positions with different distributions may depend strongly on each other, for example the digits of one number. The coupling can be measured with the model itself: reveal j, run the model, and read the new confidence at i. Done naively, this costs one forward pass per candidate. We make it cheap. A probe only needs to recompute the current block. The rest of the sequence reuses the keys and values from the step’s main forward pass. This approximation is accurate (Pearson 0.96 to 0.97 with exact probes) and reduces the work of a probe from the whole sequence to the masked positions of one block. We then run the probes inside the main forward pass of each step, so the model weights are read once for both.

Previous: Similarity Proxy

Ours: Measured Coupling Support (Negative Δ)

Token j Token A

Token B

Token i

Conflict (Positive Δ)

Measures how much revealing j changes the margin of i

Penalized for distribution overlap

Block-Local Probes Architecture (Data Flow) One forward pass: main tokens + probe rows

Sequence Evolution & Indices

Step 0

x1 (Pos j)

x3 (Pos i)

[MASK]

Main Attention

Input Base Margin ci

Calculate Coupling:

Main

The

[MASK]

quick

[MASK]

fox

Probe

main keys/values (same layer)

The (Cached)

brown (Reveal vj)

quick (Cached)

[MASK] (Read ci(j))

fox (Cached)

Δij = ci − ci(j) Mean-Field Update:

qi ← σ(ci − Σj Δij qj)

Step t

Calculate coupling between j and i

Step t + 1

Probe Attention (Recomputes masked positions only)

Input Probe Margin ci(j)

Commits S = {i : qi ≥ τ} and Updates Sequence for Next Iteration

Figure 1: Overview of measured coupling. Top: mean-field decoding penalizes two positions whose predicted distributions overlap (left). We instead measure how much revealing token j changes the margin of token i, which shows support as well as conflict (right). Bottom: one decoding step. The main tokens and K probe rows, one for each of the most confident candidates, run in a single forward pass. Each probe row reveals one candidate and recomputes only the masked positions, reading the main keys and values of the same layer everywhere else. The resulting margins give the coupling (j) ∆ij = ci − ci , which enters the mean-field update, and positions with qi ≥ τ are committed for the next step.

Our contributions are: • Measured coupling. We define the pairwise term of parallel decoding as the change in log-margin caused by committing another token, including support as well as conflict. • Fused block-local probes. A probe recomputes only the masked positions of the current block against the keys and values of the main pass, and all probes run inside that same pass. This makes measuring the coupling cheap enough to pay off in wall-clock time. • A reading of the mean-field rule. As an interpretation, under an additive model of conditional margins the mean-field fixed point with measured coupling equals the sigmoid of each token’s expected margin given the others (Proposition 1), and a standard contraction condition guarantees a unique fixed point (Proposition 2). • Results. On LLaDA-8B, LLaDA-1.5, and Dream, measured coupling needs the fewest steps on every benchmark and has the shortest time per answer in most of them. Its accuracy is 1.1 points below mean-field decoding on average and ranges from 6.1 points lower to 1.8 points higher; the drop is significant on MATH for two of the three models, and the largest drops are on Dream HumanEval and LLaDA-1.5 MBPP.

Background

Decoding a masked diffusion model. The sequence x holds a prompt followed by masked positions. Decoding fills the answer block by block. At each step the model reads x and returns a distribution πi for every masked position i in the current block. Let U be these positions. For each i ∈ U we take the most likely token vi = arg max πi and its log-margin ci = log πi (vi ) − max log πi (v). v̸=vi

(1)

The decoder commits a set S ⊆ U , writes vS into x, and repeats until the block is full. The number of decoding steps, called function evaluations (NFE), sets the cost. Mean-field parallel decoding. Zoabi et al. (2026) attach a binary variable si to each i ∈ U , with si = 1 meaning commit. They define the energy X X E(s) = − ci si + Dij si sj , (2) i

i<j

where Dij is one minus the normalized Jensen-Shannon divergence between πi and πj . A factorized distribution q with qi = P (si = 1) gives the mean-field update X qi ← σ ci − Dij qj , (3) j̸=i

run for two iterations from q = σ(c). Positions with qi ≥ τ are committed. This rule is training-free and fast, and it is the method we build on.

Measured Coupling

3.1

The coupling (j)

Definition 1 (Coupling). For positions i, j ∈ U , let ci be the log-margin of the same token vi at position i after revealing xj = vj . The coupling of j on i is (j)

∆ij = ci − ci .

(4)

A positive ∆ij means that committing j lowers the confidence at i, so committing both together is risky. A negative ∆ij means that j supports i: once j is known, i becomes more certain. The coupling is directional, since the effect of j on i can differ from the effect of i on j. We keep its sign. Support is what allows many tokens to be committed in one step. We plug the coupling into the same update as (3): X qi ← σ ci − ∆ij qj .

(5)

j̸=i

3.2

What the update computes

The next result gives a simple reading of this update. Write ci (xS ) for the log-margin of vi at position i after revealing the tokens vS . Proposition 1 (Expected margin). Assume that conditional margins are additive in the revealed tokens, X ci (xS ) = ci − ∆ij for all S ⊆ U \ {i}. (6) j∈S

Let the other positions be committed with probabilities Pr(sj = 1) = qj , independently or not. Then the expected margin of i is X Eq ci (xS ) = ci − ∆ij qj , (7) j̸=i

and every fixed point of (5) satisfies qi = σ Eq [ci (xS )] for all i. P Proof. Under (6), ci (xS ) = ci − j̸=i ∆ij sj . Taking expectations and using E[sj ] = qj gives the P first claim, by linearity of expectation. A fixed point of (5) satisfies qi = σ(ci − j ∆ij qj ), which is the sigmoid of this expectation. 3

In words, each token’s commit probability becomes a function of how confident it is expected to be once the other likely commits are in place. With ∆ in place of D, the mean-field rule of Zoabi et al. (2026) gains this direct reading. The similarity matrix D can be seen as a cheap stand-in for ∆. The proposition interprets the update; it does not show that the committed set is safe. Since ∆ is not symmetric, the update is a fixed-point rule with the mean-field form, and no longer solves a pairwise energy. The value σ(ci ) compares only the top two tokens, so it is not a calibrated probability of being correct, and σ(E[ci ]) differs from E[σ(ci )]. The proposition also averages over random commits, while the decoder commits every position above τ after two iterations. Assumption (6) is a first-order model, and our probes give estimates of each ∆ij (Section 3.4), so additivity and estimation error both enter the final decision. We test additivity directly in Appendix J. For the sets the decoder commits, the additive prediction and the jointly measured margin fall on the same side of τ for 90% to 93% of the remaining positions, and for 95% to 99% when two or three tokens are committed. The sum overstates both support and conflict, and more so for large sets. 3.3

When the update has a unique answer

P P Proposition 2 (Contraction). Let F (q)i = σ ci − j̸=i ∆ij qj and ∥∆∥∞ = maxi j̸=i |∆ij |. If ∥∆∥∞ < 4, then F is a contraction in the ℓ∞ norm. It has a unique fixed point, and the iteration q ← F (q) converges to it geometrically from any start. Proof. satisfies |σ ′ (z)| ≤ 1/4. By the mean value theorem, |F (q)i − F (q ′ )i | ≤ P The sigmoid 1 1 ′ ′ |∆ | |q − q | ≤ ij j j j 4 4 ∥∆∥∞ ∥q − q ∥∞ . The Banach fixed point theorem completes the proof. When couplings are large, synchronous updates can swing between states. A simple example is two strongly conflicting tokens that rise together, fall together, and rise again. This explains why the number of mean-field iterations matters in practice (Zoabi et al., 2026). Like mean-field decoding, we run the update for two iterations at every decoding step. In practice the measured couplings are strong, and the contraction condition holds in about a quarter of the decoding steps. The condition is therefore sufficient and rarely met in practice. Running the update for 100 iterations changes the commit decision in about one step in seven. When the update oscillates, two iterations need not be close to a fixed point. 3.4

Cheap block-local probes

Measuring ∆ij directly needs one forward pass per candidate j, over the full sequence. We reduce this cost with the key-value cache. The main forward pass of a step already computes keys and values for every position. A probe for candidate j writes vj at position j and recomputes only the positions of the current block that are still masked. All other positions, inside or outside the block, reuse the keys and values of the main pass. The probes for all candidates run as one batch. This is the same approximation that Fast-dLLM uses for its dual cache (Wu et al., 2026). Here it is applied to probes. It treats the representations of all other positions as fixed while j is revealed. Section 4.6 shows that the resulting couplings agree closely with exact probes. Probes inside the main pass. A separate probe call costs a fixed amount, mostly for reading the model weights, even when it holds few tokens. We remove this cost by running the probes inside the main forward pass of the next step. All token-wise layers (normalization, projections, MLP, and output head) process the main tokens and the probe tokens together, so each weight is read once. Only attention is split. The main tokens use the usual bidirectional attention. Each probe row attends to its own keys at the masked positions and to the main keys and values of the same layer everywhere else, including the block positions that are already committed. Since the main logits of a step are not known before the pass, the probes use the proposals vj and the confidence ranking from the previous step, including the predictions for the next block at a block boundary. A probe whose token has changed since then is dropped. At the first step of an answer, the probes run in one extra call after the main pass, so an answer takes NFE + 1 calls. We probe the K most confident candidates, which carry most of the weight qj in the mean-field sum, and set ∆ij = 0 for the others. 4

Algorithm 1 One decoding step with measured coupling Require: Sequence x, masked positions Ut of the current block, proposals vj and margins cj from the previous step, parameters τ , R, K Ensure: Commit set Ct // 1. Probe construction P ← the K positions j ∈ Ut with the largest cj x(j) ← the tokens at Ut with xj := vj , for all j ∈ P // 2. One forward pass for x and all probes (j) {ℓi }, {ℓi } ← fθ x, {x(j) }j∈P ▷ probes reuse the main keys and values outside Ut // 3. Local confidence πt,i ← softmax(ℓi ), vi ← arg maxv πt,i (v) for all i ∈ Ut (j) (j) ci ← log πt,i (vi ) − maxv̸=vi log πt,i (v), and ci from ℓi with the same token vi // 4. Measured coupling (j) ∆ij ← ci − ci for j ∈ P whose probed token is still vj , and ∆ij ← 0 otherwise; ∆ii ← 0 // 5. Mean-field relaxation q (0) ← σ(c) for r = 0 to R − 1 do q (r+1) ← σ c − ∆q (r) end for // 6. Thresholding (R) (R) Ct ← {i ∈ Ut : qi ≥ τ }; if it is empty, Ct ← {arg maxi qi } return Ct

Algorithm 1 summarizes one step. The method adds no parameters and needs no training data. Code, including the fused probe kernels, exact configurations, evaluation and timing scripts, and per-example outputs, is available at https://github.com/TalOved1/measured-coupling.

Experiments

4.1

Setup

We use LLaDA-8B-Instruct (Nie et al., 2026) and the DAWN code base (Luo et al., 2026), which the mean-field paper also uses. We also evaluate LLaDA-1.5 (Zhu et al., 2026) with the same protocol, and Dream-v0-Instruct-7B (Ye et al., 2025), a model from a different family. We evaluate on GSM8K (Cobbe et al., 2021) (1,319 problems, 5-shot), MATH (Hendrycks et al., 2021) (5,000 problems, 4-shot with the Minerva prompts (Lewkowycz et al., 2022)), HumanEval (Chen et al., 2021) (164 problems, 0-shot), and MBPP (Austin et al., 2021b) (500 problems, 3-shot). The generation length is 256 tokens. MATH answers are checked for mathematical equivalence with math-verify, and HumanEval answers are post-processed with the sanitizer from the DAWN code base. We compare against one-token-per-step decoding, Fast-dLLM with threshold 0.9 (Wu et al., 2026), DAWN (Luo et al., 2026), and mean-field decoding with the published LLaDA setting (block 20, τ = 0.85, two iterations) (Zoabi et al., 2026). Our method uses the same block size (20) and number of iterations (two) as mean-field decoding. All methods are greedy. Section 4.2 explains how we chose the remaining settings of our method. We report accuracy, the average number of function evaluations (NFE) per answer, and the step reduction over one-token-per-step decoding (256/NFE). Throughput (tokens per second, TPS) depends on the GPU, so we measure it for all methods in the same job on the same A100 GPU. We compare accuracy with mean-field decoding on the same questions: we report the paired difference with a 95% confidence interval (paired Wald interval), and use a paired exact McNemar test where we call a difference significant. When an interval includes zero, the data cannot tell which method is more accurate; it does not show that they are equally accurate. 5

Table 1: Results on LLaDA-8B-Instruct. Step reduction is 256/NFE, relative to one-token-per-step decoding, and is not a runtime speedup. TPS (tokens per second) is measured for all methods of a benchmark in one job on the same GPU (GSM8K on its first 200 questions, MATH on 105 problems with 15 per subject, HumanEval and MBPP on the full test sets). Time is the wall-clock time per answer in seconds, from the same job. Bold marks the best value in each efficiency column. Ours is measured coupling, MF is mean-field decoding, and one token is one-token-per-step decoding. The last column is the accuracy difference from mean-field decoding in points, with a paired 95% confidence interval. Bench.

Method

Acc. ↑ NFE ↓ Step red. ↑ TPS ↑ Time (s) ↓ ∆ vs. MF [95% CI]

GSM8K

One token Fast-dLLM DAWN MF Ours

77.94 78.17 77.79 76.57 76.27

256.0 78.5 55.8 65.4 40.1

1.0× 3.3× 4.6× 3.9× 6.4×

7.2 23.2 31.7 28.6 40.3

32.42 10.00 7.29 8.10 5.86

−0.3 [−2.5, +1.9]

One token Fast-dLLM DAWN MF Ours

33.82 33.36 32.30 32.02 31.18

256.0 97.0 72.6 77.2 47.9

1.0× 2.6× 3.5× 3.3× 5.3×

9.4 24.0 30.8 30.8 41.5

26.21 10.32 8.01 8.18 6.05

−0.8 [−1.8, +0.2]

One token Fast-dLLM HumanEval DAWN MF Ours

41.46 42.68 37.80 37.20 39.02

256.0 78.9 61.6 61.7 40.3

1.0× 3.2× 4.2× 4.1× 6.4×

19.8 64.5 80.6 80.4 83.7

12.27 3.76 2.97 2.93 2.77

+1.8 [−4.1, +7.8]

One token Fast-dLLM DAWN MF Ours

30.00 30.20 31.00 30.20 29.00

256.0 70.6 50.5 51.6 32.3

1.0× 3.6× 5.1× 5.0× 7.9×

6.4 23.4 30.7 28.6 31.7

26.92 7.41 5.42 5.31 4.00

−1.2 [−4.9, +2.5]

MATH

MBPP

4.2

How we chose the settings

Our method has three settings: the threshold τ , the number of probes K, and whether the probes run in a separate call or inside the main pass. We follow the protocol of mean-field decoding (Zoabi et al., 2026): we choose the settings on the first 100 questions of the GSM8K training split, separately for each model family. The candidate with the highest held-out accuracy wins, and ties go to the candidate with fewer steps. This selects K = 12 probes inside the main pass with τ = 0.9 for LLaDA-8B-Instruct, which LLaDA-1.5 shares, and K = 20 with τ = 0.95 for Dream. As a check on code, we also scored the candidates on the validation split of MBPP (90 problems), which does not overlap with the MBPP or HumanEval test sets; on LLaDA the two lists overlap but differ, since the candidates changed during development. On Dream it selects the same setting. On LLaDA its best threshold is slightly higher, τ = 0.95, two problems ahead of τ = 0.9. Tables 13 and 14 in the appendix list every candidate scored on each set. All baselines use their published settings. Mean-field decoding uses the LLaDA setting of Zoabi et al. (2026) (block 20, τ = 0.85, two iterations), and Fast-dLLM and DAWN use the settings of the DAWN code base. On Dream, mean-field decoding uses the default Dream setting of Zoabi et al. (2026) (block 20, τ = 0.95, two iterations), and Fast-dLLM and DAWN use the Dream settings of the DAWN code base. Their test results are therefore not tuned on the test sets either. During development we also evaluated other variants on the test sets, and Section 4.4 reports several of them as analysis. The main table uses only the settings chosen by the rule above. The candidate list grew once during the project: we added the probe variant that recomputes only the masked positions, and applied the same rule again to the full list. All reported results use the setting selected from the full list. 6

Table 2: Results on LLaDA-1.5. Our method uses the setting chosen on LLaDA-8B-Instruct, with no new tuning. Columns and TPS measurement as in Table 1. Bold marks the best value in each efficiency column. Bench.

Method

Acc. ↑ NFE ↓ Step red. ↑ TPS ↑ Time (s) ↓ ∆ vs. MF [95% CI]

GSM8K

One token Fast-dLLM DAWN MF Ours

80.59 81.12 81.35 81.35 80.36

256.0 75.4 54.4 64.1 40.2

1.0× 3.4× 4.7× 4.0× 6.4×

6.9 22.5 30.7 28.2 37.3

31.82 9.70 7.11 7.78 5.93

−1.0 [−2.9, +0.9]

One token Fast-dLLM DAWN MF Ours

33.96 33.96 32.20 33.28 32.02

256.0 95.6 72.7 77.3 49.5

1.0× 2.7× 3.5× 3.3× 5.2×

8.8 22.2 28.4 28.8 37.4

25.62 10.15 7.95 7.88 6.09

−1.3 [−2.3, −0.2]

One token Fast-dLLM HumanEval DAWN MF Ours

43.29 43.29 40.85 38.41 40.24

256.0 99.9 77.2 72.4 44.4

1.0× 2.6× 3.3× 3.5× 5.8×

7.1 18.3 23.5 25.2 27.8

11.77 4.56 3.58 3.29 2.83

+1.8 [−3.4, +7.0]

One token Fast-dLLM DAWN MF Ours

37.40 37.40 35.80 39.80 37.40

256.0 42.1 30.0 38.3 28.5

1.0× 6.1× 8.5× 6.7× 9.0×

2.4 14.1 19.5 16.2 18.4

26.58 4.49 3.29 3.96 3.58

−2.4 [−6.0, +1.2]

MATH

MBPP

4.3

Main results

Table 1 shows the main results. Measured coupling needs the fewest steps on every benchmark, more than a third fewer than mean-field decoding. Its accuracy differs from mean-field decoding by −1.2 to +1.8 points, and every 95% confidence interval of the paired difference includes zero. The step savings carry over to wall-clock time. On the same GPU, our method has the highest throughput of all methods on every benchmark, most of all on GSM8K, whose long prompts make the main pass dominate a step. Appendix C compares the methods at matched accuracy, with every threshold timed on the same GPU. On GSM8K and MBPP our method reaches a given accuracy in less time than mean-field decoding where both reach it, and on HumanEval the two curves cross. With the dual key-value cache of Fast-dLLM for all methods, our method at block 32 is the fastest on GSM8K and MBPP (Appendix D). Figure 3 in Appendix E shows the full trade-off between accuracy and steps, sweeping the threshold of each method. On GSM8K, HumanEval, and MBPP our curve lies to the left of the mean-field curve: it reaches a given accuracy with far fewer steps, at accuracies up to 76.7% on GSM8K. On MBPP it also rises above the best mean-field point. Table 2 repeats the comparison on LLaDA-1.5, with the setting chosen on LLaDA-8B-Instruct and no new tuning. Our method again needs the fewest steps on every benchmark, 1.3 to 1.6 times fewer than mean-field decoding. Its accuracy differs from mean-field decoding by −2.4 to +1.8 points. On MATH it is 1.3 points lower, and this difference is significant (p = 0.02). Its throughput is 1.10 to 1.32 times that of mean-field decoding, and it is the highest of all methods on GSM8K, MATH, and HumanEval. On MBPP, DAWN is slightly faster (19.5 against 18.4 tokens per second). Table 3 reports Dream-v0-Instruct-7B, with the setting calibrated on Dream. Our method needs 1.7 to 2.2 times fewer steps than mean-field decoding on every benchmark. It is less accurate than mean-field decoding on all four benchmarks, by 0.6 to 6.1 points. The largest drop is on HumanEval, where the confidence interval is wide (164 problems), and the drop on MATH (2.1 points) is significant (p < 0.001). It has the shortest time per answer on GSM8K, MATH, and HumanEval, and on MBPP DAWN is slightly faster. Tokens per second counts only tokens before the end-of-text token, and our answers end earlier (Appendix C), so our gain shows more in time per answer. 7

Table 3: Results on Dream-v0-Instruct-7B. Our method uses the setting calibrated on Dream (K = 20, τ = 0.95). Columns and measurement as in Table 1. Bold marks the best value in each efficiency column. Bench.

Method

Acc. ↑ NFE ↓ Step red. ↑ TPS ↑ Time (s) ↓ ∆ vs. MF [95% CI]

GSM8K

One token Fast-dLLM DAWN MF Ours

75.06 74.98 72.25 76.19 75.13

256.0 62.2 52.0 69.2 41.4

1.0× 4.1× 4.9× 3.7× 6.2×

4.8 19.1 22.3 17.0 21.1

27.17 6.71 5.79 7.41 5.67

−1.1 [−3.2, +1.1]

One token Fast-dLLM DAWN MF Ours

38.18 38.34 37.70 37.88 35.76

256.0 115.0 94.4 100.0 51.9

1.0× 2.2× 2.7× 2.6× 4.9×

11.2 23.5 27.9 21.6 29.4

22.37 10.65 8.82 9.03 5.97

−2.1 [−3.2, −1.0]

One token Fast-dLLM HumanEval DAWN MF Ours

52.44 55.49 54.27 54.27 48.17

256.0 90.7 75.5 76.0 40.1

1.0× 2.8× 3.4× 3.4× 6.4×

16.3 38.8 46.2 35.7 34.0

10.44 3.83 3.16 3.21 2.59

−6.1 [−13.2, +1.0]

One token Fast-dLLM DAWN MF Ours

54.00 56.00 56.80 54.20 53.60

256.0 63.6 42.9 77.2 34.4

1.0× 4.0× 6.0× 3.3× 7.4×

5.5 26.9 30.1 25.6 18.6

20.39 5.17 3.55 6.22 3.79

−0.6 [−4.0, +2.8]

MATH

MBPP

Calibration narrows the drops. We keep one setting per model family, chosen like mean-field decoding on GSM8K training questions, and τ trades steps for accuracy. Stricter thresholds narrow the drops and keep most of the step savings: on MATH, τ = 0.99 on Dream and 0.95 on LLaDA-1.5 are within 0.8 and 0.1 points of mean-field decoding (95% CI [−1.9, +0.3] and [−1.2, +0.9], so at most 2 points lower) with 39% and 29% fewer steps (Appendix G). On Dream HumanEval, τ = 0.97 (K = 12) narrows the gap to 3.7 points (CI [−10.8, +3.5]) with 45% fewer steps (Appendices E and F). 4.4

Where the savings come from

Appendix B isolates the pairwise term at a common threshold. Our coupling needs clearly fewer steps than both the confidence-only rule and mean-field decoding, because it detects support: tokens that confirm each other are committed together. Table 4 tests whether the measured pairs matter, or whether any added support would do. Each control changes only the coupling of our method, on the same measured pairs, with all other settings fixed. Shuffling the measured values across pairs keeps their distribution and breaks accuracy, by 36 points on GSM8K and 20 on MBPP. A constant support bonus, tuned on held-out GSM8K questions, keeps accuracy and needs 44% to 46% more steps, and keeping the measured conflicts with a tuned constant support needs 53% to 72% more. The gain therefore comes from knowing which pairs support each other. The two signs work together: keeping only support collapses accuracy, and keeping only conflicts needs about as many steps as mean-field decoding. Averaging the two directions where both are measured costs 2 to 3 points (significant on MBPP), so its direction carries some information. The coupling matters most when decoding is pushed hard. At about 34 steps on GSM8K, the confidence-only rule drops to 62% accuracy, while our method keeps 75% (Figure 3). 4.5

Ablations

Appendix H varies the number of probes and the block size on GSM8K. More probes save steps at the same accuracy. Larger blocks need more probes to keep the same share of candidates measured; with that, block 32 matches mean-field accuracy with far fewer steps, while block 64 loses accuracy. From 8

Table 4: Controls on the measured coupling, LLaDA-8B-Instruct, full test sets (accuracy % / NFE). Each row changes only the coupling of our method on the measured pairs. ∆ is the accuracy difference from our method with a paired 95% confidence interval. The constants (b = 0.02 and 0.05) were chosen on 100 GSM8K training questions. Appendix I defines each control. GSM8K ∆ [95% CI]

Coupling

Acc. / NFE

Measured (ours) Shuffled across pairs Support only Conflict only Partially symmetrized Constant bonus Conflicts + constant support

76.27 / 40.1 40.11 / 44.7 −36.2 [−39.1, −33.2] 53.37 / 39.8 −22.9 [−25.7, −20.1] 78.24 / 72.2 +2.0 [−0.2, +4.1] 74.22 / 37.4 −2.1 [−4.1, +0.0] 76.72 / 58.6 +0.5 [−1.7, +2.6] 77.33 / 68.9 +1.1 [−1.1, +3.2]

MBPP Acc. / NFE

∆ [95% CI]

29.00 / 32.3 8.60 / 30.9 −20.4 [−24.2, −16.6] 15.40 / 29.3 −13.6 [−16.9, −10.3] 32.20 / 51.9 +3.2 [+0.0, +6.4] 26.00 / 30.8 −3.0 [−5.5, −0.5] 29.80 / 46.4 +0.8 [−2.2, +3.8] 32.80 / 49.3 +3.8 [+0.7, +6.9]

one to four mean-field iterations, our steps change little and no accuracy difference is significant, while mean-field decoding alternates between short and long schedules with the parity of R (Appendix L). 4.6

Probe fidelity and cost

Our probes agree closely with exact full-sequence probes. During greedy decoding on GSM8K and HumanEval, the two couplings have a Pearson correlation of 0.96 to 0.97, and they agree in sign on 97% of the pairs with a clear effect (|∆ij | > 0.5; Appendix A). About half of the candidate pairs are measured (51% on GSM8K, 55% on MBPP; the rest keep ∆ij = 0), and 31% and 23% of probes are dropped because their token changed. On LLaDA, at a similar step budget, a confidence threshold commits three to four times as many tokens that are overturned by the rest of their commit set (Appendix K). The probes are also cheap. Exact probes cost about nine forward passes per step, and a separate probe call about half a pass, mostly for reading the weights. Inside the main pass, a step costs 1.2 to 1.5 times a mean-field step, and our method needs 26% to 55% fewer steps. A variant that learns the coupling from the decoder’s next step, with no probes, saves fewer steps (Appendix M).

Related Work

Confidence thresholds (Wu et al., 2026), entropy bounds, and dilated schedules (Ben-Hamu et al., 2025; Luxembourg et al., 2026) decide how many positions to commit at once. Mean-field decoding adds a similarity penalty and a variational update (Zoabi et al., 2026). DAWN and dependency-aware parallel decoding build a dependency graph from attention maps and choose an independent set (Luo et al., 2026; Kim et al., 2026). DEMASK trains a predictor of the total-variation effect of revealing a token and gives a guarantee on the joint error (Ringel et al., 2026); our coupling is close to this quantity, and its predictor is not public. AXON reveals supportive anchors on top of an existing decoder, approximating their value with attention (Abu Ayoub et al., 2026); it could run on top of ours, and our probes measure a similar quantity directly. PUNT tests independence with extra model calls and prunes every dependent token (Azangulov et al., 2026). Conditional influence is therefore not new; we add a training-free way to measure it online, with sign and direction, cheaply enough to coordinate parallel commits. Other work verifies drafts with an autoregressive model or with the model itself (Israel et al., 2026; Han et al., 2026), verifies drafted future steps (Pan et al., 2026; Shlomi et al., 2026), or remasks committed tokens (Wang et al., 2026); these ideas are complementary to ours. In speculative decoding, DCOCO trains a small module to score adjacent draft tokens (Eisenstadt et al., 2026); we measure such dependencies without training.

Limitations

A step with probes costs 1.2 to 1.5 times a mean-field step (1.4 to 1.5 times with the cache), so the wall-clock gain is smaller than the step gain; a leaner probe kernel or cheaper coupling estimation would widen it. The selected settings trade accuracy for speed on MATH and Dream HumanEval (Section 4.3). Mutual support shows self-consistency, not correctness, so wrong tokens can reinforce 9

each other. All answers have 256 tokens. With a key-value cache (Appendix D), our GSM8K accuracy is below both baselines, and at block 20 we are slightly slower than mean-field decoding.

Conclusion

The pairwise term of parallel decoding can be measured instead of approximated, cheaply, by probes inside the main pass. The measured coupling shows support as well as conflict, and on three models it needs far fewer steps than mean-field decoding with no training, with and without a key-value cache. Shuffled couplings and a uniform bonus both lose this gain.

References Giries Abu Ayoub, Mario Barbara, Lluís Pastor-Pérez, Tanja Bien, Aneesh Barthakur, Alaa Maalouf, and Loay Mualem. Supportive token revealing for fast diffusion language model decoding. arXiv preprint arXiv:2606.04236, 2026. Josh Achiam, Steven Adler, Sandhini Agarwal, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774, 2023. Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. Structured denoising diffusion models in discrete state-spaces. In Advances in Neural Information Processing Systems, 2021a. Jacob Austin et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021b. Iskander Azangulov, Teodora Pandeva, Niranjani Prasad, Javier Zazo, and Sushrut Karmalkar. Parallel sampling from masked diffusion models via conditional independence testing. In International Conference on Learning Representations, volume 2026, pages 140485–140521, 2026. Jinze Bai, Shuai Bai, Yunfei Chu, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. Heli Ben-Hamu, Itai Gat, Daniel Severo, Niklas Nolte, and Brian Karrer. Accelerated sampling from masked diffusion models via entropy bounded unmasking. In Advances in Neural Information Processing Systems, 2025. Tiwei Bie, Maosong Cao, Kun Chen, et al. LLaDA2.0: Scaling up diffusion language models to 100B. arXiv preprint, 2025. Tiwei Bie, Maosong Cao, Xiang Cao, et al. LLaDA2.1: Speeding up text diffusion via token editing. arXiv preprint, 2026. Mark Chen et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. Karl Cobbe et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, and Ilan Naiman. Contextual correction of parallel drafts for speculative decoding. arXiv preprint arXiv:2610.12570, 2026. Ligong Han, Hao Wang, Han Gao, Kai Xu, and Akash Srivastava. S2D2: Fast decoding for diffusion LLMs via training-free self-speculation. arXiv preprint arXiv:2603.25702, 2026. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, 2021. 10

Daniel Israel, Guy Van den Broeck, and Aditya Grover. Accelerating diffusion LLMs via adaptive parallel decoding. Advances in neural information processing systems, 38:52870–52888, 2026. Bumjun Kim, Dongjae Jeon, Moongyu Jeon, and Albert No. DAPD: Dependency-aware parallel decoding via attention for diffusion LLMs. arXiv preprint arXiv:2603.12996, 2026. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models. In Advances in Neural Information Processing Systems, 2022. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori Hashimoto. DiffusionLM improves controllable text generation. In Advances in Neural Information Processing Systems, 2022. Aixin Liu, Bei Feng, Bing Xue, et al. DeepSeek-V3 technical report. arXiv preprint arXiv:2412.19437, 2024. Lizhuo Luo, Zhuoran Shi, Jiajun Luo, Zhi Wang, Shen Ren, Wenya Wang, and Tianwei Zhang. DAWN: Dependency-aware fast inference for diffusion LLMs. arXiv preprint arXiv:2602.06953, 2026. Omer Luxembourg, Haim Permuter, and Eliya Nachmani. Plan for speed: Dilated scheduling for masked diffusion language models. In International Conference on Machine Learning, 2026. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, JiRong Wen, and Chongxuan Li. Large language diffusion models. Advances in Neural Information Processing Systems, 38:50608–50646, 2026. Tianxiang Pan, Baitao Gong, Mo Guang, Hongwei Yong, Tianpeng Jiang, Yaqian Li, Zheng Cao, and Kaiwen Long. Trajectory-level speculative decoding for diffusion language models. arXiv preprint arXiv:2608.27514, 2026. Liran Ringel, Ameen Ali, and Yaniv Romano. Dependency-guided parallel decoding in discrete diffusion language models. arXiv preprint arXiv:2604.02560, 2026. Eliezer Shlomi, Eliron Rahimi, Avi Mendelson, and Chaim Baskin. On the SPOT: Enhancing speculative decoding for diffusion language models via commit-order prediction. arXiv preprint arXiv:2610.22543, 2026. Guanghan Wang, Yair Schiff, Subham Sahoo, and Volodymyr Kuleshov. Remasking discrete diffusion models with inference-time scaling. Advances in Neural Information Processing Systems, 38: 147282–147339, 2026. Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. Fast-dLLM: Training-free acceleration of diffusion LLM by enabling KV cache and parallel decoding. In International Conference on Learning Representations, volume 2026, pages 57027–57051, 2026. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. Dream 7B: Diffusion large language models. arXiv preprint arXiv:2508.15487, 2025. Shaorong Zhang, Longxuan Yu, Rob Brekelmans, Luhan Tang, Salman Asif, and Greg Ver Steeg. Generation order and parallel decoding in masked diffusion models: An information-theoretic perspective. arXiv preprint arXiv:2602.00286, 2026. Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. LLaDA 1.5: Variance-reduced preference optimization for large language diffusion models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11425–11460, 2026. Tamim Zoabi, Ameen Ali, Liran Ringel, and Lior Wolf. Mean-field parallel decoding for discrete diffusion language models. arXiv preprint arXiv:2606.15805, 2026.

Implementation details

A probe row holds one copy of the still-masked positions of the current block, with the probed candidate j set to its proposal vj and the other positions left masked. In every layer, the main tokens and all probe rows go through the same normalization, projections, MLP, and output head in one batch. Each probe query attends to the main keys and values of the same layer at every position it does not recompute, and to the keys and values of its own row at the positions it does; it never sees another probe row. The main keys at the recomputed positions are masked out for the probe queries. Rotary embeddings use each position’s index in the sequence, so a probe token sits where it would in the main pass. Main tokens use the model’s usual bidirectional attention, so the main logits match a plain forward pass up to bfloat16 rounding. The fidelity numbers in Section 4.6 come from 3 test questions of GSM8K (5-shot) and 3 of HumanEval, decoded greedily for three blocks of 20; we measure at the start and middle of each block, giving 18 states and 4,230 pairs per task. The contraction statistics in Section 3.3 come from 1,805 steps on GSM8K and 1,662 on HumanEval. With K = 12, a fused step uses 0.26 GB more peak memory than a plain step on GSM8K and 0.14 GB more on HumanEval, on top of 14.9 GB of weights. Timings come from one job per benchmark on one A100 (80 GB), with batch size 1 and bfloat16. Where our method is timed at the start and end of a job, the two times differ by at most 1.3%, except in one LLaDA-1.5 GSM8K job (16%); for that benchmark we report the median over three jobs.

Effect of the pairwise term

Table 5 isolates the pairwise term. All rows use the same block size, schedule, and threshold τ = 0.85, the published threshold of mean-field decoding, so the pairwise term is the only difference. The first row has no pairwise term, so each position is committed on its own confidence. The second row is mean-field decoding with the similarity coupling, as in Table 1. The third row is our method with τ = 0.85 in place of its selected τ = 0.9, which is why its numbers differ from Table 1.

Table 5: Effect of the pairwise term (accuracy % / NFE). All rows use block 20 and τ = 0.85, so only the pairwise term changes. Our row uses the selected probes (K = 12, inside the main pass) at this common threshold. Pairwise term GSM8K HumanEval MBPP None (confidence only) Similarity D (MF) Measured ∆ (ours)

78.09 / 52.1 76.57 / 65.4 76.42 / 37.4

35.98 / 49.6 37.20 / 61.7 34.76 / 37.7

26.80 / 42.7 30.20 / 51.6 28.60 / 30.7

The similarity term makes decoding more careful: it adds steps relative to the confidence-only rule, and it helps accuracy on MBPP (+3.4 points, p = 0.027). The measured coupling moves in the other direction on steps. At the same threshold it needs 24% to 43% fewer steps than either rule.

Accuracy against time per answer

Figure 2 sweeps the threshold of our method and of mean-field decoding on LLaDA-8B-Instruct and plots accuracy against wall-clock time per answer. All points of a benchmark are timed in one job on the same A100 GPU (GSM8K on its first 200 questions, HumanEval on all 164, MBPP on its first 200), with batch size 1 and bfloat16; accuracy is from the full test sets. We timed our method at τ = 0.9 at the start and at the end of each job, and the two times differ by 0.6% to 3.1%. On GSM8K and MBPP our curve lies to the left of the mean-field curve; on GSM8K it stops at 76.7%, slightly below the best mean-field point (77.5%). On HumanEval the curves cross: our method is faster at the higher accuracies and mean-field decoding at the lower ones. Our answers are also shorter. Counting tokens before the end-of-text token, they are 5% to 56% shorter than those of mean-field decoding on Dream, 17% shorter on LLaDA-8B MBPP, and within about 5% in the other LLaDA settings. 12

Measured coupling (ours)

Accuracy (%)

GSM8K

HumanEval Fast-dLLM

DAWN

Mean-field

MBPP 32

Fast-dLLM

DAWN

26 30

75 6

Time per answer (s)

24 2.5

3.0

3.5

Time per answer (s)

4.0

Time per answer (s)

Figure 2: Accuracy against wall-clock time per answer on LLaDA-8B-Instruct. Curves sweep τ from 0.75 to 0.95 for our method and for mean-field decoding. Gray points are Fast-dLLM and DAWN at their published settings. All points of a panel are timed on the same GPU.

Results with a key-value cache

We also run every method with the dual cache of Fast-dLLM (Wu et al., 2026). A full forward pass at the start of each block fills the cache, and every later step of the block runs only the block against it. All three decoders share one implementation of this step, captured as a CUDA graph, with one fused attention kernel that applies the rotary embedding on load; our probe rows run in the same pass and kernel. All methods use block 32, where the block-start passes cost the same for every method. Table 6 reports our method with K = 4 probes and τ = 0.9 on LLaDA-8B-Instruct. Accuracy is from the full test sets, and time is measured for all methods in one job on the same GPU after a warm-up of 20 questions (GSM8K on its first 200 questions, MBPP on all 500). Our method needs 29% to 31% fewer steps than mean-field decoding and 49% to 55% fewer than Fast-dLLM, and has the shortest time per answer on both benchmarks. Its accuracy is higher than both baselines on MBPP, by 4.0 points over mean-field decoding (p = 0.03), and lower on GSM8K. This setting is the fastest of the cached grid we ran (K ∈ {4, 8}, τ ∈ {0.85, 0.9}, blocks 20 and 32). The setting chosen by our rule on GSM8K training questions uses block 20 and K = 8 (GSM8K 75.1%, MBPP 27.6%); at block 20 every method pays more block starts, and ours is 3% to 5% slower than mean-field decoding at the same block. Table 6: Methods with the dual key-value cache on LLaDA-8B-Instruct, block 32 (full test sets). Time is the wall-clock time per answer in seconds. Step time excludes the full passes at block starts, which cost about the same for every method (1.04 s per answer on GSM8K, 0.86 to 0.88 s on MBPP). Bench.

Method

Acc. ↑ NFE ↓ Time (s) ↓ Step time (s) ↓

Fast-dLLM 79.38 GSM8K MF 77.03 Ours 75.44

85.6 61.6 43.6

2.31 2.06 1.96

1.27 1.02 0.92

Fast-dLLM 26.60 MF 24.00 Ours 28.00

68.5 44.9 30.9

1.84 1.56 1.51

0.96 0.70 0.65

MBPP

Accuracy against steps on LLaDA-8B Measured coupling (ours)

Mean-field

Confidence threshold

GSM8K

Accuracy (%)

HumanEval 45

Fast-dLLM

DAWN

Fast-dLLM

One token per step (256 NFE)

MBPP

32.5

DAWN Fast-dLLM

30.0

DAWN

27.5

25.0 22.5

Steps per answer (NFE)

Figure 3: Accuracy against the number of steps per answer on LLaDA-8B-Instruct. Curves sweep the threshold τ : 0.75 to 0.95 for our method and for mean-field decoding, and 0.7 to 0.85 for the confidence-only rule (0.8 and 0.85 on code). Gray points are Fast-dLLM and DAWN at their published settings; the dashed line is one-token-per-step decoding.

Accuracy against steps on Dream

Figure 4 sweeps the threshold on Dream-v0-Instruct-7B. Our curve reaches its accuracy with about half the steps of mean-field decoding, and on MBPP it rises above the mean-field curve. Measured coupling (ours, K = 12)

GSM8K

MATH

Accuracy (%)

Fast-dLLM

DAWN

Steps per answer (NFE)

DAWN Fast-dLLM

56 55

35 34

DAWN

MBPP

Fast-dLLM

56 54

One token per step (256 NFE)

HumanEval Fast-dLLM

Ours, calibrated (K = 20, τ = 0.95)

Mean-field

48 60

100

Steps per answer (NFE)

Figure 4: Accuracy against steps per answer on Dream-v0-Instruct-7B. Curves sweep τ over 0.9, 0.95, and 0.97 at block 20, and up to 0.99 for our method on MATH. Our curve uses K = 12. The star is our calibrated setting from Table 3. Gray points are Fast-dLLM and DAWN; the dashed line is one-token-per-step decoding.

Where the MATH gap comes from

We paired every MATH problem between our method and mean-field decoding, using the saved answers of the main runs. The gap is spread over all difficulty levels and subjects instead of concentrating on the hardest problems, where long and sensitive derivations would stress the additive model most. Repetition loops occur at similar rates for both methods and explain at most 0.2 points of the gap. On Dream, a stricter threshold narrows the gap: at τ = 0.99 our method comes within 0.8 points of mean-field decoding (37.1% against 37.9%, with K = 12 and with K = 20) with nearly 40% fewer steps, and the difference is not significant (Figure 4). The gap therefore depends on the speed setting chosen on held-out data, and a stricter threshold reduces it at the cost of more steps. 14

Number of probes and block size

Table 7 varies the number of probes K and the block size on GSM8K, with all other settings fixed at the selected ones. Number of probes. Accuracy stays the same for every K we tried, while each extra probe saves steps. More measured couplings let the decoder commit more tokens together, at no cost in accuracy. Block size. At blocks 16 and 20, our method matches mean-field decoding with far fewer steps, and at block 16 it is slightly more accurate. With a fixed K, larger blocks leave most candidates unprobed: their columns of ∆ are zero, so their influence on other positions is not measured, although they still receive measured influences from the probed candidates. Accuracy drops. Scaling K with the block size keeps the share of probed candidates constant. This restores accuracy at block 32, where our method again matches mean-field decoding with far fewer steps. Very large blocks such as 64 need more than additional probes, and we leave them to future work. Table 7: Ablations on GSM8K (accuracy % / NFE). Left: number of probes K (τ = 0.9, block 20). Right: block size, for our method with K = 12, with K scaled with the block (K = 12 at blocks 16 and 20, K = 20 at 32, K = 40 at 64), and for mean-field decoding (τ = 0.85).

Ours

Block

Ours (K = 12)

Ours (scaled K)

4 8 12 20

76.42 / 45.0 76.27 / 41.3 76.27 / 40.1 76.27 / 38.8

16 20 32 64

77.03 / 43.2 76.27 / 40.1 72.55 / 34.2 67.70 / 29.1

77.03 / 43.2 76.27 / 40.1 76.19 / 33.6 67.70 / 27.4

76.95 / 69.0 76.57 / 65.4 76.42 / 56.4 75.44 / 48.3

Definition of the controls

Every control in Table 4 changes only the measured off-diagonal entries of ∆, so it has the same coverage as our method. Shuffled permutes these values at random with a fixed seed. Support only sets ∆ij ← min(∆ij , 0), and conflict only sets ∆ij ← max(∆ij , 0). Partially symmetrized replaces ∆ij by (∆ij + ∆ji )/2 where both directions are measured and keeps the other entries, so the matrix is not fully symmetric. Constant bonus sets every measured entry to −b; conflicts + constant support keeps ∆ij > 0 and sets every measured ∆ij < 0 to −b.

Testing the additive model

The decoder sums single-token effects over the committed set S (Equation 6). We test this on the sets it actually commits. At every step with |S| ≥ 2, we measure each ∆ij exactly with one full forward pass per j ∈ S, and the true margin of each remaining position with all of S revealed in one more pass. A flip means that the additive prediction and the true margin, after the sigmoid, fall on different sides of τ . Flips are rare for small sets and grow with |S| (Table 8). The sum overstates both signs. Where support dominates, the predicted margin is about 5.5 and the true one about 3.5, so support from several tokens is partly redundant. Where conflict dominates, the prediction is about −16 and the true margin about −3.5, so conflicts also saturate. For the committed tokens themselves, with the rest of their set revealed (Appendix K), the additive and the exact margin fall on the same side of τ for 96% of tokens on GSM8K and 97% on MBPP.

Commit decisions

We decode 50 test questions of GSM8K and of MBPP with our method and, at every step, compute the commit set on the same state with other couplings (Table 9). Block-local probes for all candidates change 4% of the decisions of exact probes, and measuring only the probed columns accounts for most of the rest. Most changed decisions hold back a token that exact probes would commit, which costs steps more than accuracy. 15

Table 8: Additive prediction against the jointly measured margin, LLaDA-8B-Instruct with our method (K = 12, τ = 0.9; K scaled with the block at 32 and 64). Share of remaining (uncommitted) positions whose decision at τ flips, by the size of the committed set. For the committed tokens, see the text. Run

Positions

|S| = 2–3

|S| = 4–7

|S| ≥ 8

All

GSM8K, block 20 MBPP, block 20 GSM8K, block 32 GSM8K, block 64

7,880 7,981 8,199 13,147

3.7% 4.5% 2.4% 1.4%

7.8% 7.3% 5.7% 3.7%

15.6% 14.2% 9.2% 10.8%

10.0% 9.4% 7.0% 7.2%

We then check each committed token against the rest of its commit set: we reveal the other tokens of the set, leave the token masked, and test whether its committed value is still the most likely one (Table 10). End-of-text padding is excluded. At the same number of steps, a confidence threshold commits three to four times as many overturned tokens as our method, and the gap is largest for small sets. Slower decoders with 50% to 65% more steps are overturned slightly less often. On Dream (K = 20, τ = 0.95), the threshold at a similar step budget is overturned 3.6 times as often on MATH and 1.4 times as often on HumanEval. Table 9: Commit decisions on the same decoding states, against exact full-sequence probes for every candidate (50 test questions each, our method with K = 12 and τ = 0.9; steps with only end-of-text padding left are excluded). GSM8K Coupling Block-local, all candidates Exact, probed columns only Ours (inside the main pass) No coupling

MBPP

Same set

Decisions changed

Same set

Decisions changed

70% 55% 50% 33%

4.2% 10.6% 11.8% 27.3%

71% 55% 47% 36%

4.4% 11.2% 12.5% 25.2%

Table 10: Committed tokens overturned by the rest of their commit set, by set size (50 test questions each; LLaDA-8B-Instruct unless marked Dream). Conf. is the confidence threshold.

Task

Decoder

NFE

|S| = 2–3

|S| = 4–7

|S| ≥ 8

All

GSM8K

Ours Conf. Conf. MF

0.9 0.75 0.9 0.85

41.1 41.9 66.3 68.0

0.4% 6.6% 0.2% 0.5%

0.8% 4.9% 0.3% 0.3%

0.8% 1.7% 0.2% 0.1%

0.8% 2.9% 0.2% 0.2%

MBPP

Ours Conf. Conf. MF

0.9 0.8 0.9 0.85

35.0 41.5 53.5 55.2

1.2% 5.0% 0.8% 1.3%

0.8% 3.3% 0.4% 0.1%

0.7% 1.0% 0.1% 0.0%

0.8% 2.3% 0.3% 0.3%

Dream MATH

Ours Conf. MF

0.95 0.8 0.95

60.9 61.6 116.8

0.5% 3.4% 0.0%

0.5% 1.5% 0.0%

0.4% 1.0% 0.0%

0.4% 1.5% 0.0%

Dream HumanEval

Ours Conf. MF

0.95 0.85 0.95

28.9 30.3 56.4

0.6% 2.7% 0.0%

1.1% 1.0% 0.0%

0.5% 0.4% 0.0%

0.7% 1.0% 0.0%

Number of mean-field iterations

Table 11 varies the number of mean-field iterations R on Dream-v0-Instruct-7B, with all other settings as in Table 3. For our method, the number of steps stays within 40 to 43 on GSM8K and 34 to 36 on MBPP for every R. Accuracy varies by up to 1.2 points on GSM8K and 2.4 points on MBPP, and no 16

difference from R = 2 is significant. Mean-field decoding needs 1.4 to 1.6 times more steps at odd R than at even R, which matches the oscillation reported by Zoabi et al. (2026). All main results use R = 2, as in mean-field decoding. Table 11: Number of mean-field iterations R on Dream-v0-Instruct-7B (accuracy % / NFE). The main results use R = 2 (bold). Ours (K = 20, τ = 0.95)

Mean-field (τ = 0.95)

GSM8K Acc. % / NFE

MBPP Acc. % / NFE

GSM8K Acc. % / NFE

MBPP Acc. % / NFE

1 2 3 4

75.36 / 42.9 75.13 / 41.4 74.15 / 40.2 74.22 / 39.8

56.00 / 36.1 53.60 / 34.4 56.00 / 34.3 54.80 / 33.8

75.66 / 113.3 76.19 / 69.2 75.89 / 110.3 75.82 / 70.2

57.80 / 118.3 54.20 / 77.2 58.40 / 113.9 54.00 / 79.2

Learned coupling from the next step

The decoder’s own next step contains information about the coupling. After a set S is committed, the next forward pass shows whether each remaining position changes its choice. This gives an exact label for the event that committing that position together with S would have changed the answer, at no extra cost. We fit a logistic model of this event on simple features of position pairs (distance and side, similarity of the two predictions, whether the tokens match, and the uncertainty at i), by ridge-regularized Newton steps, online and without training data, and commit positions whose predicted error probability is below a risk level ρ. This variant needs no extra forward passes and is useful on its own. With ρ tuned on GSM8K training questions, it uses fewer steps than mean-field decoding on GSM8K, HumanEval, and MBPP, with accuracy within 1.4 points. Its predicted flip rates match the observed rates within a few points for every committed-set size. This calibration is about flips, and Appendix J tests the additive margins directly. Its labels come only from positions that were not committed, so they rarely cover confident tokens. Probes reach those tokens directly, which explains the larger savings of measured coupling. Table 12 reports this variant on LLaDA-8B-Instruct. We chose ρ = 0.05 on the first 100 GSM8K training questions, the same held-out set as for our main method. It uses fewer steps than mean-field decoding on all three benchmarks. Its accuracy is 0.2 points higher on GSM8K, equal on HumanEval, and 1.4 points lower on MBPP. Table 12: Learned coupling from the next step on LLaDA-8B-Instruct (block 20, ρ = 0.05). Meanfield decoding is as in Table 1. GSM8K Method Mean-field (MF) Learned coupling

HumanEval

MBPP

Acc. % ↑

NFE ↓

Acc. % ↑

NFE ↓

Acc. % ↑

NFE ↓

76.57 76.72

65.4 53.1

37.20 37.20

61.7 54.2

30.20 28.80

51.6 48.7

Held-out selection results

Table 13: Every candidate setting of our method scored on each held-out set. “Masked only” probes recompute only the still-masked positions, and “separate call” probes run after the main pass instead of inside it. The selected setting, chosen on GSM8K, is in bold. MBPP validation is a check on code. Mean-field decoding is shown for reference only; its setting is the published one. Held-out set

Probes

Accuracy ↑

NFE ↓

GSM8K train (100)

12, inside main pass, masked only 12, inside main pass, masked only 8, inside main pass, masked only 8, inside main pass, masked only 8, separate call 8, separate call 12, separate call 12, separate call 20, inside main pass 20, inside main pass 12, inside main pass, masked only 12, inside main pass, masked only 12, inside main pass, masked only

0.90 0.85 0.90 0.85 0.85 0.90 0.85 0.90 0.85 0.90 0.93 0.95 0.97

91.0 84.0 88.0 86.0 90.0 85.0 84.0 86.0 84.0 82.0 86.0 83.0 81.0

39.3 37.2 40.1 38.4 34.3 36.2 32.8 34.7 35.6 38.0 42.0 42.8 45.5

MBPP validation (90)

12, inside main pass, masked only 8, inside main pass, masked only 20, inside main pass, masked only 12, inside main pass 12, inside main pass 20, inside main pass 20, inside main pass 12, separate call 8, separate call 12, inside main pass, masked only 12, inside main pass, masked only 12, inside main pass, masked only Mean-field (reference)

0.90 0.90 0.90 0.90 0.85 0.85 0.90 0.90 0.90 0.93 0.95 0.97 0.85

35.56 30.00 30.00 28.89 27.78 27.78 27.78 26.67 25.56 34.44 37.78 37.78 30.00

32.7 33.5 33.5 32.9 32.0 31.3 33.5 29.6 30.8 33.8 35.2 37.1 52.2

Table 14: Held-out candidates on Dream-v0-Instruct-7B, with the same held-out sets and rule as Table 13. The selected setting, chosen on GSM8K, is in bold; MBPP validation selects the same one. Mean-field decoding is shown for reference only; its setting is the published Dream one. Held-out set

Probes

Accuracy ↑

NFE ↓

GSM8K train (100)

20, inside main pass, masked only 20, inside main pass, masked only 8, inside main pass, masked only 12, inside main pass, masked only 8, inside main pass, masked only 12, inside main pass, masked only 8, inside main pass, masked only 20, inside main pass, masked only 12, inside main pass, masked only Mean-field (reference)

0.95 0.90 0.95 0.90 0.90 0.97 0.97 0.97 0.95 0.95

74.0 73.0 71.0 71.0 70.0 69.0 68.0 68.0 66.0 78.0

42.2 37.9 42.8 37.6 38.2 44.4 45.2 44.1 42.1 68.6

MBPP validation (90)

20, inside main pass, masked only 12, inside main pass, masked only 20, inside main pass, masked only 20, inside main pass, masked only 12, inside main pass, masked only 8, inside main pass, masked only 8, inside main pass, masked only 8, inside main pass, masked only 12, inside main pass, masked only Mean-field (reference)

0.95 0.97 0.90 0.97 0.95 0.97 0.95 0.90 0.90 0.95

54.44 54.44 53.33 53.33 53.33 52.22 50.00 48.89 47.78 56.67

34.1 34.9 30.4 35.5 34.1 35.2 32.9 30.3 30.4 76.0

Measured Coupling for Parallel Decoding in Diffusion Language Models

Analyze & explore related research in Conceptio

Related documents