P-Cast Precision in FP8 Attention: Sink-Induced Collapse and the Optimality of S = 28 Reed Lau Tencent [email protected]
arXiv:2606.06521v1 [cs.AR] 2 Jun 2026
Abstract FP8 (E4M3) acceleration for attention computation offers significant throughput gains, but the 3-bit mantissa introduces precision challenges when the softmax probability matrix P is cast to FP8 before the P ·V matrix multiplication. We analyze two implementation choices that affect output precision under the Attention Sink phenomenon: (1) the KV block iteration order, and (2) the static scaling factor applied to P before casting. We show that forward KV iteration causes P-collapse—to leading order a fraction Φ(∆ + δk − 6.93 − ln S) of non-sink P values underflow to zero, where the small shift δk ≈ 1 (for ksink =4) is the expected within-sink-block score maximum— and that reverse iteration removes it, with a zero-underflow guarantee when reverse is combined with S=256. We further give a constructive characterization of S = 256 = 28 as the static scale that simultaneously satisfies (i) bit-exact IEEE 754 scaling, (ii) the lower envelope of a sawtooth function dp(S) over the E4M3 number line (dp = 2−4 , the minimum worst-case quantization step), and (iii) the maximum normalrange coverage among bit-exact (2k ) scales (a non-bit-exact scale such as 448 attains slightly higher coverage; §5). Both optimizations are already deployed in FlashAttention-3/4 on engineering grounds; our contribution is a quantitative account of why these choices are good and a closed-form threshold ∆c = 6.93 + ln S − δk for predicting kernel-level precision loss. Kernel-faithful experiments (Q, K, V in FP32 to isolate the P-cast effect) show 3–10× MSE improvement at moderate sink strengths, and paired tests confirm both fixes saturate to the same precision floor when combined—which motivated updating the hpc-ops kernel from S=1 to S=256.
1
Introduction
1.1
Motivation
The relentless scaling of large language models has made low-precision arithmetic essential for both training and inference throughput. Modern GPU architectures (NVIDIA Hopper, Blackwell; AMD MI300) now provide native FP8 tensor core support, operating on two formats: E4M3 (4-bit exponent, 3-bit mantissa) for forward computation and E5M2 for gradients [5]. FlashAttention-3 [6] exploits this by casting the softmax output matrix P to E4M3 before the P · V matmul, while
keeping the output accumulator in FP32. This design creates a precision bottleneck at the P-cast step: E4M3’s 3-bit mantissa provides only 8 representable values per binade, giving a relative precision of just 12.5%—roughly 16× worse than BF16. In this regime, two implementation “details” that are inconsequential in higher precision become first-order precision determinants: 1. KV block iteration order: whether the online softmax processes KV blocks forward (0 → N ) or reverse (N → 0). 2. P-scaling factor: the constant S by which P is multiplied before the E4M3 cast (and divided back in the epilogue).
1.2
The Attention Sink Problem
The Attention Sink [8] phenomenon—where initial tokens receive disproportionately large attention weights—interacts destructively with FP8 quantization. Under forward iteration, the sink’s high logit score inflates the running softmax maximum m, forcing all subsequent P values below E4M3’s representable range. This P-collapse is a threshold effect: it activates around sink strength ∆ ≈ 6–7, where the cast zeroes the majority of non-sink P values while those positions still carry roughly half of the total probability mass.
1.3
Contributions
This paper provides a quantitative account of both implementation choices: 1. P-collapse quantification (§3): We derive a closed-form expression F (∆, S) = Φ(∆ + δk − 6.93 − ln S) (with δk a small within-sink extreme-value correction) for the fraction of non-sink P values that underflow, and a leadingorder MSE estimate showing the effect peaks in a narrow transition region. 2. Reverse iteration sufficiency (§4): We show that reverse iteration (rigorously, combined with S=256) keeps all P values representable for any practical sequence length, with an explicit probabilistic guarantee. 3. S = 256 characterization (§5): We introduce the dp(S) function—the normalized maximum quantization step— and show it forms a sawtooth with power-of-two values on its lower envelope. This yields a constructive characterization of S = 256 via three jointly imposed conditions
(bit-exactness, minimum quantization step, maximum normal coverage).
Algorithm 1 FP8 Online Softmax Attention (single query row)
4. Kernel-faithful validation (§6): Using a simulation that exactly matches production kernel semantics (gSum from FP32 pre-cast P), we measure 3–10× MSE improvement and observe saturation when both fixes are applied.
Require: Q ∈ R1×d , K, V ∈ RN ×d (FP8), scale S, block size B 1: m ← −∞, ℓ ← 0, O ← 0 ∈ R1×d ▷ FP32 2: for j ∈ BlockOrder do ▷ Forward: 0..N/B; Reverse: N/B..0 √ 3: Z ← Q · KjT / dk ▷ FP8×FP8→FP32 accumulator (scores) 4: mloc ← max(Z) 5: mnew ← max(m, mloc ) 6: α ← exp(m − mnew ) ▷ FP32 correction 7: P ← exp(Z − mnew ) ▷ FP32 local prob. 8: ℓ ← α · ℓ + sum(P ) ▷ FP32 pre-cast P 9: Pfp8 ← cast_E4M3(P · S) ▷
1.4
Positioning
Both optimizations are already deployed in FlashAttention3/4 [6] on engineering grounds (register savings, range utilization). This paper is therefore not a proposal of new techniques, but a quantitative explanation of why those choices work and a closed-form diagnostic (∆c = 6.93 + ln S − δk ) that practitioners can apply to predict P-cast failure regimes. The same analysis motivated updating the hpc-ops kernel from S = 1 to S = 256 and could inform corresponding changes in FlashInfer (S = 448) and TensorRT-LLM XQA (S = 448).
2
Background and Related Work
2.1
FP8 E4M3 Number Format
P-cast; ×S is FP32, bit-exact for S=2k 10: O ← α · O + Pfp8 · Vj ▷ FP8×FP8→FP32 accum. 11: end for 12: return O/(S · ℓ) ▷ Epilogue: unscale + normalize
2.3
Multiple studies (see, e.g., [1, 4, 7, 8]) document that pretrained LLMs allocate disproportionate attention to initial tokens (“sink tokens”), with sink-vs-normal logit gap ∆ typically reported in the range [6, 13] at context lengths of several thousand. Training-side mitigations exist (learnable sink tokens, clipped softmax), but we focus on kernel-level solutions applicable to already-trained models.
The E4M3 format [5] allocates 1 sign bit, 4 exponent bits (bias b = 7), and 3 mantissa bits. The complete positive representable set contains 126 values: Subnormals (exponent field E = 0, 7 values): v = 21−b ·
M M = 2−6 · , 23 8
M ∈ {1, . . . , 7}
spanning [2−9 , 7 · 2−9 ] = [0.00195, 0.01367]. Normals (exponent field E = 1, . . . , 15; 119 values): M E−b v=2 · 1 + 3 , M ∈ {0, . . . , 7} 2
(1)
2.4
Value
Significance
Max representable Min normal Min subnormal Round-to-zero
448 2−6 2−9 2−10
Overflow/saturation Below: subnormal Smallest nonzero Below: casts to 0
(2)
Within any normal binade [2n , 2n+1 ), the spacing (LSB) is 2n−3 , yielding exactly 8 uniformly spaced representable values. The subnormal region has uniform spacing 2−9 with only 7 values—much coarser relative precision.
2.2
Related Work
FP8 attention kernels. FlashAttention-3 [6] introduced E4M3 P-casting with S = 256 and reverse iteration on Hopper (SM90); FlashAttention-4 extends the same choices to Blackwell (SM100). FlashInfer adopts S = 448 (matching maxE4M3 ). SageAttention2 [9] uses per-block S = 448; SageAttention2++ [10] constrains S = 112 for FP16 accumulation. FP8 quantization theory. Micikevicius et al. [5] introduced the E4M3/E5M2 split. Per-tensor vs. per-channel scaling is well studied for weights/activations, but the specific structure of post-softmax P ∈ [0, 1] quantization has not received formal treatment. Attention sink analysis. Xiao et al. [8] identified the phenomenon; Sun et al. [7] linked it to massive activations; Gu et al. [4] empirically measured sink strength distributions. None analyzed the interaction with FP8 P-casting.
with E = 15, M = 7 reserved as NaN, giving max = 448 = 1.75 × 28 . Key thresholds for P quantization: Quantity
Attention Sink
Online Softmax and FP8 FlashAttention
FlashAttention [2, 3] computes exact attention via tiled online softmax. Algorithm 1 shows the FP8 variant, highlighting the P-cast step and the critical separation between ℓ (FP32 pre-cast) and O (from cast P). The key design: ℓ (line 8) uses exact FP32 probabilities, while O (line 10) uses the cast E4M3 values. This means normalization is always exact; precision loss appears only in the numerator.
3
P-Collapse Under Attention Sink
3.1
Setup and Notation
We consider a single attention head with query length q, KV length N , and head dimension d. KV blocks have size B (typically 64 or 128). The first ksink positions are sink tokens with logit scores ∆ above the mean. Non-sink scores follow s √ ∼ N (0, 1) (standard for well-trained transformers with 1/ dk scaling). 2
3.2
Table 1: P-collapse analysis (N = 4096, ksink = 4, block size 64). Both the S=1 and S=256 “frac. zeroed” columns are measured from the same kernel-faithful forward simulation (averaged over 12 seeds), so they share one mglobal = ∆ + δk convention. The measured shift is δ4 ≈ 1.0, consistent with √ E[max of 4 N (0, 1)] ≈ 1.03 (and not the asymptotic 2 ln 4 ≈ 1.67). “Eff. info loss” is non-sink mass × frac. zeroed (S=1). The closed form (7) reproduces these columns to within a few points; the residual is the per-row spread of mglobal (Corollary 2 proof).
Forward Iteration Failure Mode
In forward iteration, block 0 contains the sink tokens. After processing block 0, the running maximum is set by the largest sink-token logit. Writing the sink scores as ∆ plus the same N (0, 1) fluctuation carried by other tokens, mglobal = ∆ + δk , δk ≜ E maxi≤ksink si , (3) where δk is the expected maximum √ of ksink standard Gaussians (δ4 ≈ 1.03; the asymptotic 2 ln ksink ≈ 1.67 badly overestimates δk at the small ksink of interest, and we use the exact value). For all subsequent blocks j > 0, the local probability values are: Pj (i) = exp si − mglobal = exp si − ∆ − δk . (4) Proposition 1 (P-underflow condition). A P value p underflows to zero in E4M3 (with scale S) iff: p · S < 2−10
Frac. zeroed F (∆, S) S=1 S=256
Non-sink mass
Eff. info loss
5 6 7 8 9 10 12
22.3% 51.6% 82.0% 94.8% 99.5% ≈100% ≈100%
88.0% 74.0% 51.7% 32.2% 13.9% 5.8% 0.8%
19.6% 38.2% 42.4% 30.5% 13.9% 5.8% 0.8%
0% 0% 0% 0.3% 2.3% 11.7% 67.9%
(5)
For Pj (i) = exp(si − ∆ − δk ) with scale S, this occurs when:
random vectors with E[Vj VjT ] = σV2 Id and pairwise uncorrelated across positions. Then the expected per-dimension MSE of the kernel output vs. exact FP32 reference satisfies:
si < ∆ + δk − 10 ln 2 − ln S = ∆ + δk − 6.93 − ln S (6)
3.3
∆
Underflow Fraction: Closed Form
E[∥ε∥2 /d] =
For s ∼ N (0, 1):
σV2 X 2 1 X Pj + Pj Pk tr E[Vj VkT ], 2 ℓ dℓ2 j∈Z
j̸=k∈Z
(8) P where ℓ = all Pj is the (exact, FP32) running sum. Under the pairwise-uncorrelated assumption the cross term vanishes, yielding σ2 X 2 MSEcollapse = V2 Pj . (9) ℓ
Corollary 2 (P-collapse fraction). To leading order, the fraction of non-sink P values that underflow to zero under forward iteration with scale S is: F (∆, S) = Φ ∆ + δk − 6.93 − ln S (7)
j∈Z
where Φ is the standard normal CDF, 6.93 = 10 ln 2, and δk is the within-sink extreme-value shift of §3 (the naive δk = 0 form is a lower bound on the realized collapse).
P Proof. The output error vector is ε = 1ℓ j∈Z Pj Vj (the “missing” contribution). Expanding the squared norm and taking expectations:
Proof. By Proposition 1, Pj (i) underflows iff si < ∆ + δk − 10 ln 2 − ln S. For si ∼ N (0, 1), Pr[si < x] = Φ(x), hence the underflow fraction equals Φ(∆ + δk − 6.93 − ln S) (using 10 ln 2 = 6.9315). This treats the per-row mglobal as the fixed mean ∆ + δk ; since mglobal has nonzero spread across query rows and Φ is convex in its lower tail, the realized fraction is slightly higher than this mean-shift estimate. Table 1 therefore reports the exact simulated values, which Eq. (7) reproduces to within a few percentage points.
E[∥ε∥2 ] =
(10)
j,k∈Z
With E[VjT Vj ] = dσV2 and the pairwise-uncorrelated assumption E[VjT Vk ] = 0 for j ̸= k, dividing by d gives the stated equality. Remark 1. The pairwise-uncorrelated assumption is a standard idealization; real Vj are correlated across positions and channel-wise non-isotropic. The Pbound thus captures the leading O(σV2 ) scaling and the j∈Z Pj2 dependence, but its absolute constant can shift by an O(1) factor (calibrated empirically; §6 confirms the shape and threshold).
Table 1 evaluates this for S = 1 (direct cast) and S = 256: The effective information loss (non-sink mass × fraction zeroed) peaks at ∆ ≈ 6–7 (∼40%), where positions carrying roughly half to three-quarters of the probability mass have most of their P values zeroed.
3.4
1 X Pj Pk E[VjT Vk ]. ℓ2
Remark P 2 22. This MSE is maximized when |Z| is large and Pj /ℓ is non-negligible—which holds only in the transition region ∆ ∈ [5, 9]. At very large ∆ the non-sink mass vanishes, so even total P-collapse barely affects the output, making [5, 9] the regime where this analysis matters most.
Output MSE Bound
Proposition 3 (MSE from P-collapse). Let Z be the set of positions whose P values underflow. Assume Vj are zero-mean 3
4 Optimization 1: Reverse KV Iteration
5 Optimization 2: Scale Factor S = 256
4.1
The complete P-quantization pipeline is:
Mechanism
Reversing the iteration order to (N −1, N −2, . . . , 0) defers the sink block to the last iteration. During all preceding iterations, only non-sink tokens contribute to the running maximum: p N −ksink mpre-sink ≤ max si ≈ 2 ln(N − ksink ) (11)
P̃ =
5.1
by extreme value theory for Gaussian order statistics. The P values during these iterations are: Pj (i) = exp(si − mpre-sink ) ∈ exp(−mpre-sink − 3σ), 1 (12) with high probability, where σ = 1 is the per-token score standard deviation and the lower endpoint uses the standard 3σ tail bound (violated with √ probability Φ(−3) ≈ 1.3 × 10−3 ). For N = 8192, 2 ln N ≈ 4.25, giving Pmin ≈ exp(−7.25) ≈ 7 × 10−4 —comfortably above the S=256 round-to-zero boundary 2−10 /256 = 2−18 ≈ 3.8 × 10−6 , and only marginally below the bare 2−10 boundary (which is why S=256 adds a safety margin over reverse alone).
√
Condition 1: Bit-Exact Scaling
For non-power-of-two S (e.g., 448): 1/448 = 0.002232 . . . is not exactly representable in IEEE 754. Every ×(1/448) introduces ∼ 2−24 rounding per element. Over the ddimensional output, this accumulates to an error of O(d · 2−48 )—negligible individually, but a systematic precision disadvantage nonetheless.
5.2
(13)
Condition 2: The dp(S) Sawtooth
Definition 1 (Normalized quantization step). For scale factor S ∈ (0, 448]:
2 ln N + O(1) (pre-sink) with score s ∼ N (0, 1):
Pr[underflow] = Φ(m − 12.48) < Φ(−7.2) < 10−12 (14) dp(S) ≜
6
for N ≤ 10 . Effectively zero P values underflow. Proof. The cast-to-zero condition is p · S < 2−10 , i.e., exp(s − m) · 256 < 2−10 . Taking logs: s − m + 8 ln 2 < −10 ln 2, hence s < m − 18 ln 2. Numerically, 18 ln 2 = 12.4766 (equivalently 10 ln 2 + ln 256 = 6.9315 + 5.5452 = 12.4766), giving √ the threshold s < m−12.48. Before the sink block, m ≤ 2 ln N + O(1). For N = 106 : m ≤ 5.3, so the threshold is s < 5.3 − 12.48 = −7.18. For s ∼ N (0, 1): Φ(−7.18) ≈ 3.5 × 10−13 < 10−12 .
4.3
(16)
Proof. In IEEE 754 binary32, 2k has exponent = 127 + k and zero mantissa. Multiplication by 2k adds k to the result exponent without modifying the 23-bit mantissa. Since 1/2k = 2−k is also exactly representable, the reciprocal multiplication is likewise exponent-only.
Theorem 4 (Zero-underflow guarantee for reverse + S = 256). In reverse iteration with S = 256, a P value p = exp(s − m) survives the E4M3 cast (i.e., p · 256 ≥ 2−10 ) whenever:
For m ≤
P̃ · V ℓ
Proposition 5 (Power-of-two bit-exactness). For S = 2k and any IEEE 754 FP32 value x (with result in representable range), both x × S and x × (1/S) are exact—no rounding occurs.
Formal Sufficiency Condition
s > m − 18 ln 2 = m − 12.48
O=
where RoundE4M3 denotes round-to-nearest in E4M3. The choice of S controls the fidelity of P̃ as an approximation to P.
i=1
4.2
RoundE4M3 (P · S) , S
maxx∈[0, S] LSBE4M3 (x) S
(17)
where LSBE4M3 (x) is the spacing between consecutive representable E4M3 values in the binade containing x. For S > 448, the cast saturates at 448 and we extend the definition to also include the (one-sided) saturation step, see Remark 3. dp(S) represents the worst-case quantization step for any P ∈ [0, 1] through the pipeline (16); the maximum pointwise error is dp(S)/2.
The Final α-Correction
Remark 3 (Overflow extension of dp). For S > 448 the value P · S = S at P = 1 is clipped to 448, producing a one-sided error |1 − 448/S|. Converting this saturation error to an equivalent step (by the standard error-equals-step/2 correspondence) gives 2(1 − 448/S). The effective worstcase step for S > 448 is therefore dp(S) = max 32 (18) S , 2(1 − 448/S) ,
When the sink block is finally processed (last in reverse), the correction factor is: √ α = exp(mpre-sink − mnew ) ≈ exp 2 ln N − ∆ (15) For ∆ = 7, N = 4096: α ≈ exp(4.1 − 7) ≈ 0.055, which multiplies the FP32 accumulator (23-bit mantissa) with negligible precision loss. A paired t-test (Appendix B) confirms forward+S=256 and reverse+S=256 are indistinguishable where P-collapse dominates (∆ ≤ 9); at ∆ ≥ 10 reverse is marginally—and inconsequentially—better (∼ 10−8 vs. MSE of 10−5 –10−6 ).
where 32/S is the contribution of the largest-LSB binade [256, 512) that lies inside [0, 448]. We use this extended form only in Theorem 6(iii); for S ∈ (0, 448] the original definition applies unchanged. 4
dp(S) = max quantization step / S
0.200
dp(S) sawtooth for FP8 E4M3: S = 2k achieves minimum dp = 2 4
Scale
dp(S) (sawtooth) Overflow 4 ) Lower envelope dp(S=>2 448 S = 2k (k = 1. . 7): valley at 2 4 S = 256 (optimal, dp = 2 4) S = 448 (dp 0.071 > 2 4)
0.175 0.150 0.125
S = 64 S = 128 S = 256 S = 512
S = 448 dp = 0.071
0.100 S = 256 dp = 2 4
0.025 0.000
0
100
200
300
Scale factor S
400
−12
Coverage −4
2 /64 = 2 ≈ 2.4 × 10 Baseline 2−6 /128 = 2−13 ≈ 1.2 × 10−4 2× better 2−6 /256 = 2−14 ≈ 6.1 × 10−5 4× better Overflow (exceeds 448)
This monotonicity does not stop at 256: a non-power-of-two scale such as S = 448 pushes the normal threshold down further to 2−6 /448 ≈ 3.5 × 10−5 , i.e. strictly better coverage than S = 256 (6.1 × 10−5 ). Coverage alone therefore favors 448; 256 wins only once we additionally require bit-exactness (C1) and the minimum quantization step (C2), both of which 448 fails. The 256-vs-448 choice is thus a genuine trade-off (smaller worst-case step vs. slightly better deep-tail coverage), resolved empirically in §6—not a clean domination.
0.075 0.050
P normal threshold −6
500
Figure 1: The normalized quantization step dp(S) for E4M3. All 2k values (blue) sit on the lower envelope at dp = 2−4 . S = 256 (green star) is the rightmost before overflow. S = 448 (red) sits 14% above the envelope.
5.4
Characterization of S = 256
Theorem 7 (Characterization of S = 256). Restricted to scales S ≥ 1 (so that the cast expands the dynamic range of P ∈ [0, 1] rather than contracting it), S = 256 = 28 is the unique value satisfying all three of:
Theorem 6 (dp(S) structure). (i) For all S = 2k with integer k ∈ {0, 1, . . . , 8} (i.e. S ∈ {1, 2, 4, . . . , 256}): dp(2k ) = 2−4 .
(C1) S = 2k
(bit-exact scaling; Proposition 5)
(ii) For all S ∈ [2−6 , 448] with S ̸= 2k : dp(S) > 2−4 .
(C2) dp(S) = 2−4 rem 6)
(iii) For S > 448 (using the extended form of Remark 3): dp(S) > 2−4 .
(C3) S = max{2k : 2k ≤ 448} age among 2k scales)
Proof. (i) For S = 2k with k ∈ {0, . . . , 8}, we have S ≤ 256 < 448, so S itself is exactly representable in E4M3. The mapped range [0, S] = [0, 2k ] has its highest binade [2k−1 , 2k ) entirely in the normal region (since k − 1 ≥ −1 > −6), with LSB = 2k−4 . Thus dp(2k ) = 2k−4 /2k = 2−4 . (ii) Let n = ⌊log2 S⌋, so S ∈ [2n , 2n+1 ) with S ̸= 2n (since S is not a power of two). Because S ≥ 2−6 , the binade [2n , 2n+1 ) lies in the normal region, so it has LSB = 2n−3 . The range [0, S] overlaps with this binade, hence dp(S) = 2n−3 /S. Since S < 2n+1 , dp(S) > 2n−3 /2n+1 = 2−4 .1 (iii) For S > 448, both terms in max(32/S, 2(1 − 448/S)) exceed 2−4 over disjoint sub-ranges: 32/S > 2−4 for S < 512, and 2(1 − 448/S) > 2−4 for S > 448 · 1−21 −5 ≈ 462.5; the two intervals overlap on (448, 512), covering the full S > 448 regime.
Proof. By (C1), S = 2k for some integer k ≥ 0 (using the standing S ≥ 1 assumption). By Theorem 6, the set {2k : k ≥ 0} partitions into {20 , . . . , 28 } where dp = 2−4 and {2k : k ≥ 9} where dp > 2−4 (Theorem 6(iii)). Thus (C2) admits exactly the nine candidates {1, 2, 4, . . . , 256}. (C3) selects the maximum element of this set, k = 8, giving S = 256.
(maximum normal cover-
Remark 4 (Discriminating power of the conditions). Of the three conditions, (C3) alone—“pick the largest 2k ≤ 448”— uniquely identifies S = 256 among power-of-two scales. (C1) and (C2) jointly leave nine candidates and merely articulate why non-2k scales and 2k with k ≥ 9 are dominated. The value of (C2) is therefore explanatory rather than discriminating: it makes the lower-envelope structure of the dp sawtooth (Figure 1) explicit, and shows that the bit-exactness preference (C1) does not come at the cost of quantization-step optimality. Finally, (C3) is a within-2k statement: if bit-exactness is dropped, S = 448 achieves strictly larger normal coverage (§5). The optimality claimed here is thus for the bit-exact family, and the practical 256-vs-448 gap is the empirical dp (worst-case step) effect, not a coverage advantage.
Figure 1 shows dp(S) computed by exact enumeration of all 126 positive E4M3 values, confirming the sawtooth structure and power-of-two lower envelope.
5.3
(minimum quantization step; Theo-
Condition 3: Maximum Normal Coverage
Among the power-of-two candidates satisfying S ≤ 448 (i.e., S ∈ {1, 2, 4, . . . , 256}), larger S is better because the E4M3 normal-region lower bound in the P-domain scales as 2−6 /S:
Comparison: S = 256 vs S = 448. dp(448) = 32/448 ≈ 0.0714 exceeds dp(256) = 0.0625 by 14%. In MSE terms: (0.0714/0.0625)2 ≈ 1.30, predicting ∼30% higher MSE for S = 448. Our experiments (§6) measure 10–15% MSE difference, consistent with the bound (the bound is worst-case; average-case is milder, and 448’s better deep-tail coverage partly offsets its larger worst-case step).
1 Equivalently, dp(S) = 2n−3 /S < 2n−3 /2n = 2−3 , so dp(S) ∈ (2−4 , 2−3 ) for non-power-of-two S ∈ [2−6 , 448]. The range S < 2−6 (entirely subnormal) is excluded for cleanliness; the conclusion still holds there since the uniform subnormal LSB 2−9 gives dp(S) = 2−9 /S > 2−3 , but the practical scope of this paper is S ≥ 1.
5
Experimental Validation
6.1
Kernel-Faithful Simulation
FP8 P-cast MSE under Attention Sink (N = 4096, kernel-faithful) MSE vs FP32 reference
6
We simulate the FP8 attention kernel with semantics matched to production code (specifically, the attention_with_kvcache_prefill_fp8 kernel in hpc-ops and FlashAttention-3’s Hopper/Blackwell backend): • ℓ (running sum): FP32, from pre-cast P (line 8 of Alg. 1).
Forward, S=1 Reverse, S=1 Forward, S=448 Forward, S=256 Reverse, S=256
10 5
10 6 4
• O (output): FP32 accumulator, from post-cast Pfp8 × V . • P-cast: round-to-nearest E4M3 (values below 2−10 → 0). %
To isolate the P-cast effect, we keep Q, K, V in FP32. The QKV FP8 quantization is an orthogonal concern (handled by separate qscale/kscale/vscale) and does not interact with the P-cast precision loss.
6.3
Forward, S=1 Reverse, S=1 Forward, S=448 Forward, S=256 Reverse, S=256
fwd, no scale rev, no scale fwd, max_normal fwd, 28 rev, 28
8
9
Sink strength
10
11
12
13
P zeroed by S=256 (%) Non-sink prob. mass (%) P zeroed by S=1 (%)
4
5
6
7
8
9
10
11
12
13
Figure 2: Top: MSE vs. sink strength. Forward+S=1 exhibits a peak at ∆ = 7–8 (the transition region). All other configurations remain at the quantization-noise floor. Bottom: Diagnostic—non-sink probability mass (blue bars) and fraction of P values zeroed by S=1 cast (red bars) vs. S=256 cast (green line).
We compare five configurations spanning the design space: Parameters
7
50 0
Configurations
Label
6
Probability mass at risk vs. P-zeroing fraction
100
• Epilogue: Ofinal = O/(S · ℓ).
6.2
5
Matches hpc-ops (prior) — FlashInfer/TRT This work FA3/4
Table 2: MSE (×10−5 ) at selected configurations.
Experiment 1: Sink Strength Sweep
We sweep ∆ ∈ [4, 13] with N = 4096, d = 128, qlen = 32, B = 64, ksink = 4, over 20 seeds. Figure 2 shows both the MSE results and the underlying physics (P-zeroing fraction and mass at risk).
Configuration
∆=7 N =4096
∆=7 N =8192
∆=7 N =16384
Forward, S=1 Reverse, S=1 Forward, S=448 Forward, S=256 Reverse, S=256
5.65 1.70 1.81 1.64 1.64
4.40 0.83 0.90 0.80 0.81
2.94 0.32 0.32 0.28 0.28
Ratio (FwdS=1 / best)
3.4×
5.5×
10.5×
Key findings:
6.6 Comparison with Mainstream Implementations
1. Forward+S=1 is 3.4× worse than optimized configurations at ∆ = 7.
4. At ∆ ≥ 11, all configurations converge (non-sink mass < 2%).
Table 3 positions the implementations in terms of both design choices. Based on this analysis, we have adopted S = 256 in the hpc-ops codebase. FlashAttention-3/4 and the updated hpcops both satisfy the optimality conditions; FlashInfer and TensorRT-LLM XQA could gain an additional 10–15% precision improvement by switching from S = 448 to S = 256.
6.4
7
2. Both reverse (any S) and S = 256 (any direction) independently fix the issue. 3. S = 256 gives ∼10–15% lower MSE than S = 448, consistent with the dp ratio.
Experiment 2: Sequence Length Sweep
At ∆ = 7, we sweep N ∈ [512, 16384]. As N grows, the nonsink probability mass increases, amplifying the P-collapse effect. Figure 3 shows the results.
6.5
Discussion
7.1 Saturation: Same Mechanism, Same Floor Both optimizations address the identical failure mode: P values falling below E4M3’s representable range. Once either fix is applied, P-collapse is eliminated and the residual MSE is set by the inherent E4M3 quantization noise on representable
Numerical Results
Table 2 reports exact MSE values for key operating points: 6
MSE vs FP32 reference
FP8 P-cast MSE vs sequence length ( = 7, kernel-faithful)
scales break this alignment, leaving dp(S) on the rising portion of the sawtooth (Figure 1). The P-collapse threshold ∆c = 6.93 + ln S − δk corresponds to the sink strength at which the median non-sink P value falls below the round-tozero boundary 2−10 /S; below this threshold the cast is wellconditioned, above it most of the non-sink mass is silently zeroed.
Forward, S=1 Reverse, S=1 Forward, S=448 Forward, S=256 Reverse, S=256
10 5
7.3 512
1024
2048
4096
Sequence length N
8192
1. For forward-order kernels (e.g., TRT-LLM XQA): Add S = 256 P-scaling. Implementation: multiply P by 256 before cast; divide output by 256 in epilogue. Both operations are bit-exact (§5) and add only two FMAs whose latency is negligible against the tensor-core matmul. We have applied this change to the hpc-ops kernel.
16384
Figure 3: MSE vs. sequence length at ∆ = 7. The improvement of Forward+S=256 over Forward+S=1 grows from 1.3× (N =512) to 10× (N =16384) as non-sink probability mass increases.
2. For kernels already using S = 448 (FlashInfer, SageAttention2): Switch to S = 256 for 10–15% MSE reduction at negligible cost.
Table 3: Design choices in production FP8 attention kernels. “Optimal” denotes joint satisfaction of (C1)–(C3) of Theorem 7 and reverse iteration (which is redundant given S=256 but offers belt-and-suspenders robustness against multi-sink patterns). “Near-optimal” denotes one fix applied (here, reverse iteration eliminates P-collapse) but a non-power-of-two scale incurs the dp(448)/dp(256) = 1.14 residual penalty (§5). “Suboptimal scale” denotes the same scale issue without the iteration-order safety net. Implementation
Order
S
2k ?
Status
FlashAttention-3/4 hpc-ops (updated) hpc-ops (prior) FlashInfer TensorRT-LLM XQA SageAttention2
Rev Fwd Fwd Rev Fwd Fwd
256 256 1 448 448 448
✓ ✓ ✓ × × ×
Optimal Optimal S=1 baseline Near-optimal Suboptimal scale Suboptimal scale
3. For kernels already using reverse + S = 256 (FA3/4): No change needed—this is already optimal among staticscale strategies.
7.4
Limitations and Future Work
Worst-case vs. average-case. The dp(S) analysis provides the minimax-optimal scale assuming P uniformly spans [0, 1]. In practice, the post-softmax distribution is highly non-uniform (a few large values, many small). A dynamic perblock scale (still constrained to 2k for bit-exactness) could yield better average-case precision by adapting to the perblock P distribution. Even so, for typical small P (∼0.01– 0.05) the static S = 256 maps into mid-range binades with competitive relative precision, and its no-underflow guarantee remains the main advantage. Interaction with QKV quantization. We isolated the Pcast effect by keeping Q, K, V in FP32. In production, these are also in FP8, contributing an additive noise floor. Since both reference and kernel use identical dequantized QKV values, the P-cast MSE signal is preserved, but the relative improvement ratio is compressed at high delta where QKV noise dominates. No end-to-end metric. All our numbers are isolated output MSE against an FP32 reference; we report no perplexity or task-accuracy. The 3–10× figures show the P-cast error is real and removable, but not that removing it shifts a downstream metric—where the FP8 QKV floor dominates, the end-toend benefit may be small. We thus frame the S=256 switch as zero-cost and strictly-no-worse rather than a guaranteed quality win, to be confirmed per model. Scope of the dp(S) analysis. The dp(S) construction assumes (a) a bounded range (here P ∈ [0, 1]), (b) one static scale per cast, and (c) a bit-exactness requirement. These break down elsewhere: MXFP4’s E8M0 block exponent already forces scales to 2k (removing the freedom dp(S) addresses), and outlier-heavy activations need per-channel smoothing first. We thus make no claims beyond E4M3 Pcast.
values. Paired t-tests over 100 instances confirm that Forward+S=256 and Reverse+S=256 are statistically indistinguishable wherever P-collapse is active (∆ ≤ 9); for ∆ ≥ 10 the residuals diverge with reverse marginally better, but the absolute gap is ∼ 10−8 , three orders of magnitude below the MSE itself, so the practical conclusion is unchanged (Appendix B). This has a practical implication: either optimization alone suffices. The choice between them should be driven by engineering constraints (reverse may simplify causal mask handling; forward may offer better memory access patterns).
7.2
Practical Recommendations
Structural Interpretation
The role of S = 2k admits a simple geometric reading: every E4M3 normal binade [2n , 2n+1 ) contains 8 equispaced points, differing only in absolute scale. The quantity dp(S) = LSB(S)/S measures the normalized granularity; at S = 2k , both numerator and denominator double simultaneously, locking their ratio at 2−4 . Non-power-of-two 7
Multi-sink patterns. Our analysis assumes a single sink cluster at position 0. Some models exhibit distributed sink patterns across the sequence; the P-collapse analysis generalizes straightforwardly by replacing ∆ with the per-block maximum gap.
8
accurate attention with asynchrony and low-precision. arXiv preprint arXiv:2407.08608, 2024. [7] Mingjie Sun, Xinlei Chen, J. Zico Kolter, and Zhuang Liu. Massive activations in large language models. arXiv preprint arXiv:2402.17762, 2024. [8] Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. International Conference on Learning Representations, 2024.
Conclusion
We have analyzed two implementation-level precision considerations for FP8 E4M3 attention: (1) P-collapse under Attention Sink is a quantifiable threshold effect—activating around ∆ ≈ 6–7 with leading-order underflow fraction F = Φ(∆ + δk − 6.93 − ln S), where δk is the within-sink extreme-value shift—that both reverse iteration and S = 256 scaling independently eliminate; (2) the dp(S) sawtooth, defined over the E4M3 number line, characterizes S = 256 via three conditions—bit-exact arithmetic, the lower-envelope minimum step (dp = 2−4 ), and the largest normal-range coverage among 2k scales. Kernel-faithful experiments measure 3–10× MSE improvement at the critical transition region (∆ = 5–9, N = 4096–16384), and paired statistical tests show that both optimizations reach the same precision floor when combined. For practitioners, the actionable recommendation is simple: for any FP8 attention kernel currently using S = 1 (direct cast) or S = 448 (max-normal), switching to S = 256 is a single-constant, bit-exact change that removes P-collapse and is never worse on P-cast MSE; whether it moves an end-toend metric should be confirmed per model (§7). We make no transfer claims to other formats (MXFP4, FP4) or per-channel quantization, where the preconditions differ (§7).
[9] Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia Wei, Jun Zhu, and Jianfei Chen. SageAttention2: Efficient attention with thorough outlier smoothing and per-thread INT4 quantization. arXiv preprint arXiv:2411.10958, 2024. [10] Jintao Zhang, Xiaoming Xu, Jia Wei, Haofeng Huang, Pengle Zhang, Chendong Xiang, Jun Zhu, and Jianfei Chen. SageAttention2++: A more efficient implementation of SageAttention2. arXiv preprint arXiv:2505.21136, 2025.
A
Complete dp(S) Verification
We verify Theorem 6 by exhaustive computation over all 126 positive E4M3 values: for each S, take the binade with maximum LSB overlapping [0, S] and compute dp(S) = LSBmax /S. Power-of-two scales (k = 0, 1, . . . , 8): all give dp = 2−4 = 0.0625 exactly. Top binade is [2k−1 , 2k ) with LSB = 2k−4 . For k = 9 (S = 512): overflow, dp = 0.25. Non-power-of-two scales: dp(3) = 0.083, dp(100) = 0.080, dp(250) = 0.064, dp(300) = 0.107, dp(448) = 0.071. All strictly exceed 2−4 . See experiments/proof_mathematical.py for full enumeration.
References [1] Federico Barbero, Álvaro Arroyo, Xiangming Gu, Christos Perivolaropoulos, Michael Bronstein, Petar Veličković, and Razvan Pascanu. Why do LLMs attend to the first token? arXiv preprint arXiv:2504.02732, 2025.
B
Saturation Statistical Verification
We test H0 : MSE(Fwd+S=256) = MSE(Rev+S=256) via paired t-test (100 instances per condition).
[2] Tri Dao. FlashAttention-2: Faster attention with better parallelism and work partitioning. International Conference on Learning Representations, 2024.
∆ 7 7 10 12
[3] Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. Advances in Neural Information Processing Systems, 35, 2022.
N
Mean diff
t
Sig?
4k 16k 16k 16k
−7×10−9
−1.0 −1.1 −4.4 −3.6
No No Yes∗ Yes∗
−3×10−9 −2×10−8 −1×10−8
∗ Where significant, reverse is marginally better (not worse). All differences are ∼ 10−8 , negligible vs. MSE values of 10−5 –10−6 , confirming algorithmic equivalence.
[4] Xiangming Gu, Tianyu Pang, Chao Du, Qian Liu, Fengzhuo Zhang, Cunxiao Du, Ye Wang, and Min Lin. When attention sink emerges in language models: An empirical view. arXiv preprint arXiv:2410.10781, 2024. [5] Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, et al. FP8 formats for deep learning. arXiv preprint arXiv:2209.05433, 2022. [6] Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. FlashAttention-3: Fast and
8