Revisiting Cauchy-Schwarz Norm Bounds for SDC Detection in LLM Inference

Revisiting Cauchy–Schwarz Norm Bounds for SDC Detection in LLM Inference Anton Altenbernd

Dylan Everingham

Philipp Wiesner

Odej Kao

Technische Universität Berlin Berlin, Germany a.altenbernd@tu-berlin.de

Technische Universität Berlin Berlin, Germany everingham@tu-berlin.de

University of Cambridge Cambridge, United Kingdom pw560@cam.ac.uk

Technische Universität Berlin Berlin, Germany odej.kao@tu-berlin.de

arXiv:2610.18557v1 [cs.DC] 13 Oct 2026

Abstract Large Language Models (LLMs) execute billions of floating-point operations per output token across many hardware accelerators. At this scale, hardware-induced faults can bypass system-level detection mechanisms and cause Silent Data Corruption (SDC). Algorithm-Based Fault Tolerance (ABFT) is the state-of-the-art technique for protecting matrix multiplication from such faults, but it incurs non-negligible computational overhead and requires thresholds to distinguish hardware-induced errors from floatingpoint roundoff. In this paper, we revisit Cauchy–Schwarz Norm Bounds (CSNB) as a low-overhead alternative for detecting SDC during LLM inference. We compare CSNB against ABFT through controlled fault injection and evaluate the runtime overhead of both verification and threshold determination across multiple model sizes, batch sizes, and generation lengths. Our results show that, on average, CSNB retains 99% of the fault detection capability of ABFT for the evaluated dense linear layers while reducing verification overhead by 87% and threshold determination overhead by 67%. For Mixture-of-Experts (MoE) router gate computations, however, CSNB retains only 55% of the fault detection capability of ABFT. These findings suggest that CSNB is a promising low-overhead alternative for protecting dense linear layers in LLM inference, while checksum-based verification remains preferable for the evaluated MoE router gate.

Keywords Large Language Models, Silent Data Corruption, Algorithm-Based Fault Tolerance, Cauchy–Schwarz Inequality, Fault Detection ACM Reference Format: Anton Altenbernd, Dylan Everingham, Philipp Wiesner, and Odej Kao. 2026. Revisiting Cauchy–Schwarz Norm Bounds for SDC Detection in LLM Inference. In ACM Symposium on Cloud Computing V.2 (SoCC ’26), November 18–20, 2026, Singapore, Singapore. ACM, New York, NY, USA, 7 pages. https: //doi.org/10.1145/3828161.3857897

1

Introduction

Since the introduction of the Transformer architecture [27], Large Language Models (LLMs) have become a central component of modern artificial intelligence systems. Large-scale LLM inference is executed across many hardware accelerators [20] and requires

This work is licensed under a Creative Commons Attribution 4.0 International License. SoCC ’26, Singapore, Singapore © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2816-7/2026/11 https://doi.org/10.1145/3828161.3857897

billions of floating-point operations to generate each output token [16]. Consequently, even low-probability hardware faults can affect inference, making reliable execution an important concern. A major challenge arises from Silent Data Corruption (SDC), which silently bypasses system-level detection mechanisms such as Error-Correction Codes (ECCs) [19]. As a result, incorrect computations may remain undetected for extended periods, compromising application correctness and complicating debugging efforts [2, 8, 17]. SDC can be caused by transient faults induced by external factors such as radiation [3, 18, 29], as well as by internal hardware faults resulting from manufacturing defects or component aging [8, 12, 28]. Although these faults are relatively rare in conventional computing environments, their likelihood increases in large-scale systems due to the sheer number of devices, sustained workload-induced stress, and elevated operating temperatures [7, 28]. Emerging proposals for space-based data centers [1] further motivate research in this area. Transformer-based inference is largely composed of linear operations, including the query, key, value, and output projections in the attention mechanism, the gate, up, and down projections in the feed-forward network, the Mixture of Experts (MoE) routing, and the language modeling head [23, 27]. These operations are implemented as matrix multiplications, account for the majority of the computational workload [11], and, due to their linearity, satisfy algebraic invariants that make them a natural target for fault-tolerance mechanisms.

An alternative to ABFT. The state-of-the-art technique for protecting matrix multiplications underlying linear layers is AlgorithmBased Fault Tolerance (ABFT). It exploits the algebraic structure of matrix multiplication by augmenting the input operands with checksums, providing error detection with significantly lower overhead than redundant execution [13]. However, ABFT still introduces non-negligible computational overhead and requires threshold determination to compensate for floating-point roundoff errors, which can result in loose verification bounds and additional overhead [9]. This raises the question of whether simpler verification mechanisms can provide comparable fault detection while reducing runtime overhead for LLM inference. An alternative approach exploits the Cauchy–Schwarz inequality [24] to derive an upper bound on the magnitude of linear layer outputs that can be checked during execution. Despite its conceptual simplicity and low computational cost, the resulting bound is tight only when the corresponding vectors are linearly dependent, which can make it overly conservative for error detection. However, to the best of our knowledge, the potential of Cauchy-Schwarz

SoCC ’26, November 18–20, 2026, Singapore, Singapore

Norm Bounds (CSNB) for SDC detection during LLM inference has not been investigated. Contributions. In this paper, we revisit CSNB as a low-overhead alternative for detecting SDC during LLM inference and, to the best of our knowledge, provide the first systematic evaluation of normbound-based verification in this setting. We make the following contributions: • We compare CSNB against V-ABFT [9], a recent ABFT extension with online threshold determination, through controlled fault injection across four dense LLMs and a representative MoE model, spanning both prefill and decoding and fault multiplicities from 1 to 256 bit flips. • We quantify the runtime overhead of both the verification stage and threshold determination. The verification stage is implemented directly within fused Triton-based [26] matrix multiplication kernels, whereas threshold determination is evaluated separately using a Python implementation across varying model sizes, batch sizes, and generation lengths. • We show that CSNB achieves comparable fault detection capability to V-ABFT on the evaluated dense linear layers at substantially lower overhead. Checksum-based verification remains preferable for the evaluated MoE router gate.

2

Background

This section introduces the two verification mechanisms compared throughout the remainder of the paper. Algorithm-Based Fault Tolerance (ABFT). In the classical ABFT scheme for matrix multiplication [13], the objective is to protect the computation 𝐶 = 𝐴𝐵, (1) where 𝐴 ∈ R𝑚×𝑘 , 𝐵 ∈ R𝑘 ×𝑛 , and 𝐶 ∈ R𝑚×𝑛 . The key idea is to augment the input matrices with checksum rows and/or columns before computation. These checksums are propagated together with the original data during the matrix multiplication, allowing the correctness of the result to be verified without recomputing the entire operation. For row checksums, the augmented computation is given by     𝐴 𝐶 𝐵= ⊤ , (2) ⊤ (1 𝐴) 1 𝐶 where the last row represents the propagated checksum of the output matrix. Verification is performed by comparing this propagated checksum with the checksum recomputed from the computed result, 𝑅 = 1⊤𝐶 − (1⊤𝐴)𝐵. (3) In exact arithmetic, where 𝐶 = 𝐴𝐵, the residual satisfies 𝑅 = 0. In floating-point arithmetic, however, rounding errors accumulate during both the matrix multiplication and checksum propagation, causing 𝑅 to be nonzero even in the absence of hardware faults. Consequently, practical ABFT implementations verify the result by checking ∥𝑅∥ ≤ 𝜏, (4) where 𝜏 is a verification threshold chosen to distinguish floatingpoint roundoff from SDC [4, 9]. The effectiveness of ABFT therefore depends critically on selecting a threshold that is sufficiently large

Anton Altenbernd, Dylan Everingham, Philipp Wiesner, and Odej Kao

to avoid false positives while remaining small enough to detect faults reliably. V-ABFT [9] addresses threshold selection by directly modeling the ABFT verification difference caused by floating-point rounding. It derives a variance-based adaptive threshold using row-wise statistics such as the mean, minimum, and maximum. The authors report that this yields tighter thresholds than A-ABFT [4], an earlier analytical threshold determination method, while maintaining a zero false-positive rate in the evaluated configurations. In this work, we compare against V-ABFT with respect to fault detection capability and runtime overhead of threshold determination, as it is the most recent floating-point-aware ABFT method and provides a tighter threshold determination than A-ABFT. Cauchy–Schwarz Norm Bounds (CSNB). An alternative approach exploits the Cauchy–Schwarz inequality [24] to derive bounds on the output entries of the matrix multiplication. The Cauchy–Schwarz inequality provides an upper bound on the magnitude of an inner product. For vectors 𝑥, 𝑦 ∈ R𝑘 , it states that |𝑥 ⊤𝑦| ≤ ∥𝑥 ∥ 2 ∥𝑦 ∥ 2,

(5)

with equality if and only if 𝑥 and 𝑦 are linearly dependent. Since each entry of a matrix product is an inner product between a row of 𝐴 and a column of 𝐵, the inequality immediately yields |𝐶𝑖 𝑗 | = |𝑎𝑖⊤𝑏 𝑗 | ≤ ∥𝑎𝑖 ∥ 2 ∥𝑏 𝑗 ∥ 2,

(6)

where 𝑎𝑖⊤ denotes the 𝑖-th row of 𝐴 and 𝑏 𝑗 denotes the 𝑗-th column of 𝐵. Applying this bound to every output entry yields an 𝑚 × 𝑛 matrix of bounds, requiring the storage of all row and column norms. To reduce the storage overhead, a practical alternative is to use a single global bound based on the largest row and column norms, |𝐶𝑖 𝑗 | ≤ max ∥𝑎𝑖 ∥ 2 max ∥𝑏 𝑗 ∥ 2, 1≤𝑖 ≤𝑚

1≤ 𝑗 ≤𝑛

∀ 𝑖, 𝑗 .

(7)

In exact arithmetic, every output entry satisfies the Cauchy– Schwarz bound by construction. In floating-point arithmetic, both the computed matrix entries and the corresponding norms are affected by rounding errors. Unlike ABFT, which verifies an equality, CSNB verifies a conservative inequality. The bound is tight only when the corresponding vectors are linearly dependent and becomes even more conservative when using the global bound. In this work, we therefore evaluate outputs directly against the bound without deriving an explicit floating-point verification threshold. The resulting false-positive behavior is evaluated experimentally.

3

Methodology

We first describe the fault injection methodology before introducing the detection and runtime evaluation experiments. Experimental setup. For every experiment, the detection mechanism is incorporated by replacing selected linear layer modules with detection-enabled implementations while preserving the original model parameters and inference procedure. Inference is performed in bfloat16 precision. All experiments are conducted on NVIDIA L40S GPUs, using a single GPU for the dense models and two GPUs for the MoE model.

Revisiting Cauchy-Schwarz Norm Bounds for SDC Detection in LLM Inference

Evaluation is performed using 1,000 prompts from each of the GSM8K [5] and SQuAD v2 [22] datasets. GSM8K provides mathematical reasoning tasks, whereas SQuAD v2 consists of extractive question-answering examples. The prompts are sampled uniformly at random, and for each prompt, faults are injected during either the prefill or autoregressive decoding phase of inference and for fault multiplicities of 1, 2, 64, and 256 bit flips, resulting in 8,000 fault injection runs per model, dataset, and method. Faults are injected into the output activations immediately after the linear layer computation. If the fault is injected during decoding, the decoding step is sampled uniformly at random. A linear layer is sampled uniformly from the protected layers, followed by a random output tile (64×64 elements) and a randomly selected bit position of the bfloat16 representation. Depending on the evaluated fault multiplicity, 1, 2, 64, or 256 output activations are selected uniformly at random within the chosen output tile, and the selected bit position is flipped in each selected activation. This tile-based injection approximates the spatial locality of hardware-induced corruptions, while the affected activations and bit position are selected randomly as hardware-specific fault patterns are architecture-dependent. Detection experiments. For every injection, the generated token sequence is compared against a fault-free reference execution to determine whether the injected fault is masked or results in an SDC. We therefore evaluate whether the generated sequence changes, rather than the semantic correctness of the model output. Detection is evaluated independently of whether the generated sequence changes. Consequently, every fault injection falls into one of four categories: masked and undetected, masked and detected, SDC and undetected, or SDC and detected. The corresponding fault-free reference executions are used to assess false-positive behavior. The first experiment evaluates the SDC detection capability of CSNB on dense transformer architectures. We compare CSNB against V-ABFT using four LLMs representing different transformer families: Qwen2.5-7B [21], Mistral-7B [14], Falcon3-7B [25], and Llama3-8B [10]. For each model, every linear layer is replaced by its corresponding detection-enabled implementation. Matrix multiplications performed within FlashAttention [6] are excluded as kernel fusion with the attention computation prevents the direct application of fault detection schemes. The second experiment evaluates the SDC detection capability of CSNB for router gate computations in MoE architectures. We compare CSNB against V-ABFT using Mixtral-8x7B [15] as a representative MoE model. Unlike the first experiment, only the linear layers implementing the router gate are replaced by their corresponding detection-enabled implementations. This isolates the effectiveness of the proposed detection mechanism for routing decisions, which determine the expert assignment for each token and therefore constitute a critical point in the inference path. Runtime evaluation. We separately evaluate the runtime overhead of verification and threshold determination. The runtime evaluation uses Qwen2.5-3B, Qwen2.5-7B, and Qwen2.5-14B [21] with a fixed input prompt of 256 tokens. Preliminary runtime measurements across the evaluated dense transformer architectures showed no meaningful architectural differences. Therefore, the runtime evaluation focuses on the Qwen2.5 family and studies the impact of model size, batch size, and generation length. We do not perform a

SoCC ’26, November 18–20, 2026, Singapore, Singapore

corresponding runtime evaluation for the MoE model because fault detection is applied only to the router gate computations, which represent only a small fraction of the overall inference workload. End-of-sequence token generation is disabled to ensure that every inference run generates the configured number of output tokens. Batch sizes of 1, 2, 4, 8, 16, 32, and 64 and generation lengths of 1, 64, and 256 new tokens are considered. For every model, implementation variant, batch size, and generation length, five warm-up inference runs are performed and excluded from the reported results. Subsequently, ten inference runs are measured and averaged. The verification stage is implemented within fused Triton [26] matrix multiplication kernels. We compare the baseline implementation without SDC detection against ABFT and CSNB. Verification overhead is reported relative to the baseline implementation without SDC detection. Threshold determination is evaluated independently using a Python implementation. Since no optimized implementation of threshold determination is publicly available for V-ABFT, and its implementation overlaps threshold determination with the matrix multiplication, we evaluate both V-ABFT and CSNB independently of the matrix multiplication. This avoids confounding the comparison with implementation-specific optimizations, which could similarly be applied to CSNB, and instead compares the computational cost of the underlying threshold determination algorithms. We compare V-ABFT and CSNB, as well as variants that precompute quantities derived from the model parameters during initialization, namely parameter norms for CSNB and the parameter statistics used to derive the V-ABFT thresholds.

4

Evaluation

This section shows that CSNB achieves fault detection capability comparable to V-ABFT for the evaluated dense linear layers while substantially reducing runtime overhead, but is less effective for MoE router gate computations. Comparable detection in dense models. Figure 1a compares the overall fault detection outcomes of CSNB and V-ABFT across four transformer-based dense LLMs on GSM8K and SQuAD v2. Across both datasets and all evaluated models, the two approaches exhibit highly similar detection behavior. The majority of injected faults are masked by the network, while the fraction of undetected SDC remains nearly identical for both methods. These results indicate that replacing checksum-based verification with CSNB does not noticeably reduce fault detection capability for dense transformer inference. In addition, neither CSNB nor V-ABFT produced falsepositive detections in any fault-free execution. KV caching enables fault propagation. Although both CSNB and V-ABFT detect most large perturbations, the fault injection experiments still exhibit a fraction of undetected SDCs, including faults affecting low-order mantissa bits. We therefore investigate whether KV cache propagation contributes to the observed SDCs. Figure 1b repeats the experiment with KV caching disabled. With KV caching enabled, corrupted key–value activations are stored in the cache and reused during subsequent decoding steps, allowing faults to propagate during autoregressive generation. Disabling KV caching removes this propagation path by recomputing the key–value activations from scratch at every decoding step. As a

SoCC ’26, November 18–20, 2026, Singapore, Singapore

CSNB (darker)

Anton Altenbernd, Dylan Everingham, Philipp Wiesner, and Odej Kao

V-ABFT (lighter)

GSM8K

100%

CSNB (darker)

75%

50%

50%

25%

25%

0%

0% SQuAD v2

100%

75%

50%

50%

25%

25% Qwen2.5-7B

Mistral-7B

SDC + detected Masked + detected

Falcon3-7B

SQuAD v2

100%

75%

0%

GSM8K

100%

75%

V-ABFT (lighter)

0%

Llama3-8B

Masked + undetected SDC + undetected

Qwen2.5-7B

Mistral-7B

SDC + detected Masked + detected

(a) Fault injection with KV caching enabled.

Falcon3-7B

Llama3-8B

Masked + undetected SDC + undetected

(b) Fault injection with KV caching disabled.

Figure 1: Fault detection outcomes by model, dataset, and verification method. Stacked bars show the fraction of fault injection runs in each outcome category. Darker bars with solid outlines denote CSNB, whereas lighter bars with dashed outlines denote V-ABFT. Across the evaluated dense transformer models, CSNB achieves fault detection capability comparable to V-ABFT with and without KV caching. Disabling KV caching prevents corrupted key–value activations from propagating across decoding steps, thereby reducing the fraction of undetected SDC. result, the fraction of SDC decreases, while the relative behavior of CSNB and V-ABFT remains nearly unchanged, indicating that both approaches respond similarly regardless of whether faults propagate through the KV cache. Detection depends on location and multiplicity. Figures 2a and 2b provide a more detailed analysis by reporting detection outcomes for each flipped bit position and fault multiplicity. For dense transformer models (Figure 2a), both approaches exhibit almost identical behavior across all evaluated bit positions. As expected, faults affecting the exponent bits produce a substantially higher fraction of detected SDC than faults confined to the mantissa, while increasing the number of injected bit flips increases the overall detection rate for both methods. In contrast, the behavior differs for MoE router gate computations (Figure 2b). While both approaches detect faults affecting the most significant exponent bits with high probability, V-ABFT also detects a noticeable fraction of faults affecting the mantissa and remaining exponent bits, whereas CSNB detects only a small fraction of such corruptions. Since router outputs directly determine expert selection, comparatively small numerical perturbations may alter routing decisions, making these computations more sensitive to verification accuracy. Consequently, CSNB exhibits lower fault detection capability than V-ABFT for router gate computations. CSNB reduces runtime overhead. Figure 3 presents the results of the runtime evaluation. The reported results are averaged across batch sizes to emphasize the impact of model size and generation lengths. While threshold determination exhibits little dependence

on batch size, the verification overhead of ABFT becomes closer to that of CSNB at the largest evaluated batch size (64). Figure 3a reports the runtime overhead of the verification stage relative to the baseline implementation without SDC detection. Across all evaluated configurations, CSNB consistently incurs substantially lower verification overhead than ABFT. While ABFT verification introduces approximately 6–13% overhead depending on the model size and generation length, the overhead of CSNB remains below 5% and approaches near zero for the larger Qwen2.5-7B and Qwen2.5-14B LLMs. This reduction is expected because CSNB only evaluates a threshold during verification, whereas ABFT additionally computes and compares propagated checksums. Figure 3b presents the runtime of threshold determination, normalized to V-ABFT. Both V-ABFT and CSNB benefit from precomputing model-dependent quantities during initialization, reducing threshold determination time by 40–75% and 55–87%, respectively, compared to their online implementations. Even without precomputation, CSNB requires only about one-third of the runtime of V-ABFT, while the precomputed CSNB variant reduces this cost to approximately 4–15%.

5

Discussion

This section discusses the implications of the experimental results, highlighting where CSNB provides an effective alternative to VABFT, where checksum-based verification remains preferable, and the limitations of both approaches. CSNB as a practical alternative. For the evaluated linear layers in dense LLMs, CSNB achieves fault detection performance

Revisiting Cauchy-Schwarz Norm Bounds for SDC Detection in LLM Inference

CSNB (darker)

V-ABFT (lighter)

1 flipped bit

100%

SoCC ’26, November 18–20, 2026, Singapore, Singapore

CSNB (darker)

75%

50%

50%

25%

25%

0%

0% 2 flipped bits

100%

75%

50%

50%

25%

25%

0%

2 flipped bits

100%

75%

0% 64 flipped bits

100%

75%

50%

50%

25%

25%

0%

0% 256 flipped bits

100%

75%

50%

50%

25%

25% 1

2

3

4

5

SDC + detected Masked + detected

6 7 8 9 10 11 12 13 14 15 Bit position Masked + undetected SDC + undetected

(a) Dense transformer models.

256 flipped bits

100%

75%

0

64 flipped bits

100%

75%

0%

1 flipped bit

100%

75%

V-ABFT (lighter)

0%

0

1

2

3

4

5

SDC + detected Masked + detected

6 7 8 9 10 11 12 13 14 15 Bit position Masked + undetected SDC + undetected

(b) MoE router gate computations.

Figure 2: Fault injection outcomes by flipped bit position, with stacked bars showing the fraction of runs in each category. Darker bars with solid outlines denote CSNB, whereas lighter bars with dashed outlines denote V-ABFT. For both dense transformer models and MoE router gate computations, fault detection capability increases with the number of injected bit flips. While CSNB and V-ABFT exhibit nearly identical fault detection behavior for dense transformer models, V-ABFT achieves higher fault detection capability for MoE router gate computations, particularly for faults affecting mantissa and lower exponent bits. comparable to V-ABFT while requiring less computation during both runtime verification and threshold determination. This provides an attractive trade-off for deployment in large-scale inference systems, where every generated token requires many linear layer evaluations and even small per-layer overheads accumulate over long decoding sequences. Despite the conservative upper bound provided by the Cauchy–Schwarz inequality, one possible explanation for the comparable detection performance is that the adaptive thresholds of V-ABFT are likewise conservative for these layers. Since these thresholds are derived from statistical properties of the inputs, they may become loose depending on input distributions. Consequently, both approaches may employ verification bounds of similar practical tightness.

Verification depends on the protected computation. The reduced effectiveness of CSNB for MoE router gate computations highlights a limitation of norm-bound-based verification. Unlike the evaluated dense linear layers, the router computes logits whose relative ordering determines expert selection. Consequently, comparatively small perturbations may alter routing decisions even when the resulting activations remain within the corresponding verification bound. One possible explanation for the difference in detection capability lies in the verification thresholds. The adaptive threshold of V-ABFT depends on the number of output columns 𝑛 and the statistical properties of the input matrices [9]. The evaluated Mixtral-8x7B router gate has only 𝑛 = 8 output columns, whereas the evaluated dense linear layers have substantially larger

SoCC ’26, November 18–20, 2026, Singapore, Singapore

ABFT

CSNB

V-ABFT

1 generated token

15% 10%

Anton Altenbernd, Dylan Everingham, Philipp Wiesner, and Odej Kao

12.88%

75% 50%

5% 1.23%

64 generated tokens

15% 10%

0.21%

5%

0.22%

256 generated tokens

10%

0%

28% 30% 14%

50%

4.02% 0.18%

Qwen2.5-7B

Qwen2.5-14B

0%

(a) Verification overhead.

32% 4%

256 generated tokens 60% 34%

25% 0.02%

25% 6%

0%

75%

9.55%

8.14%

Qwen2.5-3B

34%

100%

6.11%

5%

60%

50% 0.30%

11%

10%

64 generated tokens

25%

0% 15%

15%

0%

75%

3.75%

33% 35%

33% 33%

33%

100% 9.73%

8.42%

7.64%

CSNB-Pre

55%

25%

0.84%

0%

CSNB

1 generated token

100% 11.44%

10.49%

V-ABFT-Pre

15%

Qwen2.5-3B

27% 30%

25%

32%

6%

4%

Qwen2.5-7B

Qwen2.5-14B

(b) Threshold determination overhead.

Figure 3: Runtime overhead of CSNB and ABFT. Reported runtimes are averaged across all evaluated batch sizes. Left: verification overhead relative to the baseline implementation without SDC detection. Right: threshold determination normalized to V-ABFT (100%). The precomputed (-Pre) variants cache model-dependent statistics during initialization. CSNB consistently incurs lower verification overhead than ABFT, while threshold determination requires substantially less computation than V-ABFT and is further accelerated by precomputing model-dependent statistics during initialization. output dimensions. This may result in a tighter V-ABFT threshold for the router gate, while the global CSNB bound depends on the maximum row and column norms and does not directly tighten as 𝑛 decreases. Consequently, perturbations below the CSNB verification bound may still exceed the V-ABFT threshold. These findings suggest that the suitability of a verification mechanism depends on the protected computation. While CSNB achieves comparable detection capability to V-ABFT for the evaluated dense linear layers, checksum-based verification remains preferable for the evaluated MoE router gate. Whether this limitation extends to other MoE router gates with different numbers of experts or to other linear layers within MoE models has not been evaluated in this work. Selective protection. The presented evaluation assumes that every evaluated linear layer is protected. In practice, however, this may not be necessary. Hardware fault conditions may persist across multiple kernel executions rather than affecting only a single linear layer computation. Consequently, protecting only a subset of strategically selected layers may still provide a high probability of detecting faults while further reducing runtime overhead. Limitations of local verification. Beyond the differences between CSNB and V-ABFT, the presented results also highlight a limitation shared by both approaches. Both techniques verify individual linear layer computations and therefore provide only a local view of

correctness. Faults that remain below the local verification criterion may nevertheless propagate through subsequent transformer layers and eventually affect the generated output. Similarly, once corrupted key or value activations have been written to the KV cache, subsequent decoding steps reuse the corrupted values, as neither method detects errors beyond the protected linear layer. Addressing such error propagation will likely require detection mechanisms that extend beyond single linear layer computations.

6

Conclusion

This paper revisited CSNB for detecting SDC during LLM inference. Through controlled fault injection and runtime evaluation, we compared CSNB against V-ABFT across dense transformer models and MoE routing. Our results show that, on average, CSNB retains 99% of the fault detection capability of V-ABFT for the evaluated dense linear layers while reducing verification overhead by 87% and threshold-determination overhead by 67%. For the evaluated router gate, however, CSNB retains only 55% of the fault detection capability of V-ABFT. These findings suggest that CSNB is a promising low-overhead alternative for protecting dense linear layers in LLM inference, while checksum-based verification remains preferable for the evaluated router gate.

Revisiting Cauchy-Schwarz Norm Bounds for SDC Detection in LLM Inference

References [1] 2025. Starcloud: Why data centers should move to space. https://www.starcloud. com/. Accessed: December, 2025. [2] Anton Altenbernd, Philipp Wiesner, and Odej Kao. 2026. Exploring Silent Data Corruption as a Reliability Challenge in LLM Training. In 2026 IEEE 26th International Symposium on Cluster, Cloud and Internet Computing (CCGrid). IEEE, 157–166. doi:10.1109/CCGrid68966.2026.00025 [3] Robert C Baumann. 2005. Radiation-induced soft errors in advanced semiconductor technologies. IEEE Transactions on Device and materials reliability 5, 3 (2005), 305–316. [4] Claus Braun, Sebastian Halder, and Hans Joachim Wunderlich. 2014. A-ABFT: Autonomous algorithm-based fault tolerance for matrix multiplications on graphics processing units. In 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks. IEEE, 443–454. [5] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021). [6] Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. 2022. Flashattention: Fast and memory-efficient exact attention with io-awareness. Advances in neural information processing systems 35 (2022), 16344–16359. [7] Harish Dattatraya Dixit, Laura Boyle, Gautham Vunnam, Sneha Pendharkar, Matt Beadon, and Sriram Sankar. 2022. Detecting silent data corruptions in the wild. arXiv preprint arXiv:2203.08989 (2022). [8] Harish Dattatraya Dixit, Sneha Pendharkar, Matt Beadon, Chris Mason, Tejasvi Chakravarthy, Bharath Muthiah, and Sriram Sankar. 2021. Silent data corruptions at scale. arXiv preprint arXiv:2102.11245 (2021). [9] Yiheng Gao, Qin Hua, and Zizhong Chen. 2026. V-ABFT: Variance-Based Adaptive Threshold for Fault-Tolerant Matrix Multiplication in Mixed-Precision Deep Learning. arXiv preprint arXiv:2602.08043 (2026). [10] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024). [11] Congjie He, Yeqi Huang, Pei Mu, Ziming Miao, Jilong Xue, Lingxiao Ma, Fan Yang, and Luo Mai. 2025. WaferLLM: Large Language Model Inference at Wafer Scale. In 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25). 257–273. [12] Peter H Hochschild, Paul Turner, Jeffrey C Mogul, Rama Govindaraju, Parthasarathy Ranganathan, David E Culler, and Amin Vahdat. 2021. Cores that don’t count. In Proceedings of the Workshop on Hot Topics in Operating Systems. 9–16. [13] Kuang-Hua Huang and Jacob A Abraham. 1984. Algorithm-based fault tolerance for matrix operations. IEEE transactions on computers 100, 6 (1984), 518–528. [14] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7B. arXiv preprint arXiv: 2310.06825 (2023). [15] Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024. Mixtral of Experts. arXiv preprint arXiv: 2401.04088 (2024). [16] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling Laws for Neural Language Models. arXiv preprint arXiv: 2001.08361 (2020). [17] Jeffrey Jian Ma, Hengzhi Pei, Leonard Lausen, and George Karypis. 2025. Understanding Silent Data Corruption in LLM Training. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vienna, Austria, 20372–20394. doi:10.18653/v1/2025.acl-long.996 [18] Shubhendu S Mukherjee, Joel Emer, and Steven K Reinhardt. 2005. The soft error problem: An architectural perspective. In 11th International Symposium on High-Performance Computer Architecture. IEEE, 243–247. [19] George Papadimitriou, Dimitris Gizopoulos, Harish Dattatraya Dixit, and Sriram Sankar. 2023. Silent data corruptions: The stealthy saboteurs of digital integrity. In 2023 IEEE 29th International Symposium on On-Line Testing and Robust System Design (IOLTS). IEEE, 1–7. [20] Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean. 2023. Efficiently scaling transformer inference. Proceedings of machine learning and systems 5 (2023), 606–624.

SoCC ’26, November 18–20, 2026, Singapore, Singapore

[21] Qwen: An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. 2024. Qwen2.5 Technical Report. arXiv preprint arXiv: 2412.15115 (2024). [22] Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 784–789. [23] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. In International Conference on Learning Representations (ICLR). [24] Gilbert Strang. 2000. Linear algebra and its applications. [25] TII Team. 2024. Falcon 3 family of Open Foundation Models. [26] Philippe Tillet, Hsiang-Tsung Kung, and David Cox. 2019. Triton: an intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages. 10–19. [27] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017). [28] Shaobu Wang, Guangyan Zhang, Junyu Wei, Yang Wang, Jiesheng Wu, and Qingchao Luo. 2023. Understanding silent data corruptions in a large production cpu population. In Proceedings of the 29th Symposium on Operating Systems Principles. 216–230. [29] James F Ziegler. 1996. Terrestrial cosmic rays. IBM journal of research and development 40, 1 (1996), 19–39.

Related documents

• Revisiting Cauchy-Schwarz Norm Bounds for SDC Detection in LLM Inference#1344058
• M2K: Making the Model-Kernel Interface Explicit for Reliable CUDA Kernel Verification#793612
• M2K: Making the Model-Kernel Interface Explicit for Reliable CUDA Kernel Verification#789608
• Calibrating the Checksum: An Empirical False-Positive Bound for One-Sided ABFT on bfloat16 GEMM, and What It Fails to Catch#1186554
• PipeMax: Enhancing Offline LLM Inference on Commodity GPU Servers#155244