Predict Before Replay: Joint FEC and Flight Control for Reliable Scale-Up Links

arXiv:2608.17503v1 [cs.NI] 18 Aug 2026

Predict Before Replay: Joint FEC and Flight Control for Reliable Scale-Up Links Fan Yang

Jiaqi Liu

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China yangfan2020@ict.ac.cn

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China Hangzhou Institute for Advanced Study, UCAS Hangzhou, China liujiaqi242@mails.ucas.ac.cn

Tao Jiang

Zhan Wang

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China jt@ncic.ac.cn

Institute of Computing Technology, Chinese Academy of Sciences Beijing, China wangzhan@ncic.ac.cn

Abstract

buffer. With standard replay, recovery begins at 𝑖 and retransmits the retained suffix. Thus the cost of a residual error depends not only on FEC strength but also on how many flits were admitted while the link was likely bad. A controller that changes coding but leaves exposure unchanged can still create a large replay episode; a controller that waits for an uncorrectable decode is necessarily late. Modern receivers already observe corrected errors before those errors become replay events. Error counts and FEC statistics are especially informative for burst channels caused by crosstalk, equalizer error propagation, or transient signal degradation [25]. The opportunity is to use that early signal for both redundancy and admission, while leaving standard recovery intact. We present PREFACE: pre-FEC adaptive coding and exposure control. PREFACE tracks a two-state burst posterior from pre-FEC symbol-error observations. For each next flit, it compares three coding profiles by balancing the redundancy they add against the expected replay cost of an uncorrected residual error. When the posterior selects strong protection, the controller also caps outstanding traffic by an absolute risk budget. A single burst-risk estimate therefore coordinates both decisions: which FEC profile to use, and how many flits may remain unacknowledged during risky intervals. This paper makes four contributions: • It identifies replay exposure as the missing coupling between adaptive FEC and delayed standard replay in high-rate scaleup links. • It designs PREFACE, a cost-aware burst predictor that jointly controls FEC strength and outstanding flight without changing replay correctness. • It implements a reusable ns-3 model with UALink 200G 1.0inspired standard-replay semantics and serialized reverse control, and evaluates fixed FEC, post-decode adaptation, predictive-FEC, and joint-control baselines. • It shows statistically significant link-level and modeled ring AllReduce gains, a strict joint-control ablation, sensitivity to feedback and buffer size, and a long-run reliability invariant check.

Scale-up accelerator fabrics send latency-sensitive flits over serial links at hundreds of gigabits per second. Their reliability pipeline first relies on FEC, then detects residual failures and replays unacknowledged data. At these line rates, delayed feedback lets later flits enter the replay window before a residual failure is reported, so standard replay can amplify one corrupted flit into a suffix retransmission. This paper presents PREFACE, a pre-FEC controller for temporally correlated burst errors. A two-state Bayesian filter converts corrected-symbol observations into a next-flit burst posterior and jointly selects FEC strength with an outstanding-flit cap. We implement PREFACE in ns-3 with publicly verifiable UALink 200G 1.0 replay semantics. PREFACE improves goodput by 10.52%, lowers P99 latency by 50.75%, cuts replay by 47.52%, and improves modeled ring AllReduce by 13.1–27.0%.

1

Introduction

Scale-up networks connect GPUs, accelerators, and memory devices over links whose latency is short enough that link-local recovery is attractive, yet whose line rate makes every nanosecond of delayed reaction consequential [8, 13, 15]. UALink, Scale Up Ethernet, CXLderived proposals, proprietary accelerator links, and PCIe 6.0-style high-speed I/O all combine error detection, FEC, and some form of replay or retry [10, 14, 18, 23]. These mechanisms are usually studied as separate choices: spend redundancy continuously, or detect an error and retransmit. Strong FEC is highly effective for sparse, approximately independent raw errors. PREFACE targets a different regime: short, temporally correlated bursts in which adjacent flits see elevated symbol-error counts before feedback returns. Such bursts can arise from decision-feedback equalizer (DFE) error propagation, crosstalk, transient signal-integrity loss, or equalizer instability [25]. The remaining challenge is limiting the performance cost when a residual error still activates standard replay. The separation hides a feedback-amplification problem. Suppose a transmitter can retain 𝑊 flits and a burst corrupts flit 𝑖. Until a replay request returns, later flits continue to enter the replay 1

Fan Yang, Jiaqi Liu, Tao Jiang, and Zhan Wang

Table 1: Mechanism positioning by error regime. Approach

Problem setting

Exposure implication

1) Fixed strong FEC

Sparse independent raw errors; always-on protection

2) Post-decode adapt.

Residual failure visible after FEC/CRC

3) Predictive FEC

Corrected-symbol evidence before failure

4) Link-local retry 5) PREFACE

Detected link loss with NACK/retry path Short correlated bursts with early pre-FEC evidence

Lowers residual probability; retained suffix size unchanged Reacts after a replay request and suffix have formed Raises protection early; leaves outstanding flight unchanged Optimizes retry semantics Raises protection and caps flight before feedback returns

standard replay will resend a suffix rather than only the damaged flit. Always-on strong FEC avoids many such events, but its 6.25% modeled wire overhead and 12-ns decode delay apply even in the GOOD state. Predictive FEC can remove unnecessary redundancy; however, if it keeps the same𝑊 during a BAD posterior, the retained suffix remains exposed to a rare residual failure. This motivates a joint action: Use corrected-error evidence before replay, and spend both coding and flight budget according to the predicted cost of the next residual failure.

2.3

 
𝐸𝑖 ≤ min 𝑊 , 𝑂𝑖0 + 𝑇𝑑 /𝑇𝑓 , (1) 0 where 𝑂𝑖 is occupancy at failure. The exact suffix also depends on ACK progress and request serialization, but Eq. (1) exposes the two control levers: FEC reduces the probability that an episode begins, while a flight cap limits its conditional size. For coding action 𝑘 and window 𝑊 , a first-order expected wire-cost decomposition is

2 Background and Motivation 2.1 FEC, replay, and burst errors FEC converts a bounded number of corrupt symbols into extra wire bytes and decode latency [7, 19, 20]. Replay spends bandwidth only after residual corruption, but each replay event also waits for feedback and blocks later payload flits while the retained suffix is retransmitted. RIFL demonstrates efficient link-local recovery in short data-center links with NACK-only, in-band retransmission [21]. UALink-style scale-up links expose a different replay path, with retained transmit state and standard replay behavior [23]; PREFACE evaluates predictive control on this UALink-style model. Independent bit errors are an incomplete stress model for a high-speed electrical link. Decision-feedback equalizer propagation and related channel effects can create multi-symbol bursts [25]. We therefore use the Gilbert–Elliott two-state abstraction [3, 5]: a mostly clean GOOD state and a persistent, high-error BAD state. This controlled burst model makes the mechanism stress explicit; Section 7.2 outlines how measured pre-FEC traces can calibrate the parameters for particular cables, connectors, and post-FEC reliability targets. Table 1 makes the problem boundary explicit. PREFACE targets short correlated bursts where corrected-symbol evidence arrives early enough to change both coding and outstanding flight before delayed replay feedback returns.

2.2

Replay exposure as an amplification term

The key quantity couples the probability of an uncorrectable flit with the amount of still-valid traffic that becomes coupled to it. Let 𝑖 be the first failed sequence, 𝑂𝑖 the outstanding set when its replay request is processed, and 𝑆𝑖 = |{ 𝑗 ∈ 𝑂𝑖 : 𝑗 ≥ 𝑖}| the retained suffix. Standard replay sends at least 𝑆𝑖 payload flits for that episode. If𝑇𝑓 is the flit admission interval and 𝑇𝑑 is decode plus request-return time, the number exposed before feedback is bounded approximately by

𝐶 (𝑘,𝑊 | 𝑞) ≈ 𝑜𝑘 + 𝑟𝑘 (𝑞) E[𝑆 | 𝑊 , 𝑞] + 𝑢 (𝑊 ), (2) where 𝑜𝑘 is redundancy, 𝑟𝑘 residual failure probability, and 𝑢 (𝑊 ) the underfill cost of constraining flight. Optimizing 𝑘 while holding 𝑊 fixed can reduce 𝑟𝑘 yet leave a large conditional replay suffix. Conversely, constraining𝑊 without early evidence pays 𝑢 (𝑊 ) even while the channel is healthy. PREFACE uses the same posterior to approximate this joint decision, but deliberately decomposes it into a coding selector and a two-level flight budget so that the datapath remains simple.

3 PREFACE Design 3.1 Burst posterior PREFACE needs an early link-health signal before a decode failure. A protected profile can expose a corrected-symbol record or coarser high-SER indication without requiring a full raw-bit trace. A pure FEC-off mode exposes CRC failure after the prediction window has passed. We therefore describe the weakest deployable profile as LOW: a minimal monitor/protection mode, or an equivalent PHYside pre-FEC telemetry path. In the simulator, a threshold model supplies synthetic pre-FEC symbol counts to predictive policies; Section 7.2 describes how measured telemetry can replace this signal. For the controller, let 𝑧𝑡 ∈ {0, 1} indicate whether the justreceived flit contained any raw symbol error, including errors corrected by FEC. Let 𝑞𝑡− be the prior probability that flit 𝑡 is in the BAD state, and let 𝑒𝐺 and 𝑒𝐵 be the corruption emission probabilities. The posterior is

Why post-decode adaptation is late

Let a 640-byte flit traverse a 400-Gb/s link. Its payload serialization alone is 12.8 ns. A 50-ns request delay plus propagation spans multiple new flit admissions. A post-decode controller updates only when a codeword is declared uncorrectable. By then, (1) a replay request is inevitable, (2) later flits may occupy TxReplay, and (3) 2

Predict Before Replay

Figure 1: PREFACE uses one pre-FEC burst posterior for two coupled decisions. Standard replay remains the correctness backstop.

3.3

𝑞𝑡− 𝑃 (𝑧𝑡 | 𝐵) 𝑞𝑡 = − . (3) 𝑞𝑡 𝑃 (𝑧𝑡 | 𝐵) + (1 − 𝑞𝑡− )𝑃 (𝑧𝑡 | 𝐺) With GOOD-to-BAD transition probability 𝑎 and BAD-to-GOOD probability 𝑏, the next-flit prior becomes

Let 𝑊𝐻 be the healthy TxReplay/admission window and 𝑊𝑅 an absolute risk budget. PREFACE sets ( min(𝑊𝐻 ,𝑊𝑅 ), 𝑘 ∗ = STRONG, 𝑊𝑡 = (8) 𝑊𝐻 , otherwise. The primary point uses 𝑊𝐻 = 16 and 𝑊𝑅 = 8. The healthy window is a baseline admission-capacity setting chosen for the modeled link. A 640-byte flit serializes in 12.8 ns at 400 Gb/s, while ACK return spans tens to roughly one hundred nanoseconds. Sixteen flits avoid obvious ACK-limited underfill and define about 10 KB of retained-suffix exposure; the risk cap halves this to about 5 KB. A smaller window limits suffix amplification and payload-blocking exposure, but an aggressive contraction can underfill the link. We choose 𝑊𝑅 as an absolute feedback-delay/burst budget rather than as a fixed offset from 𝑊𝐻 . It is selected on training seeds together with 𝜆 and frozen before the primary, robustness, and collective tests. The optimal risk budget can change with burst persistence, load, feedback delay, and FEC costs; online adaptation is future work. Exposure control changes performance, not correctness. ACK processing, sequence ordering, retry limits, and standard replay are unchanged. If the predictor is wrong, the link may spend excess FEC or lose throughput; detected residual errors still follow the baseline replay path.

𝑞𝑡−+1 = 𝑞𝑡 (1 − 𝑏) + (1 − 𝑞𝑡 )𝑎. (4) The primary evaluation fixes (𝑎, 𝑏, 𝑒𝐺 , 𝑒𝐵 ) before held-out testing. The code also contains an online expected-count estimator for future trace-driven extensions. The fixed binary emission model keeps the mechanism results independent of any one decoder’s analog confidence scale.

3.2

Cost-aware FEC selection

The coding menu contains LOW, LIGHT, and STRONG profiles. Profile 𝑘 has normalized overhead 𝑜𝑘 and corrects up to 𝑐𝑘 symbols. STRONG is modeled after a KP4-like RS(544,514) correction strength: 30 parity symbols correct up to 15 symbol errors [7, 9, 19]. LIGHT is modeled as a lower-overhead shortened or monitoring profile rather than a standardized Ethernet FEC mode. The evaluation uses profile-level correction thresholds, byte overheads, and decode delays rather than a bit-accurate RS/BCH decoder. The simulator’s synthetic emission model bounds error magnitude by 𝑚𝐺 or 𝑚𝐵 in each state. PREFACE estimates residual failure as  + 𝑐𝑘 𝑟𝑘 (𝑞) = (1 − 𝑞)𝑒𝐺 1 − 𝑚𝐺  + 𝑐𝑘 + 𝑞𝑒𝐵 1 − , 𝑚𝐵

3.4

(5)

Implementation considerations

The required state is one posterior, four model probabilities, three profile costs, and a current admission limit. Updates occur once per decoded flit and use additions, multiplications, and comparisons; the arithmetic is amenable to a fixed-point datapath. Pre-FEC error counts may be exported from an FEC decoder locally, avoiding a new wire protocol. Profile signaling and safe transition epochs are implementation-specific; the simulator represents them as immediate selection. Real hardware must synchronize TX/RX profile state, protect profile metadata, and define replay across profile boundaries.

(6)

where [𝑥] + = max(0, 𝑥). It selects 𝑘 ∗ = arg min{𝑜𝑘 + 𝜆𝑟𝑘 (𝑞𝑡−+1 )}.

Exposure control

(7)

𝑘

𝜆 represents the normalized cost of a residual failure under standard replay. Values are swept on training seeds 1–10, and 𝜆 = 16 is frozen before all reported seeds 101 and above. The estimator is a simple control heuristic for converting bounded symbol magnitudes into residual-risk estimates.

3

Fan Yang, Jiaqi Liu, Tao Jiang, and Zhan Wang

Table 2: Protocol-to-model mapping.

on decoded flit 𝑡 with pre-FEC observation 𝑧𝑡 : 1. update posterior 𝑞𝑡 and predict 𝑞𝑡−+1 2. for 𝑘 ∈ {LOW,LIGHT,STRONG} compute 𝑜𝑘 + 𝜆𝑟𝑘 3. select minimum-cost profile 𝑘 ∗ 4. set budget 𝑊𝑡 ← 𝑊𝑅 iff 𝑘 ∗ =STRONG 5. admit only if replay occupancy < 𝑊𝑡 on detected residual error: execute unchanged standard replay

Property

TxReplay reten- Release only on cution mulative ACK Replay request Three copies; no implicit ACK Replay service Requested sequence through retained tail Admission Block new payload during active replay Reverse control ACK/request share serializer Threshold correcFEC tion, byte overhead, delay Sequence control In-order receive and retry limit

Figure 2: PREFACE controller pseudocode. Budget contraction blocks admission; it never invalidates retained state.

3.5

Modeled behavior

Controller operation and safety envelope

For each decoded flit, the receiver forms 𝑧𝑡 from the pre-FEC correction record, updates Eq. (4), evaluates the three costs in Eq. (7), and publishes the selected profile and risk state. The transmit side applies Eq. (8) to new admission. Existing replay entries are never discarded when the budget contracts; the controller simply blocks new payload until occupancy falls below 𝑊𝑡 . This distinction is essential because forcibly trimming TxReplay would change correctness. False positives spend strong FEC and may temporarily block admission. False negatives fall back to the standard retry path. Hence prediction quality affects performance, while detected-error correctness remains governed by the underlying replay protocol. A deployment can also pin 𝑊𝑡 = 𝑊𝐻 or force STRONG FEC as a fail-safe profile if telemetry is absent, the posterior becomes invalid, or a profile transition cannot be completed safely.

Relevance / abstraction Determines suffix state Preserves requested start Core amplification effect Couples replay to throughput Exposes delayed feedback No bit-level RS/miscorrection No wrap/ambiguity/NOP cadence

Table 3: Primary link-level configuration.

4 Methodology 4.1 ns-3 link model We add a scaleup-reliability contribution to the ns-3 simulator [6]. Each link contains a serializer, propagation and decode delay, bounded TxReplay state, an in-order receive buffer, a reversecontrol serializer, and a seeded burst-error model. A failed decode schedules a request; a successful in-order receive schedules a cumulative ACK. Eleven module tests cover ordering, retry, bounded state, profile selection, replay, and control accounting. The standard-replay path follows the public UALink 200G 1.0 semantics relevant to our hypothesis [23]: a replay entry remains until cumulative ACK; a request is not an implicit ACK; replay begins at the requested sequence and includes subsequent retained payload flits; three request copies are generated; and new payload admission stops while replay is active. ACK and request bytes share a serialized reverse link. The simulator abstracts sequencenumber wrap, explicit-flit cadence, ambiguity handling, requestignore timing, NOPs, exact FEC interleave placement, and bit-level RS decoding so that the evaluation can focus on replay exposure, feedback serialization, and FEC/exposure control. Table 2 separates the protocol properties required by our hypothesis from abstractions that could change absolute timing. This mapping also separates UALink-style standard replay from NACKonly designs such as RIFL: both retry at the link layer, but their retained state, feedback, and recovery ordering differ.

Parameter

Value

Payload flit / data rate Forward / reverse rate Propagation; ACK/request base delay TxReplay window; risk cap Request copies; retry limit FEC LOW FEC LIGHT FEC STRONG GOOD→BAD / BAD→GOOD Corruption emission GOOD / BAD Offered period / flits per seed

640 B / 400 Gb/s 400 / 400 Gb/s 20 ns; 50 ns 16; 8 flits 3; 20 modeled 0 overhead, 0 ns, telemetry floor 2.5%, 5 ns, 4 symbols 6.25%, 12 ns, 15 symbols 0.004 / 0.08 per flit 0.001 / 0.6 14 ns / 10,000

4.2

Parameters and traffic

Table 3 lists the primary configuration. GOOD-state corrupt flits have at most one bad symbol; BAD-state flits have up to 24. LIGHT and STRONG profiles correct 4 and 15 symbols. The 640-byte payload unit is chosen to match the scale of a UALink-style data-link reliability unit and a KP4-like RS-FEC codeword: 640 B plus 6.25% modeled STRONG overhead is about 680 B [9, 23]. We use this scale as a modeling granularity for UALink-style reliability units; other scale-up fabrics may choose different flit formats. The 14-ns offered period corresponds to 365.7 Gb/s of payload before FEC and replay. Every compared policy receives the same seeded error sequence. Under the primary GOOD/BAD probabilities, the stationary fraction of first-attempt flits with any raw symbol error is about 0.9524 · 0.001 + 0.0476 · 0.6 ≈ 2.95%; the measured mean is 309.2 per 10,000 flits. This controlled burst-stress workload makes replay amplification observable at a 12.5-flit mean BAD duration. Section 7.2 describes the trace-calibration path for field BER and reliability targets.

4

Predict Before Replay

Table 4: Training-only selection (10 seeds, 5,000 flits/seed).

Primary link-level performance Goodput (Gb/s)

Sweep 𝜆

𝑊𝑅

4.3

Value

Goodput (Gb/s)

P99 (𝜇s)

Replay (flits)

FEC (KB)

4 8 12 16 24

285.1 294.4 301.9 315.9 315.0

20.63 17.60 15.39 11.72 11.56

1463 1312 1200 976 737

29.6 31.8 34.8 116.1 224.2

8 10 12 14 16

336.0 326.6 322.4 315.9 310.9

6.72 8.99 10.04 11.72 13.13

583 855 918 976 996

108.5 115.0 115.3 116.1 116.3

Post-dec.

293.1

Pred.

300.9 329.7 280

300

320

340

P99 latency (us, lower) Fixed

31.6

Post-dec.

34.9

Pred.

30.5

PREFACE

15.5 10

Parameter selection and data isolation

19

29

38

Figure 3: Held-out link-level means and 95% confidence intervals (30 seeds). PREFACE improves both predictive FEC alone and the two replay baselines. amplification. Tensor sizes are 1 and 8 MiB per rank. The model captures the communication schedule over replay-capable links while abstracting compute overlap, routing, switch arbitration, and kernel launch costs in a full GPU/NIC software stack.

4.6

Baselines and metrics

Statistical procedure and reproducibility

All policies in one comparison consume the same latent channelstate and symbol-error stream, so we report per-seed paired deltas with two-sided 95% normal-approximation confidence intervals. Training seeds are separated from held-out reporting seeds, and the 3×3 robustness grid is predefined. The intervals quantify simulationseed variability for this controlled mechanism study. The artifact records commands, seeds, raw aggregate CSVs, paired deltas, and plotting scripts.

We compare four policies over the same standard-replay state machine: • Fixed strong FEC: STRONG protects every attempt. • Post-decode adaptation: an EWMA with hysteresis observes only decode failures and selects LOW/LIGHT/STRONG. It represents the strongest reactive baseline we model. • Predictive FEC: Eq. (7) selects FEC from pre-FEC evidence but retains the 16-flit window. This is the strict ablation. • PREFACE: predictive FEC plus Eq. (8). Metrics are payload goodput, arrival-to-in-order-delivery P99 latency, replayed payload flits, FEC redundancy bytes, and reversecontrol bytes. The primary experiment uses 30 held-out seeds (101– 130). We report normal-approximation 95% confidence intervals; comparisons use per-seed paired differences. A 3×3 robustness matrix uses 20 seeds for offered periods 13/14/16 ns and BAD-toGOOD probabilities 0.03/0.08/0.20.

4.5

298.3

PREFACE

We reserve seeds 1–10 for parameter selection. All reported primary seeds start at 101, robustness at 701, collective experiments at 1001 or 1201, and the million-flit check at 2001. Table 4 shows the complete two-stage training sweep. First, increasing 𝜆 trades more FEC for fewer residual replays. We choose 16 rather than 24 because their goodput is within 0.3%, while 𝜆 = 24 nearly doubles redundancy. Second, with 𝜆 = 16, the 8-flit risk budget dominates the larger candidates on goodput, P99, replay, and FEC bytes. No held-out or collective result is used to change either parameter.

4.4

Fixed

5 Evaluation 5.1 Primary link-level result Figure 3 and Table 5 show the primary result. Relative to fixed strong FEC, PREFACE raises goodput from 298.32 to 329.71 Gb/s: a paired gain of 31.39 ± 1.51 Gb/s (10.52%). P99 falls from 31.56 to 15.54 𝜇s, a reduction of 16.02 ± 1.06 𝜇s (50.75%). PREFACE also removes 488.9 ± 55.6 replayed flits and spends 231.8 ± 1.5 KB less FEC. Reverse control increases by 196.4 ± 19.4 KB because faster progress produces ACKs sooner, but total FEC-plus-control still falls from 7.428 to 7.393 MB. The modeled reliability-traffic accounting therefore still favors PREFACE. Against post-decode adaptation, PREFACE gains 36.60±1.75 Gb/s (12.49%), lowers P99 by 19.40±1.39 𝜇s (55.51%), and removes 1245.5± 77.0 replayed flits (47.52%). It spends 127.4 KB more FEC but sends 154.9 KB less reverse control. The reactive baseline economizes

Modeled ring AllReduce

To test whether link gains survive synchronization, we build a ring of 8, 16, or 32 independent links. A tensor is divided evenly among ranks and a synchronous ring executes 2(𝑁 − 1) reduce-scatter/allgather steps, following the standard bandwidth-optimal collective structure used in MPI and GPU collective libraries [12, 16, 22]. Every step waits for the slowest link, which exposes replay tail 5

Fan Yang, Jiaqi Liu, Tao Jiang, and Zhan Wang

Table 5: Primary means over 30 held-out seeds.

Policy Fixed strong FEC Post-decode adapt. Predictive FEC PREFACE

Robustness vs. fixed strong FEC

Goodput (Gb/s)

P99 (𝜇s)

Replay (flits)

FEC (KB)

298.32 293.11 300.88 329.71

31.56 34.94 30.50 15.54

1864.1 2620.7 2312.4 1375.2

458.4 99.2 244.4 226.6

Table 6: Paired PREFACE deltas with 95% CI (30 held-out seeds).

Goodput gain (Gb/s)

P99 reduction (us)

0.03

0.08

0.2

0.03

0.08

0.2

13

43.8

32.3

27.1

13

17.6

8.2

5.7

14

42.3

29.7

22.4

14

16.9

7.2

4.2

16

40.8

16.6

1.8

16

16.3

4.2

1.1

Rows: offered period (ns); columns: BAD-to-GOOD probability.

Metric Goodput (Gb/s) P99 (𝜇s) Replay (flits) FEC (KB) Control (KB)

vs. fixed

vs. post-decode

vs. predictive

+31.39 ± 1.51 −16.02 ± 1.06 −489 ± 56 −231.8 ± 1.5 +196.4 ± 19.4

+36.60 ± 1.75 −19.40 ± 1.39 −1246 ± 77 +127.4 ± 2.4 −154.9 ± 18.9

+28.84 ± 1.96 −14.96 ± 1.36 −937 ± 72 −17.8 ± 1.6 −23.0 ± 11.7

Cell values are paired improvements; positive is better.

Figure 4: Paired PREFACE improvement over fixed strong FEC. Positive is better; corresponding paired intervals exclude zero. 3.29 Gb/s and lowers P99 by 17.65 ± 2.40 𝜇s versus fixed FEC. The same point gains 45.80 ± 3.58 Gb/s over post-decode adaptation. The useful effect does narrow under light load and short bursts, as predicted by Eq. (1). At 16 ns and BAD-to-GOOD 0.20, the fixedFEC comparison is only 1.81 ± 1.29 Gb/s goodput and 1.09 ± 0.32 𝜇s P99, although both remain significant. Replay falls by 55.4 ± 23.2 flits. This boundary matches the mechanism: PREFACE removes replay exposure, so the benefit naturally approaches zero when the offered stream rarely fills the healthy window or BAD episodes end before feedback accumulates a suffix.

on coding only after uncorrectable frames have already triggered costly suffix replay. Table 6 reports paired effects rather than only policy means. Every interval for the three performance metrics excludes zero. The overhead rows expose the different tradeoffs: PREFACE saves both FEC and total modeled reliability bytes versus fixed FEC; versus the low-coding reactive policy, it spends coding bytes to avoid a much larger replay and tail penalty; versus predictive FEC, it saves both FEC and reverse control because the only changed actuator is exposure. This last column is particularly important because it rules out a hidden “more redundancy” explanation for the ablation gain.

5.2

5.4

Does joint exposure control matter?

The predictive-FEC ablation holds the error trace, posterior, cost function, coding choice, and replay machine constant; only 𝑊𝑡 remains 16. PREFACE adds 28.84 ± 1.96 Gb/s, reduces P99 by 14.96 ± 1.36 𝜇s, and removes 937.2±71.6 replayed flits. FEC redundancy also decreases by 17.80 ± 1.64 KB and reverse control by 23.0 ± 11.7 KB. This is the central causal result: holding predictive FEC fixed, riskcapping reduces the amount of healthy-looking suffix traffic invalidated by a residual failure. The large difference also explains why selecting the risk window after reverse-control serialization was added was necessary: a window trained under an earlier timing model would not represent the finalized system.

5.3

Overhead and control-path interpretation

PREFACE occupies a middle point between the coding baselines. In the primary experiment it uses 226.6 KB of FEC per seed, 50.6% less than always-strong FEC and 128.4% more than post-decode adaptation. The relevant cost also includes replay: the latter baseline triggers 90.6% more replayed flits than PREFACE. Reverse-control bytes are 2.2% lower than post-decode adaptation and 2.8% higher than fixed FEC. These magnitudes show why evaluating only coding overhead, or assuming instantaneous ACK/NACK delivery, would reverse some design conclusions. The flight cap uses the existing replay-request format in our model. Its wire-visible consequence is indirect: fewer suffix replays and fewer repeated requests. A real profile negotiation mechanism may add small control messages or reserve profile epochs; deployment studies should account for those bytes explicitly.

Robustness across load and burst duration

5.5

PREFACE significantly improves goodput, P99, and replay count against both fixed strong FEC and post-decode adaptation in all 9/9 robustness scenarios. It also uses less FEC than fixed strong FEC at every point. Figure 4 shows that gains grow when bursts persist or offered load is high enough for delayed feedback to accumulate a suffix. At the hardest 13-ns/0.03 point, PREFACE gains 43.78 ±

Ring AllReduce

Figure 5 shows that link-level gains survive a synchronized collective schedule. Relative to post-decode adaptation, PREFACE reduces completion time in all six configurations: 21.9–27.0% for 1 MiB and 15.6–22.1% for 8 MiB. Relative to fixed strong FEC, reductions are 16.6–19.2% for 1 MiB and 13.1–17.8% for 8 MiB. Every paired interval excludes zero. The rank trend is also favorable: for 1 MiB, the fixed-FEC reduction rises from 12.37 ± 1.48 𝜇s at 8 ranks to 6

Predict Before Replay

retained standard replay. PREFACE is complementary: it controls coding and exposure before an existing replay machine activates. Scale Up Ethernet likewise identifies link-layer retry as recovery beyond FEC [14]. Adaptive FEC. Adaptive coding has a long history in wireless and streaming networks; for example, burst-loss measurement can drive streaming FEC redundancy [4]. PREFACE builds on adaptive coding and Gilbert–Elliott-style prediction, and adds the scale-upspecific coupling between a pre-FEC posterior and the standardreplay exposure window. The fixed-window predictor ablation isolates the value of that coupling. Combining coding and retransmission. Earlier systems demonstrate that coding and ARQ are complementary but operate at very different scales. Maelstrom aggregates wide-area traffic and applies layered interleaving so that FEC recovery can happen before a long RTT dominates [2]; Ricochet exchanges receiver-generated repair information for time-critical multicast [1]. WiLDNet explicitly studies the throughput/delay/error tradeoff between bulk link retransmissions and FEC on long-distance wireless links [17]. FlEC exposes multiple coding/retransmission choices to QUIC applications [11]. These systems decide how to recover packet loss or where to place redundancy. PREFACE instead observes corrected physical errors at a sub-packet scale and controls how much standard-replay state becomes conditionally exposed before a request returns. The relationship is conceptual: all of these systems value proactive and reactive recovery, while PREFACE applies that principle to flit-scale replay exposure. Scale-up reliability. UALink specifies CRC/FEC/replay machinery [23]; RXL proposes implicit sequence numbers and end-to-end integrity for scaling CXL-like chip networks [10]. These works define or extend the correctness path. PREFACE instead optimizes when and how much traffic reaches that path without changing its correctness semantics.

Ring AllReduce completion time Fixed

Post-dec.

PREFACE

1 MiB per rank 190

127

63

0

8

16

32

8 MiB per rank 780

520

260

0

8

16 Ranks on x-axis; completion time in us on y-axis.

32

Figure 5: Modeled synchronous ring AllReduce completion time. PREFACE is significantly faster than both baselines in all six configurations. 30.30 ± 1.84 𝜇s at 32 ranks because more synchronized links create more opportunities for a replay tail to determine the step maximum. At 32 ranks and 8 MiB, completion falls from 689.5 to 566.6 𝜇s versus fixed FEC (17.83%) and from 727.2 to 566.6 𝜇s versus postdecode adaptation (22.09%). Paired reductions are 122.92 ± 4.02 and 160.60 ± 8.03 𝜇s. Replay falls by 32,261 and 89,680 flits, respectively. This communication-schedule result shows that independent link benefits survive slowest-link synchronization, while full end-to-end training speed depends on compute overlap, software, routing, and switch effects.

5.6

7 Discussion 7.1 Alternatives and design rationale Always use the risk window. A static 8-flit TxReplay window would bound suffix exposure without a predictor, but it would also impose the underfill term 𝑢 (𝑊 ) during the overwhelmingly common GOOD state. PREFACE keeps 16 entries whenever the selected code is LOW or LIGHT and contracts only under the same evidence that justifies STRONG FEC. Its window cap is therefore a transient risk budget layered on top of ordinary credit sizing. A static-small-window baseline would answer a different question— how much throughput to sacrifice universally for a replay bound— and should be included in hardware follow-up once the actual pipeline depth is known. Encode harder but never cap flight. This is precisely the predictive FEC ablation. It avoids reacting after decode failure, yet its 300.88 Gb/s goodput remains close to fixed FEC because a residual event can still couple to the full outstanding suffix. Increasing 𝜆 raises coding pressure while leaving the exposure term unchanged: the training sweep shows that 𝜆 = 24 roughly doubles FEC bytes while goodput changes by less than 1 Gb/s. Available profiles can reduce residual probability; flight control reduces the conditional cost of the residual events that remain.

Long-run delivery invariant

Three additional seeds each offer one million flits. All three million flits (1.92 GB payload) are delivered in order with zero terminal failures, despite a mean of 29,586 initially corrupted and 130,060 replayed flits per seed. The online transition counters recover the injected 0.004/0.08 probabilities, which checks that the shorter flight budget preserves the injected error-process statistics. The check targets delivery-order conservation on the modeled detected-error path, which uses perfect detection, threshold-bounded correction, and eventual success within 20 retries. A later FIT/SDC study should extend the fault model with CRC collisions, decoder miscorrection, analog behavior, and permanent or common-mode faults.

6

Related Work

Link-local retransmission. RIFL develops a low-latency in-band, NACK-only hop-by-hop retransmission protocol and demonstrates a 112-Gb/s FPGA implementation [21]. Its continuous idle frames, bidirectional-error handling, verification code, and replay state illustrate a different link-retry design point from UALink-style 7

Fan Yang, Jiaqi Liu, Tao Jiang, and Zhan Wang

Replay only the damaged flit. Selective replay could retransmit only the failed flit or receiver-reported holes, directly reducing suffix amplification. It is a protocol redesign: the receiver must retain and validate out-of-order flits, requests need finer-grained metadata or bitmaps, and verification must cover hole tracking, duplicate replays, ACK races, and buffer reclamation. PREFACE preserves standard replay and reduces replay probability and exposure before recovery activates. The ideas are complementary; future work should compare selective replay, standard replay plus PREFACE, and their combination under the same burst traces and hardware cost model. FEC profile transition cost. The simulator abstracts profile changes as immediate decisions over a multi-profile FEC interface. Real links must synchronize which profile protects each flit or epoch, protect profile metadata, and handle replay across profile boundaries. Header IDs, epoch switching, and rate-compatible codes are possible paths, but they add control bytes, transition latency, decoder complexity, and verification cost. A conservative design can force STRONG FEC when state is uncertain. These costs affect net performance; standard replay remains the detected-error correctness backstop. Use a learned predictor. A recurrent or transformer model could consume richer PHY telemetry. The two-state filter used here keeps the observation, transition, cost, and safety behavior inspectable, and its handful of state variables is compatible with per-flit control. Trace-calibrated learned prediction is best viewed as a replaceable front end to the joint actuation policy, with explicit latency, area, and out-of-distribution fallback costs. The measured results are consistent with the exposure model. The fixed-window predictive ablation sees exactly the same error samples and coding decisions yet retains substantially more replay and tail latency. Gains increase with load, burst persistence, and rank count—the three factors that increase the number or impact of suffix replays. PREFACE also lowers FEC relative to fixed strong FEC and replay relative to post-decode adaptation, connecting the measured effect to Eq. (2) rather than to a single “more-coding” or “less-coding” explanation.

7.2

count, while the reported confidence intervals summarize held-out seed variability on the predefined robustness grid. Two deployment details deserve explicit engineering. First, profile changes are immediate in the simulator; hardware needs safe codeword transition epochs, protected profile metadata, fixed-point posterior arithmetic, timing closure, and area/power accounting. Second, LOW needs useful early telemetry, either from a monitor code that reports corrected symbols or from equivalent PHY-side pre-FEC counters. The ring experiment captures collective synchronization over modeled links; trace-driven collective simulation and FPGA or SerDes testbeds are the next steps for application-speed and reliability evaluation.

8

Future Work

Four directions are important for turning PREFACE from a controlled mechanism study into a deployable scale-up reliability mechanism.

8.1

Trace-calibrated channel and correctness models

The first priority is to replace the synthetic Gilbert–Elliott input with pre-FEC traces from high-speed electrical links. A useful campaign should span cable length, connector quality, temperature, equalizer settings, lane training, and aging. Besides correctedsymbol counts, traces should retain time, lane identity, codeword position, decoder confidence when available, and whether an event crossed the correction threshold. We can then compare the twostate filter against semi-Markov, change-point, and lightweight learned predictors using identical train/test splits. The key metric is delivered cost after inference delay and fallback behavior are included, rather than prediction accuracy alone. A bit-accurate FEC/CRC path is needed in parallel. It should model interleaving, decoder miscorrection, CRC coverage, and correlated multi-lane or common-mode faults. That model would separate three quantities currently collapsed by a threshold abstraction: corrected errors that inform prediction, detected residual errors that trigger replay, and undetected corruptions that threaten integrity. The viability condition is clear: early telemetry should predict risky intervals while preserving undetected-error probability and profiletransition safety. This work would support SDC/FIT analysis by replacing the current invariant check with a richer correctness model.

Deployment path and validation needs

The main calibration need is trace quality. The Gilbert–Elliott parameters and symbol magnitudes are synthetic; field validation should collect pre-FEC error traces across cables, connectors, temperatures, lane training, and aging, then replay those traces or fit a richer semi-Markov model. A bit-accurate RS decoder is also needed to study miscorrection and post-FEC undetected-error rates. The protocol model captures the UALink 200G 1.0-style standardreplay interactions needed by the hypothesis and keeps timingsensitive details as explicit extension points. Later or gated UALink revisions, including 2.0 materials [24], are natural follow-on validation targets. ACK cadence currently uses one 640-byte control flit per payload flit; future models should add cumulative ACK coalescing and reverse payload contention. Implementation confidence comes from deterministic seeded traces, paired policy runs, explicit counters for delivered, corrupted, corrected, replayed, and terminal frames, and 11 module tests. The long-run invariant check exercises conservation at a larger event

8.2

Protocol-complete feedback and multi-link behavior

The next simulator revision should implement cumulative ACK coalescing, explicit-flit cadence, sequence wrap and ambiguity handling, request-ignore timers, NOP behavior, and reverse payload contention. It should also cross-check the modeled state machine against an independently available specification or interoperable implementation. These additions can change the detection-to- request interval in Eq. (1), and therefore may move the optimal absolute risk budget. A useful stress test is whether joint control remains beneficial after ACK aggregation reduces feedback delay. The risk budget itself should eventually be adaptive. PREFACE fixes 𝑊𝑅 = 8 after offline training, but a production controller could 8

Predict Before Replay

Table 7: Proposed validation roadmap.

compute 𝑊𝑡 from the online posterior, estimated GOOD/BAD transitions, residual cost, offered load, and feedback delay. This would avoid over-conservative caps on benign links while tightening flight under frequent or persistent bursts. Scale-up packages also contain multiple lanes and links whose faults need not be independent. Future experiments should introduce lane striping, shared power/clock disturbances, link bonding, and failover. Controller state could be maintained per lane, per link, or hierarchically. Per-lane state reacts precisely but costs more; a shared posterior is smaller but may overprotect healthy lanes. This creates a new question beyond the current paper: whether exposure should be capped locally or coordinated across the striped transfer to prevent one risky lane from becoming the collective tail.

8.3

Deliverable

Key validation question

Trace

Public/anonymized pre-FEC burst corpus Bit-accurate FEC/CRC and complete replay FPGA/SerDes controller with fault injection Trace-driven collectives and application runs

Does early telemetry predict risky intervals?

Model

HW

Stack

Hardware prototype and safe deployment

A hardware prototype should separate fast-path inference from slow-path model maintenance. The per-flit path needs a posterior register, a small table or fixed-point evaluator for three residual costs, a minimum comparator, and an admission-limit register. Transition-probability estimation, diagnostics, and policy updates can run at a slower epoch. Profile changes must be associated with an unambiguous codeword boundary; an epoch tag, ordered management message, or prearranged profile schedule are possible choices, but selecting among them requires implementation details below the public model used here. The safest first prototype would keep strong FEC as a boot and fault profile, enable PREFACE only after both ends acknowledge telemetry, and revert on posterior saturation, telemetry loss, or control timeout. FPGA evaluation should report inference latency, maximum clock rate, logic and memory area, admission-gate critical path, energy per delivered bit, and transition-control bytes. Fault injection should test false positives, false negatives, permanent faults, and mid-transition resets. A useful success criterion should combine ns-3 goodput agreement with preservation of the baseline replay invariant and timing closure at the target flit rate.

8.4

Phase

Do joint gains survive accurate feedback? Do timing/safety costs stay below saved replay work? Do link gains survive software/topology effects?

Table 7 summarizes a staged agenda. Each phase has an explicit artifact and a decision criterion for judging whether the mechanism remains compelling. These phases also motivate shared artifacts: a pre-FEC errortrace format, a protocol-semantic replay test suite, and common reporting of FEC, replay, control, tail latency, and delivered-integrity costs. Such artifacts would let future mechanisms be compared on more than a single BER or throughput number.

9

Conclusion

Delayed standard replay couples error correction to the amount of traffic in flight. PREFACE makes that coupling explicit: a pre-FEC burst posterior selects both coding strength and a risk-capped exposure window, while standard replay remains the correctness backstop. In the final serialized-control ns-3 model, PREFACE improves goodput and tail latency over fixed strong FEC and post-decode adaptation, and a strict ablation shows that joint flight control contributes substantial gain beyond predictive FEC. Robustness and collective experiments identify both the useful region and the saturation points. The evidence supports PREFACE as a promising scale-up link-control design, with hardware and trace calibration as the essential next validation step.

Full-stack collective evaluation

The final track is to connect link behavior to a GPU/NIC communication stack. Trace-driven simulation should combine measured replay episodes with collective algorithms, routing, switch arbitration, compute overlap, and kernel launch costs. Beyond ring AllReduce, the workload set should cover trees, all-gather, all-to-all, point-to-point transfers, and mixtures of latency-sensitive control messages with bulk tensors. Model outputs should include job completion and iteration jitter in addition to average collective time, because one exposed suffix on a synchronized path can dominate a step. There is also an opportunity for cross-layer control. The link can export a coarse risk epoch without exposing raw PHY telemetry; a collective scheduler could temporarily avoid a degraded path, change chunking, or delay a noncritical transfer. Conversely, the communication layer could identify synchronization- critical flits whose replay exposure deserves a smaller budget. Such coupling must be compared against link-local PREFACE to determine whether its additional interface and policy complexity produce gain beyond the simple local design.

References [1] Mahesh Balakrishnan, Ken Birman, Amar Phanishayee, and Stefan Pleisch. 2007. Ricochet: Lateral Error Correction for Time-Critical Multicast. In Proc. USENIX NSDI. 73–86. https://www.usenix.org/legacy/events/nsdi07/tech/full_papers/ balakrishnan/balakrishnan.pdf [2] Mahesh Balakrishnan, Tudor Marian, Ken Birman, Hakim Weatherspoon, and Einar Vollset. 2008. Maelstrom: Transparent Error Correction for Lambda Networks. In Proc. USENIX NSDI. https://www.usenix.org/legacy/event/nsdi08/tech/ full_papers/balakrishnan/balakrishnan.pdf [3] E. O. Elliott. 1963. Estimates of Error Rates for Codes on Burst-Noise Channels. Bell System Technical Journal 42, 5 (1963), 1977–1997. doi:10.1002/j.1538-7305. 1963.tb00955.x [4] Ken French and Mark Claypool. 2001. Repair of Streaming Multimedia with Adaptive Forward Error Correction. In Proc. SPIE Multimedia Systems and Applications. https://web.cs.wpi.edu/~claypool/papers/adaptive-fec/ [5] Edgar N. Gilbert. 1960. Capacity of a Burst-Noise Channel. Bell System Technical Journal 39, 5 (1960), 1253–1265. doi:10.1002/j.1538-7305.1960.tb03959.x [6] Thomas R. Henderson, Mathieu Lacage, George F. Riley, Craig Dowell, and Joseph Kopena. 2008. Network Simulations with the ns-3 Simulator. In ACM SIGCOMM Demonstration. https://www.nsnam.org/

9

Fan Yang, Jiaqi Liu, Tao Jiang, and Zhan Wang

[7] IEEE. 2017. IEEE Std 802.3bs-2017: IEEE Standard for Ethernet Amendment 10: Media Access Control Parameters, Physical Layers, and Management Parameters for 200 Gb/s and 400 Gb/s Operation. https://ieeexplore.ieee.org/document/ 8207825 [8] IEEE. 2022. IEEE Std 802.3ck-2022: IEEE Standard for Ethernet Amendment 4: Physical Layer Specifications and Management Parameters for 100 Gb/s, 200 Gb/s, and 400 Gb/s Electrical Interfaces Based on 100 Gb/s Signaling. https: //standards.ieee.org/ieee/802.3ck/7322/ [9] International Telecommunication Union. 2024. ITU-T Recommendation G.709.5: Flexible OTN short-reach interfaces. Specifies RS(544,514)-based FEC interfaces. https://www.itu.int/rec/T-REC-G.709.5/ [10] Giyong Jung, Saeid Gorgin, John Kim, and Jungrae Kim. 2025. Scaling Out Chip Interconnect Networks with Implicit Sequence Numbers. arXiv preprint arXiv:2507.01988 (2025). https://arxiv.org/abs/2507.01988 [11] François Michel, Alejandro Cohen, Derya Malak, Quentin De Coninck, Muriel Médard, and Olivier Bonaventure. 2022. FlEC: Enhancing QUIC with ApplicationTailored Reliability Mechanisms. arXiv preprint arXiv:2208.07741 (2022). https: //arxiv.org/abs/2208.07741 [12] NVIDIA. 2026. NVIDIA Collective Communication Library (NCCL) Documentation. https://docs.nvidia.com/deeplearning/nccl/user-guide/index.html [13] NVIDIA. 2026. NVIDIA H100 Tensor Core GPU. Product specifications including fourth-generation NVLink bandwidth. https://www.nvidia.com/en-us/datacenter/h100/ [14] Open Compute Project. 2025. Scale Up Ethernet, Version 1.0.0. https://www. opencompute.org/documents/ocp-sue-spec-final-pdf-1 [15] Optical Internetworking Forum. 2022. Common Electrical I/O (CEI) 5.0 Implementation Agreement. Electrical and jitter interoperability agreements for 112 Gb/s I/O. https://www.oiforum.com/technical-work/implementation-agreementsias/ [16] Pitch Patarasuk and Xin Yuan. 2009. Bandwidth Optimal All-Reduce Algorithms for Clusters of Workstations. J. Parallel and Distrib. Comput. 69, 2 (2009), 117–124.

doi:10.1016/j.jpdc.2008.09.002 [17] Rabindranath Patra, Sergiu Nedevschi, Sonesh Surana, Anmol Sheth, Lakshminarayanan Subramanian, and Eric Brewer. 2007. WiLDNet: Design and Implementation of High Performance WiFi Based Long Distance Networks. In Proc. USENIX NSDI. https://www.usenix.org/legacy/events/nsdi07/tech/full_papers/ patra/patra.pdf [18] PCI-SIG. 2022. PCI Express 6.0 Specification. https://pcisig.com/pci-express6.0-specification [19] Irving S. Reed and Gustave Solomon. 1960. Polynomial Codes over Certain Finite Fields. J. Soc. Indust. Appl. Math. 8, 2 (1960), 300–304. doi:10.1137/0108018 [20] Debendra Das Sharma. 2020. The PCIe 6.0 Specification Webinar Q&A: Error Detection and Correction with FEC. PCI-SIG Blog. https://pcisig.com/blog/ pcie%C2%AE-60-specification-webinar-qa-error-detection-and-correction-fec [21] Qianfeng Shen, Jun Zheng, and Paul Chow. 2022. RIFL: A Reliable Link Layer Network Protocol for Data Center Communication. Journal of Optical Communications and Networking 14, 3 (2022), 111–126. doi:10.1364/JOCN.443448 [22] Rajeev Thakur, Rolf Rabenseifner, and William Gropp. 2005. Optimization of Collective Communication Operations in MPICH. International Journal of High Performance Computing Applications 19, 1 (2005), 49–66. doi:10.1177/ 1094342005051521 [23] Ultra Accelerator Link Consortium. 2025. UALink 200G Data Link and Physical Layer Specification, Version 1.0. Evaluation copy. https://ualinkconsortium.org/wp-content/uploads/2025/12/UALink200_ Specification_v1.0_Evaluation_Copy-v2.pdf [24] Ultra Accelerator Link Consortium. 2026. UALink Specifications. Common 2.0 and 200G DL/PL 2.0 availability page. https://ualinkconsortium.org/ specification/ [25] H. Yamazaki et al. 2020. DFE Error Propagation and FEC Interleaving for 400GbE PAM4 Electrical Lane. IEICE Transactions on Electronics E103-C, 2 (2020). doi:10. 1587/transele.2019ECP5016

10

Related documents

• Learning-Enabled Control: Statistical Foundations and Architectures#637483
• Benchmarking Basic OSPF Single Router Control Plane Convergence#500469
• Koopman-Based Robust Model Predictive Control for Nonlinear Systems with Stochastic Intermittent Measurements#1070921
• AccelMPC: High-Rate, Low-Power FPGA-Accelerated Model Predictive Control for Tiny Drones#987133
• Optimization-Based Formation Flight on Libration Point Orbits#1050833