Conceptio › Archive › arXiv CS
arXiv CS open access

ICL-SEC: Iterative Cross-Layer Semantic Error Correction

2026 · arxiv_cs
arXiv CS · Papers · 2026
Open Source ↗Direct PDF ↓
distributed-systemsinternetnetworkingprotocols
Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

1

ICL-SEC: Iterative Cross-Layer Semantic Error Correction

arXiv:2608.15207v1 [cs.IT] 15 Aug 2026

Yirun Wang, Soung Chang Liew, Fellow, IEEE, and Yuyang Du

Abstract—Iterative decoding has been central to the success of modern channel coding, where reliability information is repeatedly exchanged across decoding components to approach fundamental performance limits. This paper brings the same principle to semantic error correction by proposing iterative crosslayer semantic error correction (ICL-SEC), a framework that closes the loop between physical-layer soft channel decoder and application-layer language-model-empowered semantic decoder. In the proposed framework, a soft-input soft-output channel decoder first produces bit-level posterior probabilities, from which word-level reliabilities are derived. Words deemed reliable are exposed to a masked language model as semantic context, while unreliable words are masked. The language model then produces contextual word likelihoods, which are leveraged to generate extrinsic bit-level priors and fed back to the channel decoder for the next iteration. This iterative refinement progressively expands the set of confidently recovered words. A key contribution is our Confirm prior-update rule: once a word is judged reliable, its bits are assigned deterministic priors with probability one in subsequent iterations, making the word fully resolved side information for both the channel decoder and the language model. This successive-confirmation mechanism prevents oscillatory unmask-mask behavior and yields a reliability interpretation consistent across layers. Simulations over text transmission demonstrates that ICL-SEC substantially outperforms both conventional channel decoding and non-iterative CLSEC. In particular, the proposed Confirm scheme reduces the bit error rate by more than two orders of magnitude relative to noniterative CL-SEC, while also significantly improving the other five performance metrics. These results show that principled iterative cross-layer semantic feedback can dramatically improve reliable and meaning-preserving communication. Index Terms—Cross-layer error correction, iteration, language models, semantic communications.

I. I NTRODUCTION HE triumphs of iterative decoding in traditional communication systems demonstrate the transformative power of exchanging reliability information across decoding components. When applied to capacity-approaching codes such as turbo codes and low-density parity-check (LDPC) codes, iterative decoding has enabled performance close to the Shannon limit [1]. Inspired by these successes, this paper investigates whether a similar iterative principle can be tailored to semantic error correction (SEC), particularly to cross-layer SEC methods [2]–[5] that integrate physical-layer channel evidence with application-layer semantic information. Our results indicate that, with a principled design, iterative cross-layer SEC can

T

Y. Wang, S. C. Liew, and Y. Du are with the Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China (e-mail:{yrwang, soung, yuydu}@ie.cuhk.edu.hk). Corresponding author: Soung Chang Liew.

reduce both bit error rate (BER) and word error rate (WER) by more than two orders of magnitude relative to non-iterative cross-layer SEC decoding for text transmission. To motivate the proposed iterative design, we briefly review how SEC has evolved from post-decoding correction to crosslayer joint decoding. Although the idea of combining channel decoding with semantic inference may also be applicable to non-textual sources, such as images, this paper follows most existing SEC works and focuses on text recovery. In this setting, SEC has recently emerged as an application-layer complement to conventional forward error correction (FEC). In a conventional SEC pipeline, the physical-layer decoder first makes hard decisions on the received bits, and SEC is then applied as a post-processing step to correct the resulting corrupted text using language-model (LM) context [6]. However, this separation is generally suboptimal: once the decoder output is reduced to hard decisions, much of the physical-layer reliability information is discarded, leaving SEC without knowledge of which bits or words are more reliable. In our earlier work [2], we proposed CL-SEC, a cross-layer joint decoding framework that fuses physicallayer soft information with LM-derived semantic probabilities, thereby anchoring semantic correction to channel evidence and markedly improving information recovery over conventional post-decoding SEC. Existing cross-layer SEC methods, however, typically perform this fusion in a one-shot manner: physical-layer soft information is integrated once with contextual likelihoods produced by LM-driven semantic inference to form a crosslayer decision on the recovered text [2]–[5]. The resulting semantic information is not fed back to refine the channel decoder, and newly refined channel-decoder outputs are not repeatedly used to update the semantic priors. This oneshot design leaves the central promise of iterative cross-layer refinement largely unexploited. This limitation echoes a classical lesson from channel coding: the power of turbo and LDPC decoding lies not merely in combining distinct sources of reliability information, but in exchanging and refining that information iteratively across decoding components [1]. Motivated by this principle, we propose iterative CL-SEC (ICL-SEC), which adapts this idea to semantic error correction by closing the loop between the physical-layer channel decoder and application-layer LMdriven semantic decoder. Overview of the ICL-SEC Framework: The following steps outline the proposed ICL-SEC framework for information recovery at the receiver. Although the framework can be extended to non-textual content, this work focuses on text

2

transmission. Step 1: The channel decoder computes bit-level soft information, from which word-level posterior probabilities are derived. Words whose posterior probabilities exceed a prescribed threshold are deemed reliable and left unmasked; the remaining words are masked. Step 2: The partially masked text sequence, e.g., “Mary <mask> a little <mask>,” is provided to an LM. Conditioned on the unmasked words, the LM produces contextual probability distributions over candidate words at each word position. Step 3: The LM-derived word probabilities are leveraged to compute extrinsic bit-level priors and fed back to the channel decoder. With these updated priors, the channel decoder repeats Step 1, enabling iterative cross-layer refinement. Across iterations, the objective is to progressively increase the number of words recovered with high confidence. A central design question in this iterative loop is how to update the bit-level priors of words that have already been deemed reliable. We consider three strategies. The Naive strategy updates the priors of both unmasked and masked words at every iteration. The Hold strategy freezes the current priors of an unmasked word once it has been deemed reliable, so that only masked words continue to receive updated priors. The proposed Confirm strategy goes further: once a word is deemed reliable, the word is confirmed and its corresponding bits are assigned deterministic priors with probability one in all subsequent iterations. In this way, a confirmed word is treated as fully resolved side information rather than as a soft and revisable estimate. This above progression also reveals why the prior-update rule is central to ICL-SEC. The naive strategy exhibits unsatisfactory performance and oscillatory behavior, since words that are unmasked in one iteration may be perturbed and masked again later. The hold strategy mitigates but does not eliminate this instability, because held soft priors still do not make an unmasked word fully reliable from the channel decoder’s perspective. These observations motivate the Confirm strategy, which achieves the best performance among the three and provides a consistent interpretation of reliability across the channel decoder and the LM. Rather than a pure belief-propagation algorithm as in turbo and LDPC decoding, this successive-confirmation mechanism is more akin to successive interference cancellation (SIC) decoding in wireless networks (see Remark 5 in Section III-D). For a comprehensive assessment, we evaluate ICL-SEC using six metrics spanning transmission reliability and semantic fidelity. Experimental results show that all three iterative variants – Naive, Hold, and Confirm – outperform non-iterative CL-SEC across the metrics, demonstrating the broad benefit of iterative cross-layer refinement. Among these variants, the Confirm method performs best, reducing both the BER and WER by more than two orders of magnitude relative to non-iterative CL-SEC. These results show that the gains of ICL-SEC come not merely from repeated semantic feedback, but from a principled prior-update rule that stabilizes the interaction between channel decoding and semantic decoding.

II. P RELIMINARIES A. System Architecture Fig. 1 shows the block diagram of the overall system. At the transmitter, we remove all spaces and punctuation from the original text message. Metadata specifying the lengths of successive words is then included in the header [2]. After word recovery at the receiver, spaces are restored using this metadata, while punctuation is restored using a context-aware modern LM [2]. The remaining payload consists of N words, denoted by wn , n ∈ N ≜ {1, . . . , N }. Let Ln be the number of letters in wn . The word-length information {Ln }n∈N is embedded in the frame header so that the receiver can determine the word boundaries.1 Concatenating the words yields the character sequence (w1 , . . . , wN ), which is encoded using the ASCII code, yielding the information bit stream v = (v1 , . . . , vK ). Thus, Kn = 8Ln is the number of bits associated with word wn . Before channel encoding, an interleaver is applied to the information bit stream to disperse the bits of each word across the overall source-bit sequence so that the successive source bits presented to the following channel encoder are not from the same word or from the neighborhood of the word. In particular, the interleaver here plays a role analogous to the interleaver in the turbo code. With the interleaver, v is permuted into u ≜ (u1 , . . . , uK ) = π(v), where uπ(k) = vk , k ∈ K ≜ {1, . . . , K}. We can obtain v from u via the inverse deinterleaving v = π −1 (u). In general, the channel code can be any code that is amenable to soft-input soft-output decoding (e.g., LDPC or convolutional code decoded using BCJR algorithm [1]). As an instant of ICL-SEC, this work employs the convolutional code, for simple exposition of the principle rather optimality of the overall system. Specifically, the bit stream u is encoded by a rate-R convolutional encoder. For each information bit uk , the encoder produces M ≜ 1/R coded bits, denoted by ck = (ck,1 , . . . , ck,M ), where ck,m ∈ F2 ≜ {0, 1}, m = 1, . . . , M . The coded bits are binary phase-shift keying (BPSK) modulated as xk ≜ (xk,1 , . . . , xk,M ) = 2ck − 1M ∈ {−1, +1}M , k ∈ K. The received symbols over the additive white Gaussian noise (AWGN) channel are yk ≜ (yk,1 , . . . , yk,M ) = xk + nk , where the noises nk ∼ N (0M , σ 2 IM ). With the normalization of signal amplitude, the signal-to-noise ratio (SNR) is given by SNR = 1/σ 2 . For notational convenience, we define y = (y1 , . . . , yK ). At the receiver, ICL-SEC is performed to recover the transmitted word sequence; the detailed decoding procedure is deferred to Section III. As illustrated in Fig. 1, ICL-SEC forms an iterative loop between BCJR decoding and LM-driven cross-layer prior (CLP) update. After each BCJR pass, bit-level posterior probabilities are used to identify reliable 1 The word-length metadata can be efficiently compressed, e.g., using the canonical Huffman coding scheme [2]. We note that removing spaces and punctuation reduces the payload length more than the compressed header increases it, and thus the ICL-SEC frame can be shorter than the original fulltext frame. The saved length can be used to provide stronger FEC protection for the header.

3

Message with N words w1

w2

wN

...

vk

Character encoding

uk

Interleaving

Convolutional encoding & modulation

xk ,m Channel

Decoded message ŝ

Hard decision (HD)

r (vk ) r/p q (vk )

Masking

ŝ m LM

Deinterleaving

r (uk )

 n (w )

ŝ m

BCJR decoding

yk , m

p (uk )

Cross-layer prior (CLP) update

p (vk )

Interleaving

Fig. 1. Block diagram of the overall system, shown for an instantiation with a convolutional code. The ICL-SEC operates at the receiver for message recovery.

and unreliable words. Reliable words are retained, whereas unreliable words are masked and passed to the LM, which produces contextual likelihoods for candidate words. These contextual likelihoods are then combined with BCJR soft information by the CLP update module to produce updated bit-level priors, which are fed back to the BCJR decoder for the next iteration.

where the summation is over all possible encoder states s′ that can transition to state s. Similarly, the backward metric βk (s) is computed recursively as ∑ βk−1 (s′ ) = βk (s) γk (s′ , s). (7) s

Finally, the branch metric γk (s′ , s) can be written as γk (s′ , s) = P(uk = u)P(yk |uk = u), (s′ , s) ∈ Su ,

B. BCJR Decoding We first review the BCJR decoding to facilitate a precise description of ICL-SEC in Section III. The BCJR algorithm computes the a posteriori probability (APP) of each information bit by marginalizing over all trellis branches associated with that bit [1]. At input bit k ∈ K, let sk ∈ S be the encoder state, and uk ∈ F2 be the input bit. Denote the constraint length of the convolutional code by C, and thus |S| = 2C−1 , where C − 1 is the memory length. For a convolutional code, each trellis branch is specified by the previous state sk−1 = s′ , the input bit uk = u, and the current state sk = s; any two of these variables determine the remaining one. The APP P(uk = u|y) of uk can be expressed as ∑ ( ) P(uk = u|y) ∝ P sk−1 = s′ , sk = s, y , u ∈ F2 , (s′ ,s)∈Su

(1) where Su denotes the set of possible state transitions (s , s) associated with the input bit uk = u. Define the forward metric, the branch metric, and the backward metric as ′

αk (s) = P (sk = s, y1 , . . . , yk ) , ′

′

(2)

γk (s , s) = P(sk = s, yk |sk−1 = s ),

(3)

βk (s) = P(yk+1 , . . . , yK |sk = s).

(4)

We then have the factorization: P(sk−1 = s′ , sk = s, y) = αk−1 (s′ )γk (s′ , s)βk (s).

(5)

The forward metric αk (s) is computed in a recursive manner as ∑ γk (s′ , s) αk−1 (s′ ), (6) αk (s) = s′

(8)

where P (uk ) is the prior probability, and P(yk |uk ) = P(yk |xk ). In conventional BCJR, the bit priors are set to P(uk = 0) = P(uk = 1) = 1/2. III. ICL-SEC Fig. 2 provides a detailed view of the ICL-SEC processing, serving as a zoomed-in representation of the ICL-SEC component in the high-level system diagram of Fig. 1. In Fig. 2, the upper part is a factor graph of the convolutional code; the BCJR decoding as described in Section II-B can be derived from this factor graph by applying the belief-propagation rule. The masked language model (MLM) and CLP updater at the lower part can be viewed as a semantic decoder separate from the BCJR decoder. A. Overview Conventional BCJR assumes that the information bits are equally probable and thus set P(uk = 0) = P(uk = 1) = 1/2 in (8). However, text exhibits strong structural characteristics. Consequently, an LM can exploit syntactic structures, lexical regularities, and contextual dependencies to infer semantic information about likely words. The semantic correlations among words uncovered by LM can be leveraged to provide CLPs to BCJR for refined decoding. A principle of ICL-SEC is to iteratively refine the values of P(uk ) in (8) in successive BCJR decoding. Specifically, the generated CLPs are fed into the BCJR decoder, whose output is subsequently used by the LM-driven semantic decoder to update the CLPs. The refined priors are then used in the next decoding iteration. The three major steps, briefed introduced in Section I, are outlined below with reference to Fig. 2. Precise details are provided in Part B, Part C, and Part D.

4

y1,2

y1,1

y2,1

 ( y1,1| x1,1)

x2,1  ( y2,1| x2,1)

 ( s1 )

s0 p (u1 )

p (u1 )

MLM

x3,2

 ( s2 )

s2 q (u2 )

p (u2 )

q (u2 )

s3 q (u3 ) p (u3 )

u3 p (u2 )

q (u3 ) p (u3 )

Interleaver/deinterleaver

r (v1 )

HD & masking

x3,1

 ( y2,2| x2,2 )

u2

u1 q (u1 )

 ( y3,2| x3,2 )

x2,2

s1 q (u1 )

ŝ m

 ( y3,1| x3,1)

 ( y2,2| x2,2 )

x1,2

y3,2

y3,1

 ( y2,1| x2,1)

 ( y1,2| x1,2 )

x1,1

y2,2

r (v2 )  p (v2 )q (v2 )

q (v1 )

p (v1 )

q (v2 ) p (v2 )

r (v3 )

q (v3 )

p (v3 )

eration count is reached or an early stopping rule is met. Specifically, early stopping is triggered when no unconfirmed words remain after the HD-and-masking step, or the change in the priors relative to the previous iteration becomes sufficiently small. After the final iteration, the decoded message ŝ is (i) constructed using the decoded words {ŵn }n∈N as the final output. Note that, after the final iteration, the most-likely words at each position are left unmasked whether its probability exceeds the threshold T or not, and there are no masks in the final sequence ŝ. In the following, we elaborate on the procedure in Fig. 2. Unless otherwise stated, we describe the bits involved in the word-level operations, such as candidate construction, HD, masking, and CLP update, in the natural bit-order vk . The BCJR decoder operates in the interleaved order uk . After each BCJR pass, r(i) (uk ) and q (i) (uk ) are deinterleaved to obtain r(i) (vk ) and q (i) (vk ). After the CLP update, p(i+1) (vk ) is interleaved back to p(i+1) (uk ) for the next BCJR pass.

 n (w )

ŝ m

Cross-layer prior (CLP) updater

Fig. 2. ICL-SEC with R = 1/2 and K = 3. The γk (s′ , s) term in (8) becomes γk (s′ , s) = p(uk )P(yk,1 |xk,1 )P(yk,2 |xk,2 ), where (xk,1 , xk,2 ) ∈ {−1, 1}2 are the BPSK-modulated channel-coded symbols corresponding to the source bit uk ∈ F2 , and (yk,1 , yk,2 ) are the received BPSK symbols.

Superscripts i = 1, . . . , I in the following denote the iteration index. Initialization: Set i = 1 and initialize the interleaved-order bit-prior p(i) (uk = 0) = p(i) (uk = 1) = 1/2 for all k ∈ K. Step 1 – BCJR Decoding and Masking (see Part B): Perform BCJR decoding to produce the complete APP r(i) (uk ), as given by (1). BCJR also generates the soft information q (i) (uk ) that will be fed into the CLP updater. Note that r(i) (uk ) ∝ p(i) (uk )q (i) (uk ), where q (i) (uk ) is the intrinsic information and p(i) (uk ) is the extrinsic information from the perspective of the BCJR decoder. After deinterleaving π −1 , we have r(i) (vk ), from which word-level posterior probabilities are derived. The hard-decision (HD) and masking module in Fig. 2 examines the probability of the most likely word at each word position. If the probability exceeds a prescribed threshold T , the word is left unmasked; otherwise, it is replaced by a mask symbol. Step 2 – MLM-Driven Contextual Inference (see Part C): The sequence of unmasked words and masks, denoted by (i) ŝm in Fig. 2, is provided to a masked language model (MLM). Conditioned on the visible words and their surrounding context, the MLM produces a probability distribution over (i) candidate words for each word position n, denoted by θn (w) in Fig. 2. Step 3 – CLP Update (see Part D): As shown in Fig. 2, (i) (i) ŝm , θn (w), and q (i) (vk ) are provided as inputs to the CLP (i) (i) updater. Based on (ŝm , θn (w), q (i) (vk )), the CLP updater computes the updated bit-level priors, denoted by p(i+1) (vk ). The updated priors are fed back to the BCJR decoder as extrinsic information in its next-iteration decoding. The above procedure is repeated until the maximum it-

B. BCJR-Intrinsic Information, Hard Decision, and Masking This part elaborates on the BCJR decoding, word-wise HD, and masking mechanism in Step 1 of Part A. To ensure probabilities in the distribution summing to one, we define a normalization operator Na,A : f (a) Na,A (f (a)) = ∑ , a ∈ A. f (b)

(9)

b∈A

BCJR-Intrinsic Information: As shown in Fig. 2, given the input prior p(i) (vk ), the BCJR decoder produces the complete BCJR APP r(i) (vk ), given by ( ) r(i) (vk = v) = Nv,F2 p(i) (vk = v) · q (i) (vk = v) , v ∈ F2 . (10) The intrinsic information q (i) (vk ) (from the BCJR decoder’s perspective) is output to the CLP updater as one of its inputs. Word-Wise HD: The complete probabilities r(i) (vk ) are used to decide the hard word. Denote the complete vocabulary by W. For each word wn , n ∈ N , we construct a candidate set Cn ⊂ W where each of the words has Ln letters. Let Bn = {K̃n + 1, . . . , K̃n+1 } be the set of∑natural-order bit indices n−1 covered by word n, where K̃n = ℓ=1 Kℓ with Kl = 8Ll . For a candidate w ∈ Cn at the word position n, let bk (w) ∈ F2 denote its ASCII bit at the natural-order bit-position k ∈ Bn . Note that a sequence of bits may or may not be a valid word w ∈ Cn . To ensure that the probabilities of candidate words sum to one, we need to normalize the raw bit-sequence probabilities of a word. For each word position n ∈ N , we compute the probability of each valid-word candidate w ∈ Cn from r(i) (vk ) by ( ) ∏ ( ) (i) (i) r vℓ = bℓ (w) , w ∈ Cn . (11) µn (w) = Nw,Cn ℓ∈Bn

Then we determine the most probable candidate and its confidence: ŵn(i) = arg max µn(i) (w) , Ξn(i) = max µ(i) n (w). w∈Cn

w∈Cn

(12)

5

(i)

Mapping (ŵn )n∈N back to bits yields the HD bit stream (i) (v̂k )k∈K . (i) Masking: For each n ∈ N , if Ξn ≥ T for a threshold (i) (i) T ∈ [0, 1], then ŵn is left unmasked; otherwise, ŵn is replaced by a mask symbol (e.g., <mask>). We define U (i) ⊆ N as the set of word positions that are unmasked at iteration i = 1, . . . , I with the initialization (i) U (0) = ∅, and define its complement U = N \ U (i) . With these definitions, we form { (i) ŵn , n ∈ U (i) , (i) w̃n = (13) (i) <mask>, n ∈ U . (i)

Inserting spaces between adjacent words in {w̃n }n∈N yields (i) the sequence of unmasked words and masks, denoted by ŝm . (i) An example of ŝm is “There <mask> a beach with palm <mask> and clear blue water.” Remark 1: The BCJR-intrinsic information q (i) (vk ) is output to the CLP updater as one of its inputs. Note that, if the MLM and CLP updater in Fig. 2 are viewed as a semantic decoder separate from the BCJR decoder, then (i) q (i) (vk ) and ŝm can be interpreted as extrinsic information from the semantic decoder’s perspective. In particular, while (i) q (i) (vk ) is soft information, ŝm is hard information: a mask indicates no extrinsic information is given at the associated word position, and an unmasked word conveys the intention to fix the word without further changes (this is the case at least for the proposed Confirm scheme in Part D). In that light, the threshold T should be chosen close to one when targeting a very low word error rate. C. MLM-Driven Contextual Inference This part elaborates on the MLM-driven contextual inference in Step 2 of Part A. With reference to Fig. 2, the MLM (e.g., mmBERT [7]) takes the sequence of decoded words and (i) masks ŝm , as input, and produces a probability distribution over candidate words at each word position. We restrict the MLM output to the length-compatible candidate set Cn ∈ W. After normalization over Cn , this gives the semantic word likelihood for each word n ∈ N : ( ( )) θn(i) (w) = Nw,Cn P w|ŝ(i) , MLM , w ∈ Cn , (14) m where “MLM” in (14) represents the adoption of a specific (i) (i) instance of MLM. Both ŝm and θn (w) are forwarded to the CLP Updater as its inputs. D. Confirm CLP Update (Ultimate Proposed Scheme) This part elaborates on the CLP update in Step 3 of Part A. In this paper, we study three CLP update schemes that share the same components in Fig. 2 except the CLP updater module. This part focuses on the “Confirm CLP updater” (this paper’s ultimate proposed scheme). Part E will elaborate on the other two schemes. 1) Specification of Confirm CLP Updater As shown in Fig. 2, the CLP updater refines p(i) (vk ) (i) using the masked text sequence ŝm , MLM-produced word

(i)

distribution θn (w), and the intrinsic information of BCJR q (i) (vk ). The Confirm CLP updater follows the principle of successive confirmation. Once a word is unmasked, it is confirmed and remain unmasked in all subsequent iterations. This is achieved by setting the bit priors of unmasked words to 0/1 value (as elaborated below), forcing the subsequent BCJR decoding to assign APP with probability one to the unmasked words so that their probabilities will meet the threshold T and these words remain unmasked in the next iteration. Update of p(i) (vk ) for Unmasked Words: Specifically, suppose that the word at position n is unmasked (i.e., n ∈ U (i) ) (i) and (v̂k )k∈Bn are the HD bits of the unmasked words. Then we set that { (i) 1, v = v̂k , (i+1) p (vk = v) = k ∈ Bn , n ∈ U (i) . (15) (i) 0, v ̸= v̂k , This assignment in (15) ensures consistent reliability interpretation between the BCJR decoder and the MLM, as elaborated in Remark 4. Update of p(i) (vk ) for Masked Words: Consider a particular bit position k. With reference to Fig. 2, BCJR produces the APP r(i) (vk ). This APP is a complete probability, since it already includes the incoming prior p(i) (vk ), as in (10). Therefore, the information passed from the BCJR decoder to the CLP updater should remove the contribution of this incoming prior to avoid double dips. Following this principle, we leverage the BCJR intrinsic q (i) (vk ) rather than r(i) (vk ) to update the CLP. Specifically, we combine q (i) (vk ) with the MLM likelihood (i) θn (w) to generate the next bit prior for the masked words. For (i) a target bit k ∈ Bn within a masked word n ∈ U , we first form a candidate likelihood using all other bits’ information in the same word: ∏ ( ) (i) q (i) vℓ = bℓ (w) , w ∈ Cn . (16) ϕn,\k (w) = ℓ∈Bn ℓ̸=k

The exclusion of the target bit k ensures that the evidence carried by q (i) (vk ) is not fed back into the prior of the same bit, so that the same evidence is not counted twice. The wordlevel posterior used to update bit k is obtained by combining the MLM likelihood and the BCJR likelihood: ( ) (i) (i) ρn,\k (w) = Nw,Cn θn(i) (w) · ϕn,\k (w) , w ∈ Cn . (17) Finally, the updated CLP for the bit k covered by a masked word is obtained by marginalizing over all candidate words whose bit value at position k equals v ∈ F2 : p(i+1) (vk = v) ∑ (i) (i) ρn,\k (w) I (bk (w) = v), k ∈ Bn , n ∈ U , =

(18)

w∈Cn

where I(·) is the indicator function. The resulting naturalorder priors p(i+1) (vk ) are then interleaved by π to obtain p(i+1) (uk ), which serves as the bit prior for the BCJR decoding in the next iteration. Remark 2: The reader may wonder why q (i) (vℓ ) , ℓ ̸= k, in (16), can be treated as extrinsic information for bit k

6

by the BCJR decoder, even though q (i) (vℓ ) are actually intrinsic information produced by the BCJR decoder. The rationale is as follows. Because of the interleaver, the bits associated with the same word are not necessarily adjacent in the convolutional codeword. Exact vs. Approximate: If the interleaver is designed so that any two bits belonging to the same word are separated by at least C bit positions, where C is (i) the constraint length of the convolutional code, then ϕn,\k (w) in (16) is strictly extrinsic information. This is because, under such an interleaving constraint, the bits of the same word do not share local dependencies induced by the convolutionalcode trellis. Otherwise, treating it as such is an approximation. Design Choice: To maintain simplicity, simulations in Section IV uses an unrestricted interleaver design. We adopt this to demonstrate that very good performance can be obtained despite the approximation, suggesting that this approach can also be applied to LDPC codes, for which there is no direct notion of a constraint length. Remark 3: The reader is reminded that the computation of p(i+1) (vk ) in (16)–(18) applies to masks only. In the (i) mask-unmask word sequence ŝm presented to the MLM, no information is given by the BCJR decoder on the masked words. Specifically, the information contained in q (i) (vℓ ) with ℓ ̸= k, in (16), has not been used twice by the semantic decoder consisting of the MLM and the CLP Updater. 2) Properties of Confirm Scheme Proposition 1 (Non-Oscillatory Confirmation): In the Confirm scheme, once a word becomes unmasked and is confirmed at iteration i, it remains unmasked in all subsequent iterations, and all its bits retain the same values. Specifically, for any (i) (i) confirmed word ŵn at iteration i with bit values (v̂k )k∈Bn , (j) (j) (i) (j) (i) (j) we have ŵn = ŵn , v̂k = v̂k , and µn (ŵn ) = 1 for all iterations j > i. Thus, a confirmed word cannot oscillate between the unmasked and masked states. Proof: By induction, it suffices to prove the proposition for j = i + 1. Consider a particular confirmed word n ∈ (i+1) U (i) at iteration i. For iteration i + 1, let γπ(k) (s′ , s) be the BCJR branch metric at the natural-order bit k ∈ Bn with an associated input bit uπ(k) = vk . With reference to (8), the input-bit prior is a multiplicative factor in the branch metric, and thus every branch inconsistent with the HD value has zero (i+1) metric γπ(k) (s′ , s) = 0, whenever the input bit uπ(k) = vk ̸= (i)

v̂k , k ∈ Bn . According to (5) and (1), for every k ∈ Bn , we therefore have { (i) 1, v = v̂k , (i+1) r (vk = v) = v ∈ F2 . (19) (i) 0, v ̸= v̂k , Substituting these deterministic APPs into the word(i) candidate scoring in (11), the candidate ŵn is the only one whose bit-wise probability factors are all non-zero; every other (i) candidate in Cn differs from ŵn in at least one bit and hence receives zero score. Therefore, we have { (i) 1, w = ŵn , (i+1) µn (w) = w ∈ Cn . (20) (i) 0, w ̸= ŵn ,

(i+1)

(i)

(i+1)

With (12), we arrive at ŵn = ŵn and Ξn = 1 ≥ T. Thus, by the HD-and-masking rule, word n remains unmasked in all subsequent iterations, and all its covered bits remain the same values. Corollary 1 (Non-Decreasing Confirmation): In the Confirm scheme, the confirmed-word set is non-decreasing, i.e., Ui−1 ⊆ Ui , i = 1, . . . , I.

(21)

Proof: Implied by Proposition 1. Remark 4 (Consistent Reliability Interpretation): In the Confirm scheme, an unmasked word has the same reliability meaning for both the BCJR decoder and the MLM-empowered semantic decoder. Once the HD-and-masking component in Fig. 2 declares a word to be unmasked, the word is treated as fully reliable by the semantic decoder as well as the BCJR decoder in future iterations. Remark 5 (Analogy with Successive Interference Cancellation): The successive-confirmation mechanism is analogous to successive interference cancellation (SIC) decoding for overlapped signals in wireless networks. In SIC, once a constituent signal within a superposition of signals is decoded with sufficiently high confidence, it is fixed and subtracted from the received signal, facilitating the decoding of the remaining signals. As more signals are successively decoded and canceled, fewer unresolved signals remain, until eventually all signals are decoded. Similarly, in our successive confirmation, once certain words are deemed reliable, they are fixed and used as semantic context to help recover the remaining masked words. As confirmations proceed, the number of unmasked words increases, until all words are unmasked in the final iteration. E. Naive and Hold CLP Update We next introduce two additional CLP update schemes, referred to as Naive and Hold. Our investigation in this paper originally began with the Naive scheme. The Hold scheme was subsequently devised to mitigate the oscillatory behavior observed in Naive for all text messages. Although Hold removes this behavior in a broad sense, oscillations may still occur for specific text messages. The Confirm scheme was ultimately developed as a consistent iterative scheme with provably non-oscillatory behavior. Confirm, Naive, and Hold schemes, share the same components in Fig. 2 except the CLP updater module. Therefore, we focus the Naive and Hold CLP updater here. Before proceeding, we note that the MLM, e.g., mmBERT (i) [7], produces a probability distribution θn (w) over candidate words at every word position, including both masked and unmasked positions. The Confirm scheme, however, does not use the distribution at unmasked positions. The Naive scheme, in contrast, also makes use of the unmasked-position probability distribution, while the Hold scheme does not exploit such information like Confirm scheme. Naive Scheme: In this scheme, the MLM-empowered CLP updater refines the priors of all words at every iteration, regardless of whether they are masked or unmasked. Specifically, for

7

each word n ∈ N and each its covered bit k ∈ Bn , the prior is updated using the same marginalization rule as in (18): p(i+1) (vk = v) ∑ (i) = ρn,\k (w) I (bk (w) = v), k ∈ Bn , n ∈ N ,

(22)

w∈Cn (i)

where ρn,\k (w) used in Naive is defined in the same way as in (17). The key difference from Confirm is that the Naive scheme also updates the priors of unmasked words instead of fixing the associated bit priors as probability one. This design leads to a key shortcoming. Recall that we combine the MLM likelihood and the BCJR likelihood in (17). For a masked word n, the MLM likelihood is inferred from the surrounding context of the word (i.e., words surrounding word n). However, for an unmasked word n, the MLM can observe (i) the decoded word ŵn itself. As a result, the MLM likelihood is no longer inferred only from surrounding context; it also reuses information from word n. In (17), the information from (i) the unmasked word n also contributes to ϕn,\k (w). Therefore, the prior update in Naive creates a feedback loop and doublecounts the same evidence within each unmasked word. This loop can perturb words that were already deemed reliable. Consequently, an unmasked word may become masked again in a later iteration, leading to oscillation and instability (see the results in Section IV-C). In particular, the Naive scheme does not satisfy Proposition 1. Neither does it have the consistent reliability interpretation mentioned in Remark 4. Since an unmasked word is not interpreted using bit priors with probability one, the reliability interpretation is inconsistent between the MLM and BCJR: the MLM sees an unmasked word as truly trusted hard context, whereas BCJR still receives only soft priors. Remark 6: We refer to this scheme as Naive because it applies the sum-product belief-propagation principle in a broad but overly simplistic manner. Specifically, the sum-product operation in (22) is applied to all word positions without accounting for the particular structure of the scheme in Fig. 2. In particular, the input to the MLM is a sequence of unmasked words and masks, rather than a probability distribution over candidate words at each position. This differs from a standard belief-propagation network, in which all beliefs propagated between nodes are probabilistic beliefs. Hold Scheme: Unlike the Naive scheme, the Hold scheme does not refine the priors of unmasked words for the next iteration. Specifically, if word n ∈ U (i) is unmasked at iteration i, then for each k ∈ Bn , the prior is held: p(i+1) (vk = v) = p(i) (vk = v) , v ∈ F2 , k ∈ Bn , n ∈ U (i) . (23) For masked words, the Hold scheme uses the same CLPupdate procedure as in (16)–(18). The Hold scheme avoids the direct double-counting issue of Naive because it does not directly refine the priors for the unmasked words. However, holding a soft prior is still fundamentally different from confirming a word with probability one. An unmasked word in the Hold scheme may become masked in a later iteration even though its prior was held fixed after the iteration in which it was unmasked. This is because

the BCJR APP of that word depends not only on its own prior, but also on the forward and backward beliefs propagated through the trellis. When the priors of other words change, the information entering the current word also changes. Thus, a previously unmasked word can fail the masking threshold later. Once such a word becomes masked again, its prior is no longer held and is renewed again by the CLP updater in (16)–(18). Therefore, the Hold scheme may still exhibit unmask-to-mask transitions across iterations, which is verified by our simulations in Section IV-C. In other words, it does not provide the non-oscillatory guarantee in Proposition 1. The Hold scheme also does not satisfy the consistency property mentioned in Remark 4. When a word is unmasked, the MLM observes it as a hard, fully trusted word. However, BCJR only receives the previously held soft prior rather than a deterministic prior with probability one. Therefore, the same unmasked word has different reliability meanings in the two modules. IV. S IMULATION R ESULTS A. Experimental Setups Model and Dataset: We employ mmBERT [7] as the MLM for the CLP update. We use the 50 English passages [2] as the transmitted messages. For each tested SNR, the transmission of each passage is simulated over 100 Monte-Carlo trials with independently generated channel noise, yielding 5000 received corrupted passages. System Settings: We instantiate the physical-layer channel code using a convolutional code with rate R = 1/2 and constraint length C = 3. We apply a random interleaver π. For the iterative schemes, the maximum number of iterations is set to Imax = 10, and the prior-difference stopping threshold is set to ϵ = 10−3 . For the masking probability threshold T , we adopt an operating-point-dependent setting. Specifically, T is calibrated offline as a function of the operating SNR and stored in a lookup table, analogous to the standard adaptive modulation and coding (AMC) practice in modern receivers, where transmission parameters are selected according to the estimated channel quality. This incurs no additional online cost, since SNR estimation is routinely available at the receiver. Investigated Schemes: We compare five schemes. The first two are non-iterative, while the last three are iterative and (i) differ only in the CLP prior-update rule. Let {ŵn }n∈N be (i) the decoded words and {v̂k }k∈K be the associated HD bits across all schemes, where the iteration index i = 1 for the non-iterative schemes and i = 1, . . . , I ≤ Imax for the iterative (i) (i) schemes. The determination of {ŵn }n∈N and {v̂k }k∈K in iterative schemes has been elaborated in Section III-B. 1) BCJR Decoding: This is the conventional channel decoding baseline. The BCJR decoding is run once with uniform bit priors to obtain bit-level APPs r(1) (vk ), k ∈ K. Then the conventional bit-wise HD is performed: (1)

v̂k = arg max r(1) (vk = v) .

(24)

v∈F2

(1)

Converting {v̂k }k∈Bn for each n ∈ N to characters yields (1) decoded words {ŵn }n∈N . This pure physical-layer scheme

8

100

100

100

10-2

10-2

10-4

10-4

10-6

10-5 0

0.5

1

1.5

2

2.5

3

100 99

100 99.8 99.6 0

98

0.5

10-6 0

0.5

1

1.5

2

2.5

3

100

100

90

95

80

90

0

0.5

1

1.5

2

2.5

3

0

0.5

1

1.5

2

2.5

3

1

97 96

70 0

0.5

1

1.5

2

2.5

3

85 0

0.5

1

1.5

2

2.5

3

Fig. 3. Performance comparison with varying SNR. The first row reports (a) BER, (b) WER, and (c) CER; the second row reports (d) BERTScore, (e) BLEU, and (f) ROUGE-L.

does not use any semantic information. The bit sequence in (1) ŵn may not even be a valid word in the vocabulary. 2) CL-SEC Scheme: This is the non-iterative baseline from [2]. The BCJR decoding is first run once with uniform priors to obtain bit-level APPs r(1) (vk ), k ∈ K. Then the word-wise HD and masking in Section III-B are performed, producing (1) (1) {ŵn }n∈N and U . To replace each unreliable decoded word that is masked, CL-SEC combines the physical-layer word likelihood obtained from BCJR probabilities with the MLM likelihood acquired from semantic context. That is, ∏ (1) δn(1) (w) = r(1) (vk = bk (w)) · θn(1) (w), n ∈ U , (25) k∈Bn (1) ŵn(1) ← arg max δn(1) (w), n ∈ U . w∈Cn (1)

(26) (1)

Mapping {ŵn }n∈N to ASCII bits yields {v̂k }k∈K . Unlike ICL-SEC, this scheme performs the cross-layer correction only once and does not generate prior to feed back to BCJR decoder. 3) Naive Scheme: This scheme has been detailed in Section III-E. At each iteration, it updates the priors of both masked and unmasked words using the MLM-empowered CLP updater, as in (22). 4) Hold Scheme: This scheme has also been introduced in Section III-E. For words that are unmasked at an iteration, the scheme keeps their priors unchanged in the next iteration (see (23)); for masked words, it updates the priors using the MLM-empowered CLP updater (see (16)–(18)). 5) Confirm Scheme (Proposed): The Confirm scheme is the proposed ICL-SEC scheme with successive confirmation (see Section III-D). Once a word is unmasked with sufficient reliability, it is confirmed and its corresponding bit priors are fixed to probability one in all subsequent iterations, as in (15). Bit priors for the masked words are updated using the MLMempowered CLP updater, as in (16)–(18).

Evaluation Metrics: We evaluate both transmission reliability and semantic fidelity. For transmission reliability, we report bit error rate (BER), word error rate (WER), and character error rate (CER). For semantic fidelity, we report BERTScore [8], BLEU [9], and ROUGE-L [10]. BERTScore measures semantic similarity through contextual token embeddings. BLEU measures n-gram overlap between the recovered and true texts. ROUGE-L measures sequence-level overlap based on the longest common subsequence. The performance metrics are computed as follows for each i. (i) First, BER is computed by comparing the bit stream (v̂k )k∈K with its ground truth (vk )k∈K . Second, we obtain WER by (i) comparing {ŵn }n∈N with {wn }n∈N ; a word is counted as correct only if all its letters are recovered correctly. Third, CER (i) is evaluated by comparing the character string (ŵn )n∈N with (wn )n∈N . Finally, we obtain the recovered spaced text ŝ(i) by (i) inserting spaces between adjacent words in {ŵn }n∈N , and (i) input ŝ and its ground-truth counterpart to each semantic scorer for the semantic-fidelity computation. Since the BCJR scheme does not use any semantic information, its severely poor semantic-fidelity results are omitted in the figures for clarity. B. Varying-SNR Performance Fig. 3 compares the performance of the investigated schemes over different SNRs. The results of iterative schemes are collected from the final iteration. Across all six metrics, the Confirm ICL-SEC scheme consistently achieves the best performance. Importantly, Confirm ICL-SEC achieves a substantial improvement over the non-iterative CL-SEC. For example, at 1.5 dB SNR, the BER of non-iterative CL-SEC is 1.90×10−3 , while Confirm ICL-SEC reduces it to 8.64×10−6 , constituting a BER reduction by a factor of 220; across all considered SNRs, Confirm ICL-SEC reduces the BER by more than two orders of magnitude compared with CL-SEC.

9

100 10-2

100

10-1 10-2

10-3

10-2

10-4

10-3 1

2

3

4

5

6

7

8

9

10

100

10-4 1

2

3

4

5

6

7

8

9

10

100

100

80

90

60

80

1

2

3

4

5

6

7

8

9

10

1

2

3

4

5

6

7

8

9

10

95 100 99.5 4

5

6

90

40 1

2

3

4

5

6

7

8

9

10

70 1

2

3

4

5

6

7

8

9

10

Fig. 4. Per-iteration performance at 0 dB SNR, averaged over 5000 Monte-Carlo trials. The first row reports (a) BER, (b) WER, and (c) CER; the second row reports (d) BERTScore, (e) BLEU, and (f) ROUGE-L.

Moreover, the gain over the conventional BCJR is larger: BCJR has a BER of 2.85 × 10−2 at 1.5 dB SNR, and thus Confirm scheme achieves a BER reduction by a factor of 3300, corresponding to more than three orders of magnitude. The Confirm scheme also outperforms the other iterative approaches. For instance, compared with the Naive and Hold schemes, Confirm reduces the WER at 1.5 dB SNR by factors of 26.3 and 12.4, respectively. This indicates that iteration alone is not sufficient – the way in which the CLP is updated is critical. Both the Naive and Hold schemes may render the unmasked word become masked again in a later iteration, as shown in Fig. 4 and Fig. 5. This limitation prevents them from achieving the superior performance of the proposed successive confirmation design. From Fig. 3, the same trend is also observed in CER, which is consistent with the BER and WER results. Semanticfidelity results further confirm the advantage of Confirm ICLSEC. At 0 dB SNR, Confirm scheme achieves a BERTScore, BLEU, and ROUGE-L of 99.92, 99.57, and 99.82, respectively, demonstrating the accurate semantic recovery even at low SNR. C. Iterative Performance Fig. 4 examines the evolution of different schemes over iterations at 0 dB SNR, averaged over 5000 Monte-Carlo trials. Since BCJR and CL-SEC schemes are non-iterative, they appear as horizontal reference lines. Confirm ICL-SEC achieves the best final performance and improves monotonically without oscillation. Moreover, ICL-SEC exhibits a fast convergence rate. For 0 dB SNR, it merely requires about five iterations on average to converge. In contrast, the Naive scheme exhibits severe oscillatory behavior: its error metrics can decrease in one iteration and then increase again in a later iteration. The oscillations are persistent and uniform over the 5000 trials and manifest themselves in the results of all metrics despite the averaging

20

15

10

5

0 1

2

3

4

5

6

Fig. 5. Unmask-to-mask transitions for a single trace operated at 0 dB SNR with N = 105 words.

over the 5000 trials. From Fig. 4, we see that even iterations having better performance (e.g., BER) than odd iterations. Furthermore, the BER performance does not improvement further after iteration 2. The Hold scheme appears more stable after averaging over many trials, but it still lacks the non-oscillatory guarantee of the Confirm scheme as mentioned. From our experiments, we observe that the oscillations of Hold vary from trial to trial and is trial-specific (see Fig. 5). Fig. 5 provides a closer look at the unmask-to-mask transitions for a single trace at 0 dB SNR. Specifically, at iteration i ≥ 2, we count the number of unmasked words at iteration i − 1 which revert to masked words at iteration i. The transmitted message contains N = 105 words. Notably, the Confirm scheme has zero unmask-to-mask transitions. This result directly validates Proposition 1: once a word is confirmed, its associated bit priors are fixed to probability one, and hence the word cannot become masked again in later iterations. In contrast, the Naive scheme exhibits frequent

10

decision reversals, with 31 unmask-to-mask transitions in this trace. The Hold scheme reduces but does not eliminate such transitions, with 4 unmask-to-mask events.

codes, larger language models, adaptive threshold design, and non-textual modalities such as images, speech, and multimodal messages.

V. C ONCLUSION

R EFERENCES

This paper proposed ICL-SEC, an iterative cross-layer semantic error correction framework that integrates physicallayer soft information with application-layer language-modeldriven contextual inference through repeated feedback. Motivated by the success of turbo and LDPC decoding, the proposed approach treats semantic recovery not as a one-shot post-processing task, but as an iterative refinement process in which the channel decoder and the semantic decoder exchange complementary reliability information. The central principle of ICL-SEC is successive confirmation. After each BCJR decoding pass, word-level posterior probabilities are used to identify reliable words. These words are confirmed and then revealed to the masked language model as trusted context, while unreliable words remain masked. The language model generates contextual word likelihoods for the unresolved positions, and these likelihoods are leveraged to refine the bit-level priors for the next channel-decoding iteration. Unlike Naive iterative updates that repeatedly revise all words, the proposed Confirm scheme assigns deterministic priors with probability one to words once they are declared reliable. This design gives an unmasked word the same meaning in both layers: it is fully trusted semantic context for the applicationlayer language model and fully reliable side information for the physical-layer channel decoder. This reliability-consistent design leads to an important structural property: confirmed words cannot revert to masked words in later iterations. The resulting non-oscillatory behavior distinguishes the proposed method from the Naive and Hold variants, both of which can suffer from instability because previously unmasked words may later become unreliable again. The Confirm scheme therefore does more than add iteration; it provides a principled prior-update rule that stabilizes the interaction between physical-layer decoding and semantic inference. Simulation results demonstrate the significance of this design. Across BER, WER, CER, BERTScore, BLEU, and ROUGE-L, ICL-SEC consistently improves over conventional BCJR decoding, non-iterative CL-SEC, and the alternative iterative update approaches. The proposed Confirm scheme reduces BER by more than two orders of magnitude compared with non-iterative CL-SEC and achieves high semantic fidelity even at low SNR. These gains show that semantic information, when fed back in a carefully controlled and reliability-aware manner, can substantially enhance communication reliability. More broadly, this work suggests a new direction for semantic communications: semantic intelligence should not merely repair corrupted outputs after channel decoding, but should participate actively in the decoding loop. By aligning channel reliability and semantic confidence through iterative cross-layer exchange, ICL-SEC provides a step toward communication systems that are both error-resilient and meaningaware. Future work may extend this principle to other channel

[1] W. Ryan and S. Lin, Channel codes: Classical and modern. Cambridge Univ. Press, 2009. [2] Y. Wang, Y. Du, S. C. Liew, Y. Pan, F. Zhang, and L. Zhang, “CL-SEC: Cross-layer semantic error correction empowered by language models,” arXiv:2603.26125, 2026. [3] Z. Li, C. Yue, J. Hao, B. Vucetic, and Y. Li, “LLM-Viterbi: Semanticaware decoding for convolutional codes,” arXiv:2604.19035, 2026. [4] J. Hao, C. Yue, W. Liu, B. Vucetic, and Y. Li, “Semantic error correction and decoding for short block codes,” arXiv:2604.22269, 2026. [5] C. Yue, B. Vucetic, and Y. Li, “Semantic ordered statistics decoding,” arXiv:2605.02296, 2026. [6] J. Hao, C. Yue, H. Chang, B. Vucetic, and Y. Li, “Short wins long: Short codes with language model semantic correction outperform long codes,” in Proc. GLOBECOM, 2025, pp. 1226–1231. [7] M. Marone, O. Weller, W. Fleshman, E. Yang, D. Lawrie, and B. V. Durme, “mmBERT: A modern multilingual encoder with annealed language learning,” arXiv:2509.06888, 2025. [8] T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “BERTScore: Evaluating text generation with BERT,” in Proc. ICLR, 2020. [9] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: A method for automatic evaluation of machine translation,” in Proc. ACL, 2002, pp. 311–318. [10] C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out, 2004, pp. 74–81.

Related documents

Record · ID 459603 · SHA-256 3d9bd98bf1d17b9c
Retrieved via Conceptio: every document is proof-bundled with source, license, and retrieval metadata.
Bring this data into your agent: Connect via MCP or API · 20 free credits to start · no card required
Read the Docs → · View CLI source on GitHub · Conceptio Open Knowledge Archive