Token Encoding for Semantic Recovery
arXiv:2604.12931v1 [eess.SP] 14 Apr 2026
Jingzhi Hu and Geoffrey Ye Li Department of Electrical and Electronic Engineering Imperial College London, London, UK [email protected], [email protected]
Abstract—Token-based semantic communication is promising for future wireless networks, as it can compact semantic tokens under very limited channel capacity. However, harsh wireless channels often cause missing tokens, leading to severe distortion that prevents reliable semantic recovery at the receiver. In this article, we propose a token encoding framework for robust semantic recovery (TokCode), which incurs no additional transmission overhead and supports plug-and-play deployment. For efficient token encoder optimization, we develop a sentencesemantic-guided foundation model adaptation algorithm (SFMA) that avoids costly end-to-end training. Based on simulation results on prompt-based generative image transmission, TokCode mitigates semantic distortion and can approach the performance upper-bound, even under harsh channels where 40% to 60% of tokens are randomly lost. Index Terms—Token encoding, token communication, generative semantic communication, semantic recovery.
I. I NTRODUCTION Future wireless networks are envisioned to be ubiquitous, expanding coverage into harsh wireless environments, such as satellite, offshore, underwater, and underground scenarios, where physical channels are severely capacityconstrained [1]. Semantic communication has emerged as a promising paradigm for such scenarios, as it transmits only highly-compact semantic information rather than raw bandwidth-demanding data, thereby significantly reducing transmission load compared with conventional communication systems [2]. However, harsh wireless environments incur challenges beyond limited channel capacity: severe channel impairments can cause frequent packet losses, while long delays often make retransmission impractical. In such scenarios, conventional communication systems rely on forward error correction coding and packet-level redundancy coding to ensure reliable delivery, while the required redundancy further consumes the already severely limited payload budget. By contrast, semantic communication does not seek exact recovery of transmitted information. Leveraging the distortion tolerance of latent semantic representations, it aims at semantic recovery, i.e., recovering information that preserves the source semantics under noisy and lossy transmissions, thereby maintaining downstream task performance. Existing semantic communication systems can be broadly categorized into tensor-based and token-based, depending on the transmitted contents, which exhibit different robustness characteristics over wireless channels. Tensor-based semantic communication encodes data into tensors of continuous or
discrete symbols for transmission, which are more compact than the raw data while having the potential to tolerate channel noise and errors. In [3], DeepJSCC maps images to tensors using a convolutional neural transceiver, which is trained to be robust to low signal-to-noise-ratio (SNR). In [2], Deepsemantic communication encodes text semantics into tensors with a transformer transceiver and optimizes sentence recovery, also showing robustness to low-SNR channels. In [4], a generative model at the receiver treats corrupted tensors as conditional inputs to regenerate the source image. On the other hand, token-based SC, referred to as token communication, leverages a codebook shared by the transmitter and receiver to encode data as a sequence of tokens, where each token corresponds to a codeword in the codebook [5]. Since each token can be represented as a codeword index, the compression efficiency of token-based semantic communication systems can be substantially higher than tensor-based semantic communication systems. Nevertheless, unlike tensors, an individual token is merely a discrete index and therefore provides little intrinsic robustness to channel-induced corruption. To address this limitation, existing robust token-based semantic communication methods exploit semantic correlation among tokens to combat channel corruption, mainly relying on jointly trained transceivers or receiver-end restoration. In [6], a vector-quantized variational autoencoder is trained end-toend with token masking, enabling the receiver to infer missing tokens. In [5], a large neural model is exploited at the receiver to process token sequences and recover missing tokens under packet loss. In this article, we focus on transmitter-end token encoding for robust semantic recovery without costly end-to-end training. The following example motivates our work. If the sentence, “a cartoon rabbit riding a red bicycle on the grass,” loses either “rabbit,” “red,” or “grass,” its original meaning will be hard to infer, as many equally plausible choices may fit the remaining context. Therefore, the token sequence should be proactively encoded into a more robust sequence at the semantic level before channel transmission. Our token encoding framework (TokCode) proactively encodes token sequences for robust semantic recovery over harsh wireless channels. Specifically, we consider a token communication system for generative image transmission, where the transmitter sends the token sequences corresponding to the prompts of images and the receiver locally generates the images from the received prompts. TokCode encodes a source token sequence into an equal-length sequence in the same
Tx image Text-to-image model
Rx image Satellite
Tokenizer
𝒙
Prompt 𝒙
𝒛
Underwater
Harsh wireless channel
# 𝒙
low capacity, frequent loss, long delay
𝒛$
Token Encoder TokCode Underground
Offshore Rx image
Fig. 1: Token communication system for image transmission over harsh wireless channels. Solid and dashed lines indicate the conventional and the TokCode’s signal paths, respectively. codebook but with enhanced robustness to lossy channel. To make this token encoding practical, we develop a sentencesemantic-guided foundation model adaptation (SFMA) algorithm that requires no costly end-to-end training. The rest of this paper is organized as follows. Sec. II introduces the token communication system, and Sec. III proposes the TokCode framework. Sec. IV develops the SFMA algorithm. Sec. V presents the simulation setup and evaluation results, and Sec. VI draws a conclusion. II. T OKEN C OMMUNICATION S YSTEMS In this section, we introduces a token communication system for generative image transmission over lossy channels. Consider a communication system over a harsh wireless channel as shown in Fig. 1, where the transmitter intends to transmit a set of images to the receiver. To reduce bandwidth requirements, only the prompt text corresponding to each image I is transmitted while the image is reconstructed using its local text-to-image generative model. In the paradigm of token communication, a prompt text is further represented by its token sequence, which is essentially an integer index vector x ∈ T N , obtained by a tokenizer. Here, N denotes the sequence length, and T = {1, . . . , T } denotes the set of possible tokens, with T being its size. Associated with these tokens is an embedding codebook V ∈ RT ×D , where the ith row vi is the semantic embedding, or codeword, assigned to token i. This embedding codebook is generally established by a text foundation model learning over a comprehensive dataset. We assume that the transmitter and receiver share the same codebook and tokenizer, which is consistent with practice given the widespread adoption of general-purpose pretrained text models, such as T5 [7]. For the same reason, we also assume that the prompt processing module of receiver’s generative model is based on the same tokenizer as the transmitter. As the mapping between prompt text and its token sequence is reversible, transmitting the token sequence is equivalent to transmitting the prompt text. Denote x̃ as the token sequence
received at the receiver. Then, the receiver can directly feed the received token sequence into the generative model to reconstruct an image, i.e., I˜ = G(x̃). In this article, we further assume that the receiver’s generative model captures the transmitter’s underlying image-to-text mapping. In this case, under a lossless channel, i.e., x̃ = x, ˜ is consistent with the source image the reconstructed image, I, I.1 However, in the harsh wireless environment with a lossy channel, we generally have x̃ ̸= x, which may cause I˜ to differ significantly from I. We adopt a packet-level lossy channel model suited to general harsh channel conditions, which is commonly adopted in the token-based semantic communication literature [5], [6]. In this channel model, each transmitted packet is either received correctly or lost entirely, with the receiver being aware of which packets are lost. For the packetization, we assume that the collection of token sequences is interleaved into P packets. Specifically, to reduce the probability for consecutive token indices to be lost together, we assume a modular interleaving scheme. For x, its n-th token is allocated to the q(n)-th packet, where q(n) = 1 + (n − 1 mod P ).
(1)
With this scheme, a single packet loss leads to evenly spaced token losses rather than a contiguous burst of missing tokens. Assume there are Ploss packets dropped randomly within P packets, which corresponding to the packet loss rate, p = Ploss /P . As a result, the channel output for each token is ( xn , if the q(n)-th packet is not lost, (2) x̃n = ∅, otherwise, where ∅ denotes a missing token. For presentation simplicity, we denote the lossy channel by x̃ = H(x). The above fixed-number-of-packets dropping model avoids Bernoulli loss-ratio fluctuations and facilitate stable evaluation. III. T HE T OK C ODE F RAMEWORK In this section, we the propose a token encoding framework (TokCode) for robust semantic recovery over harsh wireless channels. We first establish the process of token encoding and then formulate the semantic recovery optimization problem. A. Process of Token Encoding As indicated by the dashed lines in Fig. 1, x is tokenencoded before transmission. The token encoder is denoted by γ(·; Θ), parameterized by Θ ∈ RDout ×Din , where Dout and Din are its output and input dimensions, respectively.2 The encoded token sequence can be expressed as z = γ(x; Θ) = [z1 , z2 , . . . , zN ] ∈ T N .
(3)
1 When this mapping is not captured by receiver’s generative model, knowledge alignment between the transmitter and receiver is essential, which is investigated in our work [8], [9]. 2 For presentation simplicity, we use Θ as a shorthand for the collection of the token encoder’s parameter matrices, which may have diverse dimensions.
We note that the token encoding in (3) is realized at the semantic level because both the input x and the output z are sequences of tokens corresponding to semantic embeddings. In particular, it does not modify the feature-wise or symbol-wise semantic representations, but directly maps between semantic tokens. In this way, the encoded token sequence remains in the same token set and works with the same embedding codebook. Consequently, TokCode can be deployed in a plug-and-play manner without synchronously updating the codebooks of the transmitter and receiver or the downstream generative model. Moreover, we note that our framework can be directly extended to the unequal-length case. Shorter or longer encoding would lead to compression or redundancy injection, respectively. For clarity, we focus on the equal-length case and leave a detailed analysis of the unequal-length case to future work. Since |z| = |x| = N , the encoded token sequence incurs no additional transmission overhead. After the transmission of the encoded token sequence over the lossy channel in (2), the receiver obtains z̃ = H(z). The received sequence z̃ is then sent to the generative model for image reconstruction, i.e., I˜ = G(z̃). Overall, the goal of TokCode is to maximize the semantic similarity between original image I and reconstructed image I˜ under lossy channel H(·). Leveraging the pre-trained model that extracts semantic features of images, e.g., CLIP [10], we can formulate the semantic recovery problem in TokCode as, Θ
˜ E[cos(ϕclip (I), ϕclip (I))],
As analyzed in Sec. III-B, the token encoder must understand the semantic information carried by the token sequence so as to identify and protect the critical tokens. To avoid the prohibitive training cost of learning such a capability, we build the token encoder upon a pretrained text foundation model, for which we choose the pretrained T5 decoder in [7]. Specifically, let γt5 (·, ·; Θ̂) denote the pretrained T5 decoder, where Θ̂ is its pretrained parameters. It takes a source and a conditioning token sequences as input and produces an output token sequence. During the pretraining, it learns conditional text generation for span infilling, endowing it with the ability to model semantic relationships among tokens. To adapt this pretrained capability to token encoding without the cost of full fine-tuning, we employ the low-rank adaptation (LoRA) technique [11] with the adapted parameters, Θ = Θ̂ + BA,
B. Semantic Recovery Problem
max
A. Problem Conversion
(4)
(5)
where B ∈ RDout ×r and A ∈ Rr×Din are low-rank matrices, with r ≪ Dout , Din being the assigned rank. Since the number of trainable parameters is reduced from Dout Din to r · (Dout + Din ), the training complexity is substantially reduced. Moreover, by fixing Θ̂ and only training the lowrank matrices, the semantic prior of the foundation model is also preserved. Based on (5), the encoding from x to z is expressed as z = γt5 (x, x; Θ̂ + BA),
(6)
IV. T OKEN E NCODING O PTIMIZATION A LGORITHM
where we use original token sequence x as the source and the conditioning input so that the token encoding can fully exploit the semantic relationships among the tokens in x. Secondly, to avoid the prohibitive cost of evaluating (4) through G(·) and ϕclip (·), we replace the image-domain objective with a sentence-domain surrogate. The motivation is that, the reconstructed image is expected to remain semantically consistent with the source image, if received token sequence z̃ has similar sentence-domain semantic information of source sequence x. To measure the sentence-domain semantic similarity, we leverage Sentence-T5 [12], whose embedding space is explicitly trained for measuring sentence-domain semantic similarity. In particular, the similarity between two sentences, i.e., two token sequences in our case, can be measured by the cosine similarity between their Sentence-T5 embedding vectors, and thus the similarity loss can be calculated by Lsent = E 1 − cos href , hrx , (7)
We develop a computationally efficient optimization algorithm for solving the semantic recovery problem in (4). We propose sentence-semantic-guided foundation model adaptation algorithm (SFMA), which optimizes the token encoder for semantic recovery with a sentence-domain semantic similarity objective and adapts, rather than train from scratch, a text foundation model. SFMA comprises two components: problem conversion and foundation model adaptation.
where href = ϕsent (x) is the embedding of original token sequence x, with ϕsent (·) being the Sentence-T5 encoder and hrx = ϕsent (z̃) is that of the received sequence after being corrupted by the channel. Nevertheless, since the cosine function is invariant to embedding magnitude, it leaves a scale ambiguity in loss function (7) and leads to unstable training due to unconstrained embedding norms. We therefore introduce a norm
where function ϕclip (·) represents the model for semantic extraction of images, and the expectation is taken over the channel randomness, the potential target images, and their corresponding prompt text. However, solving (4) remains challenging for two main reasons. First, the token encoder, γ(·; Θ), operates directly on semantic tokens and therefore must capture the semantic meaning and intricate correlations underlying token sequences, which are hard to learn and generally require massive training on large-scale text datasets. Second, although the generative model G(·) and the semantic extractor ϕclip (·) are not optimization variables, evaluating and differentiating (4) still require computing through them. In particular, optimizing Θ requires computing the gradient propagation graph through these large models, which incurs prohibitive computational costs and makes end-to-end optimization impractical.
regularization loss to suppress this ambiguity and stabilize training, i.e., h 2 i Lnorm = E ∥hrx ∥2 − ∥href ∥2 , (8)
to Sentence-T5. Using the stop-gradient operator denoted by sg(·), we compute the embedding fed into Sentence-T5 by vnste = sg vnhard − vnsoft + vnsoft . (13)
which encourages the received token sequence embedding norm to match that of the original token sequence embedding. Consequently, we convert the optimization problem in (4) in TokCode into
This surrogate makes the inference identical to using the hard discrete embedding, vnhard , while in training, the gradient is directly passed to the continuous soft embedding, vnsoft . As a result, the gradient of the sentence-domain objective can propagate through the softmax probabilities and back to the parameters of the T5 decoder’s LoRA matrices. Training Procedure. During training, lossy channel H(·) is simulated by random packet erasure following (2), and the received sequence is compacted and appended with an end-of-sentence token before being fed into Sentence-T5. All parameters, except the LoRA matrices B and A, are frozen; the complete procedure is summarized in Algorithm 1.
min B,A
L = Lsent + λ Lnorm ,
(9)
where λ > 0 is a coefficient that balances the two loss terms. Through this conversion, the optimization in (4) becomes computationally tractable, with its computational resource and GPU memory requirements being significantly reduced. B. Foundation Model Adaptation A key difficulty in LoRA training under (9) is that the text foundation model outputs discrete tokens, which are then fed into the sentence embedding model, ϕsent (·). This discretization blocks gradient back-propagation from the objective to the LoRA parameters, making their training infeasible. To overcome this difficulty, we exploit the following two key observations. Firstly, for an output token sequence z with length N , the T5 decoder does not directly output discrete tokens; instead, it first produces a logit matrix over all the T possible tokens, i.e., L = [l1 , . . . , lN ]⊤ ∈ RN ×T ,
(10)
where each ln gives the scores of the T candidate tokens at position n. The discrete encoded token, zn , is obtained by selecting the token with the highest score, i.e., zn = arg maxj ln,j , which is non-differentiable and blocks gradient back-propagation. Secondly, although the Sentence-T5 uses a codebook different with that of the T5 decoder, they share the same tokenizer. Hence, token i ∈ T represented by the T5 decoder’s output logit corresponds to the i-th semantic embedding in the Sentence-T5’s codebook. Based on these two observations, we can circumvent the non-differentiable tokenization step during training. Instead of feeding encoded tokens into ϕsent (·), we use the softmax-normalized logits to form a weighted combination of Sentence-T5 token embeddings, thereby constructing a differentiable surrogate input. We denote the codebook of ′ the Sentence-T5 by Vsent ∈ RT ×D , where D′ ̸= D is its embedding dimension, and define the combination weight vector for the n-th output token by wn = softmax(ln ).
(11)
Then, the n-th hard and soft embedding in Sentence-T5’s codebook corresponding to zn can be expressed as vnsoft = wn Vsent , and vnhard = Vsent [zn ].
(12)
Based on (12), we leverage the straight-through estimation (STE) method [13] to obtain a differentiable surrogate input
Algorithm 1 Token Encoding Optimization by SFMA. Input: Image and prompt dataset D; Pretrained T5 decoder parameters Θ̂; Sentence-T5’s codebook Vsent ; LoRA rank value r; Loss coefficient λ; Channel erasure probability p. Output: Optimized LoRA matrices B ∗ , A∗ . 1: Initialize LoRA matrices B, A for the pretrained T5 decoder. 2: for each mini-batch {x} ⊂ D do 3: Compute logits l1 , . . . , lN and encoded token sequence z. 4: Obtain soft token weights wn = softmax(ln ) and embeddings vnsoft = wn Vsent . 5: Apply STE: vnste = sg(vnhard −vnsoft )+vnsoft , where vnhard = Vsent [zn ]. 6: Simulate channel: z̃ = H(z) and identify the erased token positions. 7: Compute href = ϕsent (x) and hrx = ϕsent (z̃) from the STE embeddings corresponding to z̃. 8: Calculate Lsent , Lnorm , and L = Lsent +λLnorm ; backpropagate through the STE path and update B, A.
V. P ERFORMANCE E VALUATION Below, we first describe the experimental setup and then present the semantic recovery evaluation results of TokCode. A. Experimental Setup We instantiate the system model in Sec. II with both the transmitter and the receiver equipped with the same PixArtSigma [14] text-to-image generative model and the same T5-XXL [7] text foundation model. T5-XXL comprises a tokenizer, an encoder, and a decoder, with codebook size T = 32,128 and embedding dimension D = 4,096. PixArtSigma has a diffusion transformer-based architecture and can take T5-XXL encoder’s output as its text condition. For the token-based generative image transmission task, we consider a set of source images generated by PixArtSigma from prompts drawn from DiffusionDB [15], where each image is transmitted through the token sequence of its prompt text. For the prompts, we randomly select 3,000 training samples and 100 test samples in the DiffusionDB dataset, which have token sequence length N ∈ [20, 100].
Sample
1
2
3
4
5
6
7
8
9
10
11
12
Lossless
Baseline
T5-based infilling LLM-based prediction
TokCode
Fig. 2: Rx-generated image comparison at p = 40%. Rows from top to bottom correspond to lossless prompt, baseline, T5based infilling, LLM-based prediction, and our proposed TokCode. Each sample corresponds to a test sample. For the transmitter-end token encoder, we use the decoder of T5-XXL and adapt it with LoRA of rank r = 128, injected into the query and value linear projections of every decoder block. This results in about 1.51 × 108 trainable parameters, which is approximately 2.4% of the 6.37 × 109 parameters in the T5-XXL decoder. Before sending to the lossy channel, each token sequence is interleaved across P = 5 packets. The channel follows the packet-level model in Sec. II. The default packet loss rate is p = 40%, corresponding to randomly dropping two out of the five packets. We also test p = 20% and p = 60% cases. B. Evaluation Results Since TokCode is, to the best of our knowledge, the first transmitter-end token encoding framework for semantic recovery, there is no directly comparable benchmark. Accordingly, we compare TokCode with a Baseline method, which directly transmits source token sequence x and, after the same interleaving and random packet loss, uses the surviving tokens as the received sequence. We also compare with two receiverend generative reconstruction methods following the principle idea in [5]: a T5-based infilling method and an LLM-based prediction method. The T5-based infilling method extends the idea in [5] by using the T5-XXL decoder to fill in the lost tokens, which is aligned with its pretraining objective of token infilling. The LLM-based prediction method uses a local large language model (LLM), specifically Llama3-8B-Instruct [16], to predict the original prompt text from its partially erased version. We choose Llama3-8B-Instruct because its parameter scale and release date are comparable to those of the T5-XXL decoder, and it is also a strong open-source model widely adopted for local deployment. Fig. 2 provides a qualitative comparison under packet loss rate p = 40%. In Baseline, most samples suffer from severe semantic distortion due to substantial token loss. Moreover,
samples 5, 7, and 11 exhibit nearly complete failure, indicating that the received token sequences are incomprehensible to the generative model. For T5-based infilling, although T5XXL is pretrained for token infilling, its recovery capability remains limited under substantial token loss. It provides little improvement over Baseline and may distort the original semantics through inaccurate completions, as seen in samples 2, 6, and 8. The LLM-based prediction leads to a reasonable prompt but usually results in semantically shifted outcomes. By contrast, TokCode enables the receiver to obtain images that preserve the important semantics of the target ones in most cases. Tab. 1 further explains this difference at the text level by showing the corresponding text fed to the generative model under different methods for the first sample in Fig. 2. The Lossless row gives the lossless-channel reference, and “Sim.” measures the Sentence-T5-based sentence similarity to this reference. Baseline loses key concepts, such as “portrait” and “robot,” causing a large visual mismatch. T5-based infilling produces disordered completions, showing that missing token recovery remains challenging even for an infilling-pretrained Tab. 1: Text-domain comparison under p = 40%. Case
Corresponding Text portrait of a cyberpunk male robot, rock music, highly detailed, smooth, elegant, award winning, by sachin teng and pascal Lossless blanche and alphonse mucha and nekro and josan gonzalez, in style of vector art. of cyberunk male, music, detailed smooth„ winning, achinten and Baseline pas blanche alphonea andnkro andangonzez in style vector. ,000. of Kam cyber Meadowunk male bewusst, digest music, Kam detailed Kabel smooth,werbung,werbung winning, Fläche beT5-based wusstachin Bearbeitungten Fläche and pas representation blanchinfilling esätze alphon Mailewerbunga andwerbungnpumpedkro and Arrowanpumpedgonz lassenezaji in style Sinne vectorattach. A futuristic portrait of a winning cyberpunk male, playing music LLM-based with detailed and smooth graphics, wearing a blanche and arogan prediction zez in style, rendered in vector. robotic robotic robotic male robot rock detail highly vector vector TokCode elegant award vector byschin Tangg Pascal and tech tech G and Nee and Pas Gonzalezzales and style vector art Portrait Portrait
Sim. 1.000
0.707
0.440
0.781
0.814
② T5-based Infill.
③ LLM-based Predict.
1.0
CLIP-based Similarity
SenT5-based Similarity
① Baseline
0.8 0.6 0.4 0.2
20%
40%
60%
Packet loss rate (p)
④ TokCode
⑤ Approx. Up. Bound
1.0 0.8 0.6 0.4 20%
40%
60%
Packet loss rate (p)
(a)
(b)
Fig. 3: Semantic recovery comparison in terms of (a) Sentence-T5 (SenT5)-based sentence similarity and (b) CLIPbased image similarity. foundation model. LLM-based prediction gives a plausible but shifted prompt that still misses “robot.” In contrast, TokCode learns semantic-aware repetition for token encoding, making the received text semantically closer to the lossless reference. Figs. 3a and 3b compare results of the frameworks in Fig. 2, using CLIP-based image similarity and Sentence-T5-based sentence similarity as defined in (4) and (7), respectively. We also include an Approximate Upper Bound case: assuming that the transmitter knows the packet-loss pattern in advance, it greedily selects the source tokens that best preserve the Sentence-T5-based similarity and arranges them at token positions that will survive the channel erasure. As shown in Fig. 3a, compared with Baseline, neither T5based infilling nor LLM-based prediction effectively improves the sentence-domain similarity between the received token sequence and the original one. This demonstrates that receiverend restoration has limitation in recovering critical semantic loss, which supports our motivation in Introduction. In contrast, TokCode achieves clear gains over Baseline under severe packet loss, i.e., p = 40% and p = 60%. In terms of average, TokCode closes 29.5% and 77.3% of the gap between Baseline and Approximate Upper Bound at p = 40% and p = 60%, respectively. When p = 20%, TokCode is slightly worse than the other methods, especially Baseline, possibly because its learned redundant protection of key semantics may introduce unnecessary loss of fine-grained details. As shown in Fig. 3b, the CLIP-based similarity does not always follow the sentence-domain trend: although Baseline has higher sentence similarity than LLM-based prediction at p = 40% and p = 60%, its image similarity is lower. This is because LLM-based prediction produces in-distribution prompts for image generation, whereas Baseline’s prompts under severe loss can yield noisy and meaningless images, consistent with our observation in Fig. 2. Notably, the SentenceT5-based token encoding objective is effective for TokCode, which reduces about 60.9% of the gap between LLM-based prediction and Approximate Upper Bound at p = 40% and approaches the approximate upper bound at p = 60%. VI. C ONCLUSION This article introduced a token encoding framework (TokCode) for robust semantic recovery over harsh wireless chan-
nels. TokCode encodes the source token sequence to improve semantic recovery without additional transmission overhead or Rx-side restoration efforts. For semantic recovery optimization, we proposed SFMA, which adapts a text foundation model with LoRA to maximizing a Sentence-T5-based sentence-domain similarity objective, avoiding costly end-toend training and enabling plug-and-play deployment of the token encoder. Evaluation results on token-based generative image transmission show that, under harsh channels where 60% of tokens are randomly lost, TokCode reduces the sentence-domain semantic loss gap between the no-encoding baseline and the approximate minimum by 77.3%, while approaching the approximate minimum in terms of imagedomain semantic loss. R EFERENCES [1] M. A. Imran, M. Zennaro, O. R. Popoola, L. Chiaraviglio, H. Zhang, P. Manzoni, J. van de Beek, R. Stewart, M. A. Cox, L. L. Mendes, and E. Pietrosemoli, “Exploring the boundaries of connected systems: communications for hard-to-reach areas and extreme conditions,” Proc. IEEE, vol. 112, no. 7, pp. 912–945, Jul. 2024. [2] H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE Trans. Signal Process., vol. 69, pp. 2663–2675, Apr. 2021. [3] E. Bourtsoulatze, D. B. Kurka, and D. Gündüz, “Deep joint sourcechannel coding for wireless image transmission,” IEEE Trans. Cogn. Commun. Netw., vol. 5, no. 3, pp. 567–579, Sep. 2019. [4] P. Jiang, C.-K. Wen, X. Li, S. Jin, and G. Y. Li, “Semantic satellite communications based on generative foundation model,” IEEE J. Sel. Areas Commun., vol. 43, no. 7, pp. 2431–2445, Jul. 2025. [5] L. Qiao, M. B. Mashhadi, Z. Gao, R. Tafazolli, M. Bennis, and D. Niyato, “Token communications: a large model-driven framework for cross-modal context-aware semantic communications,” IEEE Wireless Commun., vol. 32, no. 5, pp. 80–88, Oct. 2025. [6] Q. Hu, G. Zhang, Z. Qin, Y. Cai, G. Yu, and G. Y. Li, “Robust semantic communications with masked VQ-VAE enabled codebook,” IEEE Trans. Wireless Commun., vol. 22, no. 12, pp. 8707–8722, Dec. 2023. [7] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” J. Mach. Learn. Res., vol. 21, no. 140, pp. 1–67, 2020. [8] J. Hu and G. Y. Li, “Distillation-enabled knowledge alignment for generative semantic communications of AIGC images,” arXiv:2506.19893, Jun. 2025. [9] J. Hu and G. Ye Li, “Distillation-enabled knowledge alignment protocol for semantic communication in AI agent networks,” IEEE Commun. Lett., vol. 29, no. 11, pp. 2541–2545, Nov. 2025. [10] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervision,” in Proc. ICML, Online, Jul. 2021, pp. 8748–8763. [11] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: low-rank adaptation of large language models,” in Proc. ICLR, Online, Apr. 2022. [12] J. Ni, G. Hernández Ábrego, N. Constant, J. Ma, K. B. Hall, D. Cer, and Y. Yang, “Sentence-T5: scalable sentence encoders from pre-trained text-to-text models,” in Proc. ACL, Dublin, Ireland, May 2022. [13] P. Yin, J. Lyu, S. Zhang, S. J. Osher, Y. Qi, and J. Xin, “Understanding straight-through estimator in training activation quantized neural nets,” in Proc. ICLR, New Orleans, LA, May 2019. [14] J. Chen, C. Ge, E. Xie, Y. Wu, L. Yao, X. Ren, Z. Wang, P. Luo, H. Lu, and Z. Li, “PixArt-Σ: weak-to-strong training of diffusion transformer for 4K text-to-image generation,” in Proc. ECCV, Milan, Italy, Oct. 2024. [15] X. Wang, X. Tang, X. Li, N. Ahuja, and H. S. Huang, “DiffusionDB: a large-scale prompt gallery dataset for text-to-image generation,” in Proc. ACL, Toronto, Canada, Jul. 2023. [16] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian et al., “The Llama 3 herd of models,” arXiv:2407.21783, Jul. 2024.