Lightweight PCGAE-Net: Parallel CrossGate Attention and Bottleneck AutoEncoder for Efficient 5G Channel Prediction Uma Kishore Godavarti1, 2 , K. Giridhar2 , Vanani Prince Dharmendrabhai1 , Anchit Panday1 , and Madhan Raj Kanagarathinam1 1
arXiv:2606.25401v1 [cs.NI] 24 Jun 2026
2
Network Modem Team, Samsung R&D India Bangalore Department of Electrical Engineering, Indian Institute of Technology Madras, Chennai, India
Abstract—Accurate channel state information (CSI) prediction is essential for proactive beamforming and resource management in 5G massive MIMO systems, yet the deployment of high-accuracy transformer-based predictors on base-station hardware remains challenging because the most capable models carry upwards of 30 M parameters. This paper introduces Lightweight PCGAENet, which addresses the efficiency problem not by post-hoc compression but by correcting two architectural flaws in the current state of the art. The first is a sequential attention ordering bias: in CS3T-UNet, group-wise temporal attention (GTA) always operates on features that have already been transformed by cross-shaped spatial attention (CSA), distorting what temporal information GTA can capture. We remove this dependency by routing both attention modules to the same layer-normalized input and combining their independent outputs through a learned perchannel sigmoid CrossGate. The second flaw is an uncompressed bottleneck: applying full self-attention at the deepest encoder stage, where channel depth reaches 4C, is quadratically expensive and carries redundant features. A Bottleneck AutoEncoder (BAE) with 1 × 1 convolutions halves this depth and uses an auxiliary reconstruction loss to prevent information collapse. Wrapping these components inside a shallower encoder-decoder with frequency-domain dimensionality reduction (Nf = 32, C = 48) produces a model with just 8.54 M parameters—58% fewer than the CS3T-UNet baseline—that outperforms it by up to 3.26 dB at 5 km/h and 6.0 dB at 9 km/h in single-step prediction on QuaDriGa dataset. Index Terms—CSI prediction, parallel attention, bottleneck autoencoder, dimensionality reduction, 5G MIMO, QuaDriGa
I. I NTRODUCTION The ability to predict future channel conditions from past observations has become an important signal processing step in 5G and beyond 5G systems. In massive MIMO systems, the base station needs accurate CSI to compute beamforming weights, but obtaining that information requires pilot transmission, feedback, and processing time that consume the coherence window [1]. A predictor that can accurately forecast the channel one or several time steps ahead, using only the history of observed frames, lets the base station prepare precoding matrices in advance and tolerate feedback delays that would otherwise degrade throughput [2]. For several years, recurrent architectures were the natural choice for this task. LSTM and GRU networks can model sequential temporal evolution, and their predictions improve noticeably over simple extrapolation [3]. However, recurrent
models process time steps one at a time, which limits parallelism and makes it hard to capture long-range dependencies across a channel sequence of tens of frames. CNN-based approaches [4] avoid this by treating the channel matrix as a 2D spatial image, but trade temporal modeling for spatial feature extraction rather than handling both jointly. Transformers changed this picture. The self-attention mechanism can attend to arbitrary positions in both spatial and temporal dimensions simultaneously, and several recent channel prediction systems have used transformer-based encoderdecoder architectures to reach state-of-the-art NMSE [5], [6]. CS3T-UNet [7] is among the most capable, combining a UNet backbone with cross-shaped spatial attention (CSA) and group-wise temporal attention (GTA). At 20.34 M parameters, it achieves strong results, but two aspects of its design limit both performance and efficiency. The first is the order in which it applies attention. In CS3T-UNet, CSA runs first and GTA is applied to its output, meaning GTA always sees a spatially-transformed version of the channel features. When spatial and temporal dynamics are partially coupled—as they are in MIMO systems where physical angle-of-arrival varies over time—applying spatial processing before temporal processing biases what GTA can extract. The model partially compensates during training, but the structural constraint remains. The second issue is the bottleneck: at the deepest encoder layer, feature depth reaches 4C (Where C is Embedding dimension), and CS3T-UNet applies crossshaped attention at full resolution. Self-attention cost scales quadratically with channel depth, so this is the most expensive operation in the network, applied to a feature volume that carries unavoidable redundancy at such depth. These are architectural problems, not weight-level inefficiencies, so post-hoc techniques like pruning or quantization cannot resolve them. The clean solution is to redesign the attention mechanism. This paper presents Lightweight PCGAENet, which makes the following specific contributions: 1) Parallel CrossGate Attention Block: CSA and GTA are computed simultaneously on identical layer-normalized inputs, removing the ordering dependency. Their outputs are combined by a learned per-channel sigmoid gate that adapts the spatial-temporal mixture based on the
actual feature content, with zero-initialization ensuring a B. Efficient Transformer Design balanced starting point. The tension between transformer expressiveness and com2) Bottleneck AutoEncoder with Auxiliary Supervision: putational cost has received considerable attention in vision. At the deepest encoder stage, 1×1 convolutions compress Swin Transformer [10] addressed this by partitioning the image feature depth from 4C to 2C, cutting attention cost by into local windows and computing attention only within each roughly 75%. An auxiliary reconstruction loss prevents window, which reduces complexity from quadratic in total the compressed bottleneck from discarding channel token count to quadratic in window size. The cross-shaped structure needed by the decoder. attention in CS3T-UNet follows a similar logic: each token 3) Slim Encoder-Decoder and Delay Domain Dimension- attends only to positions in the same horizontal or vertical strip. ality Reduction: Reducing the number of merge/expand We retain this efficient spatial attention design; our contribution block pairs from three to two, and cropping the input addresses how spatial and temporal attentions interact rather to Nf = 32 most energetic ADP delay bins with base than how each handles long-range spatial dependencies. channel width C = 48, brings total model size to 8.54 M For general-purpose efficient network design, MobileNet [11] parameters—73% fewer than the full PCGAE-Net and and EfficientNet [12] showed that architectures designed for 58% fewer than CS3T-UNet. efficiency from the start outperform those compressed after 4) Systematic evaluation: Five model variants are evaluated training. Depthwise separable convolutions, width multipliers, at three UE speeds (5, 9, 12 km/h) and two prediction and compound scaling rules exemplify this design-for-efficiency horizons (L ∈ {1, 5}) on QuaDriGa [8] under 3GPP philosophy. We apply the same principle to transformer-based 38.901 UMa NLOS conditions, with ablation for each channel prediction, accepting a small accuracy cost in exchange design choice. for a substantially smaller model rather than training a large The paper is organized as follows. Section II reviews related accurate model and then compressing it [13]. work. Section III describes the system model. Section IV presents the proposed architecture. Section V details the C. CSI Compression and Representation Learning experimental setup. Section VI reports results and analysis. Autoencoder-based CSI compression [4], [14] has been Section VII concludes. studied extensively for FDD feedback, where the encoder at the UE compresses CSI into a bitstream and the decoder at the base II. R ELATED W ORK station reconstructs it. The reconstruction quality is the primary A. Channel Prediction with Deep Learning objective in that setting. Our Bottleneck AutoEncoder differs The deep learning work on channel prediction adapted in two important respects. It operates inside the prediction models developed for time-series forecasting. LSTM networks network rather than between the transmitter and receiver, and were applied to OFDM channel matrices by treating the its reconstruction loss is a regularizer rather than the primary sequence of channel observations as a multivariate time training objective. The goal is not to faithfully reconstruct the series [3]. This works well at low antenna counts but does not compressed features but to prevent the compressed representascale well to massive MIMO, where the spatial dimension has tion from losing channel structure needed for an accurate hundreds of entries, and the spatial structure is as informative prediction. This distinction changes the compression ratio as the temporal one. GRU-based approaches [9] offer a lighter appropriate and how the auxiliary loss should be weighted, as recurrent alternative and remain competitive at low mobility. we discuss in Section IV. CNNs entered through the CSI feedback compression literature [4], where autoencoders learned to compress and III. S YSTEM M ODEL reconstruct channel matrices. The encoder half naturally learns A. Channel Model spatial feature representations, and it was a short step to We consider a massive MIMO downlink in which a base reuse those representations for prediction. The limitation is station with Nt = 64 antennas serves a single-antenna UE that that CNNs have fixed receptive fields and cannot model moves at speed v. The uplink channel at discrete time step t dependencies between frames that are far apart in time without is H(t) ∈ CNf ×Nt , where Nf = 64 is the number of OFDM deep architectures that carry correspondingly large parameter subcarriers. Each entry [H(t)]k,m is the complex channel gain counts. Transformers solved this by replacing convolutions with self- between UE and base station at each antenna m and subcarrier attention, which has a global receptive field by construction [5]. k. The raw channel matrices are correlated in both frequency Several groups have since applied transformer architectures and spatial dimensions. To exploit this structure, we transform to channel prediction, generally following the pattern of a to the angle-delay-power (ADP) domain: spatial feature extractor followed by a temporal attention module [6]. CS3T-UNet [7] achieves the best published NMSE HADP (t) = Ff H(t) FH (1) t , on QuaDriGa dataset by combining CSA with GTA inside a UNet encoder-decoder. Our work retains this backbone structure, where Ff ∈ CNf ×Nf and Ft ∈ CNt ×Nt are DFT matrices but restructures the attention to run in parallel rather than in applied along frequency and spatial dimensions, respectively. sequence, and adds a compressed bottleneck. In the ADP domain, channel energy concentrated in a small
number of dominant angle-delay bins corresponding to physical propagation paths. For typical urban macro scenarios, most energy sits in 15–20% of the bins. Truncating to the Nf = 32 most energetic delay indices retains nearly all channel information while halving the effective input size—the basis for our delay domain dimensionality reduction strategy. B. Prediction Task Formulation The predictor at the base station observes T = 10 consecutive ADP-domain frames and outputs predictions for the following L steps. Stacking observations along the time axis gives an input tensor X ∈ RT ×2×Nf ×Nt , where the factor of 2 separates the real and imaginary parts. The prediction target is Y ∈ RL×2×Nf ×Nt . Prediction accuracy is reported as normalized mean-squared error: ! ∥Ŷ − Y∥2F NMSE = 10 log10 , (2) ∥Y∥2F where Ŷ denotes the model output, and lower values indicate better accuracy. For practical gNB deployment, the inference time per slot must fit within the channel coherence time, which places a hard upper bound on model complexity and motivates our parameter-reduction strategy. IV. P ROPOSED A RCHITECTURE The overall structure of Lightweight PCGAE-Net follows a U-Net encoder-decoder, illustrated in Fig. 1. The encoder progressively downsamples angle and delay dimensions using convolutional merge blocks, building a feature hierarchy. The decoder symmetrically upsamples with expand blocks and skip connections from corresponding encoder levels. Three modifications distinguish our design from CS3T-UNet: the Parallel CrossGate (PCG) Attention Block replaces sequential attention at each skip level, the Bottleneck AutoEncoder (BAE) replaces vanilla attention at the deepest layer, and overall depth is reduced from three merge/expand pairs to two. A. Parallel CrossGate Attention Block CS3T-UNet applies CSA and GTA sequentially at each encoder-decoder level: X′ = X + CSA(LN(X)),
X′′ = X′ + GTA(LN(X′ )). (3)
CSA computes cross-shaped attention across spatial positions and adds it to the residual stream. GTA then applies group-wise temporal attention to the already-modified features X′ rather than to the original X. In slow-fading conditions, this ordering has little effect; as UE speed increases and temporal channel variation dominates, the temporal signatures in the original features that GTA needs to capture become harder to recover from the spatially-processed X′ . The PCG Attention Block fixes this by routing both modules to the same input: S = CSA(LN(X)),
T = GTA(LN(X)).
(4)
CSA and GTA now operate independently on the original channel representation, producing feature maps S and T
respectively. These are combined through a CrossGate—a learned per-channel sigmoid weighting: g = σ(Wg [S; T] + bg ) ,
F = g ⊙ S + (1 − g) ⊙ T, (5)
where [S; T] ∈ R2C is the channel-dimension concatenation, Wg ∈ RC×2C , σ(·) is the sigmoid function, and ⊙S denotes element-wise multiplication. The gate g ∈ (0, 1)C produces a separate weight for each channel dimension, letting different feature channels favor spatial or temporal information as the channel conditions require. We initialize Wg = 0 and bg = 0, giving g = 0.5 at training start—an equal mixture of the two streams. Starting from this balanced point avoids early-training instability that can arise when one stream dominates before the optimizer has found a good feature space. In practice, the gate values learned after training differ substantially from 0.5 in a channel-specific way, confirming that the gate does useful work. The full block output includes a feed-forward network (FFN) and a residual connection: Y = X + F + FFN(LN(X + F)).
(6)
The CrossGate itself adds only 2C 2+C parameters per block— negligible relative to the attention and FFN modules. The architectural benefit of independent feature extraction comes at essentially no parameter cost. B. Bottleneck AutoEncoder In the deepest encoder stage, feature depth has grown to 8C through the successive merge operations. CS3T-UNet applies full cross-shaped attention at this resolution, which is the single most expensive operation in the network. Many of these 8C feature channels at the bottleneck are redundant: they capture global statistics that summarize information already encoded in the encoder’s shallower representations. The Bottleneck AutoEncoder replaces this with a compression-decompression block: Z = GELU(GN(Conv1×1 (X))) ,
(7)
X̂ = GN(Conv1×1 (Z)) ,
(8)
YBAE = X̂ + X.
The encoder convolution maps from 8C to 4C channels; the decoder restores the original depth. Group Normalization (GN) is applied after each convolution, GELU provides the nonlinearity, and a residual connection ensures gradient flow. The compressed features Z ∈ R4C are passed to a smaller attention module operating at half the original bottleneck depth, cutting the attention cost at this stage by approximately 75%. Training minimizes a combined loss: L = Lpred + α Lrec ,
(9)
where Lpred is the NMSE prediction loss and Lrec = ∥X̂−X∥2F is the bottleneck reconstruction loss. We use α = 0.1. The reconstruction loss acts as a self-supervised regularizer: it penalizes compression that discards features the decoder will need, even when those features do not directly reduce the current prediction loss on the training set. This is particularly
(a) PCGAE-Net Architecture Input X
(b) PCG Block Detail Input X
Output Y
LN
PCG Block × N1
Linear Projection Skip connection 1/2
Merge Block
PCG Block × M3
Cross Shaped Spatial Attention (CSA)
Group wise Temporal Attention (GTA)
S
T
Residual
Patch Embedding
Expand Block CrossGate Fusion
PCG Block × N2
Skip connection 1/4
F
PCG Block × M2 +
LN
Expand Block
Residual
Merge Block
Feed-Forward Network (FFN)
Bottleneck AutoEncoder (BAE)
+
PCG Block × N3 PCGAE-Net Encoder
Skip connection 1/8
PCG Block × M1
Output Y
PCGAE-Net Decoder
Fig. 1. Proposed architecture. (a) Lightweight PCGAE-Net: a two-level U-Net encoder-decoder with PCG Attention Blocks at each skip level and a BAE at the deepest stage. Skip connections carry encoder features to the symmetric decoder levels. (b) PCG Block: CSA and GTA receive the same layer-normalized input independently. Their outputs S and T are fused by a per-channel sigmoid CrossGate, added to the residual stream, and processed by an FFN.
valuable when combining the BAE with aggressive channeldimension reduction (from C = 64 to C = 48), where the compressed representation must work harder to preserve the spatial-temporal structure needed for accurate multi-step prediction. C. Slim Encoder-Decoder and Dimensionality Reduction
TABLE I M ODEL CONFIGURATIONS . D EPTH = N O . OF MERGE / EXPAND BLOCK PAIRS .
Model CS3T-UNet (baseline) PCGAE (w/o AE) PCGAE (Full) PCGAE (DR, C = 48) PCGAE (Slim) Proposed (Slim+DR)
Depth
C
Nf
BAE
3 3 3 3 2 2
64 64 64 48 64 48
64 64 64 32 64 32
No No Yes Yes Yes Yes
Beyond the attention-level changes, we make two structural modifications to reduce parameter count. First, the number of merge/expand block pairs is reduced from three to two. The third pair in CS3T-UNet contributes relatively little to prediction accuracy while adding roughly 16 M parameters,and the remaining accuracy gap is largely recovered through improved attention quality from the CrossGate. Second, we and Nf = 64 OFDM subcarriers. After transformation to apply delay-domain cropping to the Nf = 32 most energetic the ADP domain, Nf and Nt correspond to delay bins and ADP bins, reducing input size by half along the delay domain. angle bins respectively, before any dimensionality reduction. The discarded 32 bins carry mostly noise at the carrier and Three UE speeds are evaluated: 5, 9, and 12 km/h, repreUE speeds we consider. Combined with reducing base channel senting pedestrian, slow vehicular, and moderate vehicular width from C = 64 to C = 48, these changes bring the total mobility.Each dataset contains 9,000 training sequences and parameter count from 31.74 M (full PCGAE-Net) to 8.54 M. 1,000 test sequences. Each sample is a sequence of 20 ADPTable I summarizes all model configurations evaluated in domain channel snapshots, over which a sliding window of size T + L is applied, yielding more independent windows for this work. L = 1 than for L = 5. The first T = 10 frames of each window V. E XPERIMENTAL S ETUP serve as input history and the remaining L frames as prediction A. Dataset and Simulation Conditions targets. The input tensor shape before delay domain cropping All experiments use channels generated by the QuaDriGa is (20, 2, 64, 64). simulator [8] under the 3GPP 38.901 UMa NLOS propagation The three speeds probe a meaningful range of Doppler model at 5 GHz carrier frequency, with Nt = 64 BS antennas conditions, while keeping experiments tractable. At 5 km/h the
TABLE II NMSE( D B). S PEED COLUMNS IN KM / H . L OWER IS BETTER . B OLD : BEST.
5
9
12
GRU CS3T-UNet PCGAE (w/o AE) PCGAE Full PCGAE DR (C=48) PCGAE Slim Proposed (Slim+DR)
4.99 M −3.6 −3.2 −3.1 −0.3 +0.0 +0.0 20.34 M −32.3 −25.6 −24.3 −22.7 −9.3 −9.6 31.47 M −36.3 −32.2 −30.2 −24.1 −15.2 −11.2 31.74 M −36.7 −32.7 −30.4 −24.3 −15.2 −11.5 19.69 M −35.2 −32.1 −30.3 −22.4 −15.1 −11.1 15.16 M −34.2 −32.0 −30.6 −23.4 −15.0 −11.3 8.54 M −35.5 −31.6 −30.2 −23.2 −14.6 −11.2
channel evolves slowly and spatial modeling dominates; at 12 km/h temporal variation becomes the dominant factor and 5-step prediction is substantially harder for all methods. The speed of 9 km/h sits at an intermediate point where spatial and temporal dynamics matter, and it turns out to be where the parallel CrossGate provides its largest benefit. B. Training Protocol
NMSE (dB)
12
20 30
5 km/h 9 km/h 12 km/h
5 10 15 20 25
40
et
-UN S3T
C
AE AE AE AE sed PCGo AE) PCG(Full) PCCG=48) PC(GSlim)Propo+DR) m (w/ (Sli (DR
et
N T-U
CS3
AE AE AE AE sed PCGo AE) PCG(Full) PCCG=48) PC(GSlim)Propo+DR) m (w/ (Sli (DR
Fig. 2. NMSE (dB) for all models at 5, 9, and 12 km/h. Bars are grouped by model; three bars per group correspond to three UE speeds. Lower is better. The proposed model consistently outperforms CS3T-UNet across all conditions despite using 58% fewer parameters. (a) Single-step (L = 1) PCGAE (w/o AE) PCGAE (Full) PCGAE (DR C=48)
10 8
6.6
6 4 2 0
4.0
4.5 3.3
2.9 2.0
7.1
6.5 6.4
PCGAE (Slim) Proposed (Slim+DR)
6.0 5.9 6.1 6.0 6.3 5.9
(b) Multi-step (L = 5)
NMSE Improvement over CS3T-UNet (dB)
9
NMSE (dB)
5
L=5
Params
(b) Multi-step Prediction (L = 5) 0
5 km/h 9 km/h 12 km/h
10
NMSE Improvement over CS3T-UNet (dB)
L=1 Model
(a) Single-step Prediction (L = 1)
0
PCGAE (w/o AE) PCGAE (Full) PCGAE (DR C=48)
10 8
5.9 5.9 5.8 5.6
6
PCGAE (Slim) Proposed (Slim+DR)
5.3
4 2 0
1.4 1.6
1.6 1.9 1.5 1.7 1.6 0.6 0.5
5 km/h 9 km/h 12 km/h 5 km/h 9 km/h 12 km/h All models are trained with AdamW at an initial learning −4 −6 rate of 2×10 with decaying cosine annealing to 10 over 3. NMSE improvement (dB) over CS3T-UNet baseline, grouped by UE 400 epochs with batch size 32. BAE models are trained using Fig. speed. At L = 1, the proposed model achieves 3.3–6.0 dB gain depending on the combined loss in (9) with α = 0.1; others use Lpred alone. speed; at L = 5, gains range from 0.5 to 5.9 dB. All experiments run on an NVIDIA RTX 6000 Ada GPU in bfloat16 precision without data augmentation. The weight initialization for CrossGate parameters follows B. Effect of UE Speed In single-step prediction (L = 1), the gain from parallel the convention in Section IV-A: Wg = 0 and bg = 0. All other CrossGate grows with speed. The improvement from PCGAE parameters use the default PyTorch scheme. All models share the same training protocol; parameter (w/o AE) over CS3T-UNet increases from 4.0 dB at 5 km/h to 6.6 dB at 9 km/h. The simplest explanation is that as temporal counts and configuration details are given in Table I. variation becomes the dominant channel characteristic, freeing VI. R ESULTS AND A NALYSIS GTA from operating on spatially-preprocessed features matters more. At 5 km/h the channel changes slowly enough that the A. Main Comparison bias in CS3T-UNet’s sequential design has limited impact; at Table II reports NMSE for all models at all speed-horizon 9 km/h it costs substantially more. The proposed model tracks combinations. Fig. 2 visualizes the same data as a grouped this trend: 3.3 dB gain at 5 km/h and 6.0 dB at 9 km/h. bar chart, Fig. 3 shows per-speed NMSE improvement over In multi-step prediction (L = 5), gains at 9 km/h are 5.9 dB CS3T-UNet, and Fig. 4 summarizes the accuracy-efficiency for PCGAE (Full) and 5.3 dB for the proposed model over the trade-off. baseline. The baseline itself drops sharply from −22.73 dB at The GRU baseline confirms that the prediction task is 5 km/h to −9.33 dB at 9 km/h as the 5-step prediction becomes genuinely non-trivial: it reaches only −3.57 dB at 5 km/h harder under the Doppler effect; the PCGAE variants maintain (L = 1), nearly 29 dB below the transformer-based methods. prediction quality much better in this regime, which is where This gap is not a parameter-count issue—GRU has 4.99 M the practical benefit of parallel architecture is most evident. parameters, not far below the proposed model at 8.54 M—but reflects the fundamental limitation of recurrent networks in C. Ablation Study jointly capturing spatial-temporal structure of MIMO channels. Table III isolates each design choice by adding one modificaAmong transformer-based models, the proposed method is tion at a time, evaluated at 5 km/h (L = 1) to avoid confounding an interesting position. It outperforms the 20.34 M CS3T-UNet with speed-dependent effects. baseline at every speed and horizon despite using 58% fewer Parallel CrossGate alone provides the largest single imparameters, and it comes within 1.2 dB of the 31.74 M PCGAE provement (4.02 dB), confirming that the sequential ordering (Full) while being less than a quarter its size. No competing dependency is the primary structural limitation. The BAE then configuration achieves this combination. contributes a further 0.44 dB at a cost of only 0.27 M additional Furthermore, the inference time is almost the same across parameters. This small gain has an outsized effect on what all model variants, confirming that the architectural changes do compression is viable: as will be apparent from the next row, not introduce additional latency overhead despite the difference reducing depth without the reconstruction regularizer causes in parameter counts. a larger accuracy drop than reported because the bottleneck
(a) NMSE vs. UE Speed
NMSE (dB) at 5 km/h, L = 1
NMSE (dB)
15
(b) Accuracy Efficiency Trade-off Solid: L = 1 Dash: L = 5
CS3T-UNet PCGAE (Full) PCGAE (Slim) Proposed (Slim+DR)
10
20 25 30 35 5
9
UE Speed (km/h)
12
CS3T-UNet PCGAE (w/o AE) PCGAE (Full) PCGAE (DR C=48) PCGAE (Slim) Proposed (Slim+DR)
33 34 35
Proposed 8.54M 35.5 dB
36 10
15
20
25
Parameters (M)
30
Fig. 4. (a) NMSE vs. UE speed for key model variants. Solid: L = 1; dashed: L = 5. (b) Parameter count vs. NMSE at 5 km/h, L = 1. The proposed model (starred) achieves the best accuracy-per-parameter. TABLE III A BLATION STUDY AT 5 KM / H , L = 1. E ACH ROW ADDS ONE DESIGN CHANGE TO THE PREVIOUS . Configuration
Params
NMSE (dB)
∆ vs. CS3T-UNet
CS3T-UNet (baseline) Parallel CrossGate (w/o BAE) + Bottleneck AutoEncoder + Slim encoder-decoder (2 levels) + DR (C = 48, Nf = 32)
20.34 M 31.47 M 31.74 M 15.16 M 8.54 M
−32.27 −36.29 −36.73 −34.24 −35.53
— +4.02 dB +4.46 dB +1.97 dB +3.26 dB
discards channel structure that the decoder needs to recover at the skip connections. Reducing depth from three levels to two costs 2.49 dB relative to the full model—the largest single accuracy loss. At 15.16 M parameters, however, even this reduced model is already 1.97 dB better than the baseline. Adding dimensionality reduction then recovers 1.29 dB at a saving of 6.62 M parameters, producing the final proposed configuration. D. Discussion Two broader observations are worth noting. The first is that architectural correctness matters more than scale. PCGAE (Full) has 56% more parameters than CS3T-UNet, yet the gain from fixing the sequential attention ordering (4.02 dB from the parallel CrossGate alone) is larger than what any parameter increase alone could plausibly explain. The structural bias is a genuine limitation, not a capacity issue, and correcting it first— before pursuing scale reduction—is what allows the proposed model to stay well above the baseline despite being less than half its size. The second observation concerns the role of the auxiliary reconstruction loss in the BAE. As shown in Table III, applying dimensionality reduction without the BAE leads to a noticeable accuracy drop, since prediction loss alone provides no explicit signal to preserve channel structure at the bottleneck. The reconstruction loss addresses this by directly regularizing the compressed representation, giving the model a second training objective that specifically rewards the retention of useful information for the decoder. This becomes especially important under aggressive compression, where the channel width is reduced from C = 64 to C = 48, halving the frequency dimension. The benefit is clearly reflected in Table III: the final row recovers 1.29 dB at a saving of 6.62 M parameters, which
is largely attributable to the reconstruction loss keeping the bottleneck representation informative under such compression. VII. C ONCLUSION This paper presented Lightweight PCGAE-Net, a 5G channel predictor built around two targeted architectural corrections to CS3T-UNet. Running cross-shaped spatial and group-wise temporal attention in parallel on the same input, with a learned per-channel CrossGate to combine their outputs, removes the sequential ordering bias that constrains GTA’s ability to capture temporal channel dynamics—an effect that grows more significant as UE speed increases. A Bottleneck AutoEncoder at the deepest encoder stage compresses feature depth by half, cutting the most expensive self-attention operation, while an auxiliary reconstruction loss prevents the compressed representation from losing prediction-relevant structure. Combining these with a shallower encoder-decoder and frequency-domain input cropping yields a model with 8.54 M parameters that outperforms the 20.34 M CS3T-UNet baseline by 3.3–6.0 dB across all evaluated conditions on QuaDriGa. Future directions include extending evaluation to higher mobility regimes (30-120 km/h) would validate the CrossGate’s robustness under severe Doppler conditions representative of vehicular and railway communication scenarios. R EFERENCES [1] K. T. Truong and R. W. Heath, “Effects of channel aging in massive MIMO systems,” J. Commun. Netw., vol. 15, no. 4, pp. 338–351, Aug. 2013. [2] D. J. Love et al., “An overview of limited feedback in wireless communication systems,” IEEE J. Sel. Areas Commun., vol. 26, no. 8, pp. 1341–1365, Oct. 2008. [3] H. Ye, G. Y. Li, and B.-H. F. Juang, “Power of deep learning for channel estimation and signal detection in OFDM systems,” IEEE Wireless Commun. Lett., vol. 7, no. 1, pp. 114–117, Feb. 2018. [4] C.-K. Wen, W.-T. Shih, and S. Jin, “Deep learning for massive MIMO CSI feedback,” IEEE Wireless Commun. Lett., vol. 7, no. 5, pp. 748–751, Oct. 2018. [5] A. Vaswani et al., “Attention is all you need,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), 2017, pp. 5998–6008. [6] Z. Yuan et al., “Transformer-based deep learning for channel estimation in massive MIMO systems,” IEEE Wireless Commun. Lett., vol. 10, no. 12, pp. 2757–2761, 2021. [7] X. Liu et al., “Cross-shaped separated spatial-temporal UNet transformer for accurate channel prediction,” in Proc. IEEE Int. Conf. Comput. Commun. (INFOCOM), 2024. [8] S. Jaeckel et al., “QuaDRiGa: A 3-D multi-cell channel model with time evolution for enabling virtual field trips,” IEEE Trans. Antennas Propag., vol. 62, no. 6, pp. 3242–3256, Jun. 2014. [9] Y. Lu et al., “Channel prediction for limited feedback beamforming in time-varying channels,” IEEE Commun. Lett., vol. 25, no. 2, pp. 432–435, Feb. 2021. [10] Z. Liu et al., “Swin Transformer V2: Scaling up capacity and resolution,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2022, pp. 12 009–12 019. [11] A. G. Howard et al., “MobileNets: Efficient convolutional neural networks for mobile vision applications,” in arXiv preprint arXiv:1704.04861, 2017. [12] M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proc. Int. Conf. Mach. Learn. (ICML), 2019, pp. 6105–6114. [13] K. Ravindran et al., “Lightweight deep learning for CSI prediction in massive MIMO systems,” IEEE Access, vol. 11, pp. 48 220–48 232, 2023. [14] T. Wang, C.-K. Wen, S. Jin, and G. Y. Li, “Deep learning-based CSI feedback approach for time-varying massive MIMO channels,” IEEE Wireless Commun. Lett., vol. 8, no. 2, pp. 416–419, Apr. 2019.