arXiv CSopen access

LOLLA: Deep Reinforcement Learning for Closed-Loop Link Adaptation Towards a GPU-Accelerated AI-RAN

1

LOLLA: Deep Reinforcement Learning for Closed-Loop Link Adaptation Towards a GPU-Accelerated AI-RAN

arXiv:2606.23110v1 [eess.SP] 22 Jun 2026

Rui Wang, Member, IEEE, Linchao Zhang, Member, IEEE, Qiang Liu, Senior Member, IEEE, and Kun Yang, Fellow, IEEE

Abstract—Outer-loop link adaptation (OLLA) is widely deployed in 5G NR to track channel variations, yet its reliance on first-order, single-bit feedback degrades performance significantly under high-mobility and fast-varying channels. This paper presents LOLLA (Learned Outer-Loop Link Adaptation), a deep reinforcement learning framework that replaces the conventional OLLA staircase with a learned, continuous SINR offset conditioned on rich PHY/MAC telemetry inaccessible to OLLA. The offset modulates the SINR-to-MCS lookup table, preserving 3GPP-compliant MCS selection and provably subsuming the conventional OLLA update rule. A Proximal Policy Optimization (PPO) policy trained under a Lagrangian block error rate (BLER) constraint automatically enforces tunable reliability targets from 1% to 15% without manual penalty calibration. The framework is realized as the first closed-loop AI-native control dApp on a GPU-accelerated 5G NR stack, achieving end-to-end control latencies under 500 µs. Evaluations under 3GPP TDL channel models demonstrate 15% to 92% throughput gains over OLLA across Doppler frequencies up to 400 Hz, while attaining a Pareto frontier that strictly dominates OLLA across all evaluated reliability targets. The learned policy generalizes to unseen channel models and scales to eight concurrent UEs under shared-resource scheduling. In the uplink formulation, the gNB directly observes decoding outcomes, enabling simulationto-deployment parity. Index Terms—5G NR, AI-RAN, closed-loop control, constrained reinforcement learning, deep reinforcement learning (DRL), GPU-accelerated RAN, link adaptation, O-RAN dApp, outer-loop link adaptation (OLLA).

I. I NTRODUCTION

T

He transition towards 5G-Advanced and 6G Radio Access Networks (RANs) is driven by a paradigm shift from fixed-function hardware to software-defined, virtualized architectures running on programmable computing platforms [1]. This evolution, exemplified by the O-RAN Alliance’s disaggregated architecture [2], enables the deployment of intelligent controllers, including xApps for near-real-time and dApps for real-time control [3], that can optimize network performance Rui Wang, Linchao Zhang, and Qiang Liu are with the School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China, and also with the Yangtze Delta Region Institute (Quzhou), University of Electronic Science and Technology of China, Quzhou 324003, China (e-mail: [email protected]; [email protected]; [email protected]). Kun Yang is with the State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210008, China, with the Institute of Intelligent Networks and Communications (NINE) and School of Intelligent Software and Engineering, Nanjing University (Suzhou Campus), Suzhou 215163, China, and also with the School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China (e-mail: [email protected]).

at unprecedented granularity. GPU-accelerated baseband processing [4] has emerged as a key enabler, providing the computational power for massive MIMO signal processing and the ability to expose fine-grained physical layer (PHY) telemetry at slot level [5]. In particular, dApps, lightweight applications co-located with the baseband processing unit, can access PHY/MAC telemetry and execute control actions at sub-millisecond timescales [6], making them uniquely suited for latency-critical Radio Resource Management (RRM) functions such as link adaptation. Link adaptation, the process of selecting the optimal Modulation and Coding Scheme (MCS) for each transmission, is a critical RRM function that directly impacts spectral efficiency and link reliability [7]. In 5G NR, link adaptation employs a two-loop structure: an inner loop that maps a Channel Quality Indicator (CQI) to an MCS index via a static SINR-to-MCS Lookup Table (LUT), and an Outer Loop Link Adaptation (OLLA) mechanism that adjusts an SINR offset based on HARQ ACK/NACK feedback to maintain a target Block Error Rate (BLER) [8]. While computationally efficient, OLLA is inherently reactive: each update consumes only one bit of feedback per transmission, rendering it slow to adapt under fast-varying channels in high-mobility scenarios [9], [10] or bursty interference environments [11]. Deep Reinforcement Learning (DRL) can address these limitations by learning nonlinear control policies from high-dimensional PHY state [12], and several recent works report substantial throughput gains over OLLA in simulation [13]–[15]; however, bridging the gap to real-time deployment remains challenging due to submillisecond control budgets [6], the risk of discarding the structured SINR-to-MCS relationship encoded in the LUT, and the overhead of transporting high-dimensional PHY telemetry to an external controller. In this paper, we present LOLLA (Learned Outer-Loop Link Adaptation), a real-time, closed-loop DRL controller for uplink link adaptation on GPU-accelerated RANs. Our agent learns a continuous SINR offset, a residual correction [16], that modulates the legacy SINR-to-MCS LUT, ensuring safe, standards-compliant default behavior when the offset is near zero while enabling the policy to exploit rich PHY/MAC telemetry inaccessible to OLLA. A Proximal Policy Optimization (PPO)-Lagrangian BLER constraint [17]–[19] replaces manual penalty tuning, enabling automatic adaptation to arbitrary reliability targets. We realize this within the E3 dApp architecture [5] via a dual-agent design with zero-copy shared-memory data transfer, achieving sub-500 µs latency. The uplink formulation ensures the gNB both controls the MCS via uplink grants [20] and directly observes decoding

2

TABLE I Q UALITATIVE COMPARISON OF LINK ADAPTATION METHODS . C ONT. = CONTINUOUS , D ISC . = DISCRETE , CL = CLOSED - LOOP, S IM = SIMULATION , OTA = OVER - THE - AIR , LUT = SINR- TO -MCS LOOKUP TABLE , Q- LEARN . = Q- LEARNING , LTS = LATENT T HOMPSON SAMPLING , SL = SUPERVISED LEARNING , B R . = BRANCHING , OPT. = OPTIMIZATION , GAT = GRAPH ATTENTION NETWORK .

System capabilities Action

BLER control

Multi-UE‡

CL§

GPU†

Validation

Heuristic RL (Q-learn.) Bandit (LTS) SL (online) DRL (DQN) Heuristic DRL (Br. DQN) DRL (GAT) Online opt. DRL (TD3) DRL (DQN)

Cont. offset → LUT Disc. MCS Disc. MCS Disc. MCS Disc. MCS Cont. offset → LUT Disc. MCS+layer Disc. MCS Cont. SINR → LUT Disc. MCS Disc. MCS

Adaptive step Reward penalty Bayesian Implicit Implicit Non-linear step Implicit Implicit Integral feedback Reward penalty Reward penalty

✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗

✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ X ✗ X

✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗ X ✗ ✗

Sim Sim Sim Sim Sim Sim Sim Sim Sim + OTA Sim Sim + OTA

DRL (PPO)

Cont. offset → LUT

Lagrangian

X

X

X

Sim

Method

Year

Type

eOLLA [8] Mota et al. [15] Saxena et al. [13] Bobrov et al. [25] Ye et al. [14] NOLLA [21] DRAGON [26] GenRL-RAN [28] SALAD [22] Gao et al. [27] DC-DQN-LA [29]

2016 2019 2022 2022 2023 2023 2024 2025 2025 2025 2026

LOLLA

2026

† GPU-accelerated PHY with identical CUDA baseband kernels as production hardware [5]. ‡ Evaluated under shared-resource multi-UE scheduling (K > 1 UEs sharing PRB resources). § Closed-loop actuation: algorithm’s decisions are executed by the PHY/MAC stack (not open-loop inference only).

outcomes, enabling simulation-to-deployment interface parity. A. Related Work The limitations of conventional OLLA have motivated enhanced heuristics such as eOLLA [8] with adaptive step sizes and NOLLA [21] with non-linear offset adjustments, as well as SALAD [22], which replaces the fixed OLLA step with online gradient descent and demonstrates overthe-air gains on a GPU-accelerated testbed. On the learning side, approaches range from multi-armed [23] and contextual bandits [24], latent Thompson Sampling [13], and supervised online learning [25], to Q-learning [15], DQN [14], branching DQN for joint MCS and MIMO-layer adaptation [26], TD3 with Bayesian optimization for joint device scheduling and link adaptation in URLLC [27], graph attention networks for cross-scenario generalization [28], and decoupled DQN addressing inference latency [29]. Table I provides a structured comparison. On the systems side, the E3 framework [5] establishes dApps for GPU-native RAN programmability with submillisecond PHY/MAC access. Recent deployments, including X5G [30], interference detection [31], spectrum classification [32], and sim-to-field transfer [33], all target open-loop inference without closed-loop actuation. Although SALAD [22] achieves closed-loop SINR estimation, it relies on modelbased gradient descent rather than learned policies. Across these works, three gaps persist: most use discrete MCS actions that bypass the SINR-to-MCS LUT, rely on fixed penalties or heuristic step sizes for BLER control rather than formal constrained optimization [18], and lack closed-loop integration with a production-grade GPU-accelerated RAN stack. B. Contributions and Paper Organization The main contributions of this paper are as follows:

1) A residual DRL formulation [16] in which the agent learns a bounded, continuous SINR offset that modulates a link-level calibrated SINR-to-MCS mapping [34], retaining this mapping as a structural prior and reducing the policy search to a one-dimensional correction over a 13dimensional PHY/MAC observation space that captures channel dynamics inaccessible to conventional OLLA. Because the offset action space strictly contains the linear OLLA update rule as a special case, the learned policy class is at least as expressive as OLLA, providing a formal performance dominance guarantee. 2) A Constrained MDP formulation [18] with PPOLagrangian [19] online dual-variable updates that replaces manual penalty tuning. The flat NACK penalty produces a non-uniform per-SNR BLER allocation that concentrates the error budget where throughput gains are largest, provably throughput-optimal under a convex, continuous MCS relaxation [35, §5.5.3]. A single training framework supports arbitrary BLER targets from eMBB (≈10%) to URLLC (< 1%) by specifying εtarget without altering the policy or training procedure. 3) The first closed-loop learning-based control dApp on the NVIDIA ARC-OTA platform. Whereas the E3 framework of Villa et al. [5] demonstrates open-loop inference dApps (e.g., sensing and PHY-layer telemetry processing), LOLLA realizes a full closed control loop in which a learned policy actuates the per-slot MCS and observes the resulting decoding outcomes. We bridge DU-Low and DU-High through a dual E3-Agent architecture connected via shared-memory data transfer through the Aerial Data Lake, and drive inference through four progressively optimized backends (PyTorch, Triton gRPC, Triton inprocess, and Triton C-API), achieving end-to-end control latencies under 500 µs at 30 kHz numerology. 4) To our knowledge, the first multi-UE closed-loop link adaptation system on a real-time GPU-accelerated RAN.

3

Prior dApp deployments [5], [31], [32] and RL-based link adaptation methods [13]–[15], [26] are limited to singleUE operation. Scaling to K concurrent UEs requires solving three tightly coupled challenges: (C1) per-UE state isolation via RNTI-indexed observation and reward tracking, (C2) MAC-RL interface isolation through a two-stage PRB-only scheduler that decouples RL MCS decisions from the production OLLA loop, and (C3) exact per-UE Transport Block size propagation from the MAC scheduler through the E3 pipeline, eliminating the approximation error inherent in uniform PRB-splitting. 5) Extensive evaluation under 3GPP TDL channel models across Doppler frequencies up to 400 Hz, multiple BLER targets (εtarget ∈ {1%, 5%, 9.1%, 12%}), multi-UE configurations (K ∈ {1, 2, 4, 8}), and four inference backends, demonstrating 15% to 92% throughput gains over OLLA while satisfying the prescribed BLER constraints, with the gain growing under faster channel dynamics and peaking at fd = 200 Hz. The remainder of this paper is organized as follows. Section II presents the system model, formulates the link adaptation problem as a constrained MDP, and reviews conventional OLLA. Section III details the proposed LOLLA framework, including the residual action design, PPO policy architecture, and Lagrangian BLER constraint. Section IV describes the real-time system architecture and inference pipeline. Section V presents simulation results and ablation studies, and Section VI concludes the paper. Notation: Boldface lowercase and uppercase letters denote vectors (o) and matrices (H). (·)⊤ is the transpose, ˆ· denotes an estimate, I(·) the indicator function. N (µ, σ 2 ) and CN (µ, C) denote real and circularly-symmetric complex Gaussian distributions. E[·] is expectation, k·kTV total variation distance, and [x]ba , max(a, min(b, x)) the projection onto [a, b]. M the MCS index set, π policies, β the discount factor, and λ the Lagrangian dual variable. II. S YSTEM M ODEL AND P ROBLEM F ORMULATION A. 5G NR Uplink System Model We consider a 5G NR system in which a gNB equipped with Nant receive antennas serves K User Equipments (UEs), each with NUE transmit antennas and Nl ≤ min(Nant , NUE ) spatial layers. The system operates with subcarrier spacing ∆f (numerology µ), yielding a slot duration Tslot . The available bandwidth spans NPRB Physical Resource Blocks (PRBs), each comprising 12 contiguous subcarriers. We focus on uplink link adaptation: the gNB selects the MCS for each UE’s Physical Uplink Shared Channel (PUSCH) transmission, assuming a CP-OFDM waveform (transform precoding disabled) to enable multi-layer MIMO, and conveys it via the uplink grant (DCI Format 0 0 or 0 1) [20], following the standard procedure of 3GPP TS 38.214 Section 6.1.4 [36]. The gNB acquires channel state information (CSI) by processing PUSCH transmissions. Let Ht,k,n ∈ CNant ×Nl denote the uplink channel matrix for UE k on subcarrier n at slot t. The received uplink signal vector at the gNB is: yt,k,n = Ht,k,n xt,k,n + nt,k,n ,

(1)

where xt,k,n ∈ CNl is the transmitted signal vector across Nl spatial layers and nt,k,n ∼ CN (0, σ 2 I) is additive white Gaussian noise. From (1), the gNB estimates the channel using uplink Demodulation Reference Signals (DMRS) [37] via linear estimation (e.g., MMSE), obtaining Ĥt,k,n together with a wideband SINR estimate γt,k and Reference Signal Received Power (RSRP) for each scheduled UE. In each slot t, the MAC scheduler allocates NPRB,k PRBs to each scheduled UE and the link adaptation mechanism selects an MCS index mt,k ∈ M = {0, 1, . . . , 27} (|M| = 28 entries), determining the modulation order Qm and target code rate r per the 64QAM MCS table [36, Table 5.1.3.1-1]. The selected MCS is signaled to the UE via the 5-bit MCS field in the uplink grant [20]; the UE encodes its PUSCH transport block accordingly. The Transport Block Size (TBS) B(m, NPRB,k ) is computed as a function of MCS index and allocated PRBs per the procedure in [36]. After PUSCH reception, the gNB decodes the transport block and verifies its CRC, obtaining an immediate transmission outcome without the signaling delay of downlink HARQ-ACK reporting via PUCCH: ( 1 (ACK), if CRC passes at the gNB ot,k = (2) 0 (NACK), otherwise. The instantaneous per-UE uplink throughput is Rt,k = B(mt,k , NPRB,k ) · ot,k / Tslot , where ot,k is the HARQ outcome (2). This uplink formulation offers a key advantage over downlink link adaptation: the gNB both selects the MCS and directly observes the decoding outcome, closing the feedback loop within a single entity without relying on TDD channel reciprocity or UE-reported CQI. B. Conventional OLLA Commercial 5G systems employ a two-loop link adaptation structure: 1) Inner loop (ILLA): Maps the wideband SINR estimate γt,k (measured by the gNB receiver) to an MCS index via a static Lookup Table (LUT): mt,k = LUT(min(γt,k , γmax ) + ∆t,k ) ,

(3)

where γmax = 25.99 dB is the SINR saturation threshold (the highest entry in the MCS lookup table) that prevents over-aggressive MCS selection at high SNR, and the min lowest entry is γLUT = −4.57 dB. 2) Outer loop (OLLA): Maintains a per-UE SINR offset ∆t,k that is adjusted based on HARQ feedback: ( δup , if ot,k = 1 (ACK) ∆t+1,k = ∆t,k + (4) −δdown , if ot,k = 0 (NACK) Under stationarity, the expected drift vanishes, yielding the well-known equilibrium BLER originally derived for outer-loop SIR control in [38] and adopted by modern OLLA [8]: δup . (5) εeq = δup + δdown The offset is initialized at ∆0,k = 0 dB. For the default step sizes δup = 0.1 dB and δdown = 1.0 dB, (5) yields εeq ≈ 9.1%.

4

While robust and widely deployed, OLLA has two fundamental limitations. First, the update (4) is a first-order integral controller with scalar state ∆t driven by a single bit of feedback per slot (ACK or NACK), ignoring the rich uplink channel state available at the gNB, including persubcarrier channel estimates from PUSCH DMRS processing, temporal fading statistics, and received signal power. Second, the convergence rate is governed by the step sizes: smaller steps ensure stability but slow adaptation, while larger steps enable faster tracking at the cost of offset oscillations [8], [15]. This trade-off is particularly acute in high-Doppler scenarios where the channel coherence time approaches the OLLA convergence time. C. Constrained Optimization Formulation The link adaptation objective is to maximize expected discounted throughput subject to a long-term BLER constraint: "∞ # X (6) max (1 − β) Eπ β t Rt π

s.t.

(1 − β) Eπ

" t=0 ∞ X t=0

t

#

β ct ≤ εtarget ,

(7)

where π is the control policy mapping observations to MCS selections, β ∈ (0, 1) is the discount factor, Rt is the per-slot throughput reward (instantiated in Section III-A), ot ∈ {0, 1} denotes the binary HARQ feedback (ACK/NACK, distinct from the observation vector ot ), ct = I(ot = 0) is the corresponding constraint cost, and εtarget is the maximum allowable BLER. The (1 − β) normalization follows the standard P CMDP convention [18, §2.2]: under ergodicity, (1 − β)E[ β t ft ] → Ess [ft ] as β → 1 for any bounded per-slot quantity ft [18, Remark 2.1], so the objective represents the average perslot throughput and the constraint directly bounds the steadystate BLER. This is a Constrained Markov Decision Process (CMDP) [18]. Although link adaptation is a continuing task, the discounted objective is a standard and well-justified choice: the discount factor induces an effective planning horizon of 1/(1 − β) steps matched to the timescale of channel dynamics, and the discounted-optimal policy converges to the average-optimal policy as β → 1 under ergodicity [18]. In practice, the gNB observes only noisy channel estimates, quantized signal quality indicators, and binary HARQ feedback, rather than the true state. Section III addresses this partial observability by extending the CMDP to a POMDP framework with historyenriched observations and Lagrangian constraint enforcement. III. LOLLA F RAMEWORK As noted in Section II-C, the true system state, comprising the instantaneous channel realization, inter-cell interference, and UE buffer occupancy, is not directly accessible to the gNB. We therefore model the constrained link adaptation problem as a Partially Observable Markov Decision Process (POMDP) [39], defined by the tuple hS, A, T, R, Ω, O, βi, where S is the latent state space (channel, interference, and buffer conditions), A the continuous SINR offset action space,

Ω the PHY/MAC observation space accessible to the gNB, T (s, a, s′ ) = Pr(s′ | s, a) the state-transition probability, O(s′ , a, o) = Pr(o | s′ , a) the observation-emission probability, R the per-slot throughput reward, and β ∈ (0, 1) the discount factor. Each element is instantiated below. Because the agent cannot condition on s ∈ S directly, an optimal POMDP policy must integrate information over the observation history o1:t to maintain a sufficient statistic, the belief state bt (s) = Pr(st =s | o1:t , a0:t−1 ), over the latent dynamics [39]. We approximate this belief state through a fixed-length observation window that embeds recent MCS indices, ACK/NACK outcomes, and a running BLER estimate directly into the observation vector (Section III-A), enabling a reactive feedforward policy to capture the most salient temporal patterns without recurrent state. The BLER constraint (7) is enforced via Lagrangian relaxation (Section III-B). A. POMDP Formulation For notational clarity, we drop the UE index k in this section; the formulation applies independently to each scheduled UE, and the multi-UE batched extension is described in Section IV-D. Observation space. The agent observes ot ∈ Rdo with do = 13, constructed from slot-level PHY/MAC telemetry (the realtime data path is detailed in Section IV): ot = [fch , fsig , hmcs , hack , ε̄t ]⊤ .

(8)

4

Channel features fch ∈ R . The uplink channel estimate Ĥ ∈ CNant ×Nl ×NSC ×NDMRS is compressed into four statistics: mean channel gain µ|H| (path loss), standard deviation of per-subcarrier gains σ|H| (frequency selectivity), 10th-percentile gain P10 (|H|) (deep-fade severity), and cross-slot Pearson correlation ρt (instantaneous Doppler indicator). These capture per-slot channel state entirely discarded by OLLA’s one-bit feedback. 2 • Signal quality features fsig ∈ R . The wideband SINR γt (dB), measured directly by the gNB L1 PHY pipeline, which is the primary LUT input, and RSRP (dBm), which disambiguates low SINR due to path loss from low SINR due to interference. ⊤ 3 • MCS history hmcs = [mt−1 , mt−2 , mt−3 ] /27 ∈ [0, 1] . The three most recent MCS selections (default 0.5), enabling detection of MCS oscillations, a key OLLA failure mode under rapid channel variation. ⊤ • HARQ history hack = [ot−1 , ot−2 , ot−3 ] ∈ {0, 1}3. The corresponding three-step binary outcomes; together with hmcs , these form action-outcome pairs (mt−i , ot−i ) that subsume OLLA’s single-bit feedback. • Running BLER ε̄t ∈ [0, 1]. A sliding-window average (window size W = 100 slots) that gives the agent direct observability of the constraint state relative to εtarget , accelerating Lagrangian convergence. Action space. The action at = δt ∈ [−δmax , +δmax ] is a continuous scalar SINR offset (in dB). The final MCS is selected via the same standards-compliant pipeline (3) used by OLLA: •

mt = LUT(min(γt , γmax ) + δt ) ,

(9)

5

where γt is the wideband SINR (in dB) and γmax is the SINR cap. Fig. 1 contrasts the three action formulations: (a) conventional OLLA adjusts its offset ∆t via a scalar ACK/NACK staircase, consuming only one bit of feedback per slot; (b) discrete RL bypasses the SINR-to-MCS LUT entirely and selects mt ∈ {0, . . . , 27} directly from the policy, forgoing the structured SINR-to-MCS mapping as a prior; (c) LOLLA retains the identical SINR-capped LUT pipeline while replacing the staircase with a policy-generated offset δt conditioned on the full 13-dimensional observation. This residual formulation [16] ensures that δt ≈ 0 recovers the SINR-only baseline (inner-loop LUT without outerloop adaptation), providing a strong inductive bias for safe operation and faster convergence compared to direct MCS prediction. Crucially, this shared pipeline establishes a formal performance guarantee: Uplink closed loop (common to all methods) SINR UE

PUSCH

MCS selection

PUSCH Processing 13D obs

(a)/(b)/(c) UL Grant

(a) Conv tional OLL

SINR-to -MCS LUT

ward MLP conditions on the current observation rather than the full history, incurring an approximation gap of O(ε/(1 − β)) (Proposition 2(iii)), where ε jointly absorbs the policy parameterization error and the history-truncation error of the feedforward MLP class; however, the enriched 13-dimensional observation enables the learned policy to outperform OLLA through proactive rather than reactive adaptation. Reward function. We define the base throughput reward as: Rt , (10) r̄t = Rnorm where Rt = B(mt , NPRB ) · ot /Tslot is the per-slot effective throughput defined in Section II-A and Rnorm > 0 is a fixed normalization constant chosen per configuration. Since Rnorm is a policy-independent positive constant, replacing Rt by r̄t in (6) only rescales the objective by a constant and leaves both the optimal policy and the constraint (7) unchanged; the resulting policy is further invariant to this scale through PPO’s advantage normalization. As derived in Section III-B, the Lagrangian relaxation of the BLER constraint yields the per-step training reward:  Rt /Rnorm , if ot = 1 (ACK) rt = r̄t − λt · ct = (11) −λ , if ot = 0 (NACK) t

where λt ≥ 0 is the Lagrangian dual variable and ct = I(ot = 0) = 1 − ot is the binary constraint cost. The flat NACK penalty −λt (independent of MCS) ensures consistency with the binary constraint cost used in the dual variable update, forming a valid Lagrangian [19].

ACK/NACK

(b) Direct MCS-RL

B. Lagrangian BLER Constraint (c)

OLLA

SINR-to -MCS LUT

Fig. 1. Link adaptation action formulations. (a) OLLA: ACK/NACK staircase adjusts ∆t . (b) Direct MCS-RL: categorical policy selects MCS index directly, bypassing the LUT. (c) LOLLA (proposed): continuous offset δt modulates the standards-compliant LUT, conditioned on PHY/MAC observations ot .

Proposition 1 (Performance Dominance). Let ΠOLLA denote the class of OLLA policies parameterized by (δup , δdown ), and let ΠRL denote the class of all measurable policies mapping observation histories o1:t to offsets δt ∈ [−δmax , +δmax ]. Assume the wideband SINR satisfies γt ≥ γmin a.s. for a min min receiver sensitivity γmin ≤ γLUT , where γLUT is the lowest SINR threshold in the MCS lookup table, and that δmax ≥ γmax −γmin . Then every OLLA policy is realizable within ΠRL (i.e., induces an identical MCS trajectory), and consequently "∞ # "∞ # X X t t β r̄t . sup (1−β) Eπ β r̄t ≥ sup (1−β) Eπ π∈ΠRL

t=0

π∈ΠOLLA

t=0

The proof (Appendix A) constructs an explicit embedding of any OLLA update rule within ΠRL . In practice, the feedfor-

To enforce the BLER constraint (7) without manual penalty tuning, we adopt the Lagrangian relaxation framework P for CMDPs [18], [19]. Denoting Jr (π) , (1 − P β) Eπ [ t β t r̄t ] with r̄t from (10) and Jc (π) , (1 − β) Eπ [ t β t ct ] as the (1 − β)-normalized reward and constraint cost [18, §2.2], the constrained objective is relaxed into an unconstrained saddlepoint problem: "∞ # X
t L(π, λ) = (1 − β) Eπ β r̄t − λ ct + λ εtarget , (12) t=0

where λ ≥ 0 is the Lagrangian dual variable. Equivalently, this
can be written as L = Jr (π) − λ Jc (π) − εtarget , the standard Lagrangian for the constraint Jc (π) ≤ εtarget . Since the (1 − β) prefactor and the additive term λεtarget are both constant with respect to π, the per-step training reward is rt = r̄t − λ ct as in (11). The policy π maximizes L via PPO, while λ is updated via dual gradient ascent on the constraint violation: λk+1 = clip(λk + η · (ε̄k − εtarget ), 0, λmax ) ,

(13)

where k indexes PPO iterations (cf. Algorithm 1), η is the dual learning rate, and ε̄k , NNACK /B is the rollout-batch BLER over the B = T N transitions collected in iteration k, distinct from the per-slot running-window feature ε̄t of Section III-A. The clip to [0, λmax ] ensures dual feasibility and numerical stability.

6

Although δt is continuous, the SINR-to-MCS LUT (Section III-A) maps the offset to one of |M| = 28 MCS indices, so the effective action space is finite. The correctness of this approach rests on the following duality result. Proposition 2 (Zero Duality Gap). Consider a finite CMDP with |S| < ∞ states and |M| < ∞ actions that approximates the link adaptation problem (6)–(7) via statespace discretization.P Denote the discounted constraint cost Jc (π) , (1 − β) Eπ [ t β t ct ]. Then: (i) Strong duality holds for the Lagrangian (12): maxπ minλ≥0 L(π, λ) = minλ≥0 maxπ L(π, λ). (ii) There exists a saddle point (π ∗ , λ∗ ) satisfying complementary slackness: λ∗ (Jc (π ∗ ) − εtarget ) = 0. (iii) If additionally Slater’s condition holds (i.e., ∃ π0 with Jc (π0 ) < εtarget ) and policies are parameterized by a function class ΠΘ that is ε-universal [40, Definition 1] R(i.e., for every policy π there exists θ with maxs∈S A |π(a|s) − πθ (a|s)| da ≤ ε), then the parametric duality gap, bounding both value sub-optimality and constraint violation, is O(ε/(1 − β)). The proof is in Appendix B: (i) follows from the minimax theorem on the convex-compact set of occupation measures [18, Theorems 3.2, 3.6]; (ii) from Kuhn–Tucker conditions [18, Theorem 3.6(iii)]; (iii) from [40] under Slater’s condition. Practically, the dual update (13) acts as a discretetime integral controller on the constraint error ε̄t − εtarget [41], driving the aggregate BLER to the target without manual penalty calibration. This primal-dual formulation offers three additional properties compared to fixed penalty coefficients. Target generality. The Lagrangian decouples policy optimization from constraint enforcement: changing εtarget modifies only the dual update without altering the policy architecture or training procedure. A single framework therefore supports requirements ranging from eMBB (εtarget ≈ 10%) to URLLC (εtarget < 1%) by specifying the desired target before training. Asymmetric enforcement dynamics. From (13), the dual gradient gt = ε̄t −εtarget is bounded in [−εtarget , 1−εtarget], so λ increases up to (1 − εtarget )/εtarget times faster upon constraint violation than it decreases during satisfaction (e.g., ≈10× at εtarget = 9.1%), a desirable “fast up, slow down” property for safety-critical link adaptation [41]. This mirrors OLLA’s asymmetric step sizes (δdown /δup = 10) [8], but emerges automatically from the constraint structure without manual tuning. Non-uniform per-SNR BLER allocation. Because the ACK reward Rt /Rnorm in (11) grows with MCS index while the flat NACK penalty −λ is fixed, the penalty-to-reward ratio decreases at high SNR, incentivizing the agent to tolerate higher BLER where aggressive MCS selection yields the largest throughput gain, and to enforce lower BLER at low SNR where the penalty dominates. A throughput-proportional penalty (scaling −λ with the same factor) would instead produce uniform BLER allocation analogous to conventional OLLA. The aggregate BLER converges to εtarget through dual feedback on λ, while the per-SNR allocation concentrates the

error budget where gains are largest. Under a continuous MCS relaxation, assuming each per-SNR throughput function Ti is differentiable and concave in εi , this is throughput-optimal by the KKT equal-marginal condition Ti′ (ε∗i ) = µ [35, §5.5.3]. C. PPO Policy Architecture We train a feedforward actor-critic network with Proximal Policy Optimization (PPO) [17]. Although the underlying system is partially observable, the observation vector ot already embeds a fixed-length history window (three-step MCS and ACK histories, running BLER) that captures the most salient temporal patterns. A feedforward (MLP) policy conditioned on this enriched observation achieves strong performance without the additional complexity of recurrent state management. Network architecture. The actor and critic are two independent two-layer MLPs with tanh activations and hidden dimension nh . The actor outputs the mean of a Gaussian policy πθ (δt | ot ) = N (µθ (ot ), σ 2 ), where ln σ is a state-independent learnable parameter; the critic shares the same architecture with independent parameters, producing a scalar value estimate V̂θ (ot ). Weights are orthogonally initialized [42] with gain 0.01 for the actor output, ensuring near-zero initial offsets δt ≈ 0 so that the initial policy approximates the SINR-only baseline. PPO objective. The policy is updated by maximizing the clipped surrogate objective [17]: i h  LCLIP (θ) = Êt min ρt Ât , clip(ρt , 1−ǫ, 1+ǫ) Ât , (14)

where ρt = πθ /πθold is the importance sampling ratio and Ât is the advantage estimated via GAE(λGAE ) [43] with donemasked bootstrapping. The full loss combines the clipped surrogate (14), a clipped value function loss [44], and a Gaussian entropy bonus H[πθ ] = 12 ln(2πe σ 2 ): L(θ) = −LCLIP (θ) + cv LVF (θ) − ce H[πθ ].

(15)

The entropy term in (15) prevents premature collapse of σ, which is critical for our one-dimensional action space where the policy can converge to a near-deterministic mapping without exploration pressure. D. Training Procedure Algorithm 1 presents the unified training loop. At each iteration k, the agent collects B = T N transitions (rollout length T across N parallel environments), performs Kepoch PPO epochs over M mini-batches, and adjusts the dual variable λ based on the observed BLER. The loop runs for Ktotal = ⌊Ttotal /B⌋ iterations. All hyperparameters, including the Adam [45] optimizer settings, are listed in Table II. Three aspects of the training setup merit further discussion. Episode structure and domain randomization (line 3). Link adaptation is a continuing task with no natural termination; episodes are truncated at a fixed horizon of Tep steps. To promote generalization, we apply SNR domain randomization [46]: the operating SNR is drawn uniformly from a prescribed range at episode start and re-sampled with probability pSNR per slot, exposing the agent to diverse

7

Algorithm 1 LOLLA Training Require: Policy πθ , initial dual variable λ0 , hyperparameters (Table II) 1: Initialize running-statistics normalizer R 2: for k = 0, 1, . . . , Ktotal − 1 do 3: for t = 0, . . . , T −1 across N environments do ⊲ Rollout 4: Update R with ot p
5: õt ← clip (ot −µ̂)/ σ̂ 2 +ǫ, −co , co 6: δt , log πt , V̂t ← πθ (õt ) 7: mt ← L UT(min(γt , γmax ) + δt ) 8: Execute mt ; observe rt , ot+1 , dt+1 9: end for 10: {Ât } ← GAE(λGAE ) with done-masked bootstrap ⊲ Sec. III-C
11: αk ← max fmin , 1−k/Ktotal · α0 ⊲ LR with floor 12: for epoch = 1, . . . , Kepoch do ⊲ PPO update 13: Flatten and randomly shuffle all B = T N transitions 14: Partition into M mini-batches 15: for each mini-batch S do 16: Recompute πθ , V̂θ on S

17: ÂS ← ÂS − mean(ÂS ) std(ÂS )+ǫ 18: θ ← θ − αk ∇θ L(θ) after clipping k∇θ Lk2 to gmax 19: end for 20: end for 21: ε̄k ← NNACK /B ⊲ Lagrangian dual update  λmax ⊲ Sec. III-B 22: λk+1 ← λk + η (ε̄k − εtarget ) 0 23: Set NACK penalty ← λk+1 24: end for

link conditions within each episode. The channel model and Doppler frequency are fixed per training run; cross-scenario generalization is evaluated in Section V-E. Observation normalization (line 5). Each feature is normalized online using a Welford running mean/variance estimator [47] (ǫ = 10−8 ), with statistics updated at every environment step. Advantages are additionally zero-mean/unitvariance normalized within each mini-batch (line 16), a standard PPO practice that reduces sensitivity to reward scale [44]. Learning rate floor (line 10). The learning rate is linearly annealed from α0 , but a minimum fraction fmin ∈ [0, 1] ensures it never falls below fmin ·α0 . Without this floor, λ may still be converging in late training while the near-zero learning rate freezes the policy, preventing the BLER constraint from being satisfied. Remark 1 (Computational Complexity). Training. Each of the Ttotal /B iterations collects B = T N transitions and performs Kepoch sweep epochs, each partitioned into M minibatch gradient steps of size B/M , yielding a total cost of O(Kepoch · Ttotal · |Θ|), where |Θ| = O(do nh + n2h ) is the parameter count of the actor-critic network. The Lagrangian dual update and GAE computation are both O(B), dominated by the network passes. Inference. Per-slot inference requires one actor forward pass of cost O(do nh + n2h ) plus a constant-

time LUT lookup, and scales linearly with the number of scheduled UEs K under batched execution. IV. R EAL -T IME S YSTEM A RCHITECTURE This section describes how LOLLA (Section III) is realized as a closed-loop dApp on a GPU-accelerated 5G NR stack. The design builds on the E3 dApp framework of Villa et al. [5], which established the first GPU-native platform for slot-level PHY/MAC telemetry and specifies an E3 control path for actuation. Its realized dApps, however, are openloop, and the reference explicitly notes that control logic is not implemented [5]. LOLLA realizes the closed control loop this framework leaves unimplemented: a dual-agent E3 architecture bridges DU-Low and DU-High through zerocopy shared memory, and a four-stage inference backend progression culminating in a Triton C-API binding meets the per-slot latency budget. Fig. 2 overviews the resulting closedloop dApp; the rest of this section details its components along the per-slot data path. MCS1

DU-Low (L1)

UE1

UEk SHM pointers

MCSk

RSRP,SINR,CRC,MCS

UE2 E3 setup

MCS2

E3 subscription

MCS

E3 Indication

dApp DU-High (L2)

SHM

RSRP,SINR CRC,MCS

MCS override

MCS E3 setup E3 control

6 dApp Intelligence

Fig. 2. LOLLA system architecture. The L1 E3 Agent exports PHY telemetry via the Aerial Data Lake (shared memory) and ZMQ; the dApp performs feature extraction and policy inference; the L2 E3 Agent delivers per-UE MCS overrides to the MAC scheduler.

A. Dual-Agent Closed-Loop Architecture The E3 framework [5] exposes PHY/MAC telemetry to ingNB dApps through E3 Agents coordinated by a co-located E3 Manager under the E3 Application Protocol (E3AP) [6]. Bulk PHY tensors traverse the Aerial Data Lake (ADL), a zerocopy POSIX shared-memory region with a double-buffered ping-pong layout, while per-slot scalars and synchronization fields ride a lightweight ZeroMQ (ZMQ) transport carrying E3 Indication and E3 Control messages. LOLLA operates a dual-agent configuration (Fig. 2) that closes the control loop: • L1 E3 Agent (DU-Low). Integrated into the CUDA PHY pipeline, it stages each per-slot channel estimate Ĥ into the active ADL ping-pong buffer and publishes the wideband SINR γt (Section II-A), RSRP, per-TB CRC

8

outcomes, timing identifiers (SFN, slot, RNTI), and the SHM pointers as an E3 Indication on a ZMQ PUB socket. • L2 E3 Agent (DU-High). A new agent co-located with the MAC scheduler that exposes a ZMQ REP endpoint accepting E3 Control messages with single-UE or batched multi-UE MCS overrides; a registered callback applies them to the next uplink grant, providing the control-plane actuation path. On the dApp side, the E3 Manager opens a ZMQ DEALER socket connected to the L2 Agent’s REP endpoint, dispatching each E3 Control as a single frame without waiting for a reply; this fire-and-forget path achieves a measured send latency of ∼ 23–25 µs (Section V-G, Op 9) while the L1 PUB-SUB telemetry path runs on independent sockets. At each slot t, the closed loop proceeds as: 1) The CUDA PHY pipeline executes the uplink receive chain (channel estimation, noise estimation, MMSE equalization with soft demapping, LDPC decoding, CRC check) on the received PUSCH, producing the telemetry tuple (Ĥ, γt , RSRP, CRC). 2) Ĥ is copied into the active ADL ping-pong sharedmemory buffer (2a); upon completion, the ADL notifies the L1 Agent with the SHM pointers (the active pingpong buffer index, the in-buffer write offset, and the tensor shape) together with the scalar fields (γt , RSRP, per-TB CRC, and timing identifiers) (2b). 3) The L1 Agent publishes the corresponding E3 Indication, carrying those SHM pointers and scalar fields, over its ZMQ PUB socket, which the E3 Manager receives (3a); the E3 Manager then memory-maps Ĥ from the indicated shared-memory segment using those pointers, exposing the tensor in its address space without any data copy (3b). 4) The dApp extracts the four channel statistics fch from Ĥ on the CPU (Section IV-B) and assembles ot together with the indication scalars and the per-UE state buffers. 5) The inference backend (Section IV-C) runs the actor network on ot and emits the SINR offset δt . 6) The dApp maps δt through the SINR-to-MCS LUT (9) and dispatches the resulting per-UE MCS override to the L2 Agent over the ZMQ DEALER socket. 7) The L2 Agent’s registered callback applies the override on the next uplink grant; the subsequent PUSCH outcome feeds back into Step 1 as fresh telemetry, yielding ot+1 and closing the loop. Steps 6 and 7 close the loop beyond the open-loop inference dApps of [5]: the dApp now actuates control and observes its consequences. Because the dApp interacts with the gNB only through E3AP messages [6] sent to configurable endpoint addresses, the same binary runs against either a GPU-accelerated PHY simulator or a production gNB by reconfiguring those addresses. B. Shared-Memory Telemetry and Feature Extraction The uplink channel estimate Ĥ ∈ CNant ×Nl ×NSC ×NDMRS is staged into POSIX shared memory by the ADL following Ops 1–3 of [5, Table 1]; for the 4 × 2 MIMO configuration of Table II (NSC = 3276, NDMRS = 1, complex64), Ĥ occupies

approximately 205 KiB per UE per slot in single-UE mode. Rather than uploading the raw tensor back to the inference GPU, the dApp compresses it in place on the CPU into the four channel statistics fch = [µ|H| , σ|H| , P10 (|H|), ρt ] defined in Section III-A via a vectorized C++ module that parallelizes across UEs with OpenMP; the cross-slot correlation ρt uses an RNTI-keyed cache of the previous slot’s per-subcarrier magnitude profile. The 13-dimensional observation ot (8) is then assembled from fch , the signal-quality features fsig = [γt , RSRP]⊤ carried as scalars in the same E3 Indication, and the per-UE history buffers (hmcs , hack , ε̄t ), which are updated before observation construction so that ot already reflects the previous slot’s outcome. The resulting 52-byte vector represents an ∼4,000× compression of Ĥ that removes bulk-PHY-tensor uploads from the per-slot critical path. Because the GPU-accelerated PHY simulator and the production ARC-OTA gNB execute the same CUDA baseband kernels [5], with only the RF frontend replaced by calibrated 3GPP TDL channel models (Section V-A), the PHY-derived components of ot are produced by identical kernels at training and at deployment, while (hmcs , hack , ε̄t ) are dApp-internal and PHY-agnostic. C. Model Export and Inference Pipeline Model export. The trained actor is exported to ONNX with optional FP16 TensorRT compilation; the observation runningmean and running-variance from training (Section III-D) are baked into the graph as constant tensors, so inference applies the same normalization as training. Dynamic batch axes let one exported model serve both single-UE and multi-UE inference, and a Triton model repository is generated automatically. Inference backend progression. Inference lies on the perslot critical path, where the gRPC round-trip to a separate Triton process is a dominant framework overhead [5]. We evaluate a four-stage backend progression that isolates each source of overhead in turn, culminating in an in-process Triton C-API path: 1) Direct PyTorch. The actor is loaded from a training checkpoint and invoked in inference mode through the PyTorch Python API, with no export or inference server. Python and framework overhead dominate, so this stage serves as a development baseline. 2) Triton gRPC. The Triton Inference Server runs as a separate process and the dApp communicates with it over gRPC, matching the gRPC inference path of [5]; this retains Triton’s model versioning, dynamic batching, and multi-backend support (ONNX Runtime, TensorRT) but pays the full Ops 5–8 round-trip cost. 3) Triton in-process. Triton is embedded directly inside the dApp process through its Python bindings, eliminating gRPC serialization and localhost networking; inference is invoked as a local function call with zero-copy DLPack tensor exchange. 4) Triton C-API. A pybind11 C++ extension drives the Triton Server C API directly, removing the remaining Python object churn on the per-slot inference path. Three optimizations target steady-state latency: (i) the inference

9

request object is built once at initialization and reused across all subsequent slots, (ii) observation and output buffers are pre-allocated in CUDA pinned host memory to avoid host-side allocation on the hot path, and (iii) a custom response allocator routes Triton’s outputs into those pre-allocated buffers. The result is the lowest percall overhead among the four backends. Section V-G reports a per-Op latency comparison across the four backends. D. Multi-UE Batched Control Practical deployments serve multiple UEs concurrently, yet neither the E3 reference dApps [5], [31], [32], which target single-UE open-loop inference, nor existing RL-based link adaptation methods [13]–[15], [26], which assume a single UE with a fixed PRB allocation, address the interaction between per-UE adaptation and a shared MAC scheduler. Extending the closed loop of Section IV-A to K concurrent UEs raises three architectural challenges. C1: Per-UE state isolation. A state manager indexed by Radio Network Temporary Identifier (RNTI) maintains for each UE its three most recent MCS indices, three most recent HARQ outcomes, and the running BLER estimate ε̄t (Section III-A) updated through an O(1) circular buffer. The RNTI-to-row mapping is locked at initialization from the order of the first batch indication and preserved throughout the dApp’s lifetime, guaranteeing strict alignment between the ith row of the observation batch O ∈ RK×13 and the i-th UE’s state. C2: MAC-RL interface isolation. The production MAC scheduler executes three stages per slot: priority-weighted round-robin UE selection, PRB allocation, and OLLA-based MCS selection. The dApp invokes only the first two through a PRB-only scheduler variant, retrieves the per-UE PRB allocations, and applies its own MCS through the L2 E3 Agent override path of Section IV-A (Step 6). Bypassing the OLLA stage prevents its internal offset from being updated on MCS values it never selected, which would otherwise corrupt any OLLA-only baseline reusing the same scheduler. C3: Non-uniform per-UE resource dimensions. When NPRB mod K 6= 0, round-robin allocates ⌊NPRB /K⌋ + 1 PRBs to a subset of UEs and ⌊NPRB /K⌋ to the rest, so per-UE H-matrix dimensions and Transport Block (TB) sizes differ. The per-UE subcarrier count is carried in each E3 indication for an independent reshape during feature extraction (Section IV-B), and the exact per-UE TB sizes computed by the MAC scheduler are propagated through the E3 pipeline rather than re-derived from the average PRB count, eliminating a worst-case TB-size error of 1/⌊NPRB/K⌋ (the relative gap between adjacent per-UE PRB counts; ≈ 2.9% at NPRB = 273, K = 8, and ≈ 17% at NPRB = 52, K = 8). At each slot the dApp assembles O, runs a batched forward pass emitting K SINR offsets, and dispatches the per-UE MCS overrides as a batched E3 Control message; at training the K UEs are batched as K rollout streams under a shared policy, so the single-UE formulation of Section III extends to multi-UE without algorithmic changes.

TABLE II D EFAULT S IMULATION AND T RAINING C ONFIGURATION

Parameter

Value

System and channel Antenna configuration / layers System bandwidth / NPRB Subcarrier spacing / center freq. MCS table Channel model / delay spread SNR range / change prob.

4 × 2 MIMO, Nl = 2 100 MHz / 273 PRBs 30 kHz / 3.5 GHz Table 5.1.3.1-1 (64QAM) [36] TDL-A / 100 ns (nominal) [9] [−5, 25] dB / 0.3 per slot

PPO and network Learning rate α0 / floor fmin Discount β / GAE λGAE Clip / entropy / value coeff. Rollout T / minibatches / epochs MLP hidden dim. / layers Action range δmax

2.5×10−4 / 0.2 0.99 / 0.95 0.2 / 0.01 / 0.5 128 / 4 / 4 64 / 2 (Tanh) ±31 dB

Lagrangian and training Initial / max dual (λ0 , λmax ) Dual learning rate η Default target BLER εtarget Parallel envs. / total steps

(1.0, 50) 0.05 0.091 8 / 4×106

V. S IMULATION R ESULTS This section evaluates LOLLA against conventional OLLA, ILLA, and a Direct MCS-RL baseline representing prior RLbased link adaptation. We assess throughput, BLER constraint tracking, and end-to-end latency under 3GPP TDL channel models, covering Doppler sweeps, BLER target sweeps, crosschannel generalization, and multi-UE scaling. A. Simulation Setup Training and inference run on a single NVIDIA L20 GPU using the GPU-accelerated PHY simulator of Section IV with 3GPP TDL channels [9]. The default configuration is in Table II, and per-experiment overrides are noted in the corresponding subsections. Baselines. LOLLA is compared against: (i) conventional OLLA with the standard step sizes (δup , δdown ) = (0.1, 1.0) dB yielding εeq ≈ 9.1%; (ii) ILLA, the LUT mapping with zero outer-loop offset; and (iii) Direct MCS-RL, a PPO agent with categorical action space A = {0, 1, . . . , 27} that shares the same network, observations, and Lagrangian mechanism as LOLLA but bypasses the SINR-to-MCS LUT, mirroring prior RL-based link adaptation methods [13]–[15], [26] and isolating the contribution of the residual formulation. We additionally report fixed MCS indices m ∈ {5, 10, 15, 20} as reference points. Evaluation protocol. Each trained policy is evaluated over 1,000 slots per seed (deterministic action mean) starting from δ0 = ∆0 = 0. Results are averaged over 20 random seeds (mean ± one standard deviation); minor deviations from Table II are noted inline. B. Training Convergence Fig. 3 plots LOLLA and Direct MCS-RL training curves averaged over three seeds at fd = 100 Hz with pSNR =

10

0.3, using 3 × 106 steps and a tighter Lagrangian range (λ0 , λmax , η) = (1.0, 10, 0.08) but otherwise following Table II. Averaged over the last 105 steps, both methods converge to 9.4% BLER (within 0.3 pp of the 9.1% target) and to comparable throughput, 252.4 ± 2.6 Mbps for LOLLA versus 244.2 ± 2.2 Mbps for Direct MCS-RL. LOLLA converges substantially faster in the early phase: at 200K steps it already reaches 233.2 ± 2.9 Mbps, whereas Direct MCS-RL lags at 206.0 ± 2.2 Mbps, reflecting the strong initialization provided by the LUT prior. The Lagrangian dual variable λ rises monotonically from λ0 = 1.0 to approximately 4.6 for LOLLA and 4.3 for Direct MCS-RL, confirming the integralcontroller behavior of Section III-B; the narrow ±1σ bands further indicate stable and reproducible training.

Fig. 4. Comparison of link adaptation methods across Doppler frequencies under TDL-A with pSNR = 0.3. (a) Per-UE throughput (Mbps). (b) Throughput gain over OLLA (%). (c) Block error rate (%) with the εtarget = 9.1% target (dashed). (d) Average MCS index. Shaded regions denote ± 1σ over 20 seeds.

D. Lagrangian BLER Constraint Control

Fig. 3. Training convergence at fd = 100 Hz (TDL-A, pSNR = 0.3). (a) Episode reward. (b) Episode throughput (Mbps). (c) Episode BLER (%) with 9.1% target (dotted). (d) Lagrangian λ. Mean ± 1σ over 3 seeds (shaded). Blue: LOLLA; orange: Direct MCS-RL.

C. Throughput vs. Channel Dynamics We train PPO policies at five Doppler frequencies fd ∈ {10, 50, 100, 200, 400} Hz (approximately 3 to 120 km/h at fc = 3.5 GHz). At fd = 10 Hz the channel is nearly static and all four adaptive methods agree to within 3% (283 to 290 Mbps), so the outer loop is essentially redundant. As fd grows, LOLLA delivers 284.0, 250.2, 179.4, and 77.0 Mbps at fd = 50, 100, 200, and 400 Hz, gains of +15%, +46%, +92%, and +75% over OLLA, while holding the empirical BLER within 8.6% to 9.3%. The gain peaks at fd = 200 Hz, where the coherence time Tc ≈ 1/(2fd) = 2.5 ms is shorter than OLLA’s multi-slot convergence horizon yet LOLLA still adapts per slot from the channel features. Fig. 4(d) corroborates this: LOLLA’s average MCS remains near 13 for fd ≤ 100 Hz, whereas OLLA collapses to 5.4 and 1.9 at fd = 200 and 400 Hz as repeated NACK-triggered down-steps drag its offset negative. Direct MCS-RL stays within ±3% of LOLLA at every operating point; the residual formulation’s advantage emerges only under distribution shift (Section V-E). The non-adaptive baselines collapse at high mobility, with ILLA exceeding 26% BLER at fd ≥ 100 Hz.

To validate the Lagrangian mechanism (Section III-B), we sweep four BLER targets εtarget ∈ {1%, 5%, 9.1%, 12%} at fd = 100 Hz, plus one unconstrained point obtained by setting the target so large that λ converges to zero. For each constrained point, OLLA is configured with step sizes yielding the same equilibrium BLER. Pareto frontier. Fig. 5(a) shows that LOLLA dominates OLLA across the Pareto frontier: at every constrained operating point, LOLLA achieves substantially higher throughput at strictly lower BLER than OLLA. At the tightest target (εtarget = 1%), OLLA overshoots the prescribed value to 2.6% whereas LOLLA tracks 1.3% precisely, confirming tight constraint satisfaction in the URLLC regime. At the unconstrained point, where both methods operate near 15.3% BLER, the throughput ceiling gap reaches 50%, 272.6 Mbps for LOLLA versus 181.2 Mbps for OLLA. Direct MCS-RL attains comparable or slightly higher throughput but with larger BLER deviations from the prescribed targets (e.g., 5.9% vs. the 5% setpoint), consistent with the coarser granularity of discrete MCS selection. Per-SNR BLER allocation. A dedicated policy trained under fixed-SNR episodes (εtarget = 5%) is evaluated at 13 SNR points from −5 to 25 dB. Both methods achieve comparable per-SNR throughput [Fig. 5(b)], so BLER redistribution does not cost spectral efficiency. However, their BLER profiles differ [Fig. 5(c)]: OLLA holds a uniform ∼ 5% across all SNR points, while LOLLA tightens BLER to 1%–2% across the 3–13 dB range and rises toward ∼ 10% at 23–25 dB, in agreement with the flat-penalty analysis of Section III-B. E. Channel Model Generalization We evaluate zero-shot cross-channel generalization (Table III) by testing the TDL-A trained policy on three un-

11

trained policy per K to match the per-UE PRB allocation (≈ ⌊273/K⌋ PRBs) and the resulting TB sizes. Fig. 6 shows that LOLLA delivers 254–258 Mbps for K ≤ 4 and 229.9 Mbps at K = 8, with gains of +27% to +48% over OLLA. Aggregate throughput stays nearly flat for K ≤ 4 and drops at K = 8, where the per-UE allocation shrinks to 34 PRBs and the loss of frequency diversity together with the higher relative DMRS overhead reduce per-UE spectral efficiency for both schemes. The relative gain narrows monotonically as OLLA’s conservative offset becomes less suboptimal in the narrowband regime, yet remains substantial at K = 8. Throughout the sweep, LOLLA holds the empirical BLER between 8.8% and 9.1%, tracking the target tightly, while OLLA stays at 9.5% to 9.6%.

Fig. 5. Lagrangian BLER constraint analysis (fd = 100 Hz, TDL-A). (a) Throughput versus BLER Pareto frontier (pSNR = 0.3); dashed lines mark prescribed targets and the rightmost RL points correspond to the unconstrained (λ = 0) case. (b) Per-SNR throughput and (c) per-SNR BLER under fixedSNR evaluation (pSNR = 0, εtarget = 5%); OLLA maintains a uniform ∼ 5% BLER while LOLLA exhibits a non-uniform allocation.

seen channel models: TDL-B (NLOS, two dominant clusters), TDL-C (NLOS, extended multipath), and TDL-D (LOS, Rician fading), all at fd = 100 Hz. Both RL methods transfer well to the NLOS channels TDL-B and TDL-C, maintaining 45% to 51% throughput gains over OLLA with BLER between 8.8% and 9.5%, within 1% of the matched TDLA performance. The LOS channel TDL-D, however, exposes a sharp robustness gap: LOLLA degrades to 27.0% BLER but still delivers 165 Mbps (+15% over OLLA) because the SINR-to-MCS LUT prior bounds the offset to structurally valid MCS indices even when the policy is out-of-distribution. Direct MCS-RL, lacking this safety net, collapses to 52.7% BLER and 112 Mbps (−23% versus OLLA), confirming that the residual formulation provides critical robustness under the Rayleigh-to-Rician distribution shift. TABLE III C ROSS -C HANNEL G ENERALIZATION ( TRAINED ON TDL-A, fd = 100 H Z )

NLOS

LOS

Method

TDL-A

∗

TDL-B

TDL-C

TDL-D

LOLLA Direct MCS-RL OLLA

254 / 9.0 259 / 9.5 171 / 9.6

254 / 8.8 260 / 9.1 175 / 9.6

257 / 8.8 261 / 9.2 177 / 9.6

165 / 27.0 112 / 52.7 144 / 9.7

Throughput (Mbps) / BLER (%). ∗ Matched (train = test).

F. Multi-UE Scaling Using the multi-UE architecture of Section IV-D, we evaluate K ∈ {1, 2, 4, 8} UEs sharing NPRB = 273 PRBs under round-robin scheduling at fd = 100 Hz, with a separately

Fig. 6. Multi-UE scaling at fd = 100 Hz, NPRB = 273, under roundrobin scheduling for K ∈ {1, 2, 4, 8} UEs. (a) Aggregate throughput with per-bar gain over OLLA annotated. (b) BLER vs. number of UEs, with the εtarget = 9.1% target (dashed). Mean over 20 seeds.

G. End-to-End Latency Breakdown Table IV decomposes the closed-loop latency into ten operations spanning the end-to-end data path (cf. [5, Table 1]) for K=1 and K=8 UEs (C-API backend; medians over N =1,000 steps, NVIDIA L20). Data collection (Ops 1–4) takes 72/93 µs at K=1/8, backend-independent, with the increase reflecting the larger aggregate H-matrix transfer. On the dApp path (Ops 5–9), Triton input preparation (Op 6, parallelized across UEs via OpenMP at K=8) and CUDA model inference (Op 7) dominate at 124/102 and 111/116 µs, together 86% of dApp time, while gRPC serialization—the dominant overhead of the gRPC backend (Ops 5 and 8)—drops to 3 µs under the C API. Total dApp latency is 272/255 µs (K=1/8), a 56–59% reduction from the gRPC backend (623/615 µs). Including the control return (Op 10), the closed-loop endto-end latency with the C API is 355/361 µs (K=1/8), within the 500 µs per-slot budget at 30 kHz SCS. Fig. 7 shows

12

TABLE IV E ND - TO -E ND C ONTROL -L OOP L ATENCY D ECOMPOSITION (µS ) Op∗

TABLE V A BLATION S TUDIES (fd = 100 H Z , TDL-A)

Protocol

K=1

K=8

Data collection 1 cuPHY copies data from GPU to CPU 2 cuBB notifies ADL data is ready 3 ADL copies data from CPU to SHM 4 E3 Agent sends pointers to E3 Mgr Subtotal (Ops 1–4)

memcpy Atomic memcpy ZMQ

35 18 18 1 72

45 22 25 1 93

dApp processing 5 Prepare input tensors (in-process)† 6 Triton accesses and prepares input data 7 AI Model performs inference (δ) 8 Extract output tensors (in-process)† 9 E3 Manager sends control to E3 Agent Subtotal (Ops 5–9)

C API SHM CUDA C API ZMQ

1 124 111 2 23 272

1 102 116 3 25 255

API

11

13

355

361

Operation

Control application 10 E3 Agent receives and applies control E2E total (C-API)

(a) Lagrangian and observation (pSNR = 0.3) Variant LOLLA (13D) (A1) Fixed λ=0.25 (A2) HARQ-only obs (7D) OLLA

Tput (Mbps)

BLER (%)

254 262 185 171

9.0 11.7 8.4 9.6

(b) H-matrix feature (mixed TDL-A/TDL-D) LOLLA (13D) No fch (9D) OLLA

TDL-A

TDL-D

Mixed

251 / 8.8 216 / 7.2 172 / 9.6

159 / 9.4 164 / 10.4 145 / 9.7

192 / 9.6 181 / 9.3 152 / 9.7

Throughput (Mbps) / BLER (%). 20 seeds, fd =100 Hz.

∗

Op numbering follows the end-to-end data path; cf. [5, Table 1]. † The C-API replaces gRPC serialization (Ops 5 and 8) with in-process tensor preparation. The dApp Subtotal also includes an 11/9 µs SINR-to-MCS LUT postprocess between Ops 8 and 9. Median over N =1,000 steps. NVIDIA L20 GPU, TDL-A, fd =100 Hz.

only the C API meets this budget; in-process is close (dApp 554/577 µs), while gRPC (623/615 µs) and direct PyTorch (1,975/2,852 µs) exceed it.

Fig. 7. Client latency (Ops 5–9) and inference latency (δ, Op 7) across four backends for K=1 (top) and K=8 (bottom). Dashed line: 500 µs slot budget (30 kHz SCS).

H. Ablation Studies We conduct two ablations at fd = 100 Hz, each removing one component of LOLLA [Table V(a)]. Disabling dual ascent and fixing λ = 0.25 throughout training (A1) drives BLER to 11.7%, overshooting the 9.1% target by 2.6 pp, whereas the

full model tracks the target at 9.0%, confirming that adaptive λ is necessary for tight constraint satisfaction. Restricting the observation to HARQ feedback only (A2, 7D: MCS history, ACK history, and running BLER) reduces throughput from 254 to 185 Mbps, with BLER settling at 8.4%, 0.7 pp below the 9.1% target. This undershoot reflects a partial-observability gap: without instantaneous channel and signal-quality features, the 7D policy cannot distinguish favorable from unfavorable slots and must hedge with a uniformly conservative offset. Since this 7D observation already subsumes OLLA’s input, the remaining +8% over OLLA is attributable to the non-linear policy, while the additional +37% confirms that PHY-layer features are the dominant source of improvement. To isolate the four H-matrix features fch , we train a 13D and a 9D policy on mixed TDL-A/TDL-D episodes and evaluate each channel separately [Table V(b)]. The 13D policy tracks the 9.1% target on both channels, indicating that the H-matrix features let the agent identify the current fading profile and adapt accordingly. The 9D policy collapses to a compromise strategy that is overly conservative on TDL-A (7.2% BLER, −35 Mbps) and overly aggressive on TDL-D (10.4%); on mixed evaluation, 13D delivers +6% throughput at comparable BLER. VI. C ONCLUSIONS We presented LOLLA, the first closed-loop deep reinforcement learning controller for per-slot uplink link adaptation on a GPU-accelerated 5G NR stack. By learning a residual SINR offset that modulates the SINR-to-MCS lookup table and enforcing reliability through a Lagrangian dual variable, the agent provably matches or exceeds conventional OLLA under the idealized history-conditioned policy class while exploiting rich PHY/MAC telemetry inaccessible to its single-bit feedback. Realized as a dual-agent dApp on the E3 framework, LOLLA strictly dominates OLLA on the throughput-versusBLER Pareto frontier, delivering 15% to 92% throughput gains over OLLA, scales to multiple concurrent UEs, and sustains sub-500 µs end-to-end control latency, demonstrating that learned per-slot link adaptation is feasible on a productiongrade GPU-accelerated RAN. Future work includes overthe-air validation with domain randomization for sim-to-real

13

transfer [33], joint MCS and MIMO-layer adaptation, multicell coordination, and extending the Lagrangian framework to other latency-critical control loops such as power control and beam management. A PPENDIX A P ROOF OF P ROPOSITION 1 We show that for every OLLA policy πOLLA ∈ ΠOLLA , there exists a policy π̃ ∈ ΠRL that produces the same MCS at every time step on every sample path, and hence achieves the same expected return. Step 1: OLLA as a mapping. A conventional OLLA policy πOLLA ∈ ΠOLLA is parameterized by step sizes (δup , δdown ) and generates offsets ∆t via the recurrence ∆t+1 = ∆t + δup ot − δdown (1 − ot ),

∆0 = 0,

where ot ∈ {0, 1} is the HARQ outcome. Given any ACK/NACK history o0:t−1 = (o0 , . . . , ot−1 ), the OLLA offset is a deterministic function of this history: ∆t = δup

t−1 X j=0

oj − δdown

t−1 X

(1 − oj ).

j=0

Step 2: Embedding in ΠRL . The RL observation ot ∈ R13 includes the HARQ outcome ot−1 as part of the ACK history hack . Since each ok contains ok−1 , the complete ACK history (o0 , . . . , ot−1 ) is recoverable from the observation sequence (o1 , . . . , ot ); hence the OLLA offset ∆t from Step 1 is a deterministic, measurable function of o1:t . The embedding policy computes ∆t from o1:t and outputs δt = clip(∆t , −δmax , δmax ). If |∆t | ≤ δmax , the MCS is identical to OLLA. Otherwise, LUT saturation guarantees the same MCS. Let γ̃t , min(γt , γmax ) ∈ [γmin , γmax ]. When ∆t > δmax : γ̃t + δmax ≥ γmin + (γmax − γmin ) = γmax ; since γmax exceeds all MCS thresholds by design, both γ̃t + δmax and γ̃t + ∆t (> γ̃t + δmax ) yield the maximum MCS. When min ∆t < −δmax : γ̃t +∆t < γ̃t −δmax ≤ γmax −δmax ≤ γmin ≤ γLUT , so both yield MCS = 0. In every case the MCS is identical; since the same MCS produces the same HARQ feedback under identical channel conditions, ∆t+1 evolves identically by Step 1. By induction the entire trajectory matches on every sample path, and the embedding policy belongs to ΠRL for any step-size pair (δup , δdown ). Our default δmax = 31 dB is the smallest integer value satisfying this condition for the LUT min bounds (γmax = 25.99 dB, γLUT = −4.57 dB). Step 3: Performance bound. Since every OLLA policy has a trajectory-equivalent representative in ΠRL , the RL optimum is at least as large: "∞ # "∞ # X X t t β r̄t . β r̄t ≥ sup (1 − β) Eπ sup (1 − β) Eπ π∈ΠRL

t=0

π∈ΠOLLA

t=0

Informally, the inequality is strict whenever there exists a state reachable under OLLA where the richer observation ot (channel features, signal quality, MCS history) enables a strictly better action than the one prescribed by the OLLA update rule conditioned only on ot−1 . This is the generic case in time-varying fading channels, where per-subcarrier channel estimates provide information about upcoming SNR transitions that binary HARQ feedback cannot capture.

A PPENDIX B P ROOF OF P ROPOSITION 2 We prove each part separately. Part (i): Strong duality. Both the objective (6) and the BLER constraint (7) are expressed in discounted form. Following Altman’s framework [18, §2.2, eq. (2.4)], which uses the (1 − β)normalized discounted cost by convention, the finite CMDP is equivalent to a linear program (LP) P over discounted occupation measures µ(s, a) , (1 − β) t≥0 β t Pr(st =s, at =a) ≥ 0 [18, Theorem 3.3], with flow-conservation constraints [18, P µ(s, a) c(s, a) ≤ eq. (3.5)] and the BLER constraint s,a εtarget . The feasible set is non-empty (any policy selecting the lowest MCS achieves Jc (π) ≈ 0 < εtarget ). The set of occupation measures Lβ is convex and compact, and coincides with the set attained by stationary policies [18, Theorem 3.2]. Since Lβ is convex-compact, L is continuous in (π, λ), and λ ≥ 0 is restricted to a bounded interval by the dual clip, the minimax theorem yields strong duality with attained optima: maxπ minλ≥0 L = minλ≥0 maxπ L [18, Theorem 3.6(i),(iii)]. Equivalently, the primal LP over occupation measures [18, §3.1] and its dual over value functions and Lagrange multipliers [18, §3.4] attain the same optimal value [18, Theorem 3.7]. Part (ii): Complementary slackness. The primal LP has a finite optimum and the feasible set is non-empty, so by LP complementary slackness [35, §5.5.2] the primal-dual optima (µ∗ , λ∗ ) satisfy λ∗ (Jc (π ∗ ) − εtarget ) = 0. Equivalently, under Slater’s condition (which holds by construction), Theorem 3.6(iii) of [18] guarantees that Lagrange multipliers exist and satisfy the Karush–Kuhn–Tucker (KKT) conditions, including λ∗ ≥ 0 and λ∗ (Jc (π ∗ ) − εtarget ) = 0. Part (iii): Function approximation. Slater’s condition holds by construction (the lowest-MCS policy is strictly feasible). If ΠΘ is ε-universal [40, Definition 1] (i.e., for every policy R π there exists θ with maxs∈S A |π(a|s) − πθ (a|s)| da ≤ ε), then Theorem 2 of [40] establishes that the parametric duality gap, bounding both optimality loss and constraint violation, is O(ε/(1 − β)). As network capacity grows, ε → 0. R EFERENCES [1] W. Jiang, B. Han, M. A. Habibi, and H. D. Schotten, “The road towards 6g: A comprehensive survey,” IEEE Open Journal of the Communications Society, vol. 2, pp. 334–366, 2021. [2] M. Polese, L. Bonati, S. D’Oro, S. Basagni, and T. Melodia, “Understanding o-ran: Architecture, interfaces, algorithms, security, and research challenges,” IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 1376–1411, 2023. [3] S. d’oro, M. Polese, L. Bonati, H. Cheng, and T. Melodia, “dapps: Distributed applications for real-time inference and control in o-ran,” IEEE Communications Magazine, vol. PP, pp. 1–7, 11 2022. [4] A. Kelkar and C. Dick, “Nvidia aerial gpu hosted ai-on-5g,” 2021 IEEE 4th 5G World Forum (5GWF), pp. 64–69, 2021. [5] D. Villa, M. Belgiovine, N. Hedberg, M. Polese, C. Dick, and T. Melodia, “Programmable and GPU-accelerated edge inference for real-time ISAC on NVIDIA Aerial Testbed,” arXiv preprint arXiv:2512.06493v2, Apr. 2026. [6] A. Lacava, L. Bonati, N. Mohamadi, R. Gangula, F. Kaltenberger, P. Johari, S. D’Oro, F. Cuomo, M. Polese, and T. Melodia, “dapps: Enabling real-time ai-based open ran control,” Computer Networks, vol. 269, p. 111342, 2025. [7] A. Goldsmith and S.-G. Chua, “Adaptive coded modulation for fading channels,” IEEE Transactions on Communications, vol. 46, no. 5, pp. 595–602, 1998.

14

[8] F. Blanquez-Casado, G. Gómez, M. C. Aguayo-Torres, and J. T. Entrambasaguas, “eolla: an enhanced outer loop link adaptation for cellular networks,” EURASIP Journal on Wireless Communications and Networking, vol. 2016, 2016. [9] 3GPP, “5G; Study on channel model for frequencies from 0.5 to 100 GHz,” 3rd Generation Partnership Project (3GPP), TS 38.901, 2024, version 18.0.0 Release 18. [Online]. Available: https://www.etsi.org/deliver/etsi tr/138900 138999/138901/ 18.00.00 60/tr 138901v180000p.pdf [10] ——, “LTE; 5G; Overall description of Radio Access Network (RAN) aspects for Vehicle-to-everything (V2X) based on LTE and NR,” 3rd Generation Partnership Project (3GPP), TS 37.985, 2024, version 17.2.0, Release 17. [Online]. Available: https://www.etsi.org/deliver/etsi tr/ 137900 137999/137985/17.02.00 60/tr 137985v170200p.pdf [11] F. J. Martı́n-Vega, J. C. Ruiz-Sicilia, M. C. Aguayo, and G. Gómez, “Emerging tools for link adaptation on 5g nr and beyond: Challenges and opportunities,” IEEE Access, vol. 9, pp. 126 976–126 987, 2021. [12] N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y.-C. Liang, and D. I. Kim, “Applications of deep reinforcement learning in communications and networking: A survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133–3174, 2019. [13] V. Saxena, H. Tullberg, and J. Jaldén, “Reinforcement learning for efficient and tuning-free link adaptation,” IEEE Transactions on Wireless Communications, vol. 21, no. 2, pp. 768–780, 2022. [14] X. Ye, Y. Yu, and L. Fu, “Deep reinforcement learning based link adaptation technique for lte/nr systems,” IEEE Transactions on Vehicular Technology, vol. 72, no. 6, pp. 7364–7379, 2023. [15] M. P. Mota, D. C. Araujo, F. H. Costa Neto, A. L. F. de Almeida, and F. R. Cavalcanti, “Adaptive modulation and coding based on reinforcement learning for 5g networks,” in 2019 IEEE Globecom Workshops (GC Wkshps), 2019, pp. 1–6. [16] T. Johannink, S. Bahl, A. Nair, J. Luo, A. Kumar, M. Loskyll, J. A. Ojea, E. Solowjow, and S. Levine, “Residual reinforcement learning for robot control,” in 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 6023–6029. [17] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” ArXiv, vol. abs/1707.06347, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID: 28695052 [18] E. Altman, Constrained Markov Decision Processes. Boca Raton, FL, USA: Routledge, 1999. [19] J. Achiam and D. Amodei, “Benchmarking safe exploration in deep reinforcement learning,” 2019, openAI Technical Report. [20] 3GPP, “5G; NR; Multiplexing and channel coding,” 3rd Generation Partnership Project (3GPP), TS 38.212, 2024, version 18.2.0, Release 18. [Online]. Available: https://www.etsi.org/deliver/etsi ts/ 138200 138299/138212/18.02.00 60/ts 138212v180200p.pdf [21] L. Zhu, C. Bockelmann, T. Schier, S. E. Hajri, and A. Dekorsy, “Nolla: Non-linear outer loop link adaptation for enhancing wireless link transmission,” in 2023 IEEE 34th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), 2023, pp. 1–6. [22] R. Wiesmayr, L. Maggi, S. Cammerer, J. Hoydis, F. A. Aoudia, and A. Keller, “Salad: Self-adaptive link adaptation,” 2025. [Online]. Available: https://arxiv.org/abs/2510.05784 [23] S. K. Pulliyakode and S. Kalyani, “Reinforcement learning techniques for outer loop link adaptation in 4g/5g systems,” 2017. [Online]. Available: https://arxiv.org/abs/1708.00994 [24] V. Saxena, J. Jaldén, J. E. Gonzalez, M. Bengtsson, H. Tullberg, and I. Stoica, “Contextual multi-armed bandits for link adaptation in cellular networks,” in Proceedings of the 2019 Workshop on Network Meets AI & ML, ser. NetAI’19. New York, NY, USA: Association for Computing Machinery, 2019, p. 44–49. [Online]. Available: https://doi.org/10.1145/3341216.3342212 [25] E. Bobrov, D. Kropotov, H. Lu, and D. Zaev, “Massive mimo adaptive modulation and coding using online deep learning algorithm,” IEEE Communications Letters, vol. 26, no. 4, pp. 818–822, 2022. [26] Q. An, R. Doost-Mohammady, R. Yang, and K. A. Sridhar, “Dragon: A drl-based mimo layer and mcs adapter in open ran 5g networks,” Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, 2024. [27] W. Gao, P. Zheng, P. Wu, Y. Hu, and A. Schmeink, “Joint link adaptation and device scheduling approach for urllc industrial iot network: A drl-based method with bayesian optimization,” 2025. [Online]. Available: https://arxiv.org/abs/2512.23493

[28] B. Demirel, Y. Wang, C. Tatino, and P. Soldati, “Generalization in reinforcement learning for radio access networks,” 2025. [Online]. Available: https://arxiv.org/abs/2507.06602 [29] L. You, N. Zhou, G. Pang, J. Huang, Y. Shao, and L. Fu, “From simulation to reality: Practical deep reinforcement learning-based link adaptation for cellular networks,” 2026. [Online]. Available: https://arxiv.org/abs/2603.00689 [30] D. Villa, I. Khan, F. Kaltenberger, N. Hedberg, R. S. da Silva, S. Maxenti, L. Bonati, A. Kelkar, C. Dick, E. Baena, J. M. Jornet, T. Melodia, M. Polese, and D. Koutsonikolas, “X5g: An open, programmable, multi-vendor, end-to-end, private 5g o-ran testbed with nvidia arc and openairinterface,” IEEE Transactions on Mobile Computing, vol. 24, no. 11, pp. 11 305–11 322, 2025. [31] N. Neasamoni Santhi, D. Villa, M. Polese, and T. Melodia, “Interforan: Real-time in-band cellular uplink interference detection with gpuaccelerated dapps,” in Proceedings of the Twenty-Sixth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, ser. MobiHoc ’25. New York, NY, USA: Association for Computing Machinery, 2025, p. 71–80. [32] F. Olimpieri, N. Giustini, A. Lacava, S. D’oro, T. Melodia, and F. Cuomo, “Libiq: Toward real-time spectrum classification in o-ran dapps,” 2025 23rd Mediterranean Communication and Computer Networking Conference (MedComNet), pp. 1–6, 2025. [33] R. Ford, H. Chen, P. Madadi, M. N. Kulkarni, X. Ma, D. Burghal, G. Chen, Y. Hu, C. Tarver, P. Skrimponis, Y. Zhang, Y. Xin, J. Zhang, S. Khunteta, Y. G. Reddy, A. K. R. Chavva, M. Kothiwale, and D. Villa, “Sim2field: End-to-end development of ai rans for 6g,” Proceedings of the 2nd ACM Workshop on Open and AI RAN, 2025. [34] S. Lagén, K. Wanuga, H. E. Elkotby, S. Goyal, N. Patriciello, and L. Giupponi, “New radio physical layer abstraction for system-level simulations of 5g networks,” ICC 2020 - 2020 IEEE International Conference on Communications (ICC), pp. 1–7, 2020. [35] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press, March 2004. [36] 3GPP, “5G; NR; Physical layer procedures for data,” 3rd Generation Partnership Project (3GPP), TS 38.214, 2024, version 18.2.0, Release 18. [Online]. Available: https://www.etsi.org/deliver/etsi ts/ 138200 138299/138214/18.02.00 60/ts 138214v180200p.pdf [37] ——, “5G; NR; Physical channels and modulation,” 3rd Generation Partnership Project (3GPP), TS 38.211, 2024, version 18.2.0, Release 18. [Online]. Available: https://www.etsi.org/deliver/etsi ts/ 138200 138299/138211/18.02.00 60/ts 138211v180200p.pdf [38] A. Sampath, P. Sarath Kumar, and J. Holtzman, “On setting reverse link target sir in a cdma system,” in 1997 IEEE 47th Vehicular Technology Conference. Technology in Motion, vol. 2, 1997, pp. 929–933 vol.2. [39] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, “Planning and acting in partially observable stochastic domains,” Artif. Intell., vol. 101, pp. 99–134, 1998. [40] S. Paternain, L. F. O. Chamon, M. Calvo-Fullana, and A. Ribeiro, “Constrained reinforcement learning has zero duality gap,” in Neural Information Processing Systems, 2019. [41] A. Stooke, J. Achiam, and P. Abbeel, “Responsive safety in reinforcement learning by pid lagrangian methods,” in International Conference on Machine Learning, 2020. [42] A. M. Saxe, J. L. McClelland, and S. Ganguli, “Exact solutions to the nonlinear dynamics of learning in deep linear neural networks,” CoRR, vol. abs/1312.6120, 2013. [43] J. Schulman, P. Moritz, S. Levine, M. I. Jordan, and P. Abbeel, “Highdimensional continuous control using generalized advantage estimation,” CoRR, vol. abs/1506.02438, 2015. [44] L. Engstrom, A. Ilyas, S. Santurkar, D. Tsipras, F. Janoos, L. Rudolph, and A. Ma̧dry, “Implementation matters in deep policy gradients: A case study on ppo and trpo,” ArXiv, vol. abs/2005.12729, 2020. [45] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds., 2015. [46] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 23–30, 2017. [47] B. P. Welford, “Note on a method for calculating corrected sums of squares and products,” Technometrics, vol. 4, pp. 419–420, 1962.