Cross-Layer Optimization and System-Level Design of Next-Generation Wireless Networks via Intelligent RAN Control

Cross-Layer Optimization and System-Level Design of Next-Generation Wireless Networks via Intelligent RAN Control

A Dissertation Presented by

arXiv:2608.02067v1 [cs.NI] 3 Aug 2026

Maria Tsampazi

to

The Department of Electrical and Computer Engineering

in partial fulfillment of the requirements for the degree of

Doctor of Philosophy

in

Electrical and Computer Engineering

Northeastern University Boston, Massachusetts

June 2026

To my family, my professors, and my friends who always believed in me.

Ithaka, by Constantine P. Cavafy (1911) As you set out for Ithaka, hope your road is a long one, full of adventure, full of discovery. Laistrygonians, Cyclops, angry Poseidon—don’t be afraid of them: you’ll never find things like that on your way as long as you keep your thoughts raised high, as long as a rare excitement stirs your spirit and your body. Laistrygonians, Cyclops, wild Poseidon—you won’t encounter them unless you bring them along inside your soul, unless your soul sets them up in front of you. [...] Keep Ithaka always in your mind. Arriving there is what you’re destined for. But don’t hurry the journey at all. Better if it lasts for years, so you’re old by the time you reach the island, wealthy with all you’ve gained on the way, not expecting Ithaka to make you rich. Ithaka gave you the marvelous journey. Without her you wouldn’t have set out. She has nothing left to give you now. And if you find her poor, Ithaka won’t have fooled you. Wise as you will have become, so full of experience, you’ll have understood by then what these Ithakas mean. Translated by Edmund Keeley and Philip Sherrard.

i

Contents List of Figures

v

List of Tables

ix

List of Acronyms

x

Abstract of the Dissertation 1

2

xvii

Introduction 1.1 Motivation & Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 How Standards and Industry Shape Next Generation (NextG) Research . . 1.1.2 Open Radio Access Network (RAN): Enabling Resilient, Reconfigurable, and Autonomous Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.3 The Importance of Reconfigurable Intelligent Surface (RIS) technology for the 5th generation (5G) and Beyond era . . . . . . . . . . . . . . . . . . . 1.1.4 The 5G-Satellite Spectrum Clash in the Upper Mid-Band . . . . . . . . . . 1.1.5 Digital Twins (DTs): A Primer for 6th generation (6G) Research . . . . . . 1.2 Contributions & Dissertation Outline . . . . . . . . . . . . . . . . . . . . . . . . .

3 4 5 5

Enabling AI at the Edge in O-RAN 2.1 Colosseum Testbed as an Open RAN Digital Twin Playground . . . . . . . . . . . 2.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Colosseum Wireless Network Emulator . . . . . . . . . . . . . . . . . . . 2.1.4 End-to-End Open RAN Twinning in Colosseum . . . . . . . . . . . . . . . 2.1.5 Radio-Frequency Twinning . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.6 Use Cases of Colosseum as Open RAN Digital Twin . . . . . . . . . . . . 2.1.7 Colosseum for O-RAN Testing and Integration . . . . . . . . . . . . . . . 2.1.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 DRL-based xApps for Slicing and Scheduling Optimization . . . . . . . . . . . . . 2.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 The PandORA System Model and Evaluation Framework . . . . . . . . . 2.2.3 Deep Reinforcement Learning (DRL) Optimization Strategies . . . . . . . 2.2.4 Experimental Setup and DRL Training . . . . . . . . . . . . . . . . . . .

7 7 7 10 10 11 13 14 14 15 15 15 19 28 29

ii

1 1 1 3

2.2.5 In-Sample Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . 2.2.6 Out-of-Sample Experimental Evaluation . . . . . . . . . . . . . . . . . . . 2.2.7 Broader Evaluation of PandORA . . . . . . . . . . . . . . . . . . . . . . . 2.2.8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Identifying Conflicts among DRL-based xApps . . . . . . . . . . . . . . . . . . . 2.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.2 Conflict Model and Distance Metrics . . . . . . . . . . . . . . . . . . . . 2.3.3 Experimental Setup and Conflict Analysis . . . . . . . . . . . . . . . . . . 2.3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

32 47 52 58 59 59 61 62 64

Optimizing Wireless Propagation with RIS 3.1 Cross-Layer System Level Experimental Evaluation of RIS . . . . . . . . . . . . . 3.1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 GBSMs with QuaDRiGa . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 Investigating Energy Efficiency with frequency-dependent GBSMs and RIS in C-V2X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.4 Performance Evaluation of RIS-Assisted C-V2X Communication Systems on Colosseum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Bridging Open RAN and RIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Optimizing Wireless Propagation with RIS . . . . . . . . . . . . . . . . . 3.2.4 Resource Allocation Strategies . . . . . . . . . . . . . . . . . . . . . . . . 3.2.5 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.6 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

66 66 66 71

90 95 96 96 98 100 102 102 104 109

4 AI-RAN Link Adaptation Leveraging Digital Twins 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 ARIADNE: Learning Link Adaptation via Reinforcement Learning . . . . . . . . 4.2.1 State Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Action Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Reward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 Transition Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 System Architecture and Link Adaptation Workflow . . . . . . . . . . . . 4.3.2 Implementation and Training . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Experimental Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Impact of Perfect and Imperfect Channel State Information . . . . . . . . . 4.4.2 Impact of the Adaptive BLER Penalty . . . . . . . . . . . . . . . . . . . . 4.4.3 Impact of Observation Window Size . . . . . . . . . . . . . . . . . . . . . 4.5 Site-Specific Training and Robustness Across Channel Scenarios . . . . . . . . . .

110 110 111 111 112 112 113 114 115 116 116 117 118 118 121 124 124

2.3

3

iii

75

4.6 4.7 5

6

Future Integration of RL on OTA 5G Testbeds . . . . . . . . . . . . . . . . . . . . 125 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

Enabling Spectrum Sharing in FR3 for Cellular and NTN Systems 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Contributions and Outline . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 System Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Terrestrial-Satellite Spectrum Sharing Strategies . . . . . . . . . . . . . . . . . . . 5.3.1 Interference Nulling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 Quality of Service (QoS)-Aware Power Control . . . . . . . . . . . . . . . 5.3.3 Joint Interference Nulling and QoS-Aware Power Control . . . . . . . . . 5.4 Simulation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 Terrestrial Channel Model . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2 Non-Terrestrial Network (NTN) Channel Model . . . . . . . . . . . . . . 5.4.3 Considered Constellation . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Key Performance Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.1 Impact of Interference Nulling on Beamforming Gain . . . . . . . . . . . . 5.6.2 QoS-Aware Power Control for Risk-Averse Scenarios . . . . . . . . . . . . 5.6.3 Joint QoS-Aware Power Control and Interference Nulling . . . . . . . . . 5.6.4 Combined QoS-Aware Power Control and Interference Nulling for Energy Efficiency Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6.5 Case for Power Control: Limitations of Interference Nulling . . . . . . . . 5.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

127 127 129 130 131 131 132 134 136 136 137 139 140 141 142 143 147 150 153 155

Conclusions

156

Bibliography

158

Acknowledgments

186

iv

List of Figures 1.1 1.2

NextG Research Framework: Requirements, Mechanisms, Architecture, Enablers, and Experimental Validation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Open RAN Paradigm: programmable, virtualized, and disaggregated NextG cellular architectures embedding intelligent control at the network edge, adapted from [1]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.1 2.2

Open RAN twinning capabilities in Colosseum. . . . . . . . . . . . . . . . . . . . PandORA framework for intent-driven DRL training, xApp on-boarding, and testing with Open RAN in Colosseum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Reference O-RAN testing architecture with focus on the case of two xApps operating at different time scales, Ti , as described in Section 2.2.5.2. . . . . . . . . . . . . . 2.4 Reference O-RAN testing architecture with focus on the case of four xApps operating at time scale, T , as described in Section 2.2.5.3. . . . . . . . . . . . . . . . . . . . 2.5 Performance evaluation under different action spaces and values of the γ parameter with the PPO DRL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Median values under different action spaces and values of γ with the PPO DRL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Performance evaluation under different hierarchical configurations with the PPO DRL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Median values under different hierarchical configurations with the PPO DRL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 Performance evaluation under the Default weight configuration for different action spaces and discount factor γ = 0.95 with the DQN DRL Architecture. . . . . . . . 2.10 Median values under the Default weight configuration for different action spaces and discount factor γ = 0.95 with the DQN DRL Architecture. . . . . . . . . . . . . . 2.11 Performance evaluation with 4 xApps and per-slice scheduling profile selection under PPO and DQN Architectures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.12 Median values obtained with a 4-xApp setup and per-slice scheduling profile selection under PPO and DQN Architectures. . . . . . . . . . . . . . . . . . . . . . . . . . 2.13 Performance evaluation under the Alternative weight configuration for different action spaces and discount factors with the PPO DRL Architecture. . . . . . . . . . 2.14 Median values under the Alternative weight configuration for different action spaces and discount factors with the PPO DRL Architecture. . . . . . . . . . . . . . . . .

v

1

4 8 18 27 28 33 33 36 36 38 38 39 39 42 43

2.15 Performance evaluation under different action spaces, values of the γ parameter and sets of RAN control timers with the PPO DRL Architecture. . . . . . . . . . . . . 2.16 Median values under different action spaces and RAN control timers with the PPO DRL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.17 Performance evaluation under mobility with Settings 1 and 2 of Table 2.10 and the PPO DRL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.18 Median values obtained under mobility with Settings 1 and 2 of Table 2.10 and the PPO DRL Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.19 Performance evaluation focusing on the case of joint-slice optimization with a single xApp and the PPO DRL Architecture under Settings 3 and 4 of Table 2.10. . . . . . 2.20 Median values obtained when focusing on the case of joint-slice optimization with a single xApp and the PPO DRL Architecture under Settings 3 and 4 of Table 2.10. . 2.21 User Equipment (UE) Satisfaction expressed in the form of PRB ratio for the Reward Designs of Table 2.11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.22 Impact of the Reward Designs from Table 2.11 on Downlink (DL) Enhanced Mobile Broadband (eMBB) Throughput, DL Massive Machine-Type Communications (mMTC) and Transmitter (TX) Packets. . . . . . . . . . . . . . . . . . . . . . . . 2.23 Impact of the Reward Designs from Table 2.11 on the median values of DL eMBB throughput, DL mMTC TX Packets, and DL Ultra Reliable and Low Latency Communications (URLLC) buffer occupancy. . . . . . . . . . . . . . . . . . . . . . . . 2.24 Selection of Physical Resource Block (PRB) Actions for the three slices from the xApp Catalog of Table 2.12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.25 Resource Utilization and UE Satisfaction for a network deployment described by three different Use Cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.26 Impact of the network deployment described by three different Use Cases on DL eMBB Throughput, DL mMTC TX Packets, and DL URLLC Buffer Occupancy. . 2.27 Impact of the network deployment described by three Use Cases on the median values of DL eMBB Throughput and DL mMTC TX Packets. . . . . . . . . . . . . 2.28 PACIFISTA architecture and workflow [2]. . . . . . . . . . . . . . . . . . . . . . 2.29 Empirical Cumulative Distribution Functions (ECDFs) of throughput of the eMBB slice for xApps a7 and a8 , highlighting the expected Key Performance Measurement (KPM) conflict for that control parameter. . . . . . . . . . . . . . . . . . . . . . . 3.1

3.2 3.3 3.4 3.5 3.6 3.7

45 46 48 48 49 50 52

53

53 55 56 57 57 60

64

Results averaged over 100 channel realizations for the link describing the green vehicle’s transmission to the red one in Willow Grove Naval Air Base in Horsham Township, Pennsylvania, USA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Uplink (UL) cellular metrics between Base Station (BS)-node 1 and UE-node 2. . . 75 RIS-assisted and Unmanned Aerial Vehicle (UAV)-enabled communications system. 76 A hierarchical Stackelberg game-theoretic approach to energy-efficient power control. 84 Evaluation of the hierarchical game-theoretic power control framework per UE, under different number of RIS elements and different bands. . . . . . . . . . . . . 85 Evaluation of the hierarchical game-theoretic power control framework on the entire system, under different number of RIS elements and different bands. . . . . . . . . 86 Path gains for two different spectrum bands. . . . . . . . . . . . . . . . . . . . . . 87

vi

3.8

Evaluation of the hierarchical game-theoretic power control framework per UE, under different numbers of RIS elements per each band. . . . . . . . . . . . . . . . 3.9 Evaluation of the hierarchical game-theoretic power control framework under two different utilities given in Table 3.2. . . . . . . . . . . . . . . . . . . . . . . . . . 3.10 Evaluation of a slice-based multi-UE Cellular Vehicle-to-Everything (C-V2X) communication system on Colosseum with and without the inclusion of RIS under different scheduling profiles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.11 Reference O-RAN testing architecture for RIS-enabled deployments, focusing on the Sched xApp Case Study, as described in Section 3.2.6.3. . . . . . . . . . . . . 3.12 Path Gains under varying numbers of RIS elements. . . . . . . . . . . . . . . . . . 3.13 Performance evaluation results for the single-UE Case Study. . . . . . . . . . . . . 3.14 Performance evaluation results for the multi-UE Case Study. . . . . . . . . . . . . 3.15 Performance evaluation results for the multi-UE O-RAN Case Study. . . . . . . . . 3.16 Performance evaluation results for the multi-UE Case Study with and without the inclusion of the Sched xApp in O-RAN. . . . . . . . . . . . . . . . . . . . . . . 4.1

4.2

4.3 4.4 4.5 4.6 4.7 4.8 4.9

5.1 5.2 5.3

89 90

92 100 103 105 106 107 107

System-level simulations using SIONNA over ray-traced channels for link adaptation with Outer Loop Link Adaptation (OLLA), ARIADNE, and Self-Adaptive Link Adaptation (SALAD). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 End-to-end execution of Modulation and Coding Scheme (MCS) selection in SIONNA. The channel matrix is generated from a pool of ray-traced channel realizations to ensure diversity. The current Signal to Interference plus Noise Ratio (SINR) is either directly used or estimated from previous observations. The post-equalization SINR is then provided as input to the Physical (PHY) Abstraction. The state st is observed by the ARIADNE framework, which selects the corresponding MCS index. . . . . 117 Mean Spectral Efficiency and Block Error Rate (BLER) for all Reinforcement Learning (RL) methods and baselines under Setup A from Table 4.1. . . . . . . . . 119 Distribution of Selected MCS Indices across Methods and UEs under Setup A from Table 4.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Cumulative Distribution Functions (CDFs) of Spectral Efficiency, BLER, and MCS under Setup A from Table 4.1, leveraging the DT predictor. . . . . . . . . . . . . . 122 Spectral Efficiency and BLER under Setup A from Table 4.1, using the DT predictor across multiple channel realizations and experiments. . . . . . . . . . . . . . . . . 123 Mean Spectral Efficiency and BLER for all RL methods and baselines under Setup A and B from Table 4.1, leveraging the DT predictor. . . . . . . . . . . . . . . . . . 123 Training reward over episodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Offline Fitted Q-Iteration (FQI) results on over-the-air (OTA) data for UL (top) and DL (bottom). Left: Q-value convergence across Bellman iterations. Right: MCS distribution of the learned policy vs. the OpenAirInterface (OAI) scheduler. . . . . 125 Interference from terrestrial cellular DL transmissions to satellite UL in terrestrialLow Earth Orbit (LEO) satellite coexistence scenario. . . . . . . . . . . . . . . . . 131 End-to-end resource management framework overview. . . . . . . . . . . . . . . . 136 Simulation topology for terrestrial-satellite spectrum sharing. . . . . . . . . . . . . 136

vii

5.4

Satellite overview with QuaDRiGa’s visualize lotlan in various geographic regions. The trajectories shown correspond to different parameterizations of the orbital elements defined in Listing 5.1. . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Probability Density Function (PDF) of the satellite elevation angles for the constellation of Nsat = 40 satellites considered in this work. . . . . . . . . . . . . . . . . . 5.6 Beamforming gain of the considered Next Generation Node Base (gNB) with a 8 × 8 Uniform Planar Array (UPA) and λ ∈ {0, 0.1, 1, 10} for time instance (tl ) along the satellites’ trajectories. As λ increases, a greater priority is given to the interference nulling than to the terrestrial network QoS, leading to deeper nulls toward the satellites and lower gain in the UE direction. However, the degradation is minimal: 18 dB at λ = 0 (Fig. 5.6a), to 17.9 dB at λ = 10 (Fig. 5.6d), close to the theoretical maximum of 18.06 dB. . . . . . . . . . . . . . . . . . . . . . . . . . . 5.7 CDF of the interference-to-noise ratio (INR) as defined in (5.6): Power Control & Standard Beamforming vs. Interference Nulling. . . . . . . . . . . . . . . . . . . . 5.8 Worst-case evaluation and fairness analysis for the terrestrial network topology described in Table 5.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.9 Utility function analysis for the terrestrial network topology described in Table 5.1. 5.10 Achieved utility and transmit power selection for the gNB DL under varying terrestrial channel conditions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.11 CDF of the INR as defined in (5.6) for all optimization strategies. . . . . . . . . . . 5.12 Worst-case Received Signal Strength (RSS) degradation as defined in (5.8) for all optimization strategies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.13 Performance evaluation of joint interference nulling and power control: (a) INR, (b) RSS degradation, (c) power savings. . . . . . . . . . . . . . . . . . . . . . . . . . 5.14 CDF of RSS degradation for λ = 1 with varying number of satellites. . . . . . . . 5.15 Angular positions for the complete topology for different antenna sizes with Nsat = 40 satellites and Interference Nulling with λ = 1. . . . . . . . . . . . . . . . . . . 5.16 Channel analysis: (a) Average signal leakage towards all satellites, and (b) UE RSS degradation over satellite movement as defined in (5.8) for all Interference Nulling techniques. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

viii

138 139

142 144 144 146 147 148 149 150 151 151

152

List of Tables 2.1 Traffic Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Weight Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Per-Slice Top performing xApps under the Default Weight Configuration . . . . . . 2.4 Hierarchical Reporting Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Feasible PRB Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Weight Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Design Options Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Design Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 RAN Control Timers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 Machine Learning (ML) Agent & Network Condition Settings . . . . . . . . . . . 2.11 DRL Reward Design Catalog for Sched & Slicing 0.5 with Proximal Policy Optimization (PPO) in Location 1 and Set 1 of RAN control timers. . . . . . . . . 2.12 xApp Catalog for Sched & Slicing 0.5 with the PPO DRL Architecture. . . 2.13 Distance functions used in PACIFISTA to evaluate conflicts in O-RAN [2]. . . . . 2.14 Direct and KPM conflict analysis taking a8 as the reference xApp. . . . . . . . . .

29 30 31 32 35 42 44 44 44 47 54 58 62 64

3.1 3.2 3.3 3.4 3.5 3.6 3.7

Channel Modeling Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Utility Functions Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Scheduling Profiles Catalog . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Scheduling Profiles Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 93 KPMs for RIS-assisted and UAV-enabled C-V2X topologies. . . . . . . . . . . . . 95 KPMs for UAV-enabled C-V2X topologies without RIS. . . . . . . . . . . . . . . 95 Catalog of Network Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . 104

4.1 4.2 4.3

Observation Size configurations for ARIADNE. . . . . . . . . . . . . . . . . . . . 118 Performance comparison with respect to the Oracle. . . . . . . . . . . . . . . . . . 119 Impact of kE on Median Spectral Efficiency, BLER, and MCS. . . . . . . . . . . . 122

5.1

Simulation Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140

ix

List of Acronyms 3GPP 3rd Generation Partnership Project 5G 5th generation 6G 6th generation AI Artificial Intelligence AoA Angle of Arrival AoD Angle of Departure API Application Programming Interface AWGN Additive White Gaussian Noise BLER Block Error Rate BS Base Station C-V2X Cellular Vehicle-to-Everything CaST Channel Emulation Scenario Generator and Sounder Toolchain CDF Cumulative Distribution Function CIR Channel Impulse Response COTS Commercial Off-the-Shelf CPI Conditional Policy Improvement CQI Channel Quality Information CSI Channel State Information CU Central Unit dCQI delayed CQI

x

DDPG Deep Deterministic Policy Gradient DL Downlink DOF Degrees of Freedom DQN Deep Q-Network DRL Deep Reinforcement Learning DT Digital Twin DU Distributed Unit E2E end-to-end ECDF Empirical Cumulative Distribution Function eMBB Enhanced Mobile Broadband eNB evolved Node Base EZ Exclusion Zone FBS First Bounce Scatterer FIR Finite Impulse Response FPGA Field Programmable Gate Array FQI Fitted Q-Iteration FR1 Frequency Range 1 FR2 Frequency Range 2 FR3 Frequency Range 3 GBSM Geometry-Based Stochastic Model gNB Next Generation Node Base GPS Global Positioning System GPU Graphics Processing Unit HARQ Hybrid Automatic Repeat reQuest IAB Integrated Access and Backhaul INR interference-to-noise ratio xi

INT Integral Area IoT Internet of Things IP Internet Protocol ITU International Telecommunication Union JFI Jain’s Fairness Index K-S Kolmogorov-Smirnov KF Kalman Filter KPI Key Performance Indicator KPM Key Performance Measurement LBS Last Bounce Scatterer LEO Low Earth Orbit LMMSE Linear Minimum Mean Square Error LOS Line-of-Sight LSP Large Scale Parameter LXC Linux Container MAB Multi-Armed Bandit MAC Medium Access Control MARL Multi-Agent Reinforcement Learning MCHEM Massive Channel Emulator MCS Modulation and Coding Scheme MDP Markov Decision Process MGEN Multi-Generator MIMO Multiple Input, Multiple Output MINLP Mixed Integer Non-Linear Programming ML Machine Learning MLP Multilayer Perceptron

xii

mMTC Massive Machine-Type Communications mmWave millimeter wave MPC Multipath Component MTC Machine-type Communications MU Multi-User Multi-RAT Multi-Radio Access Technology NextG Next Generation NLOS Non-Line-of-Sight NN Neural Network NOMA Non-Orthogonal Multiple Access NR New Radio NTN Non-Terrestrial Network OAI OpenAirInterface OCO Online Convex Optimization OLLA Outer Loop Link Adaptation OSM Open Street Map OTA over-the-air OTIC Open Testing and Integration Center PAWR Platforms for Advanced Wireless Research PDF Probability Density Function PDSCH Physical Downlink Shared Channel PF Proportionally Fair PHY Physical PL Path-Loss PPO Proximal Policy Optimization PRB Physical Resource Block

xiii

QoE Quality of Experience QoS Quality of Service QuaDRiGa QUAsi Deterministic RadIo channel GenerAtor RAN Radio Access Network RBG Resource Block Group ReLU Rectified Linear Unit RF Radio Frequency RIC RAN Intelligent Controller RIS Reconfigurable Intelligent Surface RL Reinforcement Learning RR Round Robin RSRP Reference Signal Received Power RSS Received Signal Strength RSSI Received Signal Strength Indicator RSU Road Side Unit RT Real-time RU Radio Unit RX Receiver RZF Regularized Zero-Forcing SALAD Self-Adaptive Link Adaptation SCA Successive Convex Approximation SDK Software Development Kit SDN Software-defined Networking SDR Software-defined Radio SE Spectral Efficiency SGD Stochastic Gradient Descent SIC Successive Interference Cancellation xiv

SINR Signal to Interference plus Noise Ratio SLA Service Level Agreement SM Service Model SMO Service Management and Orchestration SNR Signal-to-Noise-Ratio SQP Sequential Quadratic Programming SRN Standard Radio Node SSP Small Scale Parameter SVD Singular Value Decomposition TCP Transmission Control Protocol TD3 Twin Delayed Deep Deterministic Policy Gradient TDL Tapped Delay Line TGEN Traffic Generator ToA Time-of-Arrival TTD True-Time-Delay TTI Transmission Time Interval TX Transmitter UAV Unmanned Aerial Vehicle UDP User Datagram Protocol UE User Equipment UL Uplink UPA Uniform Planar Array URLLC Ultra Reliable and Low Latency Communications USRP Universal Software Radio Peripheral V2X Vehicle-to-everything VPN Virtual Private Network

xv

WF Waterfilling WI Wireless InSite WSR Weighted Sum Rate XPR Cross-polarization Ratio

xvi

Abstract of the Dissertation Cross-Layer Optimization and System-Level Design of Next-Generation Wireless Networks via Intelligent RAN Control by Maria Tsampazi Doctor of Philosophy in Electrical and Computer Engineering Northeastern University, June 2026 Prof. Tommaso Melodia, Advisor Recent years have seen the evolution of the traditional Radio Access Network (RAN) toward more open, programmable, softwarized, disaggregated, and intelligent architectures, a concept referred to as Open RAN. Future Next Generation (NextG) networks are envisioned to be Artificial Intelligence (AI)-native, enabling on-demand, data-driven closed-loop optimization of Base Station (BS) resources. At the same time, key enablers such as Reconfigurable Intelligent Surfaces (RISs) are emerging as transformative technologies for optimizing wireless propagation and improving spectral efficiency. However, the efficient utilization of the already crowded spectrum is becoming a critical issue for the successful commercial deployment of 6th generation (6G) and beyond networks. Therefore, efficient optimization of BS scheduling functionalities is essential for enhancing performance and resource orchestration in cellular networks. This Ph.D. dissertation focuses on the design, optimization, and experimental evaluation of NextG RANs that integrate Open RAN principles, data-driven control loops, and intelligent resource allocation mechanisms. The work emphasizes cross-layer optimization across the full protocol stack, including system-level experimental analysis for energy-efficient power control, and explores the use of AI in the RAN for network slicing, scheduling, and link adaptation. The results demonstrate the feasibility of NextG RANs that can be reconfigured in real time to meet the requirements of 6G and beyond use cases. First, the dissertation analyzes the architectural enablers, algorithms, and modeling frameworks that underpin these technologies. Next, it prototypes and evaluates the proposed solutions on experimental platforms and Digital Twins (DTs). The main contributions include: (i) the design of Deep Reinforcement Learning (DRL) solutions for network slicing and scheduling; (ii) the development of the PandORA framework for the automatic design, training, and deployment of DRL-based solutions, deployed at the network edge for xvii

real-time control of Open RAN applications within the Colosseum wireless network emulator; (iii) accurate Physical (PHY) RIS-enabled channel modeling and optimized resource allocation through comprehensive studies of energy efficiency and power control across different spectrum bands; (iv) full-stack, system-level emulation and performance evaluation of RIS-assisted channels for Enhanced Mobile Broadband (eMBB) and Ultra Reliable and Low Latency Communications (URLLC); (v) the integration of RIS within the Open RAN ecosystem; (vi) the design of online Reinforcement Learning (RL) solutions for link adaptation; and (vii) the study of spectrum sharing mechanisms between cellular and Non-Terrestrial Network (NTN) links through power control and beamforming. Overall, this dissertation addresses the ever-growing needs of both industry and academia, which stress the importance of designing novel algorithmic solutions and tools for advanced wireless research. In this work, we provide algorithmic designs for slicing, scheduling, link adaptation, energy-efficient power control, and beamforming; develop system-level frameworks; and provide insights through validation ranging from simulation and emulation in DTs and hardware-in-the-loop channel emulators to experimental evaluation in realistic over-the-air (OTA) 5th generation (5G) testbeds.

xviii

Chapter 1

Introduction 1.1

Motivation & Context

1.1.1

How Standards and Industry Shape NextG Research • Requirements

• Mechanisms Frequency • 3GPP: slicing (TS 23.501) • Scheduling • Link Adaptation • Power Control

3GPP / ITU IMT-2020 service categories

eMBB

mMTC

• Key Enablers • AI-powered xApps • RIS: PHY layer technology for capacity & Energy Savings • Non-Terrestrial Networks • Energy Efficiency: Core KPI in 3GPP and O-RAN

xApps NearRT- RIC

• Validation • Digital Twins (SIONNA) & Testbeds (Colosseum, PAWR platforms)

S Lo

Time

NLoS

S Lo

• Architecture • AI-driven resource control • Open interfaces for vendor interoperability

URLLC

URLLC mMTC eMBB

Emil Björnson, Reconfigurable Intelligent Surfaces: The Resurrection of Relaying, 2020

e nc ere l erf ne Int han C

LEO Satellite

Terr estr ial D L Terrestrial UEs Terrestrial gNB

Figure 1.1: NextG Research Framework: Requirements, Mechanisms, Architecture, Enablers, and Experimental Validation. Standards bodies and industry fundamentally shape the field of 5G and beyond cellular networks. The first requirement that needs to be addressed, as seen in Fig. 1.1, pertains to the types of services the network shall support. The 5G spectrum is shared among User Equipments (UEs) with diverse Quality of Service (QoS) requirements, where video streaming applications

1

CHAPTER 1. INTRODUCTION require high throughput, Internet of Things (IoT) devices require massive data transmission, and Vehicle-to-everything (V2X) applications require ultra-fast and ultra-reliable communication links. These become the high-level Key Performance Indicators (KPIs) that the system must satisfy, with operators targeting gigabit speeds for eMBB applications, five-nines reliability and ultra-low latency for URLLC UEs, and massive device connectivity for Massive Machine-Type Communications (mMTC) devices [3]. Once the requirements are specified, mechanisms are defined to meet these demands. 3rd Generation Partnership Project (3GPP) defines network slicing for resource isolation, while scheduling is the operational lever used to meet service KPIs. In detail, slicing refers to the allocation of available spectrum resources (e.g., Physical Resource Blocks (PRBs)) across different network slices (e.g., eMBB, mMTC, URLLC) according to their requirements and respective Service Level Agreements (SLAs) [4]. Scheduling defines the orchestration of radio resources depending on channel and traffic conditions, ranging from round-robin (Round Robin (RR)) to more fairness-oriented approaches such as Waterfilling (WF) and Proportionally Fair (PF) [5]. Important network functionalities also include link adaptation and power control. Link adaptation refers to how efficiently the spectrum is used by selecting appropriate modulation and coding schemes based on the current channel state, while power control enables efficient use of network power resources through open- and closed-loop mechanisms [6, 7]. The O-RAN Alliance defines the RAN Intelligent Controllers (RICs) for AI-driven closed-loop control and open interfaces for interoperability [1, 8]. This architecture serves as an industry playground for innovation, with vendors developing O-RAN-compliant solutions. Since standards require performance validation and industry trials, research relies on testbeds and DTs as a bridge between specification and deployment [9, 10]. Evaluating cutting-edge applications and studying coexistence techniques is necessary for successful commercial deployment of the proposed key enabling technologies. These span AIpowered xApps for on-demand network optimization [11] to RISs for improved resource utilization and energy efficiency, representing a cross-cutting KPI dimension emphasized by both 3GPP and O-RAN [3, 12]. Finally, the study of coexistence mechanisms and strategies between terrestrial networks and NTNs is essential for the successful deployment of future cellular architectures [13–15], especially for 6G, which introduces 6G-ready RIS, AI-native RAN [16], DTs, and dense Low Earth Orbit (LEO) constellations for improved resource efficiency.

2

CHAPTER 1. INTRODUCTION

1.1.2

Open RAN: Enabling Resilient, Reconfigurable, and Autonomous Networks Recent years have seen traditional RANs transitioning toward more open, programmable,

virtualized, and disaggregated architectures [1]. These concepts are at the foundation of the Open RAN paradigm, which has revolutionized how NextG networks are designed, operated, and deployed. The traditional BS, which was once a closed, monolithic opaque system, now encompasses a distributed functional unit split among the Central Unit (CU), Distributed Unit (DU), and Radio Unit (RU) components. Each unit is responsible for different network functionalities, with open interfaces enabling these units to communicate seamlessly via standardized protocols. Through softwarization, various network functionalities (e.g., PRB allocation and scheduling decisions) can now be run as network functions on general-purpose or accelerated multi-vendor hardware within virtualized environments, facilitating network scalability and continuous integration and testing. Furthermore, network intelligence can be added to the infrastructure at the network edge through RAN RICs, enabling on-demand fine-tuning of network resources. Specifically, through the implementation of rApps, xApps, and dApps, network resources can be controlled at various timescales—from coarse to fine granularity—depending on the application. Various optimization solutions, ranging from classic optimization to AI/Machine Learning (ML) algorithms, can now be embedded directly into the network. Open RAN further reinforces the notion of a truly AI-native RAN, where multi-vendor solutions for intelligent RAN control can coexist seamlessly within the network, enhancing network management and orchestration. In this way, novel algorithm designs can be tested and validated in terms of achieved resource allocation efficiency and compared to standard research approaches that have been leveraged for years and instructed by standards bodies such as 3GPP. Indeed, the Open AI RAN offers unprecedented opportunities that are poised to revolutionize the future of telecommunications and spectrum management. Finally, Fig. 1.2 summarizes the O-RAN architecture and the evolution of NextG cellular RANs into more open and distributed architectures which can now be optimized on demand through RAN intelligent control.

1.1.3

The Importance of RIS technology for the 5G and Beyond era Another emerging question pertains to the optimization of Radio Frequency (RF) prop-

agation, where the wireless environment is no longer treated as a fixed input to the optimization problem [17]. Thanks to RIS technology, the wireless environment is instead treated as an entity that can be controlled and optimized to enhance radio propagation, by intelligently steering the RIS elements’ phase shifts through dedicated controllers. RIS technology aligns with the requirements 3

CHAPTER 1. INTRODUCTION Leverage AI for closed-loop control of the RAN’s resources based on real-time statistics! rApps Non-realtime RIC

O-CU-CP

O-CU-CP

RRC PDCP

SDAP RRC PDCP RLC MAC PHY RF

O-RAN xApps

Traditional blackbox approach

Near-realtime RIC O-Cloud in Regional Cloud

Disaggregated gNB O-DU

O-CU-UP

O-DU

O-CU-UP

RLC MAC PHY-high

SDAP PDCP

Scrambling Modulation Layer mapping Precoding RE mapping

PHY-low O-RU Precoding iFFT/CP Beamforming RF

O-Cloud in Edge Cloud

DAC/ADC

Cell site

NextG cellular architectures are programmable, virtualized, and disaggregated.

Figure 1.2: The Open RAN Paradigm: programmable, virtualized, and disaggregated NextG cellular architectures embedding intelligent control at the network edge, adapted from [1]. of the 5G and beyond era, as it has been shown to improve connectivity and coverage, enhance system capacity, reduce latency, and improve both spectral and energy efficiency across diverse deployments [18].

1.1.4

The 5G-Satellite Spectrum Clash in the Upper Mid-Band Frequency Range 3 (FR3) is an emerging band for 6G applications. This mid-band

spectrum, typically spanning frequencies between 7.125–24.25 GHz, balances coverage and capacity by offering better propagation characteristics than mmWave frequencies and larger bandwidths than the sub-6 GHz band. However, this frequency range is already used by satellite systems (e.g., Starlink). Consequently, spectrum-sharing conflicts arise when 5G and beyond cellular applications attempt to access the already scarce spectrum resources [19]. Indeed, a major challenge for the successful commercial deployment of 6G stems from the operation of cellular networks that may disrupt incumbent satellite service links. Since cellular BSs employ Multiple Input, Multiple Output (MIMO) transmissions and high transmit power, the sidelobes generated by beamformed transmissions toward terrestrial UEs can result in significant interference toward satellite Uplinks (ULs). In particular, satellites operating at low elevation angles are especially vulnerable to such interference [14]. Consequently, given the shared spectrum, devising coexistence mechanisms and spectrumsharing strategies that enable the harmonious operation of terrestrial and satellite networks is essential.

4

CHAPTER 1. INTRODUCTION Such solutions must ensure satisfactory QoS for 5G and beyond cellular networks without disrupting satellite operations, ultimately enabling reliable service for terrestrial UEs while protecting passing satellites along their predefined trajectories.

1.1.5

DTs: A Primer for 6G Research 6G research is expected to emerge from simulation [20], with DTs playing a key role in

the successful testing and evaluation of AI-native RAN solutions before commercial deployment. Indeed, AI for 6G and beyond RAN requires adaptive data-driven control for applications such as link adaptation and power control across various network slices. DTs offer a cost-effective playground for AI research that accelerates the AI-RAN development cycle. As shown in Fig. 1.1, popular DT platforms include Colosseum, the Open RAN DT [21] from Platforms for Advanced Wireless Research (PAWR) [22], and SIONNA developed by NVIDIA [23]. Colosseum, hosted at Northeastern University, is the world’s largest wireless network emulator with hardware-in-the-loop capabilities. It leverages protocol stacks for the deployment of end-to-end RAN systems, supports large-scale data collection and RAN fine-tuning, and provides an O-RANcompliant control architecture for AI/ML-based xApp execution in the near-real-time RIC. NVIDIA’s SIONNA provides an all-in-one solution spanning ray tracing to system-level experimental evaluation on ray-traced channels, while also enabling seamless plug-in integration of AI/ML solutions.

1.2

Contributions & Dissertation Outline It is evident that both research and industry emphasize the need for testbeds, novel al-

gorithms, system-level frameworks, design tools, DTs, openness, and AI-native architectures. In this context, the present work contributes algorithmic designs for slicing, scheduling, link adaptation, energy-efficient power control, and beamforming, and discusses the integration of RISs into Open RAN architectures through extensive system-level experimental evaluation. Finally, the proposed approaches are validated using experimental platforms, including the Colosseum Testbed, DTs, and real OTA 5G data, thereby providing a bridge between theoretical design and practical deployment. The remainder of this dissertation is organized as follows. Chapter 2 (based on [2, 24–26]) explores the Colosseum testbed as a DT for O-RAN, focusing on DRL-based xApps for slicing and scheduling. Chapter 3 (based on [27, 28]) examines the cross-layer evaluation of RIS and its

5

CHAPTER 1. INTRODUCTION integration with Open RAN. Chapter 4 (based on [29]) details AI-RAN link adaptation using DTs and compares it against industry standards and state-of-the-art methods. Chapter 5 (based on [30]) discusses spectrum sharing in FR3, specifically power control and beamforming strategies to protect NTN systems while maintaining terrestrial service quality. Chapter 6 provides the final remarks and conclusions.

6

Chapter 2

Enabling AI at the Edge in O-RAN 2.1

Colosseum Testbed as an Open RAN DT Playground

2.1.1

Introduction Recent years have witnessed the Open RAN paradigm transforming the fundamental ways

cellular systems are deployed, managed, and optimized. This shift is driven by concepts such as openness, softwarization, programmability, interoperability, and network intelligence, which have already emerged in wired networks through Software-defined Networking (SDN) but still lag behind in cellular systems. The realization of the Open RAN vision in practical architectures, intelligent data-driven control loops, and efficient software implementations, however, is a multifaceted challenge that requires (i) datasets to train AI and ML models; (ii) facilities to test models without disrupting production networks; (iii) continuous and automated validation of the RAN software; and (iv) significant testing and integration efforts. This chapter describes how Colosseum—the world’s largest wireless network emulator with hardware in the loop—can provide the research infrastructure and tools to bridge the gap between the Open RAN vision and the deployment and commercialization of open and programmable networks. We describe how Colosseum implements an Open RAN DT through a high-fidelity RF channel emulator and end-to-end softwarized O-RAN and 5G-compliant protocol stacks, thus allowing users to reproduce and experiment with topologies representative of real-world cellular deployments. Finally, we showcase the development, prototyping, and testing of AI/ML solutions for Open RAN. Wireless networks and systems are evolving toward more programmable, open, and virtualized solutions. The transition to software-defined networking [31], which has transformed

7

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

5G SA Core Network

dApps with AI/ML

…. UE

UE

Base station

Mobility, path loss, fading, inter-cell interference

Base station

RIC Database

O-RAN E2 Termination Container 1

O-RAN E2 Manager Container 2 Docker

Cell 2

xApps with xApps with AI/ML xApps with AI/ML AI/ML Container N

O-RAN SMO and Non Real-time RIC

….

….

UE

O-RAN Near Real-time RIC

OAI CU/DU PDCP RLC MAC PHY

rApps with AI/ML

Cell 7

ML models catalog Offline training engine

xApp Deployment

6 UEs 3 slices

Base station

O-RAN E2 Termination

SRN

6 UEs 3 slices

Cell 1

SRN

RRC

6 UEs 3 slices

Virtual RF Scenario

O-RAN Digital Twin

UE (OAI) SRN

E2 interface: RIC Subscription RIC Indication RIC Control messages

O1 interface

Colosseum Quadrant

Traffic Network Fabric

SRN (x 32)

Container Contain Contain er erUSRP X310 USRP X310

Container Contain Contain er erUSRP X310 USRP X310

Container Contain Contain er erUSRP X310 USRP X310

Container Container

NVIDIA A100

NVIDIA V100

Massive Channel Emulator (MCHEM)

Experiment Website Resource Manager

Container

FPGA Fabric RF Scenario Server

GPU Nodes

SRN (x 32)

SRN (x 32)

SRN (x 32) Container Contain Contain er erUSRP X310 USRP X310

Gateways Network services NAS (Storage)

Management Network

Colosseum Infrastructure

Management Infrastructure

Traffic Generator (TGEN)

Figure 2.1: Open RAN twinning capabilities in Colosseum. management operations in the wired networking industry over the past decade, is now extending to cellular networks. While introducing software and programmability in radio systems is more challenging than in switching and routing, it also creates opportunities to deploy and configure bespoke networks that can be dynamically tuned to optimally support a variety of verticals. The Open RAN paradigm, its embodiment in the technical specifications of the O-RAN ALLIANCE, and its evolution in the 3GPP RAN and core network designs are converging toward a network architecture that is open, disaggregated, programmable, and intelligent [1, 32]. Openness, embedded in interfaces that expose RAN telemetry and control, enables the programmability and optimization of RAN functions through intelligent closed-loop control driven by the so-called RICs [8,33]. Disaggregation, implemented by splitting RAN functionalities into self-contained units across user and control planes, fosters innovation and introduces multi-vendor interoperability. Finally, programmability, realized through open Application Programming Interfaces (APIs) of software solutions, enables swift reconfiguration of the white-box components of cellular networks to adapt to ever-changing network conditions and demand. Overall, these ingredients have the potential to transform how

8

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN cellular networks are deployed, managed, and optimized. However, advancing the Open RAN vision toward a fully developed architecture with robust and reliable algorithmic components and seamless multi-vendor integration requires addressing several challenges at the architectural, algorithmic, and system levels, including the following. Need for Datasets. To develop robust and scalable AI and ML solutions that generalize well across a variety of real-world deployment scenarios, it is necessary to leverage rich datasets of RAN telemetry, data, and performance indicators [34]. While network operators are in a unique position to collect such datasets, it is often impractical or impossible to use them for research and development due to privacy and security concerns. Wireless testbeds represent a feasible path to overcome this limitation [9, 35–38]. However, they are often limited by the RF characteristics and topology of their deployment areas. End-to-end AI and ML Testing. Once trained, AI/ML-based control solutions need to be validated and tested in controlled environments to avoid disruption to production networks. At the same time, testing conditions must be realistic to obtain meaningful results that account for, for instance, user load, traffic patterns, and RF characteristics of the real-world deployments the models are intended to control. Automated Integration and Testing of Disaggregated Components. Disaggregation enables a more robust supply chain but also introduces the need to validate interoperability across vendors and devices. This is a labor-intensive and often manual process that calls for the development of automated techniques [39]. In this section, we describe how Colosseum—the world’s largest wireless network emulator with hardware in the loop [21]—can be leveraged as a DT platform to address these challenges and to develop end-to-end, fully integrated, and reliable solutions for Open RAN, as shown in Figure 2.1. Through its channel and traffic emulation capabilities, Colosseum can replicate a wide range of real-world scenarios representative of cellular deployments and generate datasets that can be used to train AI/ML models robust to network changes [11]. A combination of generic compute nodes, Software-defined Radios (SDRs), and the possibility of integrating Commercial Off-the-Shelf (COTS) devices allows the digital replication of Open RAN and 5G-and-beyond protocol stacks [40, 41]. This enables repeatable experiments in which different network configurations and protocol stacks can be tested under the same channel and traffic conditions, while also providing a safe environment for testing AI/ML solutions.

9

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

2.1.2

Related Work The development of DTs and testbeds for cellular systems, and specifically for Open RAN,

has recently received significant attention, owing to the importance of addressing the challenges described in Section 2.1.1. The O-RAN ALLIANCE itself has dedicated a study area to DTs for Open RAN within its next Generation Research Group (nGRG). DT frameworks for cellular networks are discussed in [42–49], with analyses of the challenges and opportunities of using DTs for Open RAN and beyond-5G design. Paper [50] proposes a network DT framework for 5G and develops a use case on RAN control with reinforcement learning, using a simulation-based twin that replicates a real-world scenario in Barcelona, Spain. The authors of [51] discuss the opportunity of integrating DTs into automated operations for the development of reinforcement learning algorithms for Open RAN. Reference [52] presents a simulation study framed in the context of DTs to optimize fixed wireless access use cases for cellular networks. The studies in [53, 54] leverage DTs to create virtual environments for testing network updates driven by AI/ML components. These works, however, are primarily based on simulation-driven twins, which fail to capture the complexity of real-world RF equipment. In addition, they provide limited evaluation of the DT in real-world settings, e.g., through experiments in operational networks or testbeds. The Colosseum DT described in this chapter is based on real-time channel emulation and end-to-end Open RAN protocol stacks that can be either automatically deployed and tested in the real world or used within the DT as-is, thus achieving a high level of fidelity, as demonstrated by the experiments in [10, 35]. Implementations of DTs or components of DTs for networked systems are presented in [55–57], focusing on middleware for DTs for aerial networks and on optimizing network services for the replication of real-world setups (e.g., for Industry 4.0) within a DT. However, these works do not focus on end-to-end Open RAN systems or on the development of intelligent control solutions. Testbeds and frameworks for the evaluation of Open RAN systems are discussed in [58–62], and Open RAN evaluation is also supported by the PAWR platforms [9, 36–38]. The PAWR platforms are the closest in terms of capabilities and programmability to Colosseum, being mostly based on SDRs, but they lack the emulated environment that Colosseum supports.

2.1.3

Colosseum Wireless Network Emulator The architecture of Colosseum is shown in the bottom part of Figure 2.1. Its main

components are: (i) 128 pairs of generic compute servers and SDRs, named Standard Radio Nodes 10

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN (SRNs); (ii) a channel emulation system; and (iii) a state-of-the-art AI/ML infrastructure. Each SDR is paired with a Dell PowerEdge R750 server connected to an NI/Ettus Universal Software Radio Peripheral (USRP) X310 SDR through a dedicated 10 Gbps link. Users of Colosseum can reserve these SRNs and deploy either custom or pre-configured protocol stacks through Linux Container (LXC). Container images are stored in a dedicated storage infrastructure, which is part of a larger management infrastructure that includes services such as web interfaces, a Virtual Private Network (VPN) for external connectivity, and networking functionalities. The Colosseum channel emulator, namely Massive Channel Emulator (MCHEM), is responsible for reproducing digital representations of real-world RF propagation scenarios, which represent the state of the RF channel at given time instants. Specifically, each pair of transceivers, i.e., those of the SDRs, is modeled through a Tapped Delay Line (TDL) with up to four non-zero taps that represent the multipath components of the Channel Impulse Response (CIR) for each millisecond of the captured scenario. At its core, MCHEM includes four quadrants with 64 Field Programmable Gate Arrays (FPGAs) and 128 SDRs that up-/down-convert the signal between RF and baseband. The FPGAs implement real-time convolution operations on the digital baseband signal, modeling the RF channel effects of real-world environments. These taps, which have a maximum propagation delay of 5.12 µs [63], are streamed to the FPGAs from a dedicated RF scenario server. We refer readers to [21] for a comprehensive overview of the Colosseum architecture. The AI/ML infrastructure of Colosseum, shown in yellow at the bottom of Figure 2.1, consists of two NVIDIA DGX A100 systems with 8 Graphics Processing Units (GPUs) each, providing 10 petaFLOPS of compute power, and one Supermicro Superserver 8049U-E1CR4T with 6 NVIDIA V100 GPUs and 3 TB of RAM. Compared to the virtualization system of the SRNs, the GPU nodes leverage Docker containers.

2.1.4

End-to-End Open RAN Twinning in Colosseum The emulation capabilities of Colosseum can be leveraged to deploy, study, and profile

wireless protocol stacks in controlled and repeatable environments. As an example, Figure 2.1 illustrates the deployment of an end-to-end, fully programmable Open RAN cellular system in Colosseum. The top-left portion of the figure depicts the RAN, which consists of a core network and cellular base stations—either monolithic or disaggregated into CUs and DUs—deployed over an emulated RF scenario of interest and serving mobile UEs. The top-right portion of the figure shows the Service Management and Orchestration (SMO) and O-RAN RICs, which interface with

11

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN the RAN base stations through open interfaces and augment their operation using AI/ML applications deployed therein, namely xApps and rApps. We refer readers to [1] for a detailed overview of the O-RAN architecture and its functionalities. In Colosseum, these O-RAN-aligned solutions are provided to users through ready-to-use LXC images that implement functionalities exposed via the OpenRAN Gym framework, which is described next. 2.1.4.1

Open-source Cellular Protocol Stacks Colosseum enables experimentation with different open-source 5G softwarized protocol

stacks, such as OpenAirInterface (OAI) [40] and srsRAN [41]. These are provided as ready-to-use images [64]—the same images that can be used for OTA experiments with minor variations in the configuration parameters. These open-source projects provide reference implementations of 3GPP cellular protocol stacks, including that of 5G New Radio (NR). The NR system consists of a 5G RAN and a core network that run on Linux-based general-purpose compute platforms and can control SDRs, e.g., the NI/Ettus USRPs deployed in Colosseum. The RAN applications support the instantiation of Next Generation Node Bases (gNBs) and UEs. Compared to OTA environments, however, the additional components in the Colosseum transmit and receive chains (e.g., those that are part of MCHEM) require adjusting the timing advance during the establishment of the initial connection between gNB and UEs. This can be done using the ta input argument when starting the base station application. 2.1.4.2

OpenRAN Gym OpenRAN Gym is an open-source framework for O-RAN experimentation that enables

users to instantiate softwarized RANs based on the OAI and srsRAN protocol stacks, and to control them through xApps deployed on an O-RAN Near-Real-time (RT) RIC [10]. Its components include (i) SCOPE [5], a wrapper for RAN software that simplifies the user experiment workflow and enables data collection at scale, and (ii) ColO-RAN [11], a simplified version of the O-RAN Software Community Near-RT RIC that provides an Software Development Kit (SDK) for xApp design, as well as pipelines for data collection and training of AI/ML models for RAN inference and control to be embedded into xApps. Overall, OpenRAN Gym allows users to instantiate a fully compliant Near-RT RIC using Docker containers, as well as xApps that can interact with the RAN (e.g., to receive KPIs and enforce control policies) via the E2 interface.

12

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN Examples of developed xApps include DRL agents that optimize RAN performance in real time by controlling scheduling and slicing configurations [11, 26].

2.1.5

Radio-Frequency Twinning The channel and traffic emulation capabilities, together with the cellular protocol stacks

that can be deployed on Colosseum, provide key building blocks toward the creation of high-fidelity digital replicas of Open RAN systems with RF hardware in the loop. This section describes a set of toolchains used to twin real-world RF scenarios in Colosseum. RF scenarios are generated offline and then replayed in real time by the scenario server. To enable this process, we rely on the RF twinning methodology of the Channel Emulation Scenario Generator and Sounder Toolchain (CaST) framework [65]. The framework comprises two main components: (i) a streamlined framework for creating realistic mobile wireless scenarios; and (ii) an SDR-based channel sounder for characterizing emulated RF channels. The combination of these components enables the generation and validation of DT RF scenarios, ensuring that the digital replicas closely match their real-world counterparts. The main steps of the scenario creation workflow are as follows: (i) identify the wireless environment to be captured, i.e., the physical location to be twinned, which may vary in size and type, e.g., indoor, outdoor, urban, or rural; (ii) obtain a 3D model of the environment by leveraging online databases such as Open Street Map (OSM); (iii) load the model into a ray-tracing tool, e.g., MATLAB ray tracer, Wireless InSite (WI), or SIONNA; (iv) define the location and trajectories of nodes; (v) sample the channel using the ray tracer, capturing effects such as node mobility; (vi) parse the ray-tracer output to extract the channel for each pair of nodes at each millisecond of the emulated scenario; (vii) approximate the resulting channel representation into a format suitable for the Colosseum channel emulator, i.e., 512 channel taps with at most 4 non-zero taps, spaced at 10 ns intervals and with a maximum delay spread of 5.12 µs; (viii) install the scenario in Colosseum by converting it into an FPGA-based format. This scenario creation toolchain is modular and allows users to modify or provide inputs at any of the above steps. After a scenario is installed in Colosseum, it is validated to ensure that it closely matches the expected real-world behavior. This validation relies on a channel sounder implemented in GNU Radio [66] and consists of the following steps: (i) transmission of a known code sequence used as a reference signal via the GNU Radio transmitter; (ii) reception at the GNU Radio receiver of the signal processed by MCHEM, which applies the corresponding channel characteristics; (iii) post-processing of the received signal using the known code sequence to extract the CIR and Path-Loss (PL) of the

13

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN waveform; (iv) validation of the results against the original modeled channel taps.

2.1.6

Use Cases of Colosseum as Open RAN Digital Twin Colosseum has been extensively used to study open problems in the algorithmic, archi-

tectural, and system-level design of Open RAN systems, leveraging the realism of its RF scenario emulation and the ability to run state-of-the-art software-defined protocol stacks that replicate realworld deployments. In this section, we review recent results as examples of Open RAN studies that can be conducted on Colosseum. 2.1.6.1

Slicing and Scheduling Optimization RAN slicing allows operators to dynamically partition the available spectrum bandwidth

to accommodate UEs with heterogeneous QoS, performance, and SLA requirements. A common classification of services and UEs in 5G includes eMBB, Machine-type Communications (MTC), and URLLC. To achieve flexibility in RAN control, AI/ML agents can be designed and embedded in xApps, then deployed on the Near-RT RIC and used to adapt the RAN to varying network conditions and demand. Specifically, DRL can be leveraged in the design of such control solutions for Open RAN, as it can be trained a priori and does not require prior knowledge of run-time network dynamics [26]. Since these solutions require large datasets for training prior to deployment in production RAN environments (to avoid disrupting network operations), the scale and controlled nature of DTs provide a suitable environment for their development. Through the OpenRAN Gym framework (described in Section 2.1.4.2), Colosseum provides the tools and infrastructure for data collection, design, training, testing, and evaluation of AI/ML xApps prior to their deployment in over-the-air systems.

2.1.7

Colosseum for O-RAN Testing and Integration The Colosseum Open RAN DT is also an Open Testing and Integration Center (OTIC)

center officially recognized by the O-RAN ALLIANCE for testing and integrating Open RAN components from multiple vendors. Current OTIC activities primarily focus on achieving interoperability in disaggregated Open RAN architectures, including conformance testing of O-RAN interfaces and equipment with respect to O-RAN specifications, as well as interoperability certification of component pairs and end-to-end systems. The Colosseum data center already supports such testing; its digital twinning capabilities, together with its controlled and repeatable experimental environment 14

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN and AI/ML infrastructure, make it a key enabler for the testing and eventual certification of AI-based solutions for Open RAN.

2.1.8

Conclusions This chapter reviewed how Colosseum, the world’s largest wireless network emulator with

hardware in the loop, can be leveraged as a DT for end-to-end Open RAN systems. We introduced the Colosseum architecture and described how it enables a high-fidelity digital replica of real-world RF environments, as well as the deployment of real-world protocol stacks. We then presented use cases where Colosseum’s DT capabilities have been used to advance algorithms, architectures, and testing of Open RAN systems. We now showcase how Colosseum, through its emulation capabilities and support for softwarized stacks, enables the development of effective solutions for the control and optimization of the RAN, such as network slicing and scheduling.

2.2

DRL-based xApps for Slicing and Scheduling Optimization

2.2.1

Introduction The highly heterogeneous ecosystem of NextG wireless communication systems calls for

novel networking paradigms where functionalities and operations can be dynamically and optimally reconfigured in real time to adapt to changing traffic conditions and satisfy stringent and diverse QoS demands. Open RAN technologies, and specifically those being standardized by the O-RAN Alliance, make it possible to integrate network intelligence into the once monolithic RAN via intelligent applications, namely, xApps and rApps. These applications enable flexible control of the network resources and functionalities, network management, and orchestration through data-driven intelligent control loops. Recent work has shown how DRL is effective in dynamically controlling O-RAN systems. However, how to design these solutions in a way that manages heterogeneous optimization goals and prevents unfair resource allocation is still an open challenge, with the logic within DRL agents often considered as an opaque system. In this chapter, we introduce PandORA [25], a framework to automatically design and train DRL agents for Open RAN applications, package them as xApps and evaluate them in the Colosseum wireless network emulator. We benchmark 23 xApps that embed DRL agents trained using different architectures, reward design, action spaces, and decision-making timescales, and with the ability to hierarchically control different network parameters. We test these agents on the Colosseum testbed under diverse traffic and channel

15

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN conditions, in static and mobile setups. Our experimental results indicate how suitable fine-tuning of the RAN control timers, as well as proper selection of reward designs and DRL architectures can boost network performance according to the network conditions and demand. Notably, finer decision-making granularities can improve mMTC’s performance by ∼ 56% and even increase eMBB Throughput by ∼ 99%. Programmable, virtualized, and disaggregated architectures are seen as key enablers of NextG cellular networks. Indeed, the flexibility offered through softwarization, virtualization, and open standardized interfaces provides new self-optimization capabilities through AI. These concepts are at the foundation of the Open RAN paradigm, which is being specified by the O-RAN Alliance. Thanks to the RICs proposed by O-RAN (i.e., the near- and non-real-time RICs), intelligence can be embedded into the network and leveraged for on-demand closed-loop control of its resources and functionalities [1]. This is achieved via intelligent applications, called xApps and rApps, which execute on the near- or non-real-time RICs, respectively. Through the RICs, these applications interface with the network nodes and implement data-driven closed-loop control based on real-time statistics received from the RAN, thus realizing the vision of resilient, reconfigurable and autonomous networks. Since they do not require prior knowledge of the underlying network dynamics [67], DRL techniques are usually preferred in the design of such control solutions for the Open RAN [11, 68]. 2.2.1.1

Related Work Intelligent control in O-RAN through xApps has widely attracted the interest of the research

community. For example, [69] proposes the NexRAN xApp to control and balance the throughput of different RAN slices. In [70], an O-RAN-compliant RAN platform is introduced for MultiRadio Access Technology (Multi-RAT) environments, enabling network slicing and slice-specific scheduling. The FlexSlice framework in [71] addresses RAN slicing and scheduling with finer control loop granularity for real-time efficiency. In [72], the authors discuss an O-RAN-based framework for predictive UL network slicing, leveraging a Deep Learning-based xApp for dynamic reconfiguration of RAN scheduling for URLLC services. Similarly, in [73], the authors present a Deep Learning-based approach to develop a RAN slicing xApp. The authors of [74] develop a RL xApp to assign resource blocks to certain users according to their Channel State Information (CSI) and with the goal of maximizing the aggregated data rate of the network. A deep Q-learning-based xApp for controlling slicing policies to minimize latency for URLLC slices is presented in [75], while a cooperative multi-agent RL algorithm for Open RAN slicing and radio resource management,

16

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN taking into account various SLA constraints, is discussed in [76]. The authors of [11] experimentally evaluate and demonstrate three DRL-based xApps under a variety of traffic and channel conditions, and investigate how different action space configurations impact the network performance. Finally, other research efforts focus on coordinating multiple xApps to control different parameters via a combination of federated learning and team learning [77–80]. DRL has been widely used for resource allocation [81, 82] in the wireless communications and networking field. In [83], the authors propose a QoS-oriented resource allocation scheme for network slicing and throughput maximization in 5G and beyond networks. DeepSlicing in [84] relies on a DRL approach to determine the number of resources required by the UEs in each slice to ensure their QoS demands are met. In [85], the authors present a DRL-based framework for NextG RAN slicing to improve resource utilization and meet the slices’ QoS requirements by leveraging feedback generated by the Deep Q-Network (DQN) agents. In [86], the authors discuss a two-level scheduling framework to jointly maximize the Quality of Experience (QoE) and Spectral Efficiency (SE). Specifically, the higher layer controller manages bandwidth allocation, while the lower level controller is responsible for scheduling the PRB and power allocation at a smaller timescale, utilizing MIMO antenna technology. In [87] a Multi-Agent Reinforcement Learning (MARL) approach is introduced for joint optimization of UE scheduling and power control in a wireless environment characterized by the coexistence of multiple Transmitters (TXs) with multiple associated UEs. In [88], an energy-efficient RAN slicing scheme utilizing DRL-assisted learning for resource allocation in 5G networks is introduced. Specifically, a combination of Deep Learning and DRL is employed for resource allocation on large and small time-scales, respectively. In [89] a hierarchical DRL-based approach is employed for resource allocation, coupled with a load balancing strategy to enhance energy efficiency while ensuring user fairness from a QoS perspective. 2.2.1.2

Contributions and Outline The above works clearly show that DRL and AI are catalysts in the design and development

of intelligent control solutions for the Open RAN. However, despite early results showing their success and effectiveness, designing DRL agents which are effective at controlling and optimizing complex Open RAN scenarios—characterized by the coexistence of diverse traffic profiles and potentially conflicting QoS demands—is still an open challenge that, as we describe below, we aim at addressing in this chapter. Specifically, our goal is to go beyond merely using AI, and specifically DRL, in a black-box manner. Instead, we try to address some fundamental design questions that are

17

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

Step

Auto-AI

Intent Parsing

Intent creation

Step

Step Dataset Generation

Step

ML Training Nodes 2x NVIDIA DGX A100 servers

Step

Setup DRL Environment

Step

PandORA

AI/ML Training

Training Monitoring

Colosseum collaboration network

models

xApps xApps

xApps

O-RAN

SRN UE 2 SRN UE 3

Colosseum Near-real-time RIC

RF

Colosseum SRN server

Mobility, path loss, fading, interference

SRN UE 6

O-RAN E2 Interface

Step

OpenRAN Gym

SDR Colosseum SCOPE Colosseum SRN server

USRP X310

Report & Data Analytics

SRN UE 1

…

OpenRAN Gym Datasets

Step

Massive Channel Emulator (MCHEM)

Testing

xApps AI/ML xApps

Dispatching on RIC

Catalogs

On-boarding (AI Model in xApp)

Step

Wireless channel emulation with hardware-in-the-loop

Traffic Generator (TGEN) eMBB, URLLC, mMTC Colosseum traffic network

Figure 2.2: PandORA framework for intent-driven DRL training, xApp on-boarding, and testing with Open RAN in Colosseum. key for the success of intelligence in Open RAN systems. We consider an Open RAN delivering services to URLLC, mMTC and eMBB network slices. Specifically, we use OpenRAN Gym [10]—an open-source framework for ML experimentation in O-RAN—to deploy such Open RAN on the Colosseum wireless network emulator [21], and control it through xApps using 23 different DRL agent designs. These xApps have been trained to perform slice-based resource allocation (i.e., scheduling profile selection and RAN slicing control) and to meet the diverse requirements of each slice. We investigate the trade-off between long-term and short-term rewards, we discuss and compare different design choices of action set space and DRL architecture, hierarchical decision-making policies and action-taking timescales. Finally, we show how these choices greatly impact network performance and affect each slice differently. To the best of our knowledge, in [26], we conducted the first experimental study that comprehensively evaluated the design choices for DRL-based xApps to provide insights on the design of xApps for NextG Open RANs. In this chapter, we extend our previous work [26] and present PandORA, a large-scale evaluation and profiling of DRL agents for Open RAN, leveraging a framework to automate the training of DRL agents and their on-boarding as xApps to be executed in the near-real-time RIC. This all-in-one solution, spanning from the automated end-to-end streamlining of DRL model training, to testing on an Open RAN network, is enabled by the Colosseum testbed. Moreover, we extend the analysis in [26] by considering two new directions that affect DRL-based

18

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN xApp design. Specifically, (i) we investigate the trade-offs between using a global DRL agent that takes decisions for all slices against the case of training and deploying per-slice dedicated xApps; and (ii) we analyze the effect of RAN control timing (i.e., the temporal granularity we use to compute and enforce new control actions). We explore the aforementioned directions by training a variety of DRL agents, which we then onboard on xApps. We test these agents both under network conditions similar to those encountered in the training dataset (i.e., in-sample experimental evaluation), as well as under previously unseen network conditions not encountered during the training process (i.e., out-of-sample experimental evaluation). The remainder of this chapter is organized as follows. Section 2.2.2 describes the PandORA System Model and Evaluation Framework. Section 2.2.3 presents the different DRL optimization strategies considered in this work, while Section 2.2.4 details our experimental setup and training methodology, through PandORA. Experimental results are discussed in Sections 2.2.5, 2.2.6 and 2.2.7. Finally, Section 2.2.8 draws our conclusions.

2.2.2

The PandORA System Model and Evaluation Framework Before providing thorough implementation details of the PandORA system design and

its operational procedures (Section 2.2.2.2), we first briefly introduce the O-RAN system model considered in this work (Section 2.2.2.1). 2.2.2.1

System Model and Reference Use-Case Scenario In this work, we consider an Open RAN multi-slice scenario where UEs generate traffic

with diverse profiles and QoS demands. Without loss of generality, we assume that traffic generated by UEs can be classified into eMBB, URLLC, or mMTC slices. We focus on the case where a set of xApps execute in the near-real-time RIC to intelligently control the resource allocation process in a way that satisfies the diverse QoS demands required by each slice. These xApps take these control decisions by leveraging AI/ML algorithms that can control BS parameters and functionalities such as RAN slicing (i.e., the portion of available PRBs that are allocated to each slice at any given time) and Medium Access Control (MAC) layer scheduling policies (i.e., how such PRBs are internally allocated to users belonging to each slice). It is worth mentioning that xApps can assign a certain scheduler profile (to be selected among RR, WF and PF) to each slice, but do not take Transmission Time Interval (TTI)-level scheduling decisions, which is instead left to the evolved Node Base (eNB). xApps make decisions based on UE traffic demand,

19

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN load, performance and network conditions that are given by Key Performance Measurements (KPMs) periodically reported by the RAN. 2.2.2.2

PandORA Overview and Procedures The architecture and building blocks of PandORA are illustrated in Fig. 2.2. These are

key to our extensive evaluation campaign, as they streamline the DRL design and xApp evaluation. PandORA leverages automated pipelines that cover the end-to-end lifecycle of AI development for O-RAN systems. Specifically, it embeds the following components and functionalities: • Catalogs. These include AI/ML models trained offline, xApps, and datasets that can be used to train and test data-driven solutions. • Intent-driven Auto-AI Engine. This is a fully-customizable software component that parses training intents defined in JSON format and automatically extracts the necessary data from datasets in the catalog. Once the training process is instantiated, AI solutions are trained based on the intent, and the AI training is configured to match the desired inputs, outputs, and goals. It also offers web-based dashboards (e.g., TensorFlow’s TensorBoard [90]) to monitor the training phase. • xApp On-boarding Module. Templates and pipelines are provided to convert trained AI solutions into xApps that are subsequently published to the catalog. These xApps are composed of two interconnected units, namely a service model connector and a data-driven logic unit. The former is responsible for handling the communication with the near-real-time RIC (e.g., extracts live KPIs from the RAN nodes or sends control actions, via the E2 termination), while the latter is tasked with embedding trained AI solutions [10]. • xApp Dispatcher Module. This module automates the instantiation of xApps in order to facilitate their testing process. This is done through scripts that automatically create Docker containers of the xApps to dispatch and deploy them on the near-real-time RIC. • Integration with OpenRAN Gym and Colosseum. PandORA is seamlessly integrated with OpenRAN Gym [10] and Colosseum [21] to enable end-to-end O-RAN-compliant testing of xApps in a reproducible RF environment. Via OpenRAN Gym, PandORA also offers data collection and analytics functionalities that are useful to validate xApp behavior and evaluate their performance under diverse conditions and deployments. 20

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN In a nutshell, PandORA users (e.g., telco operators, xApp developers) can generate an intent in JSON format (Step 1 in Fig. 2.2) by specifying the control objective (e.g., maximizing a certain set of KPMs for each slice), desired control parameters and observable inputs (e.g., throughput measurements, Channel Quality Information (CQI), to name a few). PandORA parses the intent (Step 2) and: (i) produces a dataset to be used to train the AI algorithms (Step 3); (ii) configures the AI algorithm to be trained, e.g., a DRL agent based on the Proximal Policy Optimization (PPO) architecture controlling slicing policies to maximize a certain reward function (Step 4); and (iii) trains the AI solution while offering a dashboard to monitor the training phase (Step 5). Once the training phase is complete, the AI/ML models are published to the Catalog, and PandORA integrates with OpenRAN Gym [10] to convert the trained AI solution into an xApp (Step 6), which then gets published to the respective xApp Catalog (Step 7). Once the publishing is complete, the xApp can be tested on Colosseum under one or more Colosseum scenarios (Step 8), and testing data is collected and stored to enable performance evaluation, xApp validation and data analytics (Step 9). PandORA is developed in Python using Tensorflow libraries [90]. Although it supports the training of any type of AI/ML model, for the scope of this work, we primarily focus on training DRL agents which are the state-of-the-art for intelligent decision making in O-RAN [75, 91, 92]. Following O-RAN specifications and requirements [1], PandORA trains AI solutions offline using an extended and customized version of the tf agents library [93] where the configuration of the training environment, the DRL agents and the datasets are generated at run-time starting from a JSON-based intent file. A simplified example of a minimal intent document is shown in Listing 2.1. The example shows how it is possible to specify the slices to be controlled by the agent, the actions that can be controlled, the KPIs for each slice that constitute the observation of the state, and which should be considered to compute the reward. PandORA intents use a modular approach where PandORA users can specify slice-specific rewards and set a global reward to combine them. In Listing 2.1, we illustrate a case where a DRL agent is trained to maximize both the average throughput and number of transmitted packets in the Downlink (DL) for the eMBB slice (Lines 4–9), while maximizing the average number of DL transmitted packets for the mMTC slice (Lines 10–15) and minimizing the maximum DL buffer occupancy for the URLLC slice (Lines 16–21).1 These three slice-specific rewards are then combined using a global reward that aims at 1

Note that the weight associated to the URLLC slice is configured in Line 25 and has negative value. Therefore, the MaxElemReward function in Line 18 effectively results in minimizing the maximum buffer occupancy. It is worth mentioning that the eNB does not have direct access to end-to-end latency measurements, therefore we use the buffer occupancy as a proxy for latency [26].

21

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN maximizing the weighted sum of the individual reward contributions for each slice (weights are defined at Line 25). Users can specify a general control action space (e.g., controlling RAN slicing policies and scheduling profile as shown in Line 23) without the need to specify the explicit values of these control actions. At run-time, PandORA parses the JSON-formatted intent and generates a dataset that fulfills the intent. This dataset includes all necessary KPMs required to compute observations and rewards. Additionally, it generates the proper action space from the available datasets. For instance, PandORA processes these datasets, automatically determining the available scheduling profiles (e.g., RR, WF, and PF as described in Section 2.2.2.1), along with the RAN slicing policies suitable for the given slices. More advanced PandORA users can also specify the subset of actions that must be considered by the agent. For example, they can force the agent to only consider PF and RR schedulers and exclude WF. Apart from offering modules to generate custom reward functions, PandORA provides a set of pre-defined reward functions that can be selected and combined to generate a global reward. This includes rewards aimed at maximizing or minimizing specific metrics through average, maximum, minimum, and median values. It enables the prioritization of certain KPMs by configuring weights, as demonstrated in Line 25. PandORA also offers pre-configured slice configurations with tailored rewards and observable KPMs that users can utilize. These configurations serve as default values for each slice, and they can be overridden by the user. It is also worth noting that intents, slice configurations, rewards, actions, and observation configurations are modular and can be combined to generate new intents. In this way, new intents specific to each slice are created, which are later combined, providing further flexibility to PandORA and its users. Via tf agents [93], PandORA provides access to a range of DRL agents, including DQN, PPO, Deep Deterministic Policy Gradient (DDPG), Twin Delayed Deep Deterministic Policy Gradient (TD3), and various others. Users can utilize PandORA to select the type of agents they want to use, along with specifying hyperparameters. Once the selection is complete, PandORA configures the agents at run-time, adjusting action and observation spaces as well as rewards based on the specified intent. Similarly to intents, users have the flexibility to customize their DRL architecture and hyperparameters or use default values pre-configured in PandORA. Listing 2.1: A simplified example of a JSON intent for PandORA. 1

{

22

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

"intent": {

2

"slices": [

3

{

4 5

"name": "embb",

6

"reward": "MaxAverageReward",

7

"reward_KPIs": ["dl_brate", "dl_tx_pkts"],

8

"observation_KPIs": ["dl_buffer", "dl_tx_pkts"]

9

},

10

{

11

"name": "mmtc",

12

"reward": "MaxAverageReward",

13

"reward_KPIs": ["dl_tx_pkts"],

14

"observation_KPIs": ["dl_brate", "dl_tx_pkts"]

15

},

16

{

17

"name": "urllc",

18

"reward": "MaxElemReward",

19

"reward_KPIs": ["dl_buffer"],

20

"observation_KPIs": ["dl_buffer", "dl_brate"] }

21 22

],

23

"actions": ["scheduling", "ran_slicing"],

24

"global_reward_type": "NestedSumWeightedReward",

25

"global_reward_weights": [0.5, 0.25, -1] }

26 27

}

2.2.2.3

DRL Agent Architectures tested in this work To control RAN slicing, and scheduling, we focus on data-driven approaches that rely on

RL. In RL, an agent iteratively interacts with the environment by performing observations, in order to learn the optimal control policy that maximizes the desired cumulative reward. More specifically, the agent explores the environment and takes actions in several environmental states, without knowing a priori which actions are more beneficial, and eventually learns the best policy through experience. The reward is a metric that defines the effectiveness of an action, while a sequence of states, actions, and rewards that result in a terminal state is called an episode. State Space S, represents all the 23

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN possible states of the environment s ∈ S, while Action Space A defines all the feasible actions a ∈ A that can be taken by the agent. Finally, in this work, we focus on DRL agents due to their ability to learn directly from experience, without relying on pre-existing models or explicit knowledge of the wireless environment [94–96] and therefore of its complex and dynamically changing conditions. Moreover, we focus on discrete actions and, for this reason, we limit our analysis to PPO [97] and DQN [98] architectures which are two state-of-the-art algorithms respectively for on-policy and off-policy DRL for discrete action spaces. PPO has been demonstrated several times to outperform other architectures [11, 74]. It is based on an actor-critic network architectural approach, where the actor and critic network “work” cooperatively to learn a policy that selects actions that deliver the highest reward possible for each state. While the actor’s task is to take actions based on current network states, the critic’s target is to evaluate actions taken by the actor network and provide feedback that reflects how effective the action taken by the actor is. In this way, the critic helps the actor in taking actions that lead to the highest rewards for each given state. The Clipped Surrogate Objective function used by PPO is defined in Eq. (2.1): h



LCLIP (θ) = Êt min qt (θ)Ât , clip (qt (θ), 1 − ϵ, 1 + ϵ) Ât

i

,

(2.1)

where Êt represents the empirical average; Ât is the estimator of the advantage function denoted as At , which assesses how well an action performed compared to the expected performance under the current policy; and qt (θ) = ππθ θ (a(at |st |st )t ) . This ratio represents the probability of taking action at at old

state st following the current policy πθ , divided by the respective probability when following the previous policy (i.e., πθold ). When multiplied by the estimator of the advantage function at time-step t (i.e., Ât ), the unclipped part of Eq. (2.1) is obtained, as shown in Eq. (2.2), where Conditional Policy Improvement (CPI) stands for the technique leveraged to avoid large policy updates [99]: " CPI

L

(θ) = Êt

#

h i πθ (at | st ) Ât = Êt qt (θ)Ât . πθold (at | st )

(2.2)

Since maximizing Eq. (2.2) directly would result in large policy updates, by clipping the probability ratio, we constrain the surrogate objective function (i.e., Eq. (2.1)) and remove the incentive to move the probability ratio (i.e., qt (θ)) outside the interval [1 − ϵ, 1 + ϵ]. ϵ is a hyperparameter that determines the clip range and is chosen in a way to ensure that the policy update will not be too large, indicating a significant divergence between the old and the current policy. Lastly, we take the minimum of the clipped and non-clipped objectives. Therefore, the final objective serves as a lower pessimistic bound of the unclipped objective. This indicates that we select either the clipped or the 24

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN non-clipped objective based on qt (θ) and the advantage. The final form of the Clipped Surrogate Objective Loss for the actor-critic implementation of PPO is given in Eq. (2.3). It is a combination of the Clipped Surrogate Objective function (i.e., LCLIP ), the squared-error value loss function (i.e., t 

targ LVF t (θ) = Vθ (st ) − Vt

2

) which is the difference between predicted and target cumulative

reward estimations, and the entropy bonus (i.e., S ′′ ), with the latter ensuring sufficient exploration, while c1 , c2 are control parameters: h

i

′′ LCLIP+VF+S (θ) = Êt LCLIP (θ) − c1 LVF t t t (θ) + c2 S [πθ ] (st ) .

(2.3)

In addition, based on a comparative analysis between models with diverse number of layers and neurons, the actor and critic networks we select for this work both consist of fully-connected neural networks with 3 layers of 30 neurons each. The hyperbolic tangent serves as the activation function while the learning rate is set to 10−3 . The DQN algorithm leverages Q-learning principles to estimate the Q-function. The Q-function represents the expected discounted cumulative reward obtained by taking action a in state s and then following the policy π. The optimal state-action pair is computed using the Bellman Equation, defined in Eq. (2.4): 

Q∗ (s, a) = r + γ max Q∗ s′ , a′ ′ a



.

(2.4)

This equation is approximated by a DQN that calculates the cumulative future reward, denoted as rt at time t, and discounted by a factor γ ∈ [0, 1], while s′ and a′ represent the state and the action taken in the next time-step. In addition, the algorithm uses a replay buffer to store experience and to cope with instabilities caused by correlation between consecutive episodes. The set of experiences is denoted as Dt = {e1 , . . . , et }, where each e = (s, a, r, s′ ) is a tuple that represents the state, action, reward, and the next state. The system transitions to the next state s′ after the agent takes an action a at each time-step t during the training phase, while the experience vector is stored in the replay buffer. In order to reduce the correlation between the Q-function value and the optimal Q∗ value, a second Q-Network is employed, namely the target Q-Network. Both the main and target Q-Network share the same structure, while the weights of the latter are periodically updated to match those of the former. During the training phase, the replay buffer is used to randomly select batches of previous experiences. These experiences are then utilized to calculate the DQN weights, denoted as θt in Eq. (2.5). This calculation involves minimizing the loss function using Stochastic Gradient Descent (SGD) method. Concludingly, the loss equation leveraged during the Q-Learning update at iteration

25

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN t is given by Eq. (2.5): "

Lt (θt ) = Es,a,r,s′

r + γ max Q ′ a



s , a ; θt− ′

′



2 #

− Q (s, a; θt )

,

(2.5)

where θt− are the weights of the target Q-Network, and γ is the discount factor that prioritizes instantaneous rewards against long-term rewards. Additionally, the ϵDQN -greedy is employed, where the DRL agent’s choice of action depends on the parameter ϵDQN ∈ [0, 1]. In detail, with a probability of 1 − ϵDQN , an action computed by the DQN is selected. Conversely, with a probability of ϵDQN , the agent selects a random action from the action space A. This strategy effectively balances the exploration of new actions, randomly selected, and the exploitation of the agent’s learned knowledge, based on actions selected by the DQN. In our analysis, the latter is implemented as a feed-forward multi-layer Neural Network (NN), consisting of 5 hidden layers of 50 neurons each. The learning rate is set to 10−3 , while the discount factor is γ = 0.95. Finally, the replay buffer memory size is set to 10,000, while ϵDQN = 0.1. We follow the same approach as in [11], where the input (e.g., KPMs) of the DRL agent is first processed by the encoding part of an autoencoder for dimensionality reduction. This also synthetically reduces the state space and makes training more efficient in terms of time and generalization. In detail, the autoencoder converts an input matrix of K = 10 individual measurements of M = 3 KPM metrics (i.e., DL throughput, buffer occupancy, and number of transmitted packets) into a single M -dimensional vector. The Rectified Linear Unit (ReLU) activation function and four fully-connected layers of 256, 128, 32 and 3 neurons are also used in the encoder. Although the channel state information (e.g., CQI, Modulation and Coding Scheme (MCS)) is not directly fed to the AI/ML agents, it is worth mentioning that its effect on performance is indirectly captured by the KPIs used to feed the DRL agents (e.g., Throughput, Buffer Occupancy, Transmitted Packets), which has been shown to be sufficient to implicitly capture channel effects on network performance [11]. In the case of the joint-slice optimization, the cumulative average reward function of the DRL agent is designed to jointly satisfy the QoS demand of the three slices with respect to their KPM requirements. For instance, eMBB users aim to maximize throughput, while mMTC users aim at maximizing the number of transmitted packets. Finally, the goal of URLLC users is to deliver packets with minimum latency. Since the base station cannot measure end-to-end application-layer latency (which is instead measured at the receiver side), we measure latency in terms of number of bytes in the transmission buffer, the smaller the buffer, the smaller the latency. The reward is

26

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN OpenRAN Gym

T1

T2 xApp 1

OpenRAN Gym

xApp 2

sched

Deployed xApps

Colosseum SCOPE

Colosseum Near-real-time RIC

Colosseum collaboration network

SRN UE 1 SRN UE 2 SRN UE 3

Mobility, path loss, fading, interference

Colosseum SRN server

Colosseum SRN server

Massive Channel Emulator (MCHEM)

…

Docker RIC cluster

PDCP RLC MAC PHY

RRC

slicing

USRP X310

sched-slicing

SDR with hardware-in-the-loop

RAN E2 Termination

xApps Catalog

Wireless channel emulation

DU

E2 Termination

RIC Platform

SRN UE 6

Traffic Generator (TGEN) eMBB, URLLC, mMTC

Colosseum traffic network

Figure 2.3: Reference O-RAN testing architecture with focus on the case of two xApps operating at different time scales, Ti , as described in Section 2.2.5.2. formulated as the weighted sum in Eq. (2.6): R=

∞ X



γt 

t=0

N X



wj · rj,t  ,

(2.6)

j=1

where t represents the training step, and N = 3 is the total number of slices, wj represents the weight associated to slice j, considered for reward maximization in the three corresponding slices. Finally, γ is the discount factor and rj,t describes the slice-specific reward obtained at each training step t. In our case, rj,t represents the average value of the KPM measured by all users of slice j at time t (e.g., throughput for the eMBB slice). Note that the weight wj for the URLLC slice is negative to model the minimization of the buffer occupancy. The models that we have designed and trained are deployed as xApps on the near-real-time RIC, as illustrated in Fig. 2.3. In the case of the per-slice optimization, the respective weighted average reward function of the DRL agent is given as follows in Eq. (2.7): R=

∞ X

γ t (wj · rj,t ) .

(2.7)

t=0

The models that we have designed and trained are deployed as xApps on the near-real-time RIC, as illustrated in Fig. 2.4.

27

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN OpenRAN Gym

RIC Platform

Slicing mMTC

URLLC

Sched

Sched

Sched

eMBB

mMTC

URLLC

slicing sched DRL Architecture PPO/DQN

Deployed xApps

Docker RIC cluster Colosseum Near-real-time RIC Colosseum SRN server

T

PDCP RLC MAC PHY

RRC

eMBB

DU RAN E2 Termination

xApps Catalog

E2 Termination

OpenRAN Gym

T

Colosseum SCOPE Colosseum SRN server

Colosseum collaboration network Figure 2.4: Reference O-RAN testing architecture with focus on the case of four xApps operating at time scale, T , as described in Section 2.2.5.3.

2.2.3

DRL Optimization Strategies Our goal is to investigate how different design choices affect the effectiveness and decision-

making of the DRL-based xApps. For this reason, we consider the following design choices, which resulted in the generation and testing of a total of 23 xApps. Short-term vs. Long-term Rewards. We train DRL agents with different values of the discount factor γ. The PPO discount factor weights instantaneous rewards against long-term rewards. A higher value prioritizes long-term rewards, while a lower γ prioritizes short-term rewards. Results of this exploration are provided in Section 2.2.5.1. Hierarchical Decision-Making. We investigate the case of two xApps configuring different parameters in parallel but at different timescales. In this way, we investigate how multiple xApps with different optimization goals and operating timescales impact the overall network performance. The findings of this investigation are provided in Section 2.2.5.2, and a practical example is illustrated in Fig. 2.3. Per-Slice Scheduling Profile Selection. We explore a hierarchical decision-making configuration of four xApps operating simultaneously at the same time granularity and reconfiguring the BS’s control parameters. In detail, one slicing xApp simultaneously allocates PRBs to all slices. 28

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN The remaining three xApps are exclusively assigned one slice each and select a dedicated scheduling profile for each slice. In this way, we aim to examine the effects of multiple xApps operating at the same timescale, and to compare a widely used state-of-the-art off-policy architecture (i.e., the DQN), with its on-policy counterpart (i.e., the PPO algorithm). A practical example is depicted in Fig. 2.4, while the results of this investigation are presented in Section 2.2.5.3. Impact of Reward’s Weights. We test different values for the weights wi of the slices in Eq. (2.6). A different weight configuration affects how DRL agents prioritize each slice. The results of this analysis are reported in Section 2.2.5.4, where we show how weights significantly impact the overall performance and can result in inefficient control strategies. Effect of RAN Control Timers. Finally, we aim at understanding how different RAN control timers (i.e., different periodicities for RAN telemetry, report and control) will have an impact on the decision-making process and how different control sets will prioritize one slice over the other. Specifically, we look into three control timers namely the KPM log time, the DU Report Timer, and the action update time, i.e., how frequently the actions sent back by the RIC get updated and enforced by the BS. The reported findings of this strategy are provided in Section 2.2.5.5. It is reminded that the action set A consists of both scheduling and RAN slicing policies. In our analysis, we consider the cases where an agent can control either scheduling or slicing decisions individually, or control both jointly. The state S is represented by the output of the autoencoder which is used to compress input KPIs collected over the E2 interface and convert them into latent representations. Finally, although in our analysis we evaluate diverse reward designs that consider long-term and short-term goals, and combine diverse target KPIs for each slice, the general form of the reward R considered in all of our DRL agents is defined in Eq. (2.6).

2.2.4

Experimental Setup and DRL Training Table 2.1: Traffic Profiles Profile Name

eMBB [Mbps]

mMTC [kbps]

URLLC [kbps]

Profile 1

1

30

10

Profile 2

4

44.6

89.3

To experimentally evaluate the DRL agents, we leverage the capabilities of OpenRAN Gym [10], an open-source experimental toolbox for end-to-end design, implementation, and testing

29

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN of AI/ML applications in O-RAN. It features: • End-to-end RAN and core network deployments through the srsRAN [41] softwarized opensource protocol stack; • Large-scale data collection, testing and fine-tuning of RAN functionalities through the SCOPE framework [5], which adds open APIs to srsRAN for the control of slicing and scheduling functionalities, as well as for KPMs collection; • An O-RAN-compliant control architecture to execute AI/ML-based xApps via the ColO-RAN near-real-time RIC [11]. The E2 interface between RAN and the RIC and its Service Models (SMs) [1] manage streaming of KPMs from the RAN and control actions from the xApps. Table 2.2: Weight Configurations Weights

eMBB

mMTC

URLLC

Default

72.0440333

0.229357798

0.00005

Alternative

72.0440333

1.5

0.00005

We deploy OpenRAN Gym on Colosseum [21], a publicly available testbed with 128 SRNs, i.e., pairs of Dell PowerEdge R730 servers and NI USRP X310 SDRs. Colosseum enables large-scale experimentation in diverse RF environments and network deployments. The channel emulation is done through the MCHEM component, which leverages FPGA-based Finite Impulse Response (FIR) filters to reproduce different conditions of the wireless environment (e.g., path loss, fading, attenuation, interference of signals) modeled a priori through ray-tracing software, analytical models, or real-world measurements. Similarly, the Colosseum Traffic Generator (TGEN), built on top of the Multi-Generator (MGEN) TCP/UDP traffic generator [100], emulates different traffic types, demand, and distributions (e.g., Poisson, periodic). We deploy a 3GPP-compliant cellular network with one base station and 6 UEs uniformly distributed across 3 different slices. These are: (i) eMBB that concerns high traffic modeling of highquality multimedia content and streaming applications; (ii) URLLC for time-critical applications, such as autonomous driving in V2X scenarios; and (iii) mMTC for IoT devices with low data rate requirements but with high need for consistent information exchange. In terms of physical deployment, we consider two different topologies both located in the urban environment of Rome,

30

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN Italy [5]. Specifically, Location 1 concerns UEs uniformly distributed within a 50 m radius from the BS, while in Location 2, the corresponding UEs are placed within a 20 m radius from the BS. The bandwidth of the BS is set to 10 MHz (i.e., 50 PRBs) and is divided among the 3 slices, with 2 users statically assigned to each slice. Slice-based traffic is generated following the specifications reported in Table 2.1. Specifically, we consider two different traffic profile configurations (i.e., Profile 1 and Profile 2) with different source bitrates. However, for both profiles, we consider a constant bitrate traffic for eMBB users, while URLLC and mMTC UEs generate traffic based on a Poisson distribution. Table 2.3: Per-Slice Top performing xApps under the Default Weight Configuration eMBB

mMTC

1)

Sched & Slicing 0.5

Slicing 0.99

2)

Sched & Slicing 0.99

Slicing 0.5

3)

Slicing 0.5

Sched 0.99

4)

Slicing 0.99

Sched & Slicing 0.99

5)

Sched 0.99

Sched 0.5

6)

Sched 0.5

Sched & Slicing 0.5

To train the DRL agents, we used the publicly available dataset described in [11]. This dataset contains about 8 GB of KPMs collected by using OpenRAN Gym and the Colosseum network emulator over 89 hours of experiments, and concerns setups with up to 7 base stations and 42 UEs belonging to different QoS classes, and served with heterogeneous scheduling policies. Each DRL model evaluated in the following sections takes as input RAN KPMs such as throughput, buffer occupancy, number of PRBs, and outputs resource allocation policies (e.g., RAN slicing and/or scheduling) for RAN control. Abiding by the O-RAN specifications [101], which do not permit the deployment of untrained data-driven solutions, we train our ML models offline on Colosseum’s GPU-accelerated environment, which includes two NVIDIA DGX A100 servers with 8 GPUs each. Notably, AI/ML solutions should be trained offline to avoid actions that can potentially lead to performance degradation in the network [1]. Then, the trained DRL-agents are onboarded on xApps inside softwarized containers implemented via Docker and deployed on the ColO-RAN near-real-time RIC.

31

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

Table 2.4: Hierarchical Reporting Setup

2.2.5

Setup ID

Slicing 0.5

Sched 0.99

1

1s

10 s

2

1s

5s

3

10 s

1s

4

5s

1s

In-Sample Experimental Evaluation In this section, we present the results of an extensive performance evaluation campaign,

with more than 38 hours of experiments in Colosseum, to profile the impact of the strategies discussed in Section 2.2.3. These results were produced by taking the median as the most representative statistical value of a dataset, and averaged over multiple repetitions of experiments in the DL direction of the communication system. Last, all the experiments of the in-sample experimental evaluation campaign concern Location 1 which is the configuration used to collect training data, and all the UEs are assumed static. Out-of-sample evaluation, i.e., testing the DRL agents against data collected in an entirely different network deployment, will be the focus of Section 2.2.6. 2.2.5.1

Impact of Discount Factor on the Action Space We explore how RAN slicing, MAC scheduling, and joint slicing and scheduling control

are affected by training procedures that favor short-term against long-term rewards. All the reported results were obtained with γ ∈ {0.5, 0.99}, while the reward’s weight configuration is shown in Table 2.2 and identified as Default. In Fig. 2.5, we report the Cumulative Distribution Function (CDF) of individual KPMs for each slice and for different xApps trained to control different sets of actions and using different values of γ. The median of such measurements for the eMBB and mMTC slices is instead reported in Fig. 2.6. The median for the URLLC slice is not reported, as this value is zero in all configurations. The best performing configurations for the eMBB and mMTC slices are instead listed in numerical order in Table 2.3 from best to worst performing. Our results show that Sched & Slicing and Slicing 0.5 favor eMBB the most, with Sched & Slicing 0.5 being the best configuration among the ones considered. Moreover, slicing is essential to ensure high throughput values (the four top-performing xApps for eMBB include

32

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.5: Performance evaluation under different action spaces and values of the γ parameter with the PPO DRL Architecture.

(a) eMBB DL Throughput

(b) mMTC Packets

Figure 2.6: Median values under different action spaces and values of γ with the PPO DRL Architecture.

33

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN slicing as a control action). We also notice that prioritizing immediate rewards (i.e., γ = 0.5) results in higher throughput values if compared to xApps embedding agents trained to maximize long-term rewards. This design option, when combined with a bigger action space (e.g., scheduling & slicing) ultimately yields a higher throughput. For the mMTC slice, the Slicing 0.99 xApp always yields the best performance. However, we notice that Sched & Slicing 0.5, which is the best-performing xApp for eMBB, yields the worst performance for mMTC. Although a larger action space and a short-term reward design are ideal for eMBB (e.g., Sched & Slicing 0.5), we notice that this performance gain comes at the expense of the mMTC slice. Indeed, in Figs. 2.6a and 2.6b, we observe that the higher the eMBB performance, the lower the mMTC’s. This is clearly illustrated when we compare the “best” per-slice policies, respectively Sched & Slicing 0.5 (eMBB) and Slicing 0.99 (mMTC). The former delivers the highest reported eMBB throughput (4.168 Mbps) but the lowest number of mMTC packets (16 packets), while the latter delivers the highest number of mMTC packets (37 packets) and one of the lowest measured eMBB throughput values (i.e., 3.636 Mbps). Hence, eMBB-mMTC slices indicate a competitive behavior, since we cannot optimally satisfy both of them without loss in their respective rewards, as they compete for the amount of packets required for transmission. Our results show that, in general, controlling scheduling only is not ideal as it strongly penalizes eMBB performance with a modest improvement in terms of number of transmitted mMTC packets. 2.2.5.2

Impact of Hierarchical Decision-Making In this analysis, we evaluate the effectiveness of making disjoint decisions to control

scheduling and slicing policies. We select the best performing single-action xApps from Table 2.3, i.e., Slicing 0.5 and Sched 0.99, and we compare their execution at different timescales. The former, provides a good balance in terms of eMBB throughput (∼ 4 Mbps) and number of mMTC packets, while the latter, provides the best performance for the mMTC slice. With this design choice, we expect to maintain high performance for both eMBB and mMTC.

34

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

Table 2.5: Feasible PRB Allocation eMBB

mMTC

URLLC

30

9

11

30

15

5

36

9

5

24

21

5

24

15

11

18

15

17

18

9

23

18

21

11

12

27

11

12

15

23

12

9

29

6

27

17

6

39

5

6

15

29

6

9

35

36

3

11

We consider four setups, summarized in Table 2.4. Each entry describes how frequently the BS reports KPMs to the RIC. For instance, in Setup 1, the xApp for slicing control receives data from the BS every 1 s, while the scheduling agent receives the respective metrics every 10 s. Despite taking into account RAN telemetry reported every 1, 5 or 10 s, the DRL decision-making process and the enforcement of a control policy on the BS occur within a granularity of tens of milliseconds, and hence the intelligent control loops are still in compliance with the timescale requirements of the near-real-time RIC. Results of this analysis are presented in Figs. 2.7 and 2.8. From Fig. 2.7a, Setup 3 delivers the best eMBB performance, Setups 1 and 2 perform almost equally, while Setup 4 performs the worst. For mMTC, in Fig. 2.7b we notice that all combinations perform similarly and deliver approximately 26 packets, with Setup 1 and Setup 4 delivering an additional packet. From Fig. 2.7c, we notice that all setups deliver the same performance for the URLLC slice and, despite not being reported in the figures, they all yield a median buffer occupancy of 0 byte, i.e., they maintain an empty buffer to ensure low latency values. In Fig. 2.8a, we notice that Setups 2 and 3 deliver the highest

35

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.7: Performance evaluation under different hierarchical configurations with the PPO DRL Architecture.

(a) eMBB DL Throughput

(b) mMTC Packets

Figure 2.8: Median values under different hierarchical configurations with the PPO DRL Architecture.

36

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN eMBB throughput. In Fig. 2.8b, instead, we notice that Setups 1 and 4 deliver the highest number of transmitted mMTC packets. Our findings on hierarchical control verify eMBB’s and mMTC’s competitive behavior for individual reward maximization. Our results show that the rewards of eMBB and mMTC slices are competing with one another, as the best configuration for eMBB corresponds to the worst configuration for mMTC, and vice versa. Among all considered configurations, Setup 3 offers the best trade-off, as it delivers the highest throughput at the expense of a single mMTC packet less being transmitted. 2.2.5.3

Impact of Per-Slice Scheduling Profile Selection We extend our prior work [26] by evaluating the coexistence of a multitude of xApps

delivering services to an Open RAN multi-slice scenario. Specifically, we consider the case where a single xApp distributes the available PRBs to all three slices, while three xApps are each tasked with selecting a dedicated scheduling profile for a single slice. All four xApps operate at the same timescale as given in Set 1 of Table 2.9, where KPMs and actions are both logged and updated every 250 ms respectively, while fresh RAN data are reported by the DU within a granularity of 1 s. We begin the evaluation by focusing on the DQN algorithm and the case of the joint-slice optimization. All results were produced with agents trained under the Default weight configuration of Table 2.2 and with a discount factor of γ = 0.95. The findings presented in Figs. 2.9 and 2.10 indicate that in the eMBB slice, DQN underperforms compared to PPO for both γ ∈ {0.5, 0.99}, with only Slicing delivering the highest reported throughput value at 2.733 Mbps, as reported in Fig. 2.10a. With respect to mMTC, Figs. 2.9b and 2.10b depict a similar performance to the one shown in Figs. 2.5b and 2.6b, when examining the actions separately under their respective discount factor (i.e., γ ∈ {0.5, 0.99}). In detail, we observe that in both cases, Slicing delivers the highest reported number of transmitted packets, followed by Sched, and Sched & Slicing. It is also shown that the Sched xApp, in the case of DQN, performs better in the mMTC compared to the case with PPO, delivering a median value of 32 transmitted packets, which is ∼ 28% more compared to Sched 0.99 with PPO, as shown in Fig. 2.6b. Finally, we observe that slicing is the action that delivers the best performance in both eMBB and mMTC. Indeed, the competitive behavior observed with PPO is mitigated in the case of DQN, as shown in Fig. 2.10, since the actions that deliver the highest eMBB throughput also deliver the highest number of mMTC transmitted packets. On the URLLC, all configurations once again delivered optimal performance, reporting a median buffer

37

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.9: Performance evaluation under the Default weight configuration for different action spaces and discount factor γ = 0.95 with the DQN DRL Architecture.

(a) eMBB DL Throughput

(b) mMTC Packets

Figure 2.10: Median values under the Default weight configuration for different action spaces and discount factor γ = 0.95 with the DQN DRL Architecture. 38

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.11: Performance evaluation with 4 xApps and per-slice scheduling profile selection under PPO and DQN Architectures.

(a) eMBB Throughput

(b) mMTC Packets

Figure 2.12: Median values obtained with a 4-xApp setup and per-slice scheduling profile selection under PPO and DQN Architectures.

39

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN occupancy of 0 byte. Both DRL architectures are model free, i.e., they do not rely on an explicit model of the environment. Instead, the DRL agent learns directly by interacting with the environment and makes decisions through trial and error. PPO employs a trust region method [97, 102] that helps in achieving more stable training, compared to DQN which can suffer from divergence and slow convergence [103]. The trust region ensures that the old and current policy do not deviate significantly, which helps mitigate divergence issues. Actor and critic networks calculate the state-value, with the critic network reducing variance in action value estimates. This reduction in variance makes policy updates more reliable and exploration safer [104]. On the contrary, DQN does not implement trust regions and uses the target network to achieve stability during training [94]. In environments affected by noisy observations and stochasticity, the aforementioned features, combined with the model’s ability to explicitly model the policy’s probability distribution over actions, make PPO capable of adapting more efficiently to varying conditions. This ability to generalize better across tasks and environments can be achieved with DQN but might require more fine-tuning and adjustments when transitioning to new tasks. Furthermore, DQN relies on epsilon-greedy exploration, which can be less efficient, especially in complex environments [105, 106]. PPO, being an on-policy algorithm, manages to optimize the current policy directly based on the most recent experiences. On the other hand, DQN, an off-policy algorithm, estimates the value of actions independently of the current policy. On-policy methods often work better when the policy is changing during training, especially in stochastic environments. Therefore, in the latter case where rapid policy changes are observed, off-policy methods may struggle to deal successfully with them, and they may require more extensive replay buffers or further fine-tuning to handle noisy environments effectively [107–109]. Lastly, PPO has been shown to work more efficiently in problems with high-dimensional state and action spaces, as demonstrated in [110], owing to its policy-based and trust region approaches. In our work, the state and action spaces are high-dimensional. In detail, a total of 43 actions are considered when resource allocation is performed jointly for all slices. For scheduling, the size of the action space is 27, while for slicing, it is 16. The former size encompasses all possible combinations of scheduling policies for all three slices (i.e., RR, WF, and PF), while the combination for PRB allocation is given in Table 2.5 for a total of 50 PRBs. Finally, the observation of the state, obtained through srsRAN, consists of 10 independent measurements of 3 KPIs, i.e., DL buffer occupancy, the DL throughput and the number of transmitted packets on the DL. From the previous discussion, we have seen that DQN performs poorly in the highdimensional state and action spaces, such as the ones considered in this work. Therefore, we test its 40

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN effectiveness in a reduced action space where the DQN agent only controls scheduling decisions. Specifically, we consider the case of one Slicing xApp that serves all slices simultaneously, while the remaining 3 xApps are dedicated to each slice and control the scheduling profile for the corresponding slice only. For instance, one xApp selects a scheduling profile for the eMBB slice, another one focuses on the mMTC slice, and the last one controls the URLLC slice only. This configuration allows for both testing and mixing different DRL architectures, such as PPO and DQN. It also enables experimentation with a setup in which the scheduling selection action follows the PRB allocation. Therefore, the preceding xApps will reconfigure a network where the action space is altered after the changes made by the Slicing xApp. In Figs. 2.11 and 2.12, we present the results obtained from testing a total of 4 xApps that perform resource allocation. For the purposes of the experiments we consider two cases, namely Case A and Case B. In both cases, the Slicing xApp embeds a DRL agent trained with PPO under the discount factor, γ = 0.5. In Case A, the remaining Sched xApps also embed PPO agents trained with γ = 0.5. In Case B, the xApps embed DQN agents with a discount factor of γ = 0.95, as described in Section 2.2.2.3. In Fig. 2.11, we observe that Case A performs better for the eMBB slice. Indeed, the results reported in Fig. 2.12 indicate that Case A achieves a median eMBB throughput value of 4.024 Mbps, which is ∼ 6.5% higher than that of Case B. Regarding the mMTC slice, the median for the corresponding KPM metric was measured at 27 transmitted packets, representing an increase of ∼ 60% compared to Case A. For URLLC, both cases delivered optimal performance with a median buffer occupancy of 0 byte. We notice that Case A performs similarly to the case of jointly performing slicing and scheduling control with γ = 0.5, as shown in Figs. 2.5 and 2.6, by delivering one additional mMTC packet, as depicted in Fig. 2.12. With regard to Case B, the obtained results closely match those reported with Hierarchical Control under Setup 1 in Fig. 2.8. Precisely, as depicted in Fig. 2.12b, the two configurations achieve identical performance on the mMTC slice, by achieving the same median value of transmitted packets (i.e., 27 packets). For the eMBB slice, the setup with 4 xApps (i.e., Case B) exhibits a minimal drop of ∼ 0.66% in the median throughput value compared to the performance achieved with Setup 1. Therefore, Case B provides a more flexible and heterogeneous setup where different DRL agents can coexist, without a significant performance degradation. Furthermore, the DQN can still be leveraged in those cases where agents independently optimize the performance of each slice thanks to the reduced action space.

41

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

Table 2.6: Weight Design weM BB αeM BB ·

2.2.5.4

wmM T C 1 A

βmM T C ·

1 B

wU RLLC   1 γU RLLC · − C

Impact of Weight Configuration

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.13: Performance evaluation under the Alternative weight configuration for different action spaces and discount factors with the PPO DRL Architecture. In this study, we consider different weight configurations to compute the cumulative average reward function in Eq. (2.6). The considered configurations are reported in Table 2.2. The Alternative weight configuration is computed by using the weights in Table 2.6, where A, B, C, αeM BB , βmM T C , and γU RLLC are used to both scale and prioritize certain slices. Specifically, A, B, C are used to scale the individual weights according to statistical information of corresponding KPMs. For example, A, B, C can represent either the average, minimum or maximum values reported KPM per slice

42

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB DL Throughput

(b) mMTC Packets

Figure 2.14: Median values under the Alternative weight configuration for different action spaces and discount factors with the PPO DRL Architecture. so as to scale the weight according to the dynamic range of the corresponding KPM. Similarly, αeM BB , βmM T C , and γU RLLC can be used to give priority to one slice or the other. We set αeM BB = 1000, βmM T C = 456 and γU RLLC = 1. As a reference for A, B and C, we choose the historically maximum reported KPM values for each slice, i.e., A = 13.88 Mbps, B = 304, and C = 20186 byte. Based on these steps, we derive their respective weights weM BB , wmM T C , wU RLLC . For 1 example, the weight of mMTC can be computed as wmM T C = βmM T C · = 456/304 = 1.5, as B reported in the Alternative configuration in Table 2.2. The goal of comparing the two Default and Alternative weight configurations is to explore and understand the dynamics between mMTC and eMBB and the overall impact on the network performance. Specifically, since previous results have shown that the mMTC can be penalized by the eMBB slice, with the Alternative configuration we aim at giving the former a weight that is 6× larger than the Default configuration. Results for the Alternative configuration are reported in Figs. 2.13 and 2.14. In Fig. 2.13a, Sched & Slicing 0.5 delivers the best eMBB performance. Similarly to the results presented in Section 2.2.5.1, Sched & Slicing 0.5 and Slicing 0.5 are the best choices, with shortterm reward design being ideal for eMBB. In Fig. 2.13b, the Alternative weight configuration results in Sched & Slicing 0.99 being the best mMTC choice and long-term rewards are better for mMTC users. For URLLC, all policies perform well, with Sched & Slicing 0.5 performing

43

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN slightly better compared to Sched & Slicing 0.99. Figs. 2.14a and 2.14b confirm that controlling scheduling alone does not improve performance in general. Similarly to our previous analysis, a high eMBB performance (i.e., Sched & Slicing 0.5) results in a degraded mMTC performance. However, if compared with the Default, the Alternative weight configuration achieves a 31.25% increase for mMTC, with the same equally good URLLC performance and a 1% throughput increase for eMBB users. Table 2.7: Design Options Catalog Option 1

Sched & Slicing 0.5 - Alternative

Option 2

Slicing 0.5 - Default

Option 3

Hierarchical Control - Setup 1

Option 4

Slicing 0.99 - Default

In Table 2.7 we summarize the design options that have delivered a good overall performance so far. Table 2.8 indicates eMBB and mMTC’s dynamic and competitive relation. Option 2 brings balance, in terms of throughput and transmitted packets, Option 1 favors eMBB, and Option 4 boosts mMTC but with a significant decrease in the QoS of the eMBB slice. Table 2.8: Design Options

2.2.5.5

eMBB [Mbps]

mMTC [packet]

URLLC [byte]

Option 1

4.208

21

0

Option 2

4.114

26

0

Option 3

3.804

27

0

Option 4

3.636

37

0

Impact of RAN Control Timers Table 2.9: RAN Control Timers Control Time

Set 1

Set 2

Set 3

DU Report

1s

250 ms

100 ms

KPMs Log

250 ms

250 ms

100 ms

Action Update

250 ms

250 ms

100 ms

44

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.15: Performance evaluation under different action spaces, values of the γ parameter and sets of RAN control timers with the PPO DRL Architecture. We look into the case where we control three different RAN control timers, as reported in Table 2.9, with the focus primarily steered onto Sets 2 and 3. In the latter cases, the KPM collection granularity (i.e., KPMs Log time) matches both the Action Update and the KPM report granularities on the BS’s DU. In detail, KPMs are directly streamed into the RIC, upon collected, with an action being sent back through the E2 termination. This is in contrast with the case of Set 1, where the DU report time is equal to 1 s, and hence some of the freshly captured data may not be reported. The findings of this evaluation are presented in Figs. 2.15 and 2.16, and all configurations concern DRL agents trained under the Default weight configuration presented in Table 2.2. In Fig. 2.15a, it is observed that all configurations result in a comparable high performance, delivering a median throughput value of ∼ 4 Mbps (Fig. 2.16a) with the only exception of Sched 0.99 under Set 2 which delivers poor throughput performance. The aforementioned configuration yields a performance similar to the one reported in Fig. 2.6a, where Sched 0.99 is evaluated under the time granularities of Set 1. Indeed, this combination achieves the same number of transmitted packets (i.e., 25 packets) with a negligible drop on the eMBB throughput, which is now reported at 45

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB DL Throughput

(b) mMTC Packets

Figure 2.16: Median values under different action spaces and RAN control timers with the PPO DRL Architecture. ∼ 2.02 Mbps, a value approximately 5% less than the 2.125 Mbps reported with Set 1. Additionally, the Sched 0.99 xApp, when tested under Set 3, attains the maximum reported throughput value for controlling scheduling using a single xApp, (reaching 4.028 Mbps, a ∼ 99% increase compared to 2.02 Mbps achieved with Set 2). The latter is achieved while transmitting 23 packets, which is two packets fewer than the maximum number of transmitted packets ever recorded when controlling scheduling alone, and with PPO-implemented agents. Based on these findings, scheduling should be controlled using small time granularities (i.e., ∼ 100 ms) to ensure high eMBB throughput, while guaranteeing no performance degradation on the mMTC slice. Furthermore, the results illustrated in Figs. 2.15b and 2.16b indicate that both scheduling and slicing, as well as scheduling under Sets 2 and 3, can achieve the same performance on the mMTC. However, Sched & Slicing 0.5 delivers higher eMBB throughput, and hence, is preferred compared to Sched 0.99. In consistency with the reported findings so far, and when focusing on the three action profiles individually, we observe mMTC’s and eMBB’s competitive behavior (e.g., they heavily compete to gain radio resources). This trend is clearly illustrated in Figs. 2.15a, 2.15b, 2.16a, and 2.16b, where Slicing 0.5 under Set 3 achieves the highest throughput value at 4.195 Mbps, but the lowest number of transmitted packets (i.e., 18 packets). The reported results in Figs. 2.16a and 2.16b also indicate that Set 3 primarily boosts eMBB’s

46

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN performance, compared to Set 2 which is a better fit for mMTC UEs. Finally, in Fig. 2.15c, we note that all configurations for the investigated set of granularities deliver identical performance on the URLLC slice and, although not shown in the figures, they consistently maintain a median buffer occupancy of 0 byte to ensure low latency. It is also observed that both Sched & Slicing 0.5 and Slicing 0.5 under Set 2 perform slightly better on the aforementioned slice, compared to Sched 0.99 under Set 2, which performs slightly worse.

2.2.6

Out-of-Sample Experimental Evaluation Table 2.10: ML Agent & Network Condition Settings

Setting ID

UE Speed [m/s]

Traffic Load

RAN Control Timers

1

3

Profile 1

Set 1

Alternative/Default

2

3

Profile 2

Set 1

Alternative/Default

3

0

Profile 2

Set 1

Alternative

4

0

Profile 2

Set 2

Default

47

Weight Configuration

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.17: Performance evaluation under mobility with Settings 1 and 2 of Table 2.10 and the PPO DRL Architecture.

(a) eMBB Throughput

(b) mMTC Packets

Figure 2.18: Median values obtained under mobility with Settings 1 and 2 of Table 2.10 and the PPO DRL Architecture.

48

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.19: Performance evaluation focusing on the case of joint-slice optimization with a single xApp and the PPO DRL Architecture under Settings 3 and 4 of Table 2.10. In the Out-of-Sample experimental performance evaluation, we focus on the case of Location 2, and we test the framework’s performance in static and mobile scenarios under two different traffic loads. Recall that our agents have been trained using data from Location 1 only. Information regarding the traffic profiles and the UE mobility can be found in Tables 2.1 and 2.10. In Figs. 2.17 and 2.18, we present the results collected when testing the framework under mobile UE conditions and diverse traffic profiles for Settings 1 and 2 as defined in Table 2.10. We juxtapose the configuration that delivered the highest eMBB throughput (i.e., 4.208 Mbps), among those tested in this work, namely Option 1: Sched & Slicing 0.5 - Alternative with the case of two xApps which jointly optimize the performance of all slices, namely Option 2: Slicing 0.5, and Sched 0.99. Finally, the set of granularities which are considered for this evaluation are given by Set 1 of Table 2.9. In Figs. 2.17a, 2.17b, and 2.17c, we observe that under the traffic conditions of Setting 1, both configurations deliver almost identical performance. This is clearly depicted in Figs. 2.18a and 2.18b, where the mentioned xApps achieve the same median values for eMBB throughput and

49

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

Figure 2.20: Median values obtained when focusing on the case of joint-slice optimization with a single xApp and the PPO DRL Architecture under Settings 3 and 4 of Table 2.10. number of mMTC transmitted packets. However, when the two configurations are tested under Setting 2 and specifically under Traffic Profile 2, a slight performance degradation is observed in the case of joint-slice optimization with 1 xApp. In detail, Figs. 2.17a and 2.18a indicate that the performance of eMBB is optimized when using 2 xApps, resulting in ∼ 16% more throughput compared to the case with a single xApp. In the mMTC slice, both configurations deliver similar performance, as shown in Fig. 2.17b. However, when focusing on the former setup, it results in delivering two more packets as shown in Fig. 2.18b. On the URLLC, it is noted that most of the configurations under both Settings resulted in a median buffer occupancy of 0 byte. However, in the case of joint optimization with 1 xApp, this value was reported at 52 byte. Hence, in case of UE mobility, Sched & Slicing 0.5 - Alternative underperforms compared to joint optimization with Slicing 0.5, and Sched 0.99. Based on the reported findings, the simultaneous operation of two xApps can more successfully tackle possible UE disconnections caused by mobile conditions, and therefore enhance network performance. In Figs. 2.19 and 2.20 we include the results collected when testing the optimization framework under Settings 3 and 4 in Location 2. In this evaluation, we once again compare Option 1:

50

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN Sched & Slicing 0.5 - Alternative with Sched & Slicing 0.5 - Default. It is noted that the main difference between these two setups pertains to the fact that the former is evaluated under granularity Set 1, while the latter under Set 2. This comparison allows us to assess how collecting and reporting KPMs as well as updating actions at smaller timescales (i.e., 250 ms) will impact the effectiveness of the optimization framework, under the traffic load defined in Profile 2. Additionally, the latter xApp is chosen due to the fact that, among all configurations evaluated in Section 2.2.5.5 in terms of RAN control timers, it has managed to provide the most significant performance boost in the network. In detail, when tested under Set 2, it has improved the performance of mMTC by ∼ 56%, achieving a median value of 25 packets, as shown in Fig. 2.16b. This is in contrast to when it was evaluated under Set 1, where it delivered a median value of 16 packets (Fig. 2.6b). With regard to the eMBB slice, it still manages to maintain good overall performance, delivering a median throughput value of 4.140 Mbps (Fig. 2.16a). The results reported in Figs. 2.19a and 2.20a indicate that both configurations deliver similar performance in the eMBB slice, with Option 1: Sched & Slicing 0.5 - Alternative performing slightly better. However, on the mMTC slice, Sched & Slicing 0.5 - Default performs better, as depicted in Fig. 2.19b, by achieving a median value of 30 transmitted packets (Fig. 2.20b). URLLC UEs, once again, achieve zero latency (i.e., buffers are emptied), and, as a result, the respective figures are omitted.

51

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

2.2.7

Broader Evaluation of PandORA

(a) eMBB PRB Ratio

(b) mMTC PRB Ratio

(c) URLLC PRB Ratio

Figure 2.21: UE Satisfaction expressed in the form of PRB ratio for the Reward Designs of Table 2.11. The goal of this section is to provide additional insights into PandORA by: (i) considering a broader set of evaluation metrics aimed at shedding light on resource utilization and UE satisfaction; and (ii) testing across scenarios with a larger number of UEs. Specifically, in the latter case, we examine the performance of our xApps in a scenario where we add additional UEs to the mMTC slice. In line with the previous sections, we will evaluate PandORA in setups previously seen during the training process (i.e., in-sample experimental evaluation in Location 1), as well as unseen conditions (i.e., out-of-sample experimental evaluation in Location 2). All subsequent results concern joint optimization with Sched & Slicing 0.5 and are tested using Sets 1 and 2 of RAN control timers from Table 2.9 and the Traffic Profiles listed in Table 2.1, both in static and mobile scenarios (e.g., 3 m/s). It is noted that so far we have considered three main reward functions for the respective slices (i.e., maximization of DL eMBB throughput and mMTC TX Packets, and minimization of DL buffer occupancy in the URLLC). In order to demonstrate PandORA’s capabilities as well as 52

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.22: Impact of the Reward Designs from Table 2.11 on DL eMBB Throughput, DL mMTC and TX Packets.

(a) eMBB DL Throughput

(b) mMTC Packets

Figure 2.23: Impact of the Reward Designs from Table 2.11 on the median values of DL eMBB throughput, DL mMTC TX Packets, and DL URLLC buffer occupancy.

53

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN to compare the reward functions evaluated so far, we additionally craft two new reward designs, which are defined in Table 2.11. In Reward Design I, all slices are given the same priority (i.e., they are all assigned a weight of 1). The reward for all slices is the maximization of the PRB Sum of Granted PRBs Ratio, defined as PRB Ratio = Sum of Requested PRBs , where PRB Ratio ∈ [0, 1], that represents the

amount of the allocated PRBs per slice. For Reward Design II, we consider the maximization of the eMBB throughput and the minimization of the URLLC buffer occupancy, with their weights defined in Table 2.2 under the Default weight configuration. For the mMTC slice, we consider the maximization of the PRB Ratio and the slice’s weight is set to 0.5. Lastly, Reward Design III pertains to the joint scheduling and slicing optimization discussed and evaluated in Section 2.2.5.1 and shown in Table 2.11. The results of this experimental evaluation are illustrated in Figs. 2.21, 2.22, 2.23 and were collected under Traffic Profile 2 (i.e., 4 Mbps eMBB throughput, 44.6 kbps mMTC throughput and 89.3 kbps URLLC throughput), in Location 1 (i.e., 50 m radius from the BS), and Set 1 of RAN control timers (see Table 2.9), for a total of 6 UEs, uniformly distributed across the slices. Table 2.11: DRL Reward Design Catalog for Sched & Slicing 0.5 with PPO in Location 1 and Set 1 of RAN control timers. Reward Design ID

eMBB

mMTC

URLLC

I

1 Max. PRB Ratio

1 Max. PRB Ratio

1 Max. PRB Ratio

II

72.0440333 Max. DL Throughput

0.5 Max. PRB Ratio

0.00005 Min. DL Buffer Occupancy

III

72.0440333 Max. DL Throughput

0.229357798 Max. DL TX Packets

0.00005 Min. DL Buffer Occupancy

In Fig. 2.21, we show the PRB Ratio for the three Reward Designs defined in Table 2.11. We observe that with Reward Design I, eMBB UEs experience lower levels of satisfaction (Fig. 2.21a), in terms of the number of allocated PRBs. On the contrary, both on the mMTC and URLLC slices (Figs. 2.21b and 2.21c), Reward Design I results in significant UE satisfaction. The latter is due to the fact that both mMTC and URLLC have lower traffic requirements to satisfy (i.e., 44.6 kbps and 89.3 kbps throughput, respectively), compared to eMBB (i.e., 4 Mbps throughput). With Reward Design II, we observe an improvement in UE satisfaction levels on the eMBB, meaning that UEs are granted the resources they request, along with equally good performance in mMTC and URLLC. The results with Reward Design III indicate that this combination of weight configuration (i.e., tailored to prioritize one slice over the other) and slice-specific rewards effectively result in good

54

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN overall performance without penalizing one slice over the other. Notably, Reward Design III is the configuration under which eMBB UEs experience more satisfaction. Finally, we can observe the competitive behavior among eMBB and mMTC. Indeed, the reward designs that deliver a higher UE satisfaction for eMBB (Fig. 2.21a) also deliver lower satisfaction for mMTC (Fig. 2.21b). In Figs. 2.22 and 2.23, we observe the impact of the Reward Designs explored in Fig. 2.21 on the three KPIs of interest, namely DL eMBB throughput, mMTC TX packets, and URLLC buffer occupancy. In detail, Figs. 2.21a, 2.22a and 2.23a demonstrate a similar trend, since a higher value of PRB Ratio yields higher throughput. A similar trend is observed in Figs. 2.21b, 2.22b and 2.23b for the mMTC slice, with Reward Designs I & II demonstrating similar performance (i.e., a median value of 22 TX packets as illustrated in Fig. 2.23b). These results indicate that the maximization of the PRB Ratio serves as ideal reward for the aforementioned slice but with a penalty on the eMBB. Specifically, even though Reward Designs I & II result in ∼ 37.5% improvement on the performance of the mMTC, they result in lower eMBB throughput values as depicted in Fig. 2.23a. On the URLLC, all Reward Designs result in empty buffers, with Reward Design I slightly resulting in higher performance, as seen in Fig. 2.22c.

(a) eMBB PRBs

(b) mMTC PRBs

(c) URLLC PRBs

Figure 2.24: Selection of PRB Actions for the three slices from the xApp Catalog of Table 2.12.

55

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN In Fig. 2.24, we include the distribution of actions in terms of selected PRBs for the xApp Catalog presented in Table 2.12. The performance evaluation results indicate that agents can make decisions resulting in diverse action distributions (summarized in Table 2.5), which is due to varying rewards and design choices, impacting the performance achieved by the system. We now consider the case where we increase the number of UEs allocated to the mMTC slice as illustrated by Use Cases II & III. Specifically, Use Case II involves a total of 7 UEs, while in Use Case III, the setup comprises a total of 8 UEs, with 3 and 4 UEs allocated to the mMTC slice, respectively. Use Case I corresponds to a setup of 6 UEs in total, and all deployments have been tested under Traffic Profile 1 of Table 2.1 and Set 1 of RAN control timers of Table 2.9 in Location 2 (i.e., 20 m radius from the BS). Finally, the xApp under evaluation corresponds to Reward Design III of Table 2.11.

(a) eMBB PRB Ratio

(b) mMTC PRB Ratio

(c) URLLC PRB Ratio

Figure 2.25: Resource Utilization and UE Satisfaction for a network deployment described by three different Use Cases. Figs. 2.25, 2.26 and 2.27 demonstrate that even when the number of UEs increases, xApps trained using PandORA still deliver good performance, with all Use Cases resulting in good resource

56

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

(a) eMBB Throughput

(b) mMTC Packets

(c) URLLC Buffer Occupancy

Figure 2.26: Impact of the network deployment described by three different Use Cases on DL eMBB Throughput, DL mMTC TX Packets, and DL URLLC Buffer Occupancy.

(a) eMBB DL Throughput

(b) mMTC Packets

Figure 2.27: Impact of the network deployment described by three Use Cases on the median values of DL eMBB Throughput and DL mMTC TX Packets.

57

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN utilization (Fig. 2.25) for the three slices. Similarly, the improved UE satisfaction is reflected on the eMBB DL throughput (Fig. 2.26a), with all Use Cases enjoying a median throughput value of ∼ 1 Mbps (Fig. 2.27a). For the mMTC slice, and specifically in Fig. 2.26b, we observe that as the number of UEs increases, the number of generated TX packets also increases, with Use Case III reporting the highest median value of 15 DL TX packets (Fig. 2.27b). In line with the results presented in the previous evaluation sections, the performance on the URLLC remains high (i.e., empty buffers), with Use Case III performing slightly better as seen in Figs. 2.25c and 2.26c.

2.2.8

Conclusions

Table 2.12: xApp Catalog for Sched & Slicing 0.5 with the PPO DRL Architecture. xApp ID I II III IV V VI

eMBB

mMTC

URLLC

Testing Conditions

Weight Config. 72.0440333 0.00005 1.5 Slice Reward Max. DL Throughput Max. DL TX Packets Min. DL Buffer Occupancy mobility, RAN Control Traffic 1 Mbps 30 kbps 10 kbps Timer Set 1, Location 2 Weight Config. 72.0440333 0.00005 0.229357798 Slice Reward Max. DL Throughput Max. DL TX Packets Min. DL Buffer Occupancy RAN Control Traffic 4 Mbps 44.6 kbps 89.3 kbps Timer Set 2, Location 2 Weight Config. 1 1 1 Slice Reward Max. PRB Ratio Max. PRB Ratio Max. PRB Ratio RAN Control Traffic 4 Mbps 44.6 kbps 89.3 kbps Timer Set 1, Location 1 Weight Config. 72.0440333 0.00005 0.5 Slice Reward Max. DL Throughput Max. PRB Ratio Min. DL Buffer Occupancy RAN Control Traffic 4 Mbps 44.6 kbps 89.3 kbps Timer Set 1, Location 1 Weight Config. 0.00005 72.0440333 0.229357798 Slice Reward Max. DL Throughput Max. DL TX Packets Min. DL Buffer Occupancy RAN Control 89.3 kbps Traffic 4 Mbps 44.6 kbps Timer Set 1, Location 1 Weight Config. 72.0440333 0.00005 1.5 Slice Reward Max. DL Throughput Max. DL TX Packets Min. DL Buffer Occupancy RAN Control Traffic 4 Mbps 44.6 kbps 89.3 kbps Timer Set 1, Location 1

In this chapter, we presented PandORA, a comprehensive evaluation with insights on how to design DRL agents for RAN control. PandORA leverages a framework to streamline and automate DRL agent training and xApp on-boarding for extensive evaluation and testing of DRL agents with Open RAN in Colosseum. We investigated the impact of DRL design on the performance of an Open RAN system controlled by xApps embedding DRL agents that make decisions in near-real-time to compute efficient slicing and scheduling control policies. We benchmarked 23 xApps trained using DRL agents with different action spaces, architectures, reward design and decision-making timescales. Additionally, we tested the DRL agents under various traffic and mobility conditions, considering different hierarchical setups and time granularities in locations both observed and unseen during the training phase. Our experimental results show that network slices with similar objectives (e.g., maximizing throughput and number of transmitted packets) might result in a competitive behavior that can be mitigated using proper weight and reward configurations, and possibly different 58

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN architectures. Additionally, we have explored how the fine-tuning of RAN control timers can boost the performance of various xApps tasked with altering different action spaces. Based on the reported findings, the PandORA evaluation shows that DRL agents adapt well to network conditions encountered during the AI/ML training phase as well as unforeseen conditions, while ensuring high system resource utilization.

2.3

Identifying Conflicts among DRL-based xApps

2.3.1

Introduction The O-RAN ALLIANCE identifies several types of conflicts, including cell ON/OFF,

beamforming, handover, antenna tilt, traffic steering, and others, and highlights the importance of conflict management frameworks [111]. Industry experience clearly shows that conflicts are a critical issue in O-RAN and require appropriate solutions to mitigate performance degradation and prevent service disruptions, outages, and potential financial losses. While these frameworks increase network complexity, they are necessary to ensure continuous and reliable operation. Given its complexity and importance, conflict management has emerged as a key research area within the community and a critical enabler for the adoption of O-RAN. The O-RAN ALLIANCE classifies conflicts into direct, indirect, and implicit categories. Early efforts in O-RAN conflict management include real-time detection of conflicting control policies [112, 113], online deconfliction [114], coordination of AI-based xApps and rApps through team learning to reduce conflict occurrence [77, 78], and orchestration of xApp and rApp selection and deployment to avoid conflicts. While these works provide concrete solutions for mitigating conflicts in O-RAN, they do not fully capture the nuances of conflict severity or their impact on KPMs. In addition, they generally do not account for conflicts that emerge only under specific operational conditions. In this section, we leverage PACIFISTA [2], an empirical and formal framework that uses statistical information from xApps to detect, characterize, and mitigate conflicts between applications. The architecture of PACIFISTA is illustrated in Fig. 2.28. PACIFISTA operates through a profiling pipeline that tests O-RAN applications in a sandbox environment under predefined operational conditions, generating statistical profiles—in the form of Empirical Cumulative Distribution Functions (ECDFs)—of control parameters and KPMs; these profiles are then analyzed through distance metrics (e.g., K-S distance for numerical parameters, χ distance for categorical ones) to detect and evaluate direct, indirect, and implicit conflicts. PACIFISTA further combines a hierarchical

59

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN graph representation with statistical models of O-RAN application behavior to identify relationships between control parameters and KPMs. This enables PACIFISTA to: (i) determine whether two or more applications will generate conflicts; and (ii) quantify the severity of such conflicts. Our contribution in this work is the set of DRL-based xApps for slicing and scheduling control introduced in [26], whose conflict behavior under this framework we analyze below. New application (e.g., xApp)

Developer

O-RAN testing in Colosseum

SMO

New application and profiles

PACIFISTA Conflict-aware instantiation policy

Profiler

X

Conflict Detection

Conflict Evaluation

Conflict Conflict Mitigation report

Catalog Applications to deploy

Network operator

Non-RT RIC

O1

Conflict policy RU

DU

A1

CU E2

RAN

Near-RT RIC

Figure 2.28: PACIFISTA architecture and workflow [2]. PACIFISTA has a modular architecture (Fig. 2.28) with four major logical blocks and a catalog of applications (e.g., xApps). PACIFISTA executes four major tasks, namely application profiling, conflict detection, evaluation, and mitigation. The Profiler runs on sandbox testing environments (e.g., DTs, emulation environments) and generates profiles that describe the statistical behavior of O-RAN applications. The Conflict Detection Module uses such profiles for detecting the occurrence of conflicts and identifying the affected parameters and KPMs. Upon detecting the existence of conflicts, the Conflict Evaluation Module, which is executed on the production network, generates a report that summarizes how severe the conflict is, and how much it impacts the KPMs. Finally, the Conflict Mitigation Module leverages the information included in the report to make informed decisions on the deployment of O-RAN applications. These decisions are made by using conflict management policies specified by the network operator, such as avoiding the deployment of an application that would generate too large of a conflict, or removing a subset of applications to reduce the severeness of conflicts below a certain threshold. 60

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

2.3.2

Conflict Model and Distance Metrics To formalize how PACIFISTA detects and evaluates conflicts among the DRL-based xApps

considered in this work, we briefly summarize the conflict model and the statistical metrics that the framework adopts [2]. Let A denote the set of O-RAN applications that can be deployed on the RICs, P the set of control parameters that they can act upon, and K the set of observable KPMs. For each application a ∈ A and parameter p ∈ P, we define an indicator αa,p ∈ {0, 1} such that αa,p = 1 if a controls p, and αa,p = 0 otherwise. Accordingly, Pa = {p ∈ P : αa,p = 1} ⊆ P identifies the subset of parameters controlled by application a. Analogously, an indicator ϵp,k ∈ {0, 1} tracks whether parameter p ∈ P impacts KPM k ∈ K, with ϵp,k = 1 if such a dependency exists, and ϵp,k = 0 otherwise. For any 2-tuple (p1 , p2 ) ∈ P × P, let πp1 ,p2 ∈ {0, 1} be an indicator parameter such that πp1 ,p2 = 1 if parameter p1 impacts parameter p2 , πp1 ,p2 = 0 otherwise. Formally, we have E P = {(p1 , p2 ) ∈ P ×P : πp1 ,p2 = 1}. We consider the case where applications are profiled on a set C of predefined operational conditions. Each operational condition c ∈ C can specify, among others, the number of UEs, cell load, and Signal to Interference plus Noise Ratio (SINR)/CQI conditions. Building on this notation, PACIFISTA formalizes the three classes of conflicts identified by the O-RAN ALLIANCE [111] as follows. Definition 2.1 (Direct Conflict). Let p ∈ P be any parameter. Let a1 and a2 be any two applications in A. We say that a1 and a2 are in direct conflict with respect to parameter p if p ∈ PaDC = Pa1 ∩ Pa2 . 1 ,a2 Definition 2.2 (Parameter Conflict). Let a1 and a2 be any two applications in A. We say that parameter p1 ∈ Pa1 controlled by application a1 generates a Parameter conflict with parameter p2 ∈ Pa2 controlled by a2 if (p1 , p2 ) ∈ E P , i.e., πp1 ,p2 = 1. Definition 2.3 (KPM Conflict). Let a1 and a2 be any two applications in A. We say that a1 and a2 generate a KPM conflict with respect to KPM k ∈ K if there exists at least one 2-tuple (p1 , p2 ) ∈ Pa1 × Pa2 , with p1 ̸= p2 , such that ϵp1 ,k = ϵp2 ,k = 1. In our analysis, the DRL-based xApps a6 , a7 , and a8 from [26] share the parameter set P (slicing and scheduling policies) and the KPM set K (throughput, buffer occupancy, transmitted packets). a6 controls the scheduling policy only, a7 controls the slicing policy only, while a8 controls both the scheduling and the slicing policies. Since all three xApps influence overlapping KPMs (i.e., ϵp,k = 1 for the same k ∈ K), KPM conflicts are also expected. To assess the severity of such conflicts, PACIFISTA profiles each xApp under each operational condition c ∈ C and extracts ECDFs of the controlled parameters and observed KPMs. 61

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN Pairs of one-dimensional generic ECDFs F1 (x) and F2 (x), with x ∈ R being a random variable, K-S , D INT , are then compared via the distance functions summarized in Table 2.13. We write Di,j i,j χ and Di,j to denote the respective distances between the statistical profiles of xApps ai and aj . For

numerical variables (e.g., PRB allocation, throughput, buffer occupancy), PACIFISTA uses the Kolmogorov-Smirnov (K-S) and Integral Area (INT) distances: s

D

K-S

= max|F1 (x) − F2 (x)|,

D

INT

=

1 L

Z

|F1 (x) − F2 (x)| dx,

(2.8)

with L = max(x) − min(x). For categorical variables such as the scheduling policy (chosen among WF, RR, and PF), PACIFISTA relies on the χ distance Dχ = 1 − p-value from Pearson’s Chi-Square test [115]. By construction, all three distances take values in [0, 1], with larger values indicating more severe conflicts. We exploit this formulation in Section 2.3.3 to quantify direct and KPM conflicts among xApps a6 , a7 , and a8 from [26]. Table 2.13: Distance functions used in PACIFISTA to evaluate conflicts in O-RAN [2]. ID

Equation max|F1 (x) − F2 (x)|

K-S INT

q

1 R L |F1 (x) − F2 (x)| dx

Description Maximum vertical distance between the two ECDFs. Integral of the absolute distance between the two ECDFs, with L = max(x) − min(x).

1 − p-value

χ

Likelihood that data from two categorical distributions differ.

2.3.3

Experimental Setup and Conflict Analysis To fairly compare all xApps against the same repeatable operational conditions c, we

benchmark them on the Rome Colosseum scenario [10]—which reproduces the real-world cellular deployment in a section of Rome, Italy. We consider 6 UEs uniformly distributed within 50 m from the BS, which allocates them in three network slices (eMBB, URLLC, and mMTC) across 10 MHz of spectrum (50 PRBs grouped into 17 Resource Block Groups (RBGs)). We leveraged the MGEN [100] tool to serve DL traffic to the slice UEs as follows: (i) eMBB UEs are served constant bit rate traffic at a rate of 4 Mbps; (ii) URLLC are served Poisson-distributed traffic at an average rate of 89.29 kbps; and (iii) mMTC UEs are served Poisson-distributed traffic at an average rate of 44.64 kbps. 62

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN Our xApps act on a combination of two control parameters: (i) the network slicing policy, by adjusting the number of PRBs allocated to each slice; and (ii) the scheduling policy used in downlink transmissions, chosen among WF, RR, and PF. These xApps are taken from [26] and embed DRL agents controlling a combination of slicing (i.e., the portion of the available PRBs allocated to each slice) and scheduling (i.e., how the PRBs are internally allocated to users of each slice) policies to satisfy slice-specific intents. Specifically, they all aim at maximizing eMBB throughput and number of mMTC transmitted packets, while minimizing DL buffer size for URLLC traffic (as a proxy of latency). They all target this goal via different action spaces. a6 controls the scheduling policy only, a7 controls the slicing policy only, while a8 controls both the scheduling and the slicing policies. These are referred to as Sched 0.5, Slicing 0.5, Sched & Slicing 0.5 in [26], respectively. These xApps embed

agents trained using PPO, a state-of-the-art RL architecture [116], and are deployed inside the Near-RT RIC. They receive real-time KPMs via the E2 interface, make decisions based on network conditions such as downlink throughput, buffer occupancy, and the number of transmitted packets, and continuously adapt their control policies based on real-time feedback from the RAN. It is noted that the model-free architecture and trial-and-error approach of the PPO algorithm are an ideal fit for enhancing the resource allocation process in stochastic environments, such as wireless channels. Ultimately, our goal is to demonstrate that xApps with similar intents, even when controlling different parameters, are prone to generating minimal conflicts. We consider xApps a6 -a8 which use DRL agents with diverse action spaces to improve slice-specific KPMs. We present conflict analysis for eMBB and URLLC slices. We report DL throughput values for the former, and DL buffer size for the latter. Table 2.14 shows the K-S and INT distances for numerical parameters (i.e., slicing policies) and KPMs, and χ distance for scheduling policies, as well as the impact of conflicts on relevant KPMs. Recall that a8 controls both slicing and scheduling policies, while a6 and a7 respectively control scheduling and slicing only. Therefore, there is no direct conflict on scheduling between a8 and a7 , and no direct conflict on slicing between a8 and a6 . Since a7 and a8 both control slicing policies, they generate a direct conflict with K-S and INT distance equal to 0.11 and 0.23, respectively. The same holds for a6 and a8 which produce a direct conflict with respect to scheduling with a χ distance of 0.61, suggesting that the two xApps select different scheduling policies. In general, conflicts have low values due to the shared goal. However, we notice that controlling slicing policies (i.e., a7 and a8 ) results in lower KPM conflicts K−S for both throughput and buffer size (i.e., the largest distance in this case is D8,7 = 0.14), suggesting

that controlling slicing policies under a shared goal makes it possible to achieve higher performance 63

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN

Table 2.14: Direct and KPM conflict analysis taking a8 as the reference xApp. Slice

a8 -a6

Variable

a8 -a7

K−S D8,6

INT D8,6

χ D8,6

K−S D8,7

INT D8,7

χ D8,7

eMBB

PRBs

0

0

−

0.11

0.23

−

eMBB

Scheduling

−

−

0.61

−

−

0

eMBB

Throughput

0.46

0.27

−

0.14

0.13

−

URLLC

PRBs

0

0

−

0.10

0.16

−

URLLC

Scheduling

−

−

0.19

−

−

0

URLLC Buffer Size

0.06

0.01

−

0.04

0.01

−

1

K-S = 0.14 𝐷7,8

ECDF

0.8 0.6

INT = 0.13 𝐷7,8

0.4 0.2 0

0

1

2

3

4

5

A7 A8 K-S 𝐷7,8 INT 𝐷7,8 6

7

8

Throughput [Mbps] Figure 2.29: ECDFs of throughput of the eMBB slice for xApps a7 and a8 , highlighting the expected KPM conflict for that control parameter. than scheduling control alone [26], which allows the xApps to better satisfy the shared intent and produce less conflicts. We also notice that the larger action space (i.e., scheduling and slicing) allows a8 to improve performance. For example, Fig. 2.29 shows how a8 delivers higher eMBB throughput than a7 .

2.3.4

Conclusions In this section, we leveraged PACIFISTA [2], a framework to detect, characterize, and

mitigate conflicts in the O-RAN ecosystem. PACIFISTA leverages statistical information on O-RAN applications and a set of hierarchical graphs to determine the likelihood of conflict emergence and the corresponding severity. We focused our analysis on DRL-based xApps for slicing and scheduling 64

CHAPTER 2. ENABLING AI AT THE EDGE IN O-RAN control, showing how PACIFISTA uses a formal, data-driven model to capture the impact of conflicts on control parameters and target KPMs, thus providing useful insights on how conflicts affect the network performance and the operator’s intents. Specifically, we showed that the DRL-based xApps from [26], despite operating on overlapping control parameters, produce low-severity conflicts due to their shared optimization intent—with KPM distances remaining below 0.5 across all slice–variable pairs considered, and direct conflict distances on slicing as low as DaK-S = 0.11. Future work 8 ,a7 will extend this analysis to larger xApp sets, richer action spaces, and more heterogeneous network deployments.

65

Chapter 3

Optimizing Wireless Propagation with RIS 3.1

Cross-Layer System Level Experimental Evaluation of RIS

3.1.1

Introduction RISs are a promising technique for enhancing the performance of NextG wireless communi-

cation systems in terms of both spectral and energy efficiency, as well as resource utilization. However, current RIS research has primarily focused on theoretical modeling and PHY layer considerations only. Full protocol stack emulation and accurate modeling of the propagation characteristics of the wireless channel are necessary for studying the benefits introduced by RIS technology across various spectrum bands and use-cases. In this chapter, we propose, for the first time: (i) accurate PHY layer RIS-enabled channel modeling through Geometry-Based Stochastic Models (GBSMs), leveraging the QUAsi Deterministic RadIo channel GenerAtor (QuaDRiGa) open-source statistical ray-tracer; (ii) optimized resource allocation with RISs by comprehensively studying energy efficiency and power control on different portions of the spectrum through a single-leader multiple-followers Stackelberg game theoretical approach; (iii) full-stack emulation and performance evaluation of RIS-assisted channels with SCOPE/srsRAN for eMBB and URLLC applications in the world’s largest emulator of wireless systems with hardware-in-the-loop, namely Colosseum [27]. Our findings indicate (i) the significant power savings in terms of energy efficiency achieved with RIS-assisted topologies, especially in the millimeter wave (mmWave) band; and (ii) the benefits introduced for Sub-6 GHz band UEs, where the deployment of a relatively small RIS (e.g., in the order of 100 RIS elements)

66

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS can result in decreased levels of latency for URLLC services in resource-constrained environments. Undoubtedly, research in 5G wireless communication systems has been following an upward trend in recent years. 5G promises better connectivity anywhere, anytime, for everyone and everything, with increased coverage, better system capacity, extremely low latency [117], improved performance in terms of both energy and spectral efficiency [118, 119], while also simultaneously satisfying the diverse QoS demands of its heterogeneous users [120] across a variety of deployment scenarios and network topologies [121]. Hence, 5G and beyond technologies are expected to become an indispensable component of future smart environments [122–128]. Another prominent technology anticipated to accommodate the needs of future NextG communication networks are RISs. RISs have recently attracted widespread interest in the research community as a part of a smart city environment for 5G, and beyond 5G wireless communication systems [17, 118, 129–141]. Indeed, RISs technologies have been proven to extend coverage [133–135, 139], enhance both spectral and energy efficiency [118, 142–144], while at the same time enabling the vision of smart and reconfigurable propagation environments [17, 138, 145], which can be controlled through SDR techniques [146, 147]. Therefore, due to both the tremendous advantages that they introduce (e.g., decreased levels of latency [148, 149]) and due to their presence in a multitude of use-cases scenarios (e.g., resource allocation in multi-UE Non-Orthogonal Multiple Access (NOMA) systems [150], localization and positioning [151], assisting anti-jamming methods [152], and enhancing Unmanned Aerial Vehicle (UAV)-enabled communications [149, 153–158] among others), there is a clear need for further understanding of RIS-assisted channels through effective performance evaluation campaigns in different spectrum bands and use cases. In this section, we aim to provide, for the first time, a holistic, system-level evaluation of RIS-assisted channels for NextG communication systems in Cellular Vehicle-to-Everything (C-V2X) environments across different portions of the spectrum, while also paving the way for their evaluation across various network slices on experimental platforms. Our key contributions are as follows: 1. We leverage the capabilities of the fully reconfigurable, MATLAB-based [159] statistical ray-tracer QuaDRiGa [160], which uses RF data collected under various conditions to cascade the wireless links and include the effect of a RIS in the topology. 2. We focus on a C-V2X setup comprising a UAV serving as a flying gNB for multiple UEs. 3. Through a single-leader multiple-followers Stackelberg game we jointly optimize the overall received signal strength at the gNB, and each UE’s individual energy efficiency in a NOMA system. 67

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS 4. Through this hierarchical game-theoretic approach, we determine both the phase shifts of the RIS reflecting elements, as well as the UL transmission power of the UEs for both the Sub-6 GHz and the mmWave bands. 5. We install the generated wireless topologies for the band that requires only a minimal number of RIS elements to achieve a significant path gain enhancement in the Colosseum Wireless Network and Channel Emulator [21]. 6. We modify the RAN functionalities in terms of scheduling profiles (i.e., RR and WF) and slicing (i.e., allocation of the available PRBs) and we observe the performance gain provided through the RIS technology on the eMBB and URLLC network slices. 3.1.1.1

Related Work The current literature hosts a plethora of RIS simulators for PHY layer channel modeling

across various operating frequencies. Additionally, substantial efforts have been reported in the design, creation, and real-world deployment of such surfaces. In [161, 162], the authors present the mmWave SimRIS simulator for indoor and microcell environments. The aforementioned works state that accurate PHY layer simulators that can operate in a variety of frequencies and applications, ideally using real data, are of utmost importance. The authors note that further research is needed to develop practical channel models for various propagation environments and to identify compelling use cases. Finally, the authors emphasize the need to explore the coexistence of eMBB and URLLC UEs in various frequency domains, as well as the importance of standardization and integration into existing networks. In [163], a commercial RIS simulator is leveraged to perform system-level simulation in the mid and mmWave bands within a real-world urban environment. The authors conclude by mentioning the importance of analytical frameworks for system-level analysis by using tools leveraging stochastic geometry. Regarding the sub-6 GHz band, in [164], a 3D GBSM relying on the statistical properties of the wireless channels is discussed, taking into account both multipath propagation and continuous-time evolution. A GBSM channel model incorporating a RIS is also considered in [165], highlighting the importance of leveraging small panels. Other NextG RISassisted channel models, which discuss path-loss propagation and take into account geometry, are also presented in [149, 166]. Rayleigh and Rician channels are used to model the multipath effect, and their integration in RIS-assisted communication systems is discussed in detail in [167, 168]. In [169], a frequency-independent channel model for RIS is discussed, whereas a RIS-assisted channel model for V2X environments that incorporates the Rician channel matrix is presented 68

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS in [170]. A RIS-assisted channel modeling approach for high frequencies is discussed in [171], while a signal processing perspective on multipath propagation with RIS is given in [172]. Physics-based PHY models are presented in [173, 174], while other approaches include deploying RIS panels in the Sub-6 GHz [175] and mmWave [176] bands [177, 178], with a focus on energy efficiency [179]. Additionally, the contribution of RIS technology to energy efficiency through power control is a well-investigated topic. In [180], a non-convex optimization framework for energy efficiency that jointly optimizes the TX power of the BS and the reflecting coefficients of the RIS on the DL is considered. In [181], the paper tackles energy efficiency maximization in an UL RIS-aided mmWave NOMA network by jointly optimizing the UE TX power and the RIS phase shifts using iterative optimization techniques. In [182], the usage of both RISs and UAV is discussed in terms of providing energy-efficient communications on the DL while ensuring the QoS demands of the UEs. The proposed method uses Successive Convex Approximation (SCA) to iteratively determine a joint optimal solution for the UAV trajectory, RIS phase shifts, and the UAV TX power. In [183], the problem of energy efficiency maximization is modeled as a Mixed Integer Non-Linear Programming (MINLP) problem, with the objective of enhancing overall energy efficiency on the DL by optimizing the BS’s TX power and RIS phase shifts. The work in [184] discusses the topic of energy efficiency maximization in a RIS-assisted network, where convex optimization and sequential approximation methods are employed to jointly optimize the TX powers of the UEs and the reflection coefficients of the RIS in the UL. In [185], a ML approach based on the PPO algorithm for the maximization of energy efficiency in RIS-aided networks is discussed. The optimization problem is formulated under constraints of the rate requirement of the UEs, the power budget at the BS, and the discrete phase shift coefficients of each reflecting element at the RIS on the DL. In this approach, the BS’s TX power and the phase shift matrix of the RIS are jointly optimized. Similarly, in [186], the joint energy efficiency maximization problem for a DL NOMA network is discussed, with the goal of jointly optimizing the beamforming vectors at the BS and the coefficient matrices at the RIS, while also controlling the TX power at the BS using DRL. In [154], a two-tier Stackelberg game is used to jointly maximize the received signal strength at the eNB by calculating the RIS phase shifts and the UEs’ UL transmission power in RIS-assisted, UAV-enabled networks. However, the aforementioned framework does not consider the frequency of operation, and therefore provides limited insights on energy-efficient resource allocation on different portions of the spectrum. In [187], a Stackelberg game is also discussed for end-to-end (E2E) energy-efficient optimization through power control and RIS reconfiguration in UL RIS-assisted, UAV-enabled Integrated Access and Backhaul (IAB) networks. However, the focus is directed towards the low Sub-6 GHz portion of the spectrum, 69

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS disregarding emerging 5G frequency bands, such as the mmWave. Lastly, the generation of RF scenarios for Wireless Network Channel Emulators has been a trending topic. In [188], a framework for creating RF scenarios for large-scale, real-time wireless emulators such as Colosseum, leveraging a commercial ray-tracer is introduced. It leverages efficient clustering techniques and channel impulse response re-sampling to scale down the large input set of RF data to fewer parameters. In [65], the CaST Toolchain is presented. It comprises a framework for creating RF scenarios from ray-tracing models for FPGA-based emulation platforms and a SDN channel sounder to characterize the emulated channels. In [189], a framework for creating mobile RF scenarios, including the presence of UAVs, based on real collected data, is discussed. The contributions discussed above clearly demonstrate the significance of RIS-assisted NextG channel models, as well as the substantial gains achieved in terms of energy efficiency through the deployment of RISs in various heterogeneous scenarios and use cases. At the same time, the interest in the validation and testing of channel models, leveraging the full cellular protocol stack, in high-fidelity and accurate network channel emulators is ever-growing. However, despite the established contributions of hierarchical game-theoretic approaches to energy efficient power control in wireless networks [190, 191], there has been limited research on the benefits of energy-efficient power control in RIS-assisted and UAV-enabled C-V2X networks in the Frequency Range 1 (FR1) and Frequency Range 2 (FR2). Moreover, despite the availability of frameworks for generating RF scenarios in channel emulators, no prior work has focused on installing RIS-assisted scenarios in these emulators. Consequently, a research gap exists in cross-layer experimental evaluation of RIS-assisted channels utilizing the full protocol stack on experimental platforms. Therefore, the evaluation of RIS-assisted networks across different network slices (e.g., eMBB and URLLC) is currently missing. This section aims to fill these gaps in the literature with the following original contributions: (i) we propose accurate PHY layer RIS-enabled channel modeling through GBSMs and QuaDRiGa; (ii) we cascade the generated channels to include the effect of a RIS; (iii) we leverage hierarchical game theory to optimize the RIS elements’ phase shifts; (iv) we study energy-efficient power control across different spectrum bands; (v) we implement the RIS-assisted channels on the Colosseum testbed; and (vi) we perform full-protocol stack emulation using srsRAN [192] to evaluate the performance gain in eMBB and URLLC network slicing by fine-tuning the BS’s network functions. The remainder of this section is organized as follows. Section 3.1.2 provides an overview on GBSM-based NextG channel modeling with QuaDRiGa. Section 3.1.3 discusses the simulation of a RIS-assisted C-V2X setup with QuaDRiGa, as well as its evaluation on energy efficiency in 70

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS the Sub-6 GHz and the mmWave bands. Section 3.1.4 details the installation and evaluation of a RIS-assisted topology on the Colosseum wireless network and channel emulator, as well as its evaluation on the eMBB and URLLC network slices. Finally, Section 3.1.5 concludes the section and discusses possible future directions.

3.1.2

GBSMs with QuaDRiGa

3.1.2.1

Channel Modeling with QuaDRiGa

-70

4

20. . .

∎ ∎∎ ∎

🚦

➜

➜

➜

🚦

es

➜

🚦

➜

Path Gain [dB]

3

➜ 0m 15 . es . . od n 20

∎∎ ∎ ∎

m d 87 no

➜

∎ ∎∎🚦 ∎

2

1 ∎ ∎ ∎∎

➜

green

red

m 61

Random Averaged

-75

NLOS

LOS

-80 -85 -90 -95

0

1

2

3

4

5

6

7

8

9

Time [s]

(a)

Location

from

Google

(b) Path Gain.

(c) Path Power vs. Delay vs. Dis-

Maps [193].

tance. NLOS

LOS

NLOS

LOS

(d) Delay vs. Distance.

(e) Path Power vs. Distance.

Figure 3.1: Results averaged over 100 channel realizations for the link describing the green vehicle’s transmission to the red one in Willow Grove Naval Air Base in Horsham Township, Pennsylvania, USA. The implementation of GBSMs is fairly simple and computationally inexpensive, and can be done through code-based simulators (e.g., NYUSIM [194], WINNER II/+ [195] and 3GPP3D [196]), and Standardized Channel Models (e.g., the Spatial Channel Model (SCM), IMTAdvanced, METIS, mmMAGIC and the 5G-ALLSTAR [197]). QuaDRiGa extends the WINNER and 3GPP-3D models [198], with the advantage that it achieves a high level of precision [199]. It takes as input configuration files to describe a propagation scenario, and each file contains a parameter table with statistical information about the wireless channels (i.e., delay spread, Rician K-factor, shadow fading, Cross-polarization Ratio (XPR), angular spread values and their properties such as correlation 71

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS and distribution). The aforementioned files were created through the process of channel sounding either with measurements performed in urban cities [200, 201], or by the incorporation of statistical tables such as those included in [202]. The modeling procedure can be divided into two parts: (i) a stochastic part that generates the Large Scale Parameters (LSPs), e.g., path loss, shadowing and fading, and (ii) a geometry-based part that calculates the updated Small Scale Parameters (SSPs), e.g., delay spreads [203]. In contrast to the classical ray-tracing approach, QuaDRiGa does not use an exact geometric representation of the propagation environment. The distribution of the scattering clusters (e.g., tree foliage and buildings) is random. This adds simplicity into the implementation and decreases the computational cost without accuracy loss. QuaDRiGa simulates complicated NextG heterogeneous environments [204], by considering effects such as dual-mobility, 3D propagation and continuous time evolution. The generated parameters are spatially correlated, while the widesense stationary (WSS) properties are kept both within a segment and during mobility. Finally, the simulation steps can be summarized as follows [199, 200, 205]. 1. Definition of the UE Input Variables. The topology layout is defined (i.e., TXs/Receivers (RXs)), location (i.e., coordinates) and antenna characteristics (i.e., carrier frequency and type). Second, the nodes’ trajectories and their speed profile are determined. Last, a propagation scenario is set (Line-of-Sight (LOS)/Non-Line-of-Sight (NLOS), or a combination). Examples of supported scenarios [199, 200, 206] are: (a) Berlin/Dresden UMa • Terrestrial macrocell parameters extracted from measurements in Germany. (b) WINNER model • UMa C1 & C2: For macrocell BSs in urban & sub-urban areas, respectively. • UMi B1: For microcell BSs in urban areas. (c) 3GPP TR 37.885 Highway V2X • NLOS due to dynamic blockages (e.g., vehicles). (d) 3GPP TR 38.901 NR model - UMa & UMi • For center frequencies between 0.5 and 100 GHz. 2. Calculation of LSPs. Out of the aforementioned parameter tables, the LSPs are calculated. At this step, sources of scattering have been created, according to the multi-bounce approach, comprising a First Bounce Scatterer (FBS) and a Last Bounce Scatterer (LBS). 72

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS 3. Generation of the XPR. Antenna characteristics and polarization effects are taken into account. 4. Calculation of the Channel Coefficients and Path Gain Generation. The results of the previous steps along with SSPs (e.g., angular characteristics of the paths) are combined to calculate the channel coefficients (i.e., the Multipath Components (MPCs)). A last fine-tuning of the shadow fading and path loss parameters, along with the Rician factor are applied to produce the final path gains. 5. Repetition of steps 1 − 4 for successive channel traces and final Propagation Channels. In case of mobility the process is repeated. All individual components produced by every step of the aforementioned procedure are combined to obtain the final channels. 3.1.2.2

Experimentation through Simulation with QuaDRiGa To test the software’s effectiveness, we generate a C-V2X setup [207] in which two vehicles

approach each other under the 3GPP 37.885 Highway LOS transmission conditions (Section 3.1.2.1). Both have a steady speed of 66 km/h for a distance of 150 m. The vehicles are equipped with omni-directional antennas mounted on their roof at 1.5 m and operate at the center frequency of 5.9 GHz. As it can be seen from the topology in Fig. 3.1a, scattering objects such as 4 Road Side Units (RSUs) along with 40 nodes on the highway’s sides, placed 5 m apart, representing other parked cars, are also present. The simulation’s location corresponds to the Willow Grove Naval Air Base in Horsham Township, Pennsylvania, USA (Global Positioning System (GPS) coordinates for latitude and longitude are [40.202951, −75.149008]), and the conversion from GPS to Cartesian coordinates has been done with MATLAB’s [159] latlon2local function. Given the stochastic nature of the channels, and in order to obtain a good estimate, we observe the results every 0.1 s and average them over 100 channel realizations. In Fig. 3.1, we present the simulation’s results, when the green transmits to the red vehicle, as depicted in Fig. 3.1a. In Fig. 3.1b the continuous bell-shaped curve represents the averaged path gains, while the dotted represents the path gains for a random setup. The log-distance path loss [207] model leveraged in the channel generation, explains the increase in path gains as the path loss decreases, particularly for a minimal distance between the vehicles. In the final path gain values in Fig. 3.1b, the main contribution comes from the direct LOS component, while the contribution of the NLOS MPCs is weighted to be insignificant, based on the input configuration file. In Fig. 3.1c, we include a 3D plot of the generated path powers versus delay and travelled distance, while in Fig. 3.1d and in Fig. 3.1e, we observe Fig. 3.1c from two different perspectives. 12 paths are generated, one for the direct path, and 11 for the multiple copies of the 73

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS signal, for each position of the mobile TX/RX (i.e., Fig. 3.1e). In Fig. 3.1d, the path power of the LOS path attains a lower delay value compared to its NLOS counterparts for a given distance. As the vehicles begin to approach each other, the delay values decrease, reaching their minimum around the midpoint of the track, where the two vehicles cross. 3.1.2.3

GBSMs for Wireless Network Channel Emulators We install the aforementioned scenario in Colosseum, a publicly available testbed compris-

ing 128 SRNs. It consists of pairs of Dell PowerEdge R730 servers and NI USRP X310 SDRs that enable large-scale experimentation in diverse network deployments via the MCHEM component. MCHEM leverages FPGA-based FIR filters to replicate wireless environment conditions such as path loss, fading, attenuation, mobility, and interference pre-modeled through ray-tracing software, analytical models, or real-world measurements. The Colosseum TGEN, built on top of the MGEN Transmission Control Protocol (TCP)/User Datagram Protocol (UDP) traffic generator [100], emulates different traffic profiles (e.g., multimedia content), demand, and distributions (e.g., Poisson, periodic), by emulating Internet Protocol (IP) traffic flows between the SRNs. We focus on nodes 1 → 4 as depicted in Fig. 3.1a, which share a bandwidth of 10 MHz. Nodes 1 → 2 are deployed through the srsRAN protocol stack [192], while nodes 3 → 4 are deployed with the Wi-Fi Stack, which is based on the open-source GNU Radio implementation of the IEEE 802.11a/g/p standard [208]. Cellular node 1 and Wi-Fi node 3 are BSs, while nodes 2 and 4 serve as the respective cellular and Wi-Fi UEs. Finally, UDP traffic is generated with iPerf. In Figs. 3.2a and 3.2b, we illustrate the UL cellular SINR and throughput between the cellular nodes. Initially, an average SINR of 27 dB is observed, which drops to 22 dB, when the WiFi nodes start transmitting at 20 dB (at approximately t = 120 s), before slightly recovering to ∼ 24 dB. Similarly, the UL throughput becomes unstable once WiFi transmissions commence. Conclusively, with this simple yet richly detailed use-case, we are able to verify the suitability of QuaDRiGa and GBSMs for generating RF scenarios for Colosseum, as well as the efficiency of the latter in generating and managing traffic among heterogeneous nodes. Notably, although the aforementioned scenario comprises multiple nodes, the focus is directed towards the aforementioned to demonstrate the software’s capabilities, and hence an extensive evaluation is omitted. Readers interested in the creation of RF scenarios for Colosseum can refer to [21, 65, 188].

74

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

28 Instantaneous Average

7

26

SINR [dB]

Uplink Throughput [Mbps]

8

6

5

4

22

WiFi nodes start 0

50

100

WiFi nodes start

24

150

200

250

20

300

0

50

100

150

200

250

300

Time [s]

Time [s]

(a) UL Throughput [Mbps].

(b) UL SINR.

Figure 3.2: UL cellular metrics between BS-node 1 and UE-node 2.

3.1.3 Investigating Energy Efficiency with frequency-dependent GBSMs and RISs in C-V2X Evidently, QuaDRiGa is effective in providing accurate geometry-based stochastic channel modeling. Additionally, considering that the generated channels are frequency-dependent, QuaDRiGa serves as an ideal candidate for investigating energy efficient power allocation across different spectrum bands and various use-cases. In this Section, we focus particularly on a C-V2X usecase [209] involving a UAV acting as a flying gNB serving multiple UEs, to investigate energy efficiency in the Sub-6 GHz and mmWave bands. Without loss of generality, we assume that the BS simulated with QuaDRiGa is a static UAV hovering at a certain height, while the generated channels are cascaded to simulate the inclusion of a RIS in the topology. We believe that the strong LOS links provided by the UAV and the cascaded channels offered by RIS technology fit into the description of a C-V2X scenario for NextG wireless environments and smart cities as described in [154]. 3.1.3.1

System Model: A RIS-assisted C-V2X setup with QuaDRiGa As illustrated in Fig. 3.3, a RIS-assisted and UAV-enabled wireless communications system

is evaluated on the UL direction. Specifically, the topology comprises a UAV, a building facade hosting a RIS and a set of mobile UEs, denoted as I = {1, . . . , i, . . . , |I|}, which communicate directly with the UAV. The RIS consists of |M | reflecting elements, the set of which is defined as M = {1, . . . , m, . . . , |M |}, and each RIS element’s phase shift is θm ∈ [0, 2π], ∀ m ∈ M . Finally, the corresponding diagonal phase-shift matrix is defined as Θ = diag(ejθ1 , . . . , ejθm , . . . , ejθ|M | ). The incoming signals at the UAV are combined, as in [154], so that each UE’s i signal is the outcome of the coherent addition of the direct and the reflected signal. The channel gain of the direct link between 75

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

𝒉!

ℎ#

$

𝑼

𝒉 !𝑹

Figure 3.3: RIS-assisted and UAV-enabled communications system. a user i and the UAV is described as hiU , the channel gain between a user i and the RIS is defined as hiR , where hiR can also be written as hiR = [|hiR,1 |ejω1 , . . . , |hiR,m |ejωm , . . . , |hiR,|M | |ejω|M | ]T , and finally, the channel gain of the reflected link from the RIS to the UAV is denoted as hRU . H Θh . Last, the steering vector for the UE i to the Subsequently, the cascaded channel is given as hRU iR 2π

2π

LOS = [1, e−j λ dϕiR , . . . , e−j λ (|M |−1)dϕiR ]T , while the steering vector RIS LOS link is defined as hiR 2π

2π

LOS = [1, e−j λ dϕRU , . . . , e−j λ (|M |−1)dϕRU ]T . for the RIS to the UAV LOS link is denoted as hRU

Moving forward, we keep using the same notation as in [154], which is also reported in Table 3.1. The first RIS element, i.e., m = 1, with its respective coordinates, i.e., (xR , yR , zR ) [m], is used as a reference point in the following calculations, while for the UAV, its coordinates are given as follows (xU , yU , zU ) [m]. It is noted that in [154], the featured channel model is frequencyindependent. In our study, we aim at observing the impact of the frequency of operation in terms of energy efficiency on realistic RIS-assisted wireless channels on two different 5G bands, namely the Sub-6 GHz and the mmWave band. It is highlighted that RISs are expected to work across multiple frequency ranges, including both the FR1 and FR2 bands [18], to ensure interoperability across diverse deployment scenarios and applications. Given that QuaDRiGa can reliably support the simulation of channels up to 100 GHz, as mentioned in Section 3.1.2, the Sub-6 GHz and mmWave bands are chosen. Therefore, we experiment with two different values for the center frequency, i.e., 5.9 GHz and 28 GHz, respectively. Finally, both the UEs and the UAV are equipped with singleantenna omni-directional TXs/RXs. It is noted that although omni-directional antennas are commonly used in the Sub-6 GHz band [210–215], and they can be deployed on mmWave communication

76

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS systems [216–220], this is not typical [221]. Instead, high-directional antennas are most commonly used in such scenarios [222–224], to combat severe propagation loss and achieve high antenna gains. Within the scope of this work, we focus on the use of omni-directional antennas for the mmWave to provide insights into the performance enhancement introduced by RIS in cases where omnidirectional antennas are convenient for deployment, such as in a C-V2X environment [225]. While it is indeed true that in a dynamic environment such as the aforementioned, a directional antenna can improve Received Signal Strength Indicator (RSSI), this increased directionality comes at the cost of precise alignment with the RIS and UEs. Therefore, continuous beam tracking and adaptation are needed, which introduce additional overhead and latency that may outweigh the benefits of directive gains. Additionally, highly accurate and precise channel modeling with directional antennas in urban C-V2X environments, where the effects of multipath propagation are strong, can prove rather challenging, mainly due to the narrow mmWave beams, which lead to position deviations, inaccuracies, and alignment errors, resulting in beam misalignment, signal loss, and significant performance drops.

77

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

Table 3.1: Channel Modeling Notation Symbol

Description

|I|

Number of mobile UEs

I

Set of mobile UEs

|M |

Number of reflecting elements on the RIS

M

Set of RIS’s reflecting elements

θm , ∀m ∈ M

Reflecting element’s phase shift

hRU

Channel gain for the RIS to UAV reflected link

Θ

Diagonal phase-shift matrix

hiR

Channel gain for the UE i to RIS link

H hRU ΘhiR

Cascaded channel gain

hiU

Channel gain for the UE i to UAV direct link

dRU [m]

Euclidean distance between the RIS and the UAV

diR [m]

Distance between the UE i and the RIS

diU [m]

Distance between the UE i and the UAV

ϕRU

Cosine of the signal’s Angle of Departure (AoD) for the RIS to UAV link

ϕiR

Cosine of the signal’s Angle of Arrival (AoA) for the UE i to RIS link

λ [m]

Carrier wavelength

d [m]

Antenna separation

LOS hRU LOS hiR h′xy |h′xy |, φ′xy

Steering vector for the RIS to UAV LOS link

Magnitude and phase of the MPC

|N |

Number of generated MPCs

Steering vector for the UE i to RIS LOS link MPC gain, where x ∈ {i, R}, y ∈ {R, U }

Last, the channels in [154], from which our use-case is inspired, rely on the Rice and Rayleigh mathematical models to emulate the wireless links. However, their simple yet effective implementation may not accurately represent real-world wireless environments [165]. For these reasons, we proceed by creating the channels with QuaDRiGa. The channel generation process is described in Section 3.1.2.1, where for each generated link a number of MPCs is produced based on the Channel Generator’s input scenario, in our case the 3GPP-compliant scenario, 3GPP 38.901 UMa. Finally, we assume that the links between the UEs and the UAV are NLOS, while those remaining, i.e., the channels between the UEs and the RIS, and the RIS to UAV, are LOS. The transmission conditions of the aforementioned links were selected to ensure that the cascaded channel created 78

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS by the RIS is strong enough to redirect the UEs’ signals to the UAV, for both the Sub-6 GHz and, especially, the mmWave band, which suffers from transmission losses. The wireless channels are observed every 0.1 s and for their creation, the steps listed below are followed: • Generate scenario-based MPCs (i.e., h′xy ) for every link. • Perform coherent summation of the MPCs [188] to obtain the link’s channel gain, as reported in (3.1): h′′xy =

|N | X

′

h′xy · ejφxy .

(3.1)

j=1

• Average the channel gains (i.e., h′′xy ) over 100 different channel realizations to obtain the link’s final channel gain (e.g., hiU ). LOS and hLOS , • Multiply the generated channels with the corresponding steering vectors, i.e., hRU iR

to get the respective final channel gains (i.e., hRU , hiR ). The overall channel power gain for the UE i to UAV link is denoted as Gi = |hiU + H Θh |2 , and the UEs are able to communicate concurrently with the UAV thanks to the NOMA hRU iR

technique. The received channel gains (i.e., Gi ) at the UAV are sorted and decoding starts from the UE with the highest channel gain. Finally, the UAV’s RX decodes the received superposed signal with the help of the Successive Interference Cancellation (SIC) technique, which is implemented on its end. The interference sensed by each UE is defined as follows in (3.2): Ii =

X

Gj P j + I 0 ,

(3.2)

j<i

where Pj [W] is the UL TX power of the UE j and I0 is the power of the zero-mean Additive White Gaussian Noise (AWGN). Each UE’s achieved SINR is given as follows in (3.3): γi = 3.1.3.2

Pi G i . Ii

(3.3)

Hierarchical Game-Theoretic Power Control In this Section, we aim to study how the frequency-dependent GBSM-based system model

presented in Section 3.1.3.1 will impact the energy efficiency in both the Sub-6 GHz and mmWave bands. In brief, both the UEs and the UAV indulge in a single-leader multiple-followers Stackelberg Game, where the target is to jointly maximize the UAV’s overall received signal strength and each 79

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS UE’s energy efficiency in a distributed manner. In detail, the UAV acting as a leader computes the RIS elements’ effective phase shifts (and intelligently steers the signals reflected by the RIS) with a goal to enhance its received signal quality. In turn, the UEs, acting as followers, observe the leader’s action and through the formulation of a non-cooperative game aim at maximizing their energy efficiency, by determining their optimal UL TX power. Each UE’s utility function is defined in (3.4): M

Ui (Pi , P−i ) =

W · (1 − e−αγi ) , Pi

(3.4)

where P−i is the TX power vector of all the UEs except for UE i, W [Hz] is the system’s bandwidth, and α, M ∈ R+ are parameters which control the utility function’s slope. The adopted utility function for energy efficiency represents the trade-off between the achieved QoS satisfaction, i.e., SINR, and the invested UL TX power [154, 226]. In detail, the numerator of the utility function, i.e., (1 − e−αγi )M , is a sigmoidal curve which expresses the UE satisfaction, with respect to its QoS demands, while the denominator expresses the associated cost with reference to the transmission power levels. Because of the nature of the sigmoidal curve, small increases in γi (by increasing the TX power) will initially improve the utility. However, increasing γi further requires significant power while providing diminishing returns in utility. As a result, the optimal strategy is to transmit at lower power levels in order to maintain higher energy efficiency. Finally, the joint optimization problem is formulated as follows, with the respective problem solved by the UAV written as: max θ

s.t.

X

H Pi |hiU + hRU ΘhiR |2

(3.5a)

0 ≤ θm ≤ 2π, ∀m ∈ M.

(3.5b)

∀i∈I

With regard to the optimization problem solved by the UEs, the latter ones participate in a non-cooperative game defined as G = [I, {Ai }∀i∈I , {Ui }∀i∈I ]. In detail, I is the set of UEs (e.g., players), Ai = [Pimin , Pimax ] is each UE’s strategy set, and Ui is their utility function. Pimin [W] and Pimax [W] correspond to each UE’s minimum and maximum UL TX power levels respectively, where Pimin , is determined by the SIC prerequisite at the UAV’s RX as follows: Gi Pi − Ii ≥ Ptol , ∀i ∈ {2, . . . , i, . . . , |I|},

(3.6)

such that each UE’s signal is decoded successfully. According to (3.6), each UE’s signal strength, i.e., Gi Pi , should be greater than or equal to the UE’s i sensed interference Ii , which is broadcasted by the UAV to all the UEs, plus a minimum acceptable power level Ptol ∈ R+ , which represents 80

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS the SIC RX’s tolerance/sensitivity. Finally, the maximum UL TX power level Pimax is imposed by the maximum power budget of the UE’s device. The optimization problem is finally determined as follows: M

W · (1 − e−αγi ) , ∀i ∈ I Pi

(3.7a)

Gi Pi − Ii ≥ Ptol , ∀i ∈ {2, . . . , i, . . . , |I|}

(3.7b)

max Ui (Pi , P−i ) =

Pi ∈Ai

s.t.

Pi ≤ Pimax , ∀i ∈ I.

(3.7c)

The non-cooperative game’s, G, outcome is a Nash equilibrium, which determines the ∗ ]. UEs’ optimal TX power vector, i.e., P∗ = [P1∗ , . . . , Pi∗ , . . . , P|I|

The solution of the optimization problem taking place on the UAV’s side and defined in (3.5a)-(3.5b) is the RIS elements’ effective phase shifts’ vector θ ∗ . It is noted that to maximize the overall received signal strength, the UAV only needs to calculate the effective RIS elements’ phase shifts (i.e., θ ∗ ) that maximize the overall channel power gain of the UEs. The aforementioned optimization problem is non-convex, making it challenging to find a globally optimal solution. As described in [154, 155], when there is a single UE in the system, the optimal phase shifts of the RIS elements are given by the following closed-form solution: ∗ θm = ∠hiU + ωm +

2π d(m − 1)ϕRU , ∀m ∈ M. λ

(3.8)

To derive the closed-form phase-shift solution in (3.8) and maximize the received signal strength at the UAV, we aim to achieve phase alignment of the received signals from different transmission paths. Notably, this can be achieved through coherent signal construction of the incoming signals at the UAV. Indeed, the UE’s channel power gain is maximized when the direct signal and the signal reflected by the RIS are perfectly aligned and coherently combined at the UAV’s RX [155]. This generally holds true when the phase shifts of the direct and the cascaded signals are equal, as follows 



H Θh ∗ ∠hiU = ∠ hRU iR . Therefore, there’s an optimal 1 × |M | phase-shift vector θ = ∠v for

the single-UE case. Similarly, in the multi-UE case, there exists a distinct reflection-coefficient vector vi = [vi,1 , . . . , vi,|M | ] ∈ C|M |×1 for each UE i that maximizes its channel power gain. In the multi-UE case, the solution to the leader’s maximization problem is a linear combination of the phase shifts of the RIS elements that enhances each UE’s signal strength, with the reflection-coefficient vectors vi being distinct for each UE. On the other hand, the optimization problem solved on the UEs’ side will result in the determination of their optimal UL TX power vector P∗ , as mentioned, since given the RIS elements’ 81

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS effective phase shifts, each UE determines its optimal UL TX power Pi∗ via non-cooperatively interacting with the rest of the UEs in the action space. The Stackelberg equilibrium of this hierarchical game-theoretic approach can be given as (θ ∗ , P∗ ). Definition 1. (Nash Equilibrium) The UL TX power vector P∗ = [P1∗ , . . . , Pi∗ , . . . , PI∗ ] is a Nash equilibrium of the non-cooperative game G = [I, {Ai }∀i∈I , {Ui }∀i∈I ] if for every UE i ∈ I, it holds that Ui (Pi∗ , P−i ) ≥ Ui (Pi , P−i ), ∀Pi ∈ Ai . At the Nash equilibrium, the UE has no incentive to improve its achieved utility by altering its transmission power strategy, given the strategies of the rest of the players (i.e., the other UEs). Theorem 1. (Existence & Uniqueness of a Nash Equilibrium) For the non-cooperative game given as G = [I, {Ai }∀i∈I , {Ui }∀i∈I ], there exists a unique Nash equilibrium point, which is defined as follows: Pi∗ = max



Pimin , min

 ∗ γi · I i

W Gi

, Pimax



, ∀i ∈ I,

(3.9)

where γi∗ represents the unique positive solution to the equation ∂f∂γ(γii ) γi − f (γi ) = 0, f (γi ) = (1 − e−αγi )M . The Theorem’s proof relies on the quasi-concavity property of the utility function Ui (Pi , P−i ) with respect to the UL TX power Pi . The quasi-concavity property is proven and further explained in [226], which also provides a reference guide with detailed steps of the proof. Therefore, given ∗ , ∀m ∈ M , and the optimal transmission power vector the RIS elements’ effective phase shifts θm

Pi∗ , ∀i ∈ I, the Stackelberg equilibrium is iteratively determined as presented in Fig. 3.4. Notably, the convergence of the UEs’ strategies to the Nash equilibrium point is achieved through the implementation of a Best Response Dynamics algorithm [226], as further detailed in Algorithm 1. Note that the superscript (j) indicates the iterations required for the non-cooperative game played among the UEs. Additionally, it is highlighted that the maximum TX power levels for the UEs (as defined in Step 3 of Algorithm 1) are determined by 3GPP standards and are set to 23 dBm [227–229]. The corresponding minimum value is set to −20 dBm. We consider the following setup, where a UAV hovers at the point (xU = 25, yU = 50, zU = 25) [m] of the three-dimensional space serving a NOMA cluster of |I| = 5 users. Their distance from the RIS is given as di,R = [20, 27, 37, 58, 66] [m] and all are randomly distributed around the RIS’s reference point, which is given as (xR = 30, yR = 40, zR = 20) [m]. The system’s bandwidth is equal to W = 10 MHz, and the AWGN power is I0 = −140 dBm for the Sub-6 GHz and I0 = −160 dBm for the mmWave band, respectively. It is noted that AWGN noise is already 82

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS Algorithm 1 Stackelberg Game-Theoretic Optimization 1: Initialize the network topology, including the locations of UEs, RIS, and the UAV. 2: Generate the channels using QuaDRiGa, as described in Section 3.1.3.1. 3: Initialize randomly Pi ∈ [10−5 , Pimax ]. 4: Determine RIS elements’ phase-shift adaptation by solving (3.5a)–(3.5b) and calculate Gi , ∀i ∈

I. 5: Sort users according to Gi , so decoding starts from the UE with the highest channel gain. 6: Set j = 0. 7: repeat 8:

Set j = j + 1.

9:

for i ∈ I do

10: 11:

Determine the optimal UL TX power Pi∗ (j) by solving (3.7a)-(3.7c). end for

12: until |Pi∗ (j) − Pi∗ (j − 1)| ≤ ϵ, ∀i ∈ I, where ϵ ≈ 10−4 .

added in the final path gain values through QuaDRiGa [199]. Specifically, in QuaDRiGa, AWGN noise is used to generate spatially correlated LSPs, which statistically characterize the propagation environment. These LSPs indirectly shape the multipath environment and influence path gain. Thus, the AWGN values (i.e., I0 ), which are included in the calculation of the UE’s interference, represent reference noise levels. Indeed, these two distinct values are a correction applied to the UEs’ sensed interference, i.e., Ii , defined in (3.2), so that the optimization framework can be evaluated for the respective spectrum bands under the same configuration of the utility’s slope control parameters, defined as α = 0.3 and M = 3. Finally, the frequencies are centered to 5.9 and 28 GHz for the respective bands, the mobile UEs’ maximum power budget is set to Pimax = 23 dBm, and the sensitivity of the UAV’s RX is set to be equal to Ptol = −150 dBm. It is finally noted that in the following, the performance evaluation of the RIS-assisted and UAV-enabled communication system depicted in Fig. 3.3 will take place under 10, 100, and 1000 RIS elements for both the Sub-6 GHz and the mmWave band. Although it generally holds true that for a given RIS dimension, a mmWave RIS will host a higher number of RIS elements than a Sub-6 GHz RIS, given the same element spacing (e.g., λ/2), increasing the number of elements onboarded on the RIS also increases the complexity of its handling [179]. Consequently, a trade-off emerges between hosting thousands of RIS elements and maintaining ease of operation, particularly when experimenting in the mmWave band, where location inaccuracies are of utmost importance. 83

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

Start Randomly initialize each UE’s UL TX power, 𝑃! , within the feasible range UAV (Leader): Determine the effective RIS elements’ phase shifts, Θ∗ UEs (Followers): Determine the optimal UL TX power levels, 𝑃∗

Converged?

NO

YES End: Stackelberg Equilibrium (𝑃∗ ,𝛩∗ )

Figure 3.4: A hierarchical Stackelberg game-theoretic approach to energy-efficient power control. For the aforementioned reasons, investigating the number of the elements onboarded on the RIS is deemed necessary. The outcomes of the hierarchical game-theoretic power allocation in the Sub-6 GHz and mmWave bands are observed by considering the allocated UL TX power level and achieved UE utility both independently and collectively. In Fig. 3.5, we observe each UE’s allocated UL TX power (Figs. 3.5a, 3.5c) and achieved utility (Figs. 3.5b, 3.5d), versus their respective index under different number of RIS’s reflecting elements both for the Sub-6 GHz and the mmWave band. All UEs are sorted in descending order with reference to their channel gains, where the lowest index represents the UE with the highest channel gain, and the highest UE index corresponds to the one with the lowest channel gain. Notably, the framework’s power control [154] enables low channel gain UEs to transmit with lower power levels and achieve higher utilities. This behavior can be explained by the utility function which is affected by the UE’s channel-gain-to-interference ratio, i.e., GIii , at the exponent of the utility’s numerator (see (3.4)). In detail, low channel gain UEs usually exhibit a low channel-gain-to-interference ratio which forces the maximum point of the utility function to take a low power value (i.e., Pi ). This pattern is clearly illustrated in Fig. 3.5 where we also observe that as the number of RIS’s reflecting elements increases, their achieved utilities also increase (Figs. 3.5b, 3.5d), whereas the UEs’ allocated UL TX powers decrease (Figs. 3.5a, 3.5c), as expected. With regard to the two bands under study, in Fig. 3.5a we observe that for the Sub-6 GHz band, 100 84

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

1011

no RIS 10 elements on RIS 100 elements on RIS 1000 elements on RIS

10-1

User's Utility

User's Power [W]

100

10-2

10-4

1

2

3

4

106

5

1

2

3

4

User's index

User's index

(a) Sub-6 GHz: allocated UL TX power per UE.

(b) Sub-6 GHz: achieved utility per UE. 1011

10-2

10-3

10-4

no RIS 10 elements on RIS 100 elements on RIS 1000 elements on RIS

1

2

3

4

108

5

5

no RIS 10 elements on RIS 100 elements on RIS 1000 elements on RIS

User's Utility

10-1

User's Power [W]

no RIS 10 elements on RIS 100 elements on RIS 1000 elements on RIS

10-3

1

2

3

4

User's index

User's index

(c) mmWave: allocated UL TX power per UE.

(d) mmWave: achieved utility per UE.

5

Figure 3.5: Evaluation of the hierarchical game-theoretic power control framework per UE, under different number of RIS elements and different bands. reflecting elements on the RIS are enough for efficient power control, while for the mmWave band, a minimum of 1000 elements is needed as seen in Fig. 3.5c. This is due to the high propagation losses of the latter band [230], which stress the need for bigger RIS surfaces [176] to enable effective beamforming and therefore more efficient power allocation. The overall performance gain of the entire system (e.g., the NOMA cluster) is illustrated in Fig. 3.6, where we observe both the sum UEs’ allocated power levels and utilities as a function of a different number of RIS elements both for the Sub-6 GHz and the mmWave band. We can deduce that for a small number RIS elements (e.g., 101 RIS elements), the system’s performance is comparable to the case without a RIS (i.e., 100 RIS elements). When the number of RIS elements increases, the sum UEs’ powers decreases while their sum utility/satisfaction increases, as expected. 85

10 0

10 11

Sum users' utility

Sum users' power [W]

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

10 -1 Sub-6 GHz band mmWave band

10 -2 10 0

10 1

10 2

Sub-6 GHz band mmWave band

10 10

10 9 10 0

10 3

Number of RIS elements

10 1

10 2

10 3

Number of RIS elements

(a) Sum allocated power across the NOMA cluster.

(b) Sum utility across the NOMA cluster.

Figure 3.6: Evaluation of the hierarchical game-theoretic power control framework on the entire system, under different number of RIS elements and different bands. Notably, the increased number of RIS elements results in the reduced sum of UEs’ powers and thus in their increased sum utility/satisfaction. We also observe that even if the system follows the same power allocation trend in the two bands, the sum of the allocated powers to the mmWave UEs is lower compared to the case of their Sub-6 GHz counterparts. It is reminded that the devised utility function for energy efficiency, captures the trade-off between increasing γi (which improves the QoS) and minimizing Pi (which reduces power consumption). Since the increase in γi is limited by a low GIii , increasing Pi will lead to diminishing returns in the numerator of the utility function. i In detail, when the ratio GIii is low, the SINR i.e., γi = PiIG becomes small. In this case, the i

exponential term e−αγi approaches 1, meaning that even if the UE increases Pi , the utility does not increase significantly because the SINR is limited by the high interference. Thus, the utility function reaches its maximum at a lower power level Pi because increasing power doesn’t significantly improve the SINR. Therefore, UEs with low GIii transmit at lower power levels to avoid unnecessary energy consumption, as increasing Pi will not substantially improve their utility. Our power control framework favors UEs with lower GIii to transmit with lower power levels. In our work, mmWave UEs experience low GIii levels, and hence have a lower CQI compared to the UEs of the Sub-6 GHz band they are compared to. As a result when ultimately compared, given the low GIii levels of the former, they get to transmit with lower power levels compared to their Sub-6 GHz counterparts. Finally, in Fig. 3.7, we present the path gains, P Gi , for every UE, i, with and without the inclusion of the RIS both for the Sub-6 GHz and the mmWave band. All the UEs are sorted in descending order, and thus the UE with the lowest index is the UE with the best channel gain

86

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

NO RIS

-75

10 elements on RIS

1000 elements on RIS

9.

0. -8

-7

93 8. 0 .6 05

3 User's index

-1

2

2 .1

9 .8

06

-1

04

-1

8 .4 03

-1

-105 1

1 .9 01

-1

62

7.

-9

03 6.

56 9.

80 9.

-9

-9

9

4 5.

-9

-9 27 8.

64

8.

-9

-9

-100

-110

-8

0

31

02

4.

38 4.

-9

-95

1 8.

1. -9

-90 -9

Path Gain [dB]

-85

-8

03

4.

-8

00

44

-80

100 elements on RIS

4

5

(a) Sub-6 GHz path gains. -95

NO RIS

10 elements on RIS

100 elements on RIS

1000 elements on RIS

9.

-9 4 10

2

2

3 User's index

4

23

47

1

9 .0

25

. 25

-130

-1

-1

-125

. 22

-1

6 .6 16 -1 2 .5 17 -1 2 .6 17 -1

-120

.3

3

7 .8

.6 0 .0 16 -1 4 .6 16 2 .7 16

-1

-1

-115

11 -1

-1

10

3

4

57

-1

. 08

-1

.4

.3

09

-110

09

-1

-1

Path Gain [dB]

.2 03

7 .3 04 -1 1 .0 05 9 .0 05

-1

-1

-105

-1

69

-100

5

(b) mmWave path gains.

Figure 3.7: Path gains for two different spectrum bands. conditions, while the UE with the biggest index corresponds to the UE with the worst channel. Specifically, Fig. 3.7a depicts the path gains for the Sub-6 GHz band with the carrier frequency centered at 5.9 GHz. Based on the reported findings, the performance achieved with a minimum of 10 RIS elements closely resembles the performance without the RIS. A noticeable difference at ∼ |1.41| dB between the aforementioned cases is only observed for index 4, which, within the scope of this work, indicates a UE with poor CQI. Discernible differences can be observed for all the UEs when focusing on the cases of 100 or 1000 RIS elements. In detail, an average difference of ∼ |4| dB is observed when collectively comparing all the UEs without and with the existence of RIS in the topology under 100 RIS elements. Specifically, among all the UEs, the UE with index 4, achieves the most notable difference, by achieving a |7.27| dB higher value at −97.62 dB in the presence of 100 RIS elements. In the absence of RIS, the corresponding value is reported at −104.89 dB. Last, the benefits of including 1000 elements on RIS are clearly illustrated in Fig. 3.7a, where the average path gain value for all the UEs considered is marked at −84.1 dB, ∼ 16.5% higher compared to the 87

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS average value of −100.77 dB achieved without the RIS inclusion. Fig. 3.7b depicts the path gains obtained for the mmWave band with the carrier frequency centered at 28 GHz. Contrary to the findings of the conducted evaluation in the Sub-6 GHz, in the mmWave band, a minimum number of 1000 RIS elements is required to achieve distinguishable differences between the cases with and without a RIS. In this band, the UE that significantly benefits from the RIS technology with 1000 onboarded elements is the UE that experiences the worst CQI conditions, identified by UE ID 5. Specifically, the latter UE reports a |14.15| dB higher value at −111.32 dB compared to the case without the RIS, where the path gain is marked at −125.47 dB. On average, UEs under the presence of 1000 RIS elements in the topology report path gains ∼ |7.72| dB or ∼ 7% higher than in the absence of RIS. Precisely, the average path gain value of the UEs in the case of 1000 RIS elements is reported at −107.15 dB, while in the case of RIS absence this value is indicated at −114.87 dB. Conclusively, in the Sub-6 GHz band, the average path gain value reported with 100 RIS elements is marked at −96.5 dB, a value ∼ 4% higher than the average path gain of −100.77 dB reported in the absence of RIS. In the mmWave band, with 1000 RIS elements, the corresponding increase is found to be ∼ 7%. Based on these findings, a minimum number of 100 RIS elements is deemed necessary to support the deployment of RIS in the Sub-6 GHz band, while in the mmWave this number is found to be 1000. It is finally noted that the performance evaluation of the Stackelberg game-theoretic optimization process involved elements in increasing orders of magnitude, representing successive powers of 10. The aforementioned numbers not only represent typical values encountered in the literature [172, 179], but also meet the requirements outlined in [18], which state that existing RIS elements can range from 10 elements and above. In the following, we also provide a performance evaluation under different sets of RIS elements for the respective bands, aiming to determine the minimal number of RIS elements each band requires to achieve higher levels of energy efficiency. Based on the experimental results in Fig. 3.8, it is noted that for the Sub-6 GHz band, 500 RIS elements result in similar performance to 1000 (Figs. 3.8a and 3.8b), while discernible performance improvements are observed from 100 elements and above, with 300 being an indicative value. For the mmWave band, it is noted that the respective values are 700 (resulting in similar performance to 1000) and 500 (indicative value for performance improvement), correspondingly (Figs. 3.8c and 3.8d).

88

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

no RIS 100 RIS el. 300 RIS el. 500 RIS el. 1000 RIS el.

10-1

1011

User's Utility

User's Power [W]

100

10-2

10

10-4

1

2

3

4

106

5

2

3

User's index

(a) Sub-6 GHz: power.

(b) Sub-6 GHz: utility.

4

5

4

5

1011

no RIS 100 RIS el. 500 RIS el. 700 RIS el. 1000 RIS el.

10-3

1

2

no RIS 100 RIS el. 500 RIS el. 700 RIS el. 1000 RIS el.

User's Utility

10-2

10-4

1

User's index

10-1

User's Power [W]

no RIS 100 RIS el. 300 RIS el. 500 RIS el. 1000 RIS el.

-3

3

4

108

5

1

2

3

User's index

User's index

(c) mmWave: power.

(d) mmWave: utility.

Figure 3.8: Evaluation of the hierarchical game-theoretic power control framework per UE, under different numbers of RIS elements per each band. 3.1.3.3

Comparison with the state-of-the-art In this subsection, we evaluate our optimization framework using two different utilities

for energy efficiency, as shown in Table 3.2. The utility function proposed in our work is labeled Proposed Utility and is compared with the utility presented in [187], referred to as Literature Utility. To assess the performance of our proposed method in comparison with [187], we also consider a NOMA cluster of 4 UEs, a bandwidth of 5 MHz, and the same noise levels as in [187], and we evaluate the performance in the Sub-6 GHz band. Finally, we consider two cases: one without the inclusion of a RIS and the other with 100 RIS elements onboard.

89

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

Table 3.2: Utility Functions Comparison

M

Proposed Utility:

10-2

1011

3

Literature Utility Proposed Utility

Sum users' utility

Sum users' power [W]

Literature Utility:

W · (1 − e−αγi ) Ui (Pi , P−i ) = Pi W · log2 (1 + γi ) EEi (Pi , P−i ) = Pi

10-3

10-4

Literature Utility Proposed Utility

2 1 0

e e d d se se tur tur po era ropo era o t t i r i - L l. - P -L -P el. e RIS o RIS S I IS o N N 0 R 00 R 0 1 1

ure osed rop -P

rat

IS

R No

(a) Allocated UL TX power per UE.

-

e Lit

IS

R No

e Lit

el. S I IS 0 R 00 R 10 1

ure osed rat rop -P . l e

(b) Achieved utility per UE.

Figure 3.9: Evaluation of the hierarchical game-theoretic power control framework under two different utilities given in Table 3.2. The performance evaluation results reported in Fig. 3.9 clearly indicate the suitability of the proposed utility function in yielding lower transmission power levels (Fig. 3.9a) and achieving higher energy efficiency (Fig. 3.9b). This is due to the flexible design of our proposed utility through the control parameters, as well as its effective capture of the trade-off between achieving higher QoS and lower costs (e.g., reduced TX power levels).

3.1.4 Performance Evaluation of RIS-Assisted C-V2X Communication Systems on Colosseum As mentioned in Section 3.1.3.2, the optimization problem solved by the UAV and defined in Eqs. (3.5a)–(3.5b), will result in the maximization of its total received signal strength through the individual maximization of each UE’s incoming signal strength. Indeed, the latter is achieved through a linear combination of the RIS elements’ effective phase shifts as given in (3.8), which will enable H Θh . In consistency with Section 3.1.3.1, the creation of the cascaded channels denoted as hRU iR

90

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS all the incoming UE signals at the UAV are the result of the coherent summation of the direct, i.e., hiU , and reflected signals. It is reminded that the path gains of the corresponding generated channels, without and with the inclusion of a different number of RIS elements are provided and illustrated in Fig. 3.7. As a next step, we proceed by installing the generated channels in Colosseum. In summary, the reference input scenario to be installed includes two key components: the path gain, represented as a complex coefficient value in the form of FIR, and the Time-of-Arrival (ToA) value. The path gain is derived from the coherent summation of generated MPCs across the wireless links and has been averaged across multiple experiments, as detailed in Section 3.1.3.1. Meanwhile, the ToA values have been computed based on the Euclidean distance between the entities considered in the topology (i.e., UAV, RIS, and UEs), given the speed of light (i.e., 3 × 108 [m/s]). These values collectively contribute to generating the time-variant CIR for each node pair within the topology. Finally, a tutorial on the installation of RF scenarios in Colosseum can be found in [21, 65], but as it is not the focus of this section, the detailed explanation is omitted. We focus on the case of the Sub-6 GHz band, where all the wireless channels were simulated with QuaDRiGa at the center frequency of 5.9 GHz. We only consider the scenario with 100 RIS elements for all the RIS-assisted generated channels. This approach allows us to explore how a relatively small number of RIS elements (typically large arrays are required [231, 232]), and an average numerical increase of ∼ 4% in the path gains between cases with and without RIS in the topology, can significantly improve network performance. Additionally, since previous results (see Section 3.1.3) have shown that mmWave UEs are mostly benefited in terms of energy efficiency (i.e., they get to transmit with lower power levels, especially for 1000 RIS elements, while simultaneously satisfying their QoS demands), we aim at examining the advantages introduced by RIS technology in a realistic NextG RAN scenario tested in the Sub-6 GHz band. It is noted that although the channels were created at the frequency of 5.9 GHz using the QuaDRiGa channel modeling simulator, the testing frequency for this scenario in Colosseum is set to 1 GHz, which is the frequency that the MCHEM is optimized to work [233]. We leverage the capabilities of the SCOPE framework [5] to deploy an end-to-end RAN and core network through the srsRAN [41] softwarized open-source protocol stack on Colosseum. Specifically, we deploy a 3GPP-compliant cellular network comprising a single BS (i.e., the UAV) and 5 UEs distributed across 2 different slices. These are: (i) eMBB that concerns high traffic modeling of high-quality multimedia content and streaming applications, and (ii) URLLC for time-sensitive applications, such as autonomous driving in V2X scenarios as detailed in [26]. Slice-based traffic is 91

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

(a) eMBB DL throughput CDF — same scheduling policy(b) URLLC buffer occupancy CDF — same scheduling policy across slices.

across slices.

(c) eMBB DL throughput CDF — different scheduling poli-(d) URLLC buffer occupancy CDF — different scheduling cies across slices.

policies across slices.

Figure 3.10: Evaluation of a slice-based multi-UE C-V2X communication system on Colosseum with and without the inclusion of RIS under different scheduling profiles. generated by the MGEN TCP/UDP traffic generator [100], based on the following specifications: eMBB UEs request 4 Mbps constant bitrate, while URLLC UEs generate 89.3 kbps Poisson traffic. The bandwidth of the BS is set equal to 10 MHz (i.e., 50 PRBs), and it is divided between the 2 slices. Specifically, we allocate 45 PRBs on the eMBB, while the remaining ∼ 5 PRBs are assigned on the URLLC. In this way, we allocate 90% of the available resources to eMBB, which has the greatest demand for higher throughput. Consequently, 10% of the available resources are allocated to support the requirements for low latency on the URLLC. Among the 5 users considered, 3 of them are URLLC UEs, while the remaining 2 are assigned on the eMBB slice. It is also noted that the UEs of the eMBB are those who reported the highest path gain values, while the UEs of the URLLC are the ones with the lowest reported CQI values. With this allocation, we aim at exploring how the presence of the RIS in the topology will benefit the URLLC slice in a scenario where limited resources are available to serve 60% of the available UEs, especially when they experience poor transmission conditions (e.g., in the absence of RIS in the topology). Finally, the KPM observed

92

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS for eMBB is the DL throughput, while the respective value for URLLC is the buffer occupancy as a proxy for latency [26]. By using SCOPE’s open APIs, we fine-tune the RAN’s functionalities, in terms of scheduling profile selection. Specifically, two MAC-layer scheduling algorithms are compared, namely the RR and a fairer one, the WF, in all possible combinations. In Table 3.4, we present all the configurations considered in terms of scheduling profiles (defined in Table 3.3) for the eMBB and URLLC slices, which define how PRBs are internally allocated to UEs belonging to each slice [25]. Table 3.3: Scheduling Profiles Catalog

RR:

Round Robin

WF:

Waterfilling

Table 3.4: Scheduling Profiles Configuration

Setup ID

eMBB

URLLC

1

RR

RR

2

RR

WF

3

WF

RR

4

WF

WF

The results of this comparative performance evaluation can be seen in Fig. 3.10. Finally, all subsequent results were produced by taking the median as the most representative statistical value of a dataset, and averaged over multiple repetitions of experiments in the DL direction of the communication system1 . In Figs. 3.10a and 3.10b, we focus on the case where both slices are served under the same scheduling policy. Based on the reported findings, when focusing on the eMBB slice, all configurations, both with and without the RIS inclusion, deliver a median throughput value of 1

Note that all channels generated with QuaDRiGa in the RIS-assisted and UAV-enabled scenarios are reciprocal. The focus of this Section is the evaluation of a RIS-assisted and UAV-enabled communication system on the DL direction.

93

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS ∼ 4 Mbps. However, Setup 1 achieves one of the highest reported throughput values of 4.236 Mbps with 100 RIS elements. The corresponding value in the absence of RIS is found to be 4.091 Mbps. With Setup 4 and 100 onboarded RIS elements, the median throughput value reaches 4.2 Mbps. The respective value in the absence of RIS is reported at 4.172 Mbps. On the URLLC slice, however, in the absence of RIS, both Setups 1 and 4 underperform compared to the case where RIS is included in the topology. Specifically, with Setup 1, the median value of the buffer size is reported at 158 byte, while for Setup 4, the corresponding value is marked at 127 byte. In both setups with RIS, a median value of 0 byte buffer occupancy is achieved, indicating zero latency. In Figs. 3.10c and 3.10d we include the results collected when both slices are served under different scheduling policies. On the eMBB, Setup 2 achieves the highest throughput value reported in the presence of 100 elements on RIS at 4.238 Mbps, while Setup 3 achieves one of the lowest at 4.184 Mbps. The respective values in the absence of RIS are reported at 4.163 Mbps and 4.048 Mbps, correspondingly. On the URLLC, in the presence of RIS, all setups resulted in empty buffers, reporting a median value of 0 buffer occupancy. In the absence of RIS, with Setup 2 the median value for buffer occupancy is marked at 105 byte, while with Setup 3 the corresponding value is found to be 412 byte. In Tables 3.5 and 3.6, all the results obtained without and with the inclusion of 100 RIS elements under different combinations of scheduling policies are collectively presented. Based on the reported findings in Table 3.5 we observe that in the presence of RIS in the topology, Setups 1 and 2 deliver almost identical performance on the eMBB, reporting median throughput values of 4.236 Mbps and 4.238 Mbps, respectively. Setups 3 and 4 deliver slightly worse performance, at 4.184 Mbps and 4.2 Mbps, correspondingly. On the URLLC, all setups resulted in optimal performance by achieving a median value of 0 byte buffer occupancy. The findings of Table 3.6 indicate that in the absence of RIS in the topology, Setups 2 and 4 yield similar performance, achieving median eMBB throughput values of 4.163 Mbps and 4.172 Mbps. The aforementioned setups also lead to improved URLLC performance among those compared, achieving buffer occupancies of 105 and 127 byte, respectively. Setups 1 and 3 achieve eMBB throughputs of 4.091 Mbps and 4.048 Mbps eMBB, respectively. The median values for buffer occupancy on the URLLC are 158 and 412 byte, with the latter being the highest recorded in the absence of RIS in the topology. It is noted that all setups resulted in empty buffers on the URLLC when RIS was included in the topology. In the absence of RIS, the reported median values for buffer occupancy vary around the average value of 200 byte and are included in Table 3.6. When RIS is not included in the topology, all configurations result in increased buffer sizes, indicating higher levels of latency in the 94

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS communication links. Table 3.5: KPMs for RIS-assisted and UAV-enabled C-V2X topologies.

Setup ID

eMBB [Mbps]

URLLC [byte]

1

4.236

0

2

4.238

0

3

4.184

0

4

4.2

0

Table 3.6: KPMs for UAV-enabled C-V2X topologies without RIS.

Setup ID

eMBB [Mbps]

URLLC [byte]

1

4.091

158

2

4.163

105

3

4.048

412

4

4.172

127

Conclusively, in resource-constrained environments (e.g., limited availability of PRBs), where the UEs may experience relatively poorer CQI conditions, the presence of RIS enhances the QoS experience of the URLLC by resulting in a buffer occupancy of 0 byte.

3.1.5

Conclusions In this work, we have leveraged QuaDRiGa’s open-source code to simulate a RIS-assisted

and UAV-enabled C-V2X topology which we evaluated under a different number of RIS elements both in the Sub-6 GHz and the mmWave portion of the spectrum. Through the formulation of a hierarchical - one leader - multiple followers - Stackelberg-based game-theoretic optimization framework, we were able to study the achieved energy efficiency of the topology on the aforementioned bands. Our 95

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS results on power control indicate significant power savings and UE satisfaction, specifically for the mmWave band. Finally, while focusing on the case of 100 elements mounted on the RIS and the Sub6 GHz band, we installed the generated channels on the Colosseum wireless network emulator and conducted an evaluation campaign under different setups of scheduling profiles, considering a fixed PRB allocation for both the eMBB and URLLC network slices. The performance evaluation results from network emulation on Colosseum indicate that in resource-limited conditions (i.e., a low number of allocated PRBs), the presence of RIS will counteract potential network degradation by resulting in empty buffers, thus ensuring low latency levels on the URLLC slice. Some interesting extensions of the current work include focusing on the study of stochastic UE mobility on the Colosseum Network Emulator and exploring the expected improvements introduced by the coexistence of RISs and UAVs, particularly in eliminating potential network coverage holes. Additionally, comparing QuaDRiGa RIS-assisted channels with other RIS-aided schemes and traditional beamforming represents an interesting direction for future research.

3.2

Bridging Open RAN and RIS

3.2.1

Introduction RISs serve as a transformative technology to revolutionize the cellular architecture of

NextG RANs. Previous studies have demonstrated the capabilities of RISs in optimizing wireless propagation, achieving high spectral efficiency, and improving resource utilization. At the same time, the transition to softwarized, disaggregated, and virtualized architectures, such as those being standardized by the O-RAN ALLIANCE, enables the vision of a reconfigurable Open RAN. In this work [28], we aim to integrate these technologies by studying how different resource allocation policies enhance the performance of RIS-assisted Open RANs. We perform a comparative analysis among various network configurations and show how proper network optimization can enhance the performance across the eMBB and URLLC network slices, achieving up to ∼ 34% throughput improvement. Furthermore, leveraging the capabilities of OpenRAN Gym, we deploy an xApp on Colosseum, the world’s largest wireless system emulator with hardware-in-the-loop, to control the BS’s scheduling policy. Experimental results demonstrate that RIS-assisted topologies achieve high resource efficiency and low latency, regardless of the BS’s scheduling policy. The Open RAN paradigm for O-RAN networks is pivotal in transforming cellular network design. Grounded in the principles of disaggregation, virtualization, and programmable radios,

96

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS components are interconnected through open, standardized interfaces, ensuring multi-vendor interoperability. This approach enables the deployment of flexible architectures built on cloud-native principles. The reconfigurability of the RAN is further enhanced through RICs, streaming RAN telemetry and implementing control policies via a closed-loop mechanism [1]. Both near- and non-real-time RICs, deployed at the RAN’s edge, leverage xApps and rApps to optimize network management and resource allocation. This functionality further advances the integration of AI for the RAN and AI on the RAN, fostering the development of intelligent and adaptive RAN architectures [16]. At the same time, RISs are envisioned to be an integral part of NextG cellular architectures, enabling smart propagation environments that can be controlled through programmable RF components [146]. Thanks to their reconfigurability, RISs can manipulate electromagnetic waves, reshaping wireless propagation to optimize spectrum utilization and efficiency, particularly in multi-operator networks [234]. This reconfiguration capability aligns with O-RAN’s objectives of developing adaptable and resource-efficient RANs. Intelligent control loops with xApps in O-RAN have attracted widespread attention from the research community [10]. In [25], the authors explore the impact of xApps embedding DRL agents to manage the BS’s slicing and scheduling control policies across various network deployments on the Colosseum testbed. In addition, several efforts have been conducted in the optimization of wireless propagation with RISs. Some previous works [154, 155, 187] have focused on the use of RISs and constructive beams to achieve coherent signal construction at the BS, for maximizing the achieved RSSI. Finally, a system-level experimental evaluation of RIS-enabled deployments and their cross-layer optimization, leveraging the full protocol stack on the Colosseum testbed has been discussed in [27]. 3.2.1.1

Contributions and Outline However, limited research has focused on bridging O-RAN and RISs at a system level. In

this study, we aim to address this research gap by evaluating network performance across various RIS-assisted topologies using the Colosseum Open RAN testbed [24]. We consider an Open RAN delivering services to the eMBB and URLLC network slices. Using OpenRAN Gym [10]—an opensource framework for experimentation in O-RAN—we deploy this Open RAN on the Colosseum network emulator and manage it through an xApp that controls the BS’s scheduling policy. By investigating various spectrum allocation policies and RIS deployments, we demonstrate how network

97

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS performance and resource efficiency are impacted differently for each slice. With this work, we aim to complement prior studies [27] by providing a comprehensive evaluation of RIS-assisted Open RANs on an O-RAN-compliant testbed. We believe that our findings and insights will contribute to a deeper understanding of the integration of RISs in NextG Open RANs. The remainder of this chapter is organized as follows. Section 3.2.2 describes the system model. Section 3.2.3 presents the optimization framework for shaping wireless propagation with RISs, while Section 3.2.4 discusses the resource allocation strategies. Section 3.2.5 details the experimental setup, while Section 3.2.6 discusses the experimental results. Finally, Section 3.2.7 draws our conclusions and outlines directions for future work.

3.2.2

System Model

3.2.2.1

Open RAN Framework We investigate an Open RAN system where UEs generate traffic with diverse profiles,

which is classified into the eMBB and URLLC network slices. We leverage an xApp tasked with reconfiguring the BS’s MAC-layer scheduling policies. The xApp selects a dedicated scheduling profile among multiple resource allocation algorithms (i.e., RR, WF, and PF), influencing how PRBs are internally allocated to the UEs of each slice [25]. It is noted that for the RF environment we consider various RIS-assisted network deployments. In Fig. 3.11, we outline the reference architecture considered in our work. 3.2.2.2

Channel Model We consider a topology consisting of a RIS, a BS, and a set of UEs, denoted by I =

{1, . . . , i, . . . , |I|}, which establish direct communication with the BS. The number of elements on the RIS is given as |M | and their set is defined as M = {1, . . . , m, . . . , |M |}. The first RIS element, i.e., m = 1, with its respective coordinates, i.e., (xR , yR , zR ) [m], is used as a reference point in the following arithmetic calculations, while for the BS, its coordinates are given as follows (xA , yA , zA ) [m]. The channel gain corresponding to the direct link between a UE i and the BS is given as hiA , the channel gain between a UE i and the RIS is defined as hiR , where hiR = [|hiR,1 |ejω1 , . . . , |hiR,m |ejωm , . . . , |hiR,|M | |ejω|M | ]T , and finally, the channel gain of the reflected link from the RIS to the BS is denoted as hRA . The diagonal phase-shift matrix is defined as Θ = diag(ejθ1 , . . . , ejθm , . . . , ejθ|M | ), while each RIS element’s phase shift is θm ∈ [0, 2π], ∀ m ∈ M , H Θh . Finally, the steering vector for the UE i to and the cascaded channel is therefore given as hRA iR

98

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS 2π

2π

′′′ = [1, e−j λ dϕiR , . . . , e−j λ (|M |−1)dϕiR ]T , while the steering vector for the RIS link is given as hiR 2π

2π

′′′ = [1, e−j λ dϕRA , . . . , e−j λ (|M |−1)dϕRA ]T . In the above the RIS to the BS link is denoted as hRA

calculations, d is the Euclidean distance, λ[m] is the carrier wavelength and d[m] is the antenna separation. Additionally, ϕRA is the cosine of the signal’s AoD for the RIS to the BS link, and ϕiR is the cosine of the signal’s AoA for the UE i to RIS link. It is noted that we consider the links between the UEs and the BS to be NLOS, while the channels connecting the UEs to the RIS and the RIS to the BS are assumed to be LOS. These transmission conditions are chosen to ensure that the cascaded channel formed by the RIS provides a sufficiently strong link for redirecting the UEs’ signals to the BS. Such placement can represent a maneuverable flying RIS mounted on an UAV, capable of dynamically establishing LOS to UEs that would otherwise encounter NLOS relative to a fixed-position BS. Lastly, both the BS and the UEs are equipped with single-antenna omnidirectional TXs/RXs, with their center frequency set to 5.9 GHz, where RISs are considered for deployment[18]. For the generation of the wireless links, we leverage QuaDRiGa’s [200] statistical ray-tracing capabilities, which is based on channel sounding and geometry-based channel modeling. We observe the channels every 0.1 s and select the 3GPP 38.901 UMa scenario for the channel generator. A number of MPCs (denoted as |N |) are produced for each generated link, with the channel generation procedure based on the following flow adopted from [27]: 1. Generate scenario-specific MPCs (denoted as h′xy ) for each link, leveraging QuaDRiGa. 2. Combine the MPCs coherently to compute the channel gain for every link, following Eq. (3.10): ′′ = hxy

|N | X

′

h′xy · ejφxy ,

(3.10)

j=1

The MPC gain is denoted as h′xy , where x ∈ {i, R}, y ∈ {R, A}, and |h′xy |, φ′xy are the respective MPC magnitude and phase. ′′ ) over 100 different channel realizations to 3. Calculate the average of the channel gains (i.e., hxy

determine the link’s final channel gain (i.e., hiA ). ′′′ and 4. Multiply the computed channels by their corresponding steering vectors, denoted as hRA ′′′ , to obtain the respective final channel gains (i.e., h hiR RA , hiR ).

5. Calculate the overall channel power gain for the UE i to BS link as follows Gi = |hiA + H Θh |2 . hRA iR

99

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

Figure 3.11: Reference O-RAN testing architecture for RIS-enabled deployments, focusing on the Sched xApp Case Study, as described in Section 3.2.6.3.

3.2.3

Optimizing Wireless Propagation with RIS In this work, the wireless environment is not treated as an uncontrollable entity but is

instead subject to optimization. For this purpose, we leverage the capabilities of RISs to reshape the wireless propagation by effectively adapting the phase shifts of the RIS elements. For these reasons, we formulate the following optimization problem defined in Eqs. (3.11a)-(3.11b), as follows: max θ

s.t.

|I| X

H |hiA + hRA ΘhiR |2

(3.11a)

0 ≤ θm ≤ 2π, ∀m ∈ M.

(3.11b)

i=1

It is noted that the solution to the aforementioned optimization problem, which takes place on the BS’s side, is the effective phase shift vector of the RIS elements, denoted as θ ∗ . In order to maximize the overall channel power gain of the UEs, the BS only needs to calculate the effective phase shifts of the RIS elements (i.e., θ ∗ ). Given the non-convexity of the aforementioned optimization problem, computing a globally optimal solution is challenging. To derive a closed-form phase-shift solution, we aim to achieve coherent signal alignment for the incoming signals from different transmission paths at the BS. Indeed, the UE’s channel power gain is maximized when the direct signal and the signal reflected by the RIS are perfectly aligned and coherently combined at the BS’s RX [154, 155]. This condition is generally satisfied when the phase shifts of the direct and 100

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS 



H Θh cascaded signals are aligned as ∠hiA = ∠ hRA iR . Therefore, in the single-UE case there’s an

1 × |M | phase-shift vector θ ∗ = ∠v, where the optimal phase shifts of the RIS elements are given as follows: 2π d(m − 1)ϕRA , ∀m ∈ M. (3.12) λ In the multi-UE case, the solution to the maximization problem is also a linear combination ∗ θm = ∠hiA + ωm +

of the phase shifts of the RIS elements, with the reflection-coefficient vectors vi being distinct for each UE. Therefore, there exists a distinct reflection-coefficient vector vi = [vi,1 , . . . , vi,|M | ] ∈ C|M |×1 for each UE i that maximizes its channel power gain. To determine the set of suitable weights for each UE that will maximize the aggregate channel power gain across all UEs as defined in Eq. (3.11a), we adopt the research methodologies proposed in [187, 235]. For each UE i, we define an appropriate weight factor wi ∈ [0, 1] and compute the linear combination of the overall UEs’ reflection-coefficients vi as follows: v=

|I| X

wi vi , i=1 P |I|

(3.13)

wi = 1. In the multi-UE case, the goal is to

such that the weight factors satisfy the condition

i=1

determine the optimal values of the weight factors (i.e., wi ) for the reflection-coefficient vector (i.e., v) of Eq. 3.13, which maximize the overall channel power gain of the UEs, as defined in Eq. (3.11a). Therefore, in order to compute the effective RIS elements phase shifts, the optimization problem defined in Eqs. (3.11a)-(3.11b) is formulated as follows in Eqs. (3.14a)-(3.14c): max w

s.t.

|I| X

H ΘhiR |2 |hiA + hRA

(3.14a)

i=1

0 ≤ wi ≤ 1, ∀i ∈ I

(3.14b)

|I|

X

wi = 1

(3.14c)

i=1

Through backward induction [187], the RIS reflection matrix Θ = diag(ej∠v ) is computed to optimize the phase alignment. This approach ensures that the final RIS configuration Θ maximizes both the channel gain of each UE and the total channel power gain across all UEs. Given that w = [w1 , . . . , wi , . . . , w|I| ] is the vector of the UEs’ assigned weight factors, the optimization problem described in Eqs. (3.14a)-(3.14c) consists of a non-negative linear objective function and a set of constraints, which can be intuitively optimized to derive the optimal weights w∗ . The derivation of the optimal weights w∗ leads to an effective RIS elements’ reflection-coefficient vector v∗ = [v1 , . . . , vm , . . . , v|M | ], which subsequently determines the corresponding phase shifts of the RIS elements, expressed as θ ∗ = [θ1 , . . . , θm , . . . , θ|M | ]. 101

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

3.2.4

Resource Allocation Strategies We consider three different case studies, encompassing both single and multi-UE scenarios,

which are evaluated under various resource management policies in terms of PRB allocation and scheduling profile selection. Case Study A. Following the requirements outlined in [18], which specify that existing RIS panels can host a minimum of 10 elements, we evaluate three network configurations, where the number of RIS elements increases by orders of magnitude. We consider a single UE allocated to the eMBB network slice, which is assigned ∼ 1/3 of the available bandwidth resources. The CSI is varied to evaluate the contribution of RIS to network performance given the same resource availability, which is considered fixed. The results of this analysis are provided in Section 3.2.6.1. Case Study B. Considering the trade-off between hosting thousands of RIS elements and maintaining operational simplicity, we focus on a RIS with 100 elements, serving multiple UEs, which are distributed across the eMBB and URLLC network slices. By dividing the available spectrum between the two, we aim to demonstrate how a RIS can significantly enhance network performance, improve UE satisfaction, and increase PRB utilization based on the capacity demands of each slice. The findings of this investigation are provided in Section 3.2.6.2. Case Study C. This case study focuses on an O-RAN infrastructure where an xApp controls the scheduling profile selection of the BS’s DU. The xApp updates the scheduling profile of the BS’s scheduler through a control loop by alternating its policy given the following options: RR, WF, and PF. A multi-UE use case is considered, where eMBB UEs are provisioned with 90% of the available bandwidth, while URLLC UEs are allocated the remaining 10% of the spectrum resources. With this case study, we aim to study the impact of an xApp frequently changing the scheduling profile from fair scheduling (i.e., RR) to spectral efficiency-driven (i.e., WF), or balanced scheduling (i.e., PF), analyzing its effect on resource allocation and overall network performance in both RIS-assisted and non-RIS-assisted topologies. The results of this exploration are provided in Section 3.2.6.3.

3.2.5

Experimental Setup To experimentally evaluate the RIS-assisted channels within an O-RAN ecosystem, we

leverage the capabilities of OpenRAN Gym, an experimental toolbox for the end-to-end development, implementation, and testing of diverse solutions—including, but not limited to, AI/ML applications— within O-RAN. The software offers capabilities such as RAN and core network deployments using 102

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS the srsRAN [41] protocol stack. It supports large-scale data collection, testing, and fine-tuning of RAN functionalities by integrating open APIs for slicing and scheduling control, as well as KPMs collection. Additionally, the software incorporates an O-RAN-compliant control architecture for executing xApps in the near-real-time RIC. Through the E2 interface, which bridges the RAN and RIC, along with its SMs [1], the streaming of KPMs from the RAN and the execution of control actions by the xApps are enabled. We deploy OpenRAN Gym on Colosseum, which features 128 SRNs consisting of pairs of Dell PowerEdge R730 servers and NI USRP X310 SDRs, supporting large-scale experimentation in diverse network deployments. The testbed’s MCHEM component emulates the wireless environment by leveraging FPGA-based FIR filters. These filters simulate RF conditions, including path loss, fading and attenuation based on models created through various frameworks (e.g., ray-tracing software, analytical models, or real-world measurements). Similarly, the Colosseum MGEN TCP/UDP traffic generator [100] emulates a variety of network traffic profiles (e.g., multimedia content) and demand distributions (e.g., Poisson, periodic). We consider a cellular network comprising one BS and up to 5 UEs distributed across 2 network slices. These are: (i) eMBB, for high-traffic scenarios involving multimedia content and streaming applications; and (ii) URLLC, designed for time-critical applications such as vehicle

00

44

0.

9.

-8

-7

03

4.

-8 93

8.

10

8.

-8

-8

1

.3

1 -9 02 4. -9 8 3 4.

-9

0 -1

80

56

9.

9.

-9

-9

62

7.

3

-9

.0

9 .4

6 -9

5 -9 64

27

8.

8.

-9

-9

1. 0

2

103

.6

.1

06

05

-1

-1

8

9

.8

04

.4

03

91

-1 -1

Figure 3.12: Path Gains under varying numbers of RIS elements.

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS coordination in C-V2X environments. The maximum bandwidth considered in this work is set to 10 MHz (i.e., 50 PRBs) and is divided among the 2 slices. Traffic is slice-based and generated according to the following specifications: eMBB UEs request a constant bitrate of 4 Mbps, while URLLC UEs generate Poisson traffic at 89.3 kbps. In terms of physical deployment, the BS is placed at the point (xA = 25, yA = 50, zA = 25) [m] of the three-dimensional space. Their distance from the RIS is given as di,R = [20, 27, 37, 58, 66] [m] and all are uniformly distributed around the RIS’s reference point, denoted as (xR = 30, yR = 40, zR = 20) [m]. We install the generated links in Colosseum. The Colosseum scenario includes the path gain (i.e., a complex FIR coefficient) and the ToA values, which are calculated based on the Euclidean distance between the active entities (i.e., BS, RIS, and UEs), using the speed of light. These values are used to generate the time-variant CIR for each node pair in the network.

3.2.6

Experimental Evaluation Table 3.7: Catalog of Network Configurations

Config. Number ID of UEs

UE ID(s)

Slice Type

Bandwidth [MHz]

RIS Assisted

xApp Enabled

I

1

5

eMBB

3.6

No RIS

No

II

1

5

eMBB

3.6

100 RIS el.

No

III

1

1

eMBB

3.6

10 RIS el.

No

IV

1

1

eMBB

3.6

1000 RIS el.

No

V

5

{1, 2} & {3, 4, 5} eMBB & URLLC

5&5

No RIS

No

VI

5

{1, 2} & {3, 4, 5} eMBB & URLLC

5&5

100 RIS el.

No

VII

5

{1, 2} & {3, 4, 5} eMBB & URLLC

9&1

No RIS

Yes

VIII

5

{1, 2} & {3, 4, 5} eMBB & URLLC

9&1

100 RIS el.

Yes

In Fig. 3.12, we present the path gains for each UE, i, with and without a RIS, generated through the channel modeling process and the optimization procedure as described in Sections 3.2.2.2 and Section 3.2.3, respectively. All the UEs are sorted in descending order, such that the UE with the lowest index corresponds to the UE with the best channel, while the UE with the highest index corresponds to the UE with the worst. Based on the reported results, the performance with as few as 10 RIS elements is nearly identical to that observed without the RIS. Improvements in path gains are evident across all UEs when examining scenarios with 100 or 1000 RIS elements. 104

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

(a)

(b)

(c)

Figure 3.13: Performance evaluation results for the single-UE Case Study. In Table 3.7, we present a catalog of the various network configurations evaluated within the scope of this work. Configs. I-IV focus on a single UE, whereas configs. V-VIII address the multi-UE case study, where scheduling decisions influence intra-slice resource allocation. Regarding configs. VII-VIII, an xApp tasked with reconfiguring the scheduling profile selection is evaluated. The xApp updates the scheduling profile of the BS at a granularity of 1 s, assigning one of three scheduling profiles (i.e., RR, WF, and PF) to the UEs of the eMBB and URLLC slices, cycling through all possible combinations. It is noted that all the results presented in this chapter have been averaged over multiple repetitions of the experiments. 3.2.6.1

Single-UE Case Study A In Fig. 3.13, we compare the UE with the best channel conditions, as shown in Fig. 3.12

(i.e., UE with ID 1), to the UE with the worst (i.e., UE with ID 5). In Fig. 3.13a, focusing on Configs. I and II (Table 3.7), which correspond to the UE with the worst channel, we observe that the median throughput reported with 100 RIS elements is 3.834 Mbps, representing an increase of ∼ 10%

105

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

(a)

(b)

(c)

Figure 3.14: Performance evaluation results for the multi-UE Case Study. compared to the 3.486 Mbps observed with no RIS. Similarly, when focusing on the UE with the best channel (Configs. III and IV), we observe that the median throughput reported with 1000 RIS elements is 4.154 Mbps, representing an ∼ 10% increase compared to the 3.8 Mbps achieved with only 10 RIS elements. It is reminded that according to the experimental results presented in Fig. 3.12, the performance achieved with 10 RIS elements is nearly identical to that observed without a RIS in the topology. Based on the aforementioned experimental evaluation, we observe that increases in the order of 102 in the size of the panels (e.g., from no RIS to 102 RIS, or from 101 RIS to 103 RIS) can result in a ∼ 10% increase in the reported throughput under the same resource availability (e.g., 18 PRBs or 3.6 MHz bandwidth). In Figs. 3.13b and 3.13c, we present the corresponding CQI and MCS values as reported by the protocol stack. The results in Fig. 3.13b demonstrate the channel improvement introduced by the RIS in the topology, particularly for the UE with the best channel conditions (Configs. III-IV), which reports a ∼ 13% improvement in the median CQI value (from 8 to 9). Similarly, the MCS (Fig. 3.13c) increases from a median value of 13 to 16, enabling higher data rates given the same PRB availability. Regarding the UE with the worst channel (Configs. I-II), the MCS shows an increase of ∼ 18%, rising from 11 to 13, while the CQI remains relatively stable around 7, with a

106

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS

(a)

(b)

(c)

(d)

Figure 3.15: Performance evaluation results for the multi-UE O-RAN Case Study.

(a)

(b)

Figure 3.16: Performance evaluation results for the multi-UE Case Study with and without the inclusion of the Sched xApp in O-RAN.

107

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS modest increase of ∼ 8% for the RIS topology. 3.2.6.2

Multi-UE Case Study B We examine the multi-UE case study, where UEs are allocated to specific slices based

on their traffic demands. To express UE satisfaction (which corresponds to high resource utilization), we define a new metric, i.e., the PRB Ratio. Its definition is given as PRB Ratio = Sum of Granted PRBs Sum of Requested PRBs , where PRB Ratio ∈ [0, 1], and it represents the amount of allocated PRBs to the

slice according to the channel and traffic conditions. The analysis focuses on Configs. V and VI (Table 3.7), where the UEs with the best reported channel conditions (Fig. 3.12) are grouped together on the eMBB slice, while the remaining UEs are assigned to the URLLC. The WF algorithm is selected for the scheduling profile of the eMBB (which has the highest throughput requirements), while RR is used for the URLLC (which has the lowest). Fig. 3.14 presents the complete performance evaluation results for the aforementioned case. Fig. 3.14a illustrates the median eMBB throughput for the RIS and non-RIS assisted topologies. The median throughput with a RIS is 2.973 Mbps, ∼ 34% higher than the 2.219 Mbps achieved without a RIS, under the same resource availability. Fig. 3.14b shows that both topologies (with and without a RIS) result in a median buffer occupancy of 0 in the URLLC, indicating significantly reduced latency levels.2 Fig. 3.14c shows that although both topologies indicate near-zero latency, the inclusion of a RIS (Config. VI) leads to higher utilization efficiency of the available PRBs at the BS’s scheduler. 3.2.6.3

Multi-UE O-RAN Case Study C In Fig. 3.15, focusing on Configs. VII and VIII, as detailed in Table 3.7, we present the

results of the experimental evaluation where the Sched xApp controls the BS’s scheduling policy. Fig. 3.15a shows that the median eMBB throughput with a RIS in the topology is 4.234 Mbps, compared to 4.1 Mbps without a RIS. These high throughput values are reasonable, as the eMBB UEs are allocated 45 PRBs, representing 90% of the available resources. Fig. 3.15b, demonstrates that in the presence of a RIS, the median URLLC buffer occupancy is 0, whereas in its absence, the corresponding value is 688, indicating high latency levels. Therefore, the results show that the presence of 100 RIS elements significantly reduces latency levels, regardless of the BS’s scheduling policy. On the contrary, non-RIS-assisted topologies in resource-limited environments (i.e., with only ∼ 10% of the available bandwidth) experience performance degradation, even when the BS 2

It is noted that since srsRAN does not directly measure latency, buffer occupancy is used as a proxy [25].

108

CHAPTER 3. OPTIMIZING WIRELESS PROPAGATION WITH RIS frequently updates its scheduler from fair (e.g., RR) to more efficient approaches (e.g., WF or PF) to accommodate the UEs based on their distinct QoS demands. Finally, Figs. 3.15c and 3.15d demonstrate that RIS-enabled topologies achieve higher resource utilization for both the eMBB and URLLC slices. Figs. 3.16a and 3.16b compare the previously discussed topologies to the case where no xApp reconfigures the BS’s scheduling policy. Notably, the keyword baseline, as shown in Fig. 3.16, refers to srsRAN’s default scheduling policy, which is RR. Fig. 3.16a shows that updating the BS’s scheduling profile increases resource efficiency for URLLC in the absence of a RIS (Config. VII). However, in the presence of a RIS (Fig. 3.16b), resource utilization and hence UE satisfaction improve significantly (Config. VIII). Therefore, both the presence of RIS in the topology and frequent updates of the BS’s scheduling policy enhance the resource utilization efficiency.

3.2.7

Conclusions In this work, we focused on an Open RAN and investigated the impact of various network

management policies on the performance of RIS-assisted topologies, leveraging the capabilities of OpenRAN Gym on Colosseum. Our experimental results demonstrate that a RIS deployment can enhance network throughput and resource utilization under the same bandwidth availability. In addition, our experimentation in an emulated O-RAN system, where the BS’s scheduling policy is controlled by an xApp, demonstrates that RIS-assisted topologies achieve superior network performance and higher resource efficiency compared to their non-RIS-assisted counterparts, regardless of the scheduling policy employed. Part of our current and future work focuses on designing AI/ML solutions to dynamically control the selection of scheduling policies in such Open RANs, leveraging CSI and traffic conditions in the decision-making processes. Extensions to the O-RAN control plane for carrying RIS weights, determined by the scheduler for real-time network adaptation, are also left for future work.

109

Chapter 4

AI-RAN Link Adaptation Leveraging DTs 4.1

Introduction AI-powered RAN networks have attracted significant attention from both industry and

academia. Meanwhile, DTs offer a safe playground for experimenting with AI/ML-based solutions for advanced AI-RAN research. By enabling the testing of online algorithms before deployment on the RAN, they reduce costs and safety risks associated with physical field testing. In this chapter, we propose ARIADNE, an online RL-based module that seamlessly integrates with SIONNA and is tasked with performing link adaptation. We explore different design choices and demonstrate how ARIADNE can surpass industry-standard and state-of-the-art methods by achieving up to 11% and 20% improvements in Spectral Efficiency, respectively. Finally, we show that RL learns a MCS selection strategy that diverges from Outer Loop Link Adaptation (OLLA), exhibiting either more conservative or more aggressive behavior depending on the configuration, a trend further corroborated by training offline on 5G OTA measurements. Recent years have seen collaboration among leading industry entities such as Nokia and NVIDIA to advance AI-RAN research [236]. AI-native 6G is expected to emerge from simulation, with DTs playing a key role in the train–simulate–deploy–optimize lifecycle [20]. Indeed, DTs are envisioned to enable faster innovation by allowing the evaluation of “what-if” scenarios for future technologies that are not yet available in hardware [24]. By reducing the “concept-to-live” cycle, dense urban environments and complex 6G use cases can be simulated with greater accuracy,

110

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS supporting the design and delivery of NextG systems while accelerating the adoption and deployment of advanced AI solutions [20]. In this context, NVIDIA’s SIONNA DT environment [23] provides an all-in-one platform for wireless research, enabling the execution of system-level simulations over ray-traced channels. In particular, SIONNA-SYS allows the on-the-fly integration of plugin AI/ML solutions, facilitating the testing of key features and capabilities essential to AI-native 6G research. Simulated, controlled environments often provide a practical first step for developing AI/ML solutions before conducting OTA experiments, allowing for algorithm refinement and parameter optimization. At the same time, a trending topic that has gained widespread attention in the research community is the optimization of the gNB’s MAC-layer scheduler. This involves the optimal allocation of PRBs and selection of the MCS index, as well as the tuning of power control mechanisms (e.g., 3GPP-compliant open and closed-loop power control) that adjust transmit power based on predefined Signal-to-Noise-Ratio (SNR) targets to meet QoS requirements for different slices and verticals. Importantly, by jointly optimizing power control and MCS selection, Spectral Efficiency can be significantly improved, demonstrating the benefits of a coordinated approach for enhanced network management.

4.1.1

Related Work Numerous works have focused on RL for MCS selection [237], ranging from offline

approaches [238] to contextual Multi-Armed Bandit (MAB) solutions [239, 240]. The authors in [241] introduce GrGym, an RL-based framework for MCS selection in WiFi, which leverages a custom gym environment. In [242], the authors formulate an RL framework where a low discount factor enforces myopic MCS selection based on the current channel state. The industry standard for link adaptation is OLLA [243], and modifications involving RL have been introduced for intelligent on-the-fly adaptation [244]. More recently, the authors in [245] propose a gradient descent approach with a learning rate that self-adapts online through knowledge distillation. However, none of the works mentioned above focuses on the integration of an AI module on the fly within high-fidelity DTs.

4.1.2

Contributions Motivated by the suitability of system-level simulations for DT environments, we propose

ARIADNE [29], a framework that leverages NVIDIA’s platform SIONNA to integrate an RL-module

111

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS for AI-RAN networks, enabling learning-based link adaptation through adaptive MCS selection on channels simulated via SIONNA ray tracing. Our module seamlessly integrates with the SIONNASYS platform, enabling direct comparison with state-of-the-art link adaptation algorithms (such as the industry-standard OLLA [243] and NVIDIA’s proposed link adaptation scheme, entitled Self-Adaptive Link Adaptation (SALAD) [245]). In this chapter, we train and evaluate ARIADNE on a variety of high-fidelity ray-traced channels generated with SIONNA, and we demonstrate that online RL outperforms both OLLA and SALAD in MCS selection in terms of performance, as measured by the achieved Spectral Efficiency. Finally, we evaluate the suitability of RL on offline 5G data collected from an OTA 5G testbed, demonstrating its effectiveness on real-world measurements.

4.2

ARIADNE: Learning Link Adaptation via Reinforcement Learning Our RL agent operates at the Fast Link Adaptation level, where decisions (i.e., the selection

of MCS levels for all UEs) are enforced on a per-slot basis. This requires precise knowledge of the channel quality. However, CQI reports are subject to feedback delay [246]. In addition, CQI is typically reported over a wide band, while transmission may occur over a narrower band. As a result, the true SINR is not directly observable and must be estimated online at each slot using only past feedback. Consequently, link adaptation requires jointly inferring the channel quality and selecting an appropriate MCS [245]. Therefore, for the current slot’s channel quality, we consider two estimation modes: an Oracle mode that establishes an upper bound using SIONNA’s simulated environment to calculate the SINR, and a predictor mode for SINR estimation that leverages previous CQI reports. Markov Decision Process (MDP) Formulation and Temporal Resolution. We formulate the MCS selection problem as an MDP defined by the tuple (S, A, R, P, β). Critical to our design is a 1 : 1 mapping between the RL environment and the 5G NR PHY-layer, where each discrete environment step corresponds exactly to one 5G NR time slot.

4.2.1

State Space At each time slot t, the agent observes a state vector st ∈ R(Ncqi +Nharq +5)·U , where U is

the number of UEs. The state is composed of per-UE feature vectors: h

(1)

(U )

st = st , . . . , st 112

i

,

(4.1)

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS (u)

where each per-UE component st (u)

st

h

is given by

(u)

(u)

(u)

(u)

(u)

(u)

(u)

= γ̃post,t , γ̃eff,t−k , ht−k , m̃t−1 , B̂t , ∆offset,t , āt

i

,

(4.2)

and the respective components are defined as follows: (u)

• γ̃post,t : An estimate of the current slot’s normalized post-equalization SINR, computed as the mean SINR across allocated resource elements after Linear Minimum Mean Square Error (LMMSE) equalization. In Oracle mode, this is derived from the current channel realization; in predictor mode, it is estimated from past measurements. (u)

• γ̃eff,t−k , k = 1, . . . , Ncqi : The Ncqi most recent effective normalized SINR values reported by the PHY-layer Abstraction. (u)

• ht−k , k = 1, . . . , Nharq : A window of Nharq prior Hybrid Automatic Repeat reQuest (HARQ) outcomes, where h ∈ {−1, 0, 1} denotes unscheduled, NACK, and ACK, respectively. (u)

• m̃t−1 : The normalized previously selected MCS index. (u)

• B̂t : The sliding-window Block Error Rate (BLER) estimate over the last Nbler scheduled (u)

slots, calculated as B̂t

=

P

NACK /

P

Scheduled.

(u)

• ∆offset,t : A normalized asymmetric feedback accumulator updated by +0.1 on ACK and −0.9 on NACK.1 (u)

• āt : The running ACK rate over the full episode.

4.2.2

Action Space The agent performs link adaptation for the 5G NR Physical Downlink Shared Channel

(PDSCH). The action at ∈ {0, 1, . . . , 28}U is a vector of MCS indices, one per UE, as defined in the 5G NR MCS Table 1 [6]. The mapping is performed directly by the policy network as follows in (4.3): 

(1)

(U )

at = π(st ) = mt , . . . , mt 1

(u)

(u)

Both ∆offset,t and āt



.

(4.3)

are derived from HARQ feedback; the former weights NACK outcomes 9× more heavily than (u)

ACK, making it more sensitive to link failures than the symmetric ACK rate āt .

113

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS

4.2.3

Reward We aim to capture the fundamental MCS selection tradeoff. Higher MCS indices result in

higher Spectral Efficiency, but simultaneously increase the BLER and, consequently, the probability of decoding failure. Conversely, a conservative MCS lowers the BLER but also reduces the Spectral Efficiency. To target the practical throughput–reliability tradeoff, we maximize a reward that captures the effective Spectral Efficiency, which is directly proportional to system goodput. Instantaneous Reward. At each environment step (i.e., each 5G NR time slot), the agent selects an MCS index for each UE, and the environment subsequently returns a scalar reward based on the successfully delivered Spectral Efficiency in that slot. The instantaneous reward rt at slot t is defined as the total achieved Spectral Efficiency across all UEs, as expressed in (4.4): U X

rt =

(u)

n

(u)

(u)

Q(mt ) · Rc (mt ) · ⊮ Ht

o

= ACK ,

(4.4)

u=1 (u)

(u)

where Q(mt ) and Rc (mt ) denote the modulation order and the effective code rate, respectively, (u)

associated with the MCS index mt

selected for UE u at slot t. The term ⊮{·} denotes the (u)

indicator function, which evaluates to 1 if the HARQ outcome Ht

for the u-th UE is a successful

acknowledgment (ACK), and 0 otherwise. Expected Reward. In the SIONNA-SYS simulator, the HARQ feedback is generated by (u)

the PHY Abstraction module as a Bernoulli outcome with success probability Pr(ACKt

=1|

(u) (u) (u) st , at ) = 1 − BLER(mt , γt ), where the BLER depends on the selected MCS index mt and (u) the effective SINR γt computed from the post-equalization SINR at slot t. (u) (u) (u) Given that the nominal Spectral Efficiency is defined as SEnom,t ≜ Q(mt ) · Rc (mt ) for a chosen

MCS, the expected per-slot reward is given in (4.5): E[rt | st , at ] =

U X u=1

(u)

(u)

(u)

(u)

Q(mt ) · Rc (mt ) · 1 − BLER(mt , γt ) . |

{z

SEnom (mt )

(4.5)

}

Reward Interpretation. The expectation in (4.5) evaluates the effective Spectral Efficiency, which is directly proportional to the system goodput. Our RL agent is tasked with maximizing the immediate per-slot reward, which over an episode is equivalent to maximizing the time-average effective throughput. Therefore, our reward formulation naturally captures the fundamental link adaptation tradeoff, with the RL agent learning a policy that selects the MCS to maximize the product SEnom · (1 − BLER), thereby learning the rate–reliability tradeoff through trial and error. Adaptive BLER Penalty. Although the reward in (4.4) implicitly penalizes BLER through zero Spectral Efficiency on NACK, this signal alone may not be sufficient to prevent persistently 114

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS aggressive MCS selection. Link adaptation mechanisms such as OLLA target a fixed BLER level (typically around 10% [6, 243, 245]), balancing throughput and reliability. To incorporate a similar (u)

notion of reliability control, we additionally augment the reward with a per-UE penalty weight λt , governed by an integral controller that increases when the cumulative BLER exceeds a target τ and remains zero otherwise, as defined in (4.6)–(4.7): rt =

Xh

(u)

(u)

SEnom,t ⊮{Ht

(u)

(u)

i

= ACK} − λt ⊮{Ht

= NACK} ,

(4.6)

(u)

i

(4.7)

u

where (u)

λt

h

= max 0, kE

X

⊮{Hi

= NACK} − τ .

i≤t (u) Hi ̸=∅ (u)

When λt

4.2.4

= 0 for all u ∈ {1, . . . , U }, (4.6) reduces to (4.4).

Transition Dynamics The environment transitions P (st+1 |st , at ) are determined by the SIONNA-SYS simulator

as detailed in Section 4.3.

115

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS

4.3

System Model

20

40

gNB UE1

UE1

UE 2

gNB

UE 2

gNB

gNB

UE 3

SINR Coverage Map

0

UE3

2 UE

1 UE

SIONNA-RT

dB 20

-40

a) Simple Street Canyon

UE 3

c) Étoile, Paris, France

b) Munich, Germany

Digital Replica of Real-World

Simple Street Canyon

Munich, Germany

gNB gNB

Étoile, Paris, France

SIONNA-SYS-LEVEL

# decoded bits Topology (3D Scene)

PF Scheduler

Power Control

User Mobility

Sionna RT Channel

RZF Precoding + LMMSE Equalization

post-eq SINR

PHY Abstraction Effective SINR MCS

BLER Tables HARQ

Link Adaptation Channel Quality

OLLA Δ-SINR offset

SALAD

ARIADNE

GD SINR est.

π(s) → MCS

slot++

Figure 4.1: System-level simulations using SIONNA over ray-traced channels for link adaptation with OLLA, ARIADNE, and SALAD. We compare three link adaptation strategies, namely OLLA, SALAD, and ARIADNE. All operate within the same SIONNA-SYS pipeline, and the decision-making component resides within the gNB’s MAC-layer scheduler. Independent of the specific algorithm employed, this reflects a centralized architecture in which the gNB collects CQI and HARQ feedback from all UEs, executes the link adaptation algorithm internally, and assigns MCS indices to scheduled UEs for the upcoming transmission. The per-slot pipeline is illustrated in Fig. 4.1.

4.3.1

System Architecture and Link Adaptation Workflow The end-to-end pipeline is shown in Fig. 4.2. At the beginning of each simulation, a

3D scene (e.g., Urban Street Canyon, Munich, or Étoile in Fig. 4.1) is loaded into SIONNA’s ray tracer to compute the channel frequency response Ht per slot. At each slot, the simulator executes 116

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS proportional-fair scheduling, power control, and Regularized Zero-Forcing (RZF) precoding with LMMSE equalization, yielding post-equalization SINR values. Link adaptation then selects an MCS index based on the SINR and HARQ feedback. This is the only component that differs across methods (Fig. 4.1). OLLA maintains a per-UE SINR offset updated via ACK/NACK feedback and selects the highest MCS under a target BLER, using the latest SINR report and HARQ outcome. SALAD primarily relies on ACK/NACK feedback, estimating the SINR online via recursive updates and adapting the BLER target through a feedback control loop. ARIADNE observes a window of past CQI and HARQ feedback, together with a predicted SINR, and directly maps this state to MCS indices via a trained policy network. Finally, PHY Abstraction maps the selected MCS and post-equalization SINR to a BLER, from which the HARQ outcome is sampled, yielding the achieved Spectral Efficiency. All steps except link adaptation are identical across methods, ensuring that any performance difference is solely due to the MCS selection strategy. for i=1:N_channel_realizations

Channel Generation SIONNA-RT 𝑎 , 𝜏!

i)load_scene() Physics-based 3D Geometry 1. ‘simple_street_canyon’ 2. ‘etoile’ 3. ‘munich’

gNB MAC Layer Processing SIONNA-SYS

𝑯 * 𝑇 for i=1:N_slots

ii)PathSolver() !

iii)paths.cfr()

Ray Tracing Engine 1. Multipath propagation 2. Reflection, diffraction etc. 3. ITU-R materials

Bridging RT with Signal Processing 1. RT paths converted to CFR on OFDM grid 2. MIMO-OFDM channel matrix

compute_sinr 𝐇 predict_sinr() List of SINR predictors:

• • • •

end ΜΑC-Layer Link Adaptation SIONNA-SYS

ARIADNE Episode Start: H =pool[randint()] (k)

for i=1:N_slots 𝜋) 𝑠" (MlpPolicy)

end

𝑟"

env.step(m( )

𝑚"

_get_obs()

MCS Enforcement

𝑠"#$

OLLA

% & 𝑚#$(

𝑠"

PPO Update %

Decision Tree Random Forest Kalman Filter OCO

end Bernoulli BLER Calc.

SALAD

' harq $ {ℎ"#! }!%&

i)PFSchedulerSUMIMO ii)downlink_fair_power _control iii)RZFPrecodedChannel iv)LMMSEPostEqualizati onSINR &

)

SINR history buffer

𝛾post, #$% %'(

PHY Abstraction SIONNA-SYS T.B. size Calc.

MCS Decode

&

𝛾eff, #

EESM

Figure 4.2: End-to-end execution of MCS selection in SIONNA. The channel matrix is generated from a pool of ray-traced channel realizations to ensure diversity. The current SINR is either directly used or estimated from previous observations. The post-equalization SINR is then provided as input to the PHY Abstraction. The state st is observed by the ARIADNE framework, which selects the corresponding MCS index.

4.3.2

Implementation and Training We train the agent using PPO [97] as implemented in Stable-Baselines3 [247] within a

custom Gymnasium environment [248] built on top of the SIONNA system-level simulator, with ARIADNE handling agent–environment interaction. The policy is parameterized by a two-layer

117

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS Multilayer Perceptron (MLP) with 64 hidden units per layer, a learning rate of 3 × 10−4 , a clip range of ε = 0.2, and an entropy coefficient of 0.05 to encourage exploration. We set the discount factor β = 0, treating the problem as a contextual bandit. This reflects the observation that MCS selection is primarily driven by instantaneous channel state, and temporal credit assignment for future slots does not improve performance. Finally, in our setup, we assume three mobile UEs while the ray-traced channels correspond to SIONNA’s Simple Street Canyon scenario.

4.4

Experimental Evaluation ARIADNE observes a configurable window of Ncqi past effective SINR reports and Nharq

prior HARQ outcomes, as highlighted in Section 4.2.1. We evaluate two configurations, as shown in Table 4.1. In Setup A, the observation window leverages Ncqi = 3 reports and Nharq = 10 outcomes, providing the agent with temporal context. In Setup B, the observation is limited to a single past SINR report and a single HARQ outcome. Finally, all experimental results reported below have been collected over multiple experiments and channel realizations. Table 4.1: Observation Size configurations for ARIADNE. Parameter

4.4.1

Setup A

Setup B

CQI window (Ncqi )

3

1

HARQ window (Nharq )

10

1

Impact of Perfect and Imperfect Channel State Information We begin our experimental evaluation by quantifying the impact of various SINR estimators

on system performance, where the predicted current slot’s normalized post-equalization SINR, (u)

denoted as γ̃post,t , serves as input to the RL model, as described in Section 4.2.1. Performance is evaluated in terms of the achieved mean SE and mean BLER. In detail, when using the Oracle as input to ARIADNE’s RL state, we rely on the ground-truth SINR at the current slot, as reported by SIONNA-SYS. When using one of the predictors, namely DT [249], Kalman Filter (KF) [250], RF [251], and Online Convex Optimization (OCO) [246], we replace the Oracle with the predicted SINR in order to assess how closely the resulting performance matches that of the Oracle, which serves as an upper bound. We then compare all the aforementioned RL-based methods against state-of-the-art baselines, namely OLLA and SALAD. Finally, we also discard the current slot’s 118

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS predicted SINR as input to the RL model (denoted as delayed CQI (dCQI) in the plots) and instead

le rac

O

DT

KF

2 0. 10 3 0. 10 1 0. 10 6 0. 10 0 0. 12 0

41 0. 04 9

0.15

Target

0.10

0. 0

3

Mean BLER

94

2. 8

4

3. 24 5

0.20

0. 11

3. 68 8 3. 64 5 3. 63 0 3. 61 3 3. 60 9 3. 60 7

Mean SE [bps/Hz]

rely solely on past CQI observations, evaluating the resulting performance across all methods.

0.05 0

I RF CQ CO LLA LAD d O O SA

le

ac Or

DT

KF

I RF CQ CO LLA LAD d O O SA

Figure 4.3: Mean Spectral Efficiency and BLER for all RL methods and baselines under Setup A from Table 4.1. Regarding the inputs used by each SINR predictor,2 KF, DT, and RF leverage past SINR observations, KF and OCO incorporate the most recent HARQ feedback, while DT and RF rely on (u)

the HARQ accumulator (i.e., ∆offset,t ) to capture feedback trends. In contrast, in our implementation, OCO does not rely on explicit SINR history and instead operates using only HARQ outcomes and the previously selected MCS index [246]. Table 4.2: Performance comparison with respect to the Oracle. Method

SE [bps/Hz] ∆SE vs. Oracle [%] BLER

Oracle RL

3.688

0.0

0.112

DT RL

3.645

-1.17

0.103

KF RL

3.630

-1.57

0.101

RF RL

3.613

-2.03

0.106

dCQI RL

3.609

-2.14

0.100

OCO RL

3.607

-2.20

0.120

OLLA

3.245

-12.01

0.041

SALAD

2.894

-21.53

0.049

2

We do not aim to directly compare the predictors. Instead, we assess whether the performance of the RL agent is affected by the predictor choice, focusing on the robustness of the approach. The KF is a constant-velocity filter with HARQ-driven bias correction, adaptive process noise, and innovation gating. The DT and RF are trained online on past SINR observations, and retrained every Nslots . The OCO follows the ACK/NACK-only feedback setting of [246], using mirror descent with Fixed-Share expert mixing over 12 experts defined on the grid η ∈ {0.5, 1, 2, 3}, β ∈ {0, 0.15, 0.3}, with α = 0.

119

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS In Fig. 4.3, we observe that ARIADNE, regardless of the predictor used, consistently outperforms OLLA and SALAD. Even without prediction of the current slot’s SINR, denoted as dCQI in the plots, the RL agent relying only on past CQI reports outperforms OLLA by 10% in terms of spectral efficiency, achieving a mean spectral efficiency of 3.609 bps/Hz compared to 3.245 bps/Hz, at the cost of a higher mean BLER of 0.1, whereas OLLA achieves a BLER of 0.041. Finally, OLLA outperforms SALAD by approximately 11%, while achieving similar BLER values. For the configuration of SALAD, we use the default parameters as specified in [245]. It is worth noting that SALAD may require further parameter tuning for the specific scenario at hand, and therefore the parameterization used here may not be optimal for the current setup. This further highlights the advantage of RL in adapting to changing channel conditions and dynamically adjusting its policy.

Relative Frequency Relative Frequency

UE 1 0.85

0.85

0

Oracle DT KF RF OCO dCQI

5

10 15 20 25 MCS Index

Oracle DT KF RF OCO dCQI

0.4 0.2 0

5

10 15 20 25 MCS Index

UE 1 OLLA SALAD

UE 3

UE 2

5

10 15 20 25 MCS Index

Oracle DT KF RF OCO dCQI

0.5 0

5

UE 2 1

0

10 15 20 25 MCS Index UE 3

OLLA SALAD

0.5 0

1

1

OLLA SALAD

0.5

5

10 15 20 25 MCS Index

0

5

10 15 20 25 MCS Index

Figure 4.4: Distribution of Selected MCS Indices across Methods and UEs under Setup A from Table 4.1. In Table 4.2, we provide a summary of how closely all methods perform relative to the Oracle when used as input to ARIADNE. We observe that DT achieves a Spectral Efficiency closest to the Oracle, with only ∼ 1% lower performance, reporting a mean Spectral Efficiency of 3.645 bps/Hz compared to 3.688 bps/Hz for the Oracle, followed by KF, whose Spectral Efficiency is reported at 3.630 bps/Hz. All RL methods achieve a Spectral Efficiency close to the Oracle, within ∼ 2%. 120

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS Finally, OLLA reports a mean Spectral Efficiency that is ∼ 12% lower compared to the Oracle, while SALAD underperforms all methods by ∼ 20%. However, all RL methods report a mean BLER of 0.1, indicating the tradeoff between high throughput and reliability. In contrast, OLLA and SALAD maintain the BLER well below the 10% target, at the cost of lower Spectral Efficiency. Fig. 4.4 shows the relative frequency distribution of selected MCS indices across different SINR predictor methods and UEs. Notably, the mean SINR observed per UE is ∼ 26 dB for UE 1, ∼ 6 dB for UE 2, and ∼ 28 dB for UE 3. We observe that, for UE 1 and UE 3, all RL methods select the highest MCS indices, indicating agreement across methods in high-SINR regimes. In contrast, for UE 2, which experiences the lowest SINR, the RL agent adopts a more aggressive strategy, selecting MCS indices between 5 and 15, with most selections concentrated around 8–12. On the other hand, OLLA and SALAD predominantly select low MCS indices, concentrated below 10, indicating a more conservative approach.

4.4.2

Impact of the Adaptive BLER Penalty We evaluate the impact of different kE values, as defined in (4.6)–(4.7). Since the previous

results do not explicitly constrain the BLER and yield values above 10% for all predictor modes and the Oracle, in contrast to SALAD and OLLA, which maintain BLER below 0.1, we vary kE ∈ {0, 0.025, 0.1, 0.5}. Here, kE = 0 corresponds to no penalty, kE = 0.025 provides a moderate penalty, and kE = 0.1 and kE = 0.5 impose increasingly strong penalties on the median BLER. The resulting performance is summarized in Table 4.3 and further illustrated in Fig. 4.5. As expected, increasing kE leads to more conservative MCS selection, reducing Spectral Efficiency while improving reliability. In particular, kE = 0 achieves the highest Spectral Efficiency but at the cost of a higher BLER, while larger values of kE progressively reduce the median BLER at the expense of Spectral Efficiency. Among all configurations, kE = 0.1 provides the best tradeoff, achieving high Spectral Efficiency while consistently maintaining the BLER below the 10% target (Fig. 4.5). Finally, OLLA and SALAD remain the most conservative.

121

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS

ke = 0.5

ke = 0.025 1

ke = 0.1

0.75

0.75

0.75

0.5

0.5

CDF

OLLA 1

CDF

ke = 0

CDF

1

0.25

0.25

0.25

0 2.4 2.8 3.2 3.6 SE [bps/Hz]

4

0

0.04

0.1 BLER

SALAD

Target

0.5

0.16

0

16

20 MCS

Figure 4.5: CDFs of Spectral Efficiency, BLER, and MCS under Setup A from Table 4.1, leveraging the DT predictor.

Table 4.3: Impact of kE on Median Spectral Efficiency, BLER, and MCS. kE

SE [bps/Hz]

BLER

MCS

0

3.547

0.096

21

0.025

3.520

0.072

20

0.1

3.421

0.047

19

0.5

3.342

0.037

18

OLLA

3.131

0.041

17

SALAD

2.882

0.049

16

122

Mean SE [bps/Hz]

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS

ke = 0

ke = 0.025

0

1

ke = 0.1

OLLA

SALAD

Target

4 3.6 3.2 2.8 2.4 2

3

4

5

6

7

8

5

6

7

8

Channel Index

0.15

Mean BLER

ke = 0.5

0.1

0.05 0

0

1

2

3

4

Channel Index

Figure 4.6: Spectral Efficiency and BLER under Setup A from Table 4.1, using the DT predictor

3.8

3.645

3.604

3.6 3.4

S

p etu

A S

p etu

Mean BLER

Mean SE [bps/Hz]

across multiple channel realizations and experiments.

Target

0.12

0.103

0.111

0.1

B

p etu

S

pB u t e

A S

Figure 4.7: Mean Spectral Efficiency and BLER for all RL methods and baselines under Setup A and B from Table 4.1, leveraging the DT predictor. Finally, in Fig. 4.6, we observe the mean Spectral Efficiency across each channel realization, averaged over multiple experiments. We observe that RL with kE = 0.1 consistently meets the 10% BLER target while achieving one of the highest Spectral Efficiency values among all considered methods, outperforming the industry-standard OLLA. In contrast, RL with kE = 0 and kE = 0.025 achieves higher Spectral Efficiency but often reaches or exceeds the BLER target.

123

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS

4.4.3

Impact of Observation Window Size We now proceed by evaluating how different observation window sizes impact the mean

performance of the RL agent. Once again, for the current slot’s SINR prediction, we leverage the DT, as it achieved the closest performance to the Oracle in terms of Spectral Efficiency in the previous evaluation. The parameterization for this step is given in Table 4.1. As shown in Fig. 4.7, a smaller observation window (Setup B) results in a slightly lower mean Spectral Efficiency of 3.604 bps/Hz compared to 3.645 bps/Hz achieved by Setup A. Although both Spectral Efficiency values are comparable, Setup A achieves a lower BLER (0.103) than Setup B (0.111). Both values remain above the 10% BLER target, which is expected since the adaptive penalty reward is not leveraged in these experiments. Overall, Setup A is preferred, as it achieves slightly higher Spectral Efficiency while maintaining a lower BLER, enabling the RL agent to select actions without significantly exceeding the BLER target.

4.5

Site-Specific Training and Robustness Across Channel Scenarios

Reward

1,000 800 Simple Street Canyon

600

0

100

Étoile

200 300 Episode

Munich

400

500

Figure 4.8: Training reward over episodes. Fig. 4.8 shows the training reward evolution across three different channel scenarios in SIONNA. The RL agent consistently improves its performance and converges in all considered environments. This indicates that the learned policy adapts to varying channel characteristics.

124

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS

4.6

Future Integration of RL on OTA 5G Testbeds To examine whether reward-driven MCS selection extends beyond simulation, we train

an Fitted Q-Iteration (FQI) agent [252] on OTA measurements collected from a 5G testbed [59], using both DL and UL data. The state consists of CQI, Reference Signal Received Power (RSRP), and instantaneous BLER; the reward is a proxy for achieved throughput derived from logged measurements; and the Q-function is an RF with γ = 0.5. As shown in Fig. 4.9, Q-function targets stabilize across Bellman iterations, and the learned policy shifts MCS selections away from the scheduler’s preferred indices. Since these datasets reflect only the OAI OLLA scheduler’s choices, closed-loop evaluation is left for future work. The fact that both this offline agent and the online PPO agent independently deviate from OLLA suggests that reward-driven MCS selection tends to identify

1,600 1,200 800 0

10 20 30 FQI Iteration

21 22

20

18 19

17

16

15

14

13

10 20 30 FQI Iteration

OAI Scheduler FQI Policy 11 12

0

60 40 20 0

MCS Index

60 40 20 0

OAI Scheduler FQI Policy 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

100

Percentage [%]

150

Percentage [%]

Avg. Q-value

Avg. Q-value

operating points that rule-based schedulers do not.

MCS Index

Figure 4.9: Offline FQI results on OTA data for UL (top) and DL (bottom). Left: Q-value convergence across Bellman iterations. Right: MCS distribution of the learned policy vs. the OAI scheduler.

4.7

Conclusions We developed ARIADNE, a framework that seamlessly integrates with SIONNA, leverag-

ing online RL for MCS selection. The use of system-level simulators, which enable the on-the-fly integration of AI modules, has facilitated the exploration of various approaches, spanning reward formulation and observation window design, to study the impact of different design choices on link

125

CHAPTER 4. AI-RAN LINK ADAPTATION LEVERAGING DIGITAL TWINS adaptation. Although the SIONNA integration provides a practical first step toward the design of RL-based link adaptation, future work will focus on the deployment of ARIADNE on OTA 5G testbeds. Preliminary results on both DL and UL 5G data demonstrate the potential of learning-based approaches in real-world scenarios.

126

Chapter 5

Enabling Spectrum Sharing in FR3 for Cellular and NTN Systems 5.1

Introduction FR3, encompassing frequencies between 7.125 and 24.25 GHz, is an emerging frequency

band for 6G applications. The upper mid-band, as it is frequently referred to, represents the sweet spot between coverage and capacity, providing better range than mmWaves and higher bandwidth than the sub-6 GHz band. Despite these advantages, the spectrum is already occupied by incumbent systems such as satellites (e.g., Starlink), and sharing it with terrestrial cellular applications results in spectrum conflicts, only exacerbating the existing spectrum scarcity. This chapter [30] investigates the impact of two state-of-the-art methods, namely QoS-Aware Power Control and Interference Nulling, as well as their joint application, on interference mitigation toward non-terrestrial links while maintaining acceptable QoS on terrestrial networks. Our simulation results demonstrate the advantages and disadvantages of each method, pinpointing how interference nulling can maintain high average performance and how power control is more appropriate for risk-averse scenarios to enhance fairness in terrestrial QoS. Finally, we showcase how the two can complement each other to enhance fairness in terrestrial QoS and increase the gNB’s energy efficiency, while suppressing interference toward incumbents. Indeed, FR3 has been identified as the golden band for 6G, as it satisfies both capacity and coverage requirements [19]. However, the rapid deployment of dense satellite communication systems introduces new challenges that necessitate robust interference management strategies. In

127

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS particular, coexistence between terrestrial and non-terrestrial networks remains a critical issue for the successful commercial deployment of 6G systems. Therefore, the implementation of advanced interference management techniques is essential. The challenge of interference mitigation in coexistence scenarios has been extensively studied in the literature [253]. Proposed strategies range from the use of highly directional antennas to adaptive beamforming and null steering designed to suppress radiation in undesired directions. Furthermore, spectrum masking, power control, and diversity techniques are commonly employed to minimize interference effects [15]. While these approaches are effective for managing interference in the FR3, they often face practical challenges related to hardware limitations and limited spectrum availability [15]. Specifically, in [254, 255], the authors underscore the importance of site-specific deployment strategies and context-aware policies in ensuring efficient coexistence between terrestrial and satellite systems. In [254], a spectrum-sharing scenario is examined in which a terrestrial network operating in the upper mid-band interferes with incumbent satellite communications. Interference mitigation is achieved via a closed-loop feedback mechanism that enables real-time adaptation of spectrum-sharing policies. This approach balances satellite protection with efficient spectrum utilization by dynamically adjusting the Exclusion Zone (EZ) radius whenever interference levels exceed predefined thresholds. In [255], spectral overlap is discussed with a focus on LOS links and angular alignment between the gNB and satellite RXs, which are identified as critical factors affecting interference. In [256], the authors propose an optimization framework to maximize the Weighted Sum Rate (WSR) in multi-UE multibeam LEO satellite networks by optimizing the beamforming weights to balance signal quality and interference mitigation. The work in [257] optimizes beamforming and power in dense LEO satellite networks to maximize system fairness. Beamforming vectors are calculated using SCA to maximize the minimum UE rate, while satisfying power and interference constraints. If power limits are exceeded, power scaling is applied to adjust the beamforming vectors. In [14], the authors focus on the trade-off between terrestrial SNR maximization and interference suppression, represented by the tunable parameter λ, and solve it as a joint maximization–minimization problem. Finally, in [258], the authors address satellite-terrestrial coexistence in FR3 by designing hybrid True-Time-Delay (TTD) array precoders at the gNB that steer wideband nulls toward satellites while preserving main-lobe gain for terrestrial UEs.

128

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

5.1.1

Contributions and Outline However, despite the aforementioned contributions, prior work on spectrum sharing and

coexistence has not thoroughly compared widely adopted interference suppression techniques, such as power control and spatial nulling. In this chapter, we aim to address this gap by considering the worst-case interference caused to satellites, which results from DL transmissions in the terrestrial network leaking into the satellite UL [14], as shown in Fig. 5.1. Additionally, contrary to prior works [14, 258], which focus on a limited number of satellites (e.g., up to 10 satellites within a constellation) sharing spectrum with terrestrial cellular networks, we consider deployments involving denser constellations. This allows us to evaluate the impact of interference suppression strategies on scalable coexistence scenarios, thereby pinpointing their strengths and limitations. Our contributions can be summarized as follows: • We systematically investigate the impact of Interference Nulling on the trade-off between terrestrial QoS and interference suppression at LEO satellites. • We formulate an optimization problem to suppress interference at the incumbents by minimizing the gNB’s DL transmit power, subject to throughput and interference-to-noise ratio (INR) constraints. • We provide a comparative analysis between power control strategies and interference nulling techniques, pinpointing the strengths and limitations of each method while identifying suggested use-cases. • We propose a joint interference nulling and power control strategy and benchmark it against standalone baselines. • We conduct our analysis in the context of dense satellite constellations, considering 40 LEO satellites to ensure the practical applicability of the proposed solutions under challenging scenarios. • We also evaluate the system’s performance using the energy efficiency KPM, a critical metric for designing 6G terrestrial networks. The remainder of this chapter is organized as follows. Section 5.2 describes the system model, and Section 5.3 presents the spectrum sharing strategies. Section 5.4 details the simulation setup, Section 5.5 introduces the KPMs used for performance evaluation, and Section 5.6 discusses 129

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS the simulation results. Finally, Section 5.7 concludes the chapter and outlines directions for future work.

5.2

System Model We focus on a topology comprising a terrestrial gNB serving K UEs in the DL. The

gNB is equipped with an Naz × Nel uniform planar array with Nt = Naz · Nel transmit antennas, (R)

(R)

while each UE employs an Naz × Nel

(R)

(R)

array with Nr = Naz · Nel

receive antennas. Both

the gNB and the UEs employ beamforming, selecting transmit and receive beamforming vectors wt and wr , respectively. It is noted that we employ time-division scheduling, where the gNB serves one UE per time resource and computes beamforming vectors wt and wr for the scheduled UE. We additionally consider Nsat LEO satellites visible from the gNB location, sharing the same spectrum. Each satellite is assumed to have a beam focused on the same area covered by the terrestrial gNB, making the satellite susceptible to interference from terrestrial transmissions. Finally, the k ∈ CNr ×Nt , for each UE MIMO channel matrix between the gNB and UE k is denoted as Hter

k ∈ {1, . . . , K}, while hj ∈ CNt ×1 denotes the MIMO channel vector from the gNB to the j-th satellite, where j = 1, . . . , Nsat . The time interval during which satellite trajectories are visible to the gNB is given as Nslots . Therefore, at time slot ti and for user k, the terrestrial channel matrix is denoted as Hkter (t) ∈ CNr ×Nt .

130

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

ce n e r rfe nnel e t In ha C Terr est ria

LEO Satellite

l DL

Terrestrial UEs

Terrestrial gNB

Figure 5.1: Interference from terrestrial cellular DL transmissions to satellite UL in terrestrial-LEO satellite coexistence scenario.

5.3

Terrestrial-Satellite Spectrum Sharing Strategies We begin by comparing two state-of-the-art methods, namely Interference Nulling and

QoS-Aware Power Control, to investigate their impact on terrestrial network QoS, while ensuring that the interference at the satellite remains below the International Telecommunication Union (ITU) reference levels in satellite–terrestrial spectrum sharing scenarios. Finally, we propose and evaluate a new coexistence method that combines Interference Nulling and QoS-Aware Power Control. The performance of the proposed algorithm is systematically evaluated and compared against the standalone methods.

5.3.1

Interference Nulling The first algorithm we consider is Interference Nulling [14], where the beamformed gain at

the terrestrial UE is traded off against the interference leakage toward the satellites via a regularization parameter, λ ≥ 0. The solutions to this optimization problem are the terrestrial transmit and receive opt

opt

beamforming vectors wt and wr , respectively, as detailed in (5.1): 131

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

 opt wropt , wt = arg max |wrH H̃ter wt |2 − λ w ,w r

subject to

t

∥wt ∥2 = 1

and

Nsat X

 2 ∥h̃H j wt ∥

j=1

(5.1)

∥wr ∥2 = 1.

H̃ter and h̃j denote the normalized versions of the MIMO multi-path channels Hter and hj , respectively, as normalizing the channel matrices compensates for the substantial path loss difference between terrestrial and non-terrestrial (i.e., satellite) links, thereby maintaining λ within a manageable range. To solve the optimization problem in (5.1), we follow the approach in [14], where opt

opt

wr is selected as the maximum left singular vector of H̃ter , and wt is chosen as the maximum H w w H H̃ − λ eigenvector of M = H̃ter r r ter

PNsat

H j=1 h̃j h̃j . It is noted that performing Singular Value

Decomposition (SVD) on Hter is equivalent to performing SVD on M for λ = 0, which is also equivalent to applying eigenvalue decomposition to M for λ = 0. Therefore, all the aforementioned techniques yield the same solution for the gNB’s transmit beamforming vector (i.e., wt ) when only beamforming is applied, without accounting for signal leakage toward the satellites. This equivalence serves as a numerical verification of our implementation, confirming that the framework from [14] correctly reduces to standard SVD-based beamforming when λ = 0.

5.3.2

QoS-Aware Power Control In this section, we propose and formulate a power control optimization problem that

minimizes the gNB’s transmit power while maintaining terrestrial QoS, in order to ensure satellite protection. We focus on the optimization of the DL transmit power of a single gNB, and we assume that power allocation decisions are made in a time-division manner, where, at each time slot, a single UE is scheduled. The interference-plus-noise levels experienced in the terrestrial network due to nearby operating gNBs are given in (5.2): Iiter =

X

Gn P n + σ 2 ,

(5.2)

n̸=i

where σ 2 is the AWGN power and

P

n̸=i Gn Pn is the sum of inter-cell interference power from all

neighboring cell sites. The SINR for each gNB-UE link is given in (5.3): SINRter i =

Pi |wr Hter wt |2 , Iiter

(5.3)

while the achievable rate for bandwidth W is given by the Shannon Capacity formula as follows in (5.4): 132

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

Ri (Pi ) = W log2 1 + SINRter . i

(5.4)

QoS-Aware Utility Function. The utility function we consider in our optimization problem is given in (5.5) as: 

ter

W · 1 − e−α SINRi Ui (Pi ) =

Pi

M

,

(5.5)

and is defined based on a network economics approach [27, 226], capturing the trade-off between the terrestrial network’s QoS satisfaction and the transmission power invested by the gNB. This formulation ensures that the gNB selects transmission power levels that strike a balance between meeting QoS demands in terms of achieved terrestrial SINR and power savings. In detail, due to the sigmoidal nature of the utility function numerator, small increases in transmission power initially improve the utility by enhancing the SINR. However, as the SINR increases, the utility function begins to saturate, meaning that further increases in transmission power yield diminishing returns in QoS satisfaction. Beyond this saturation point, increasing power levels further provides little improvement in utility, effectively preventing unnecessary power expenditure. Consequently, the sigmoidal function drives power allocation toward an optimum where the desired QoS is achieved with less power consumption. Finally, α and M are control parameters governing the slope of the utility function. To ensure that the utility function remains quasi-concave with an interior optimum over the feasible power range, these parameters are tuned based on the interference-plus-noise levels experienced in the terrestrial network [226], as given in (5.2). For satellite j and a gNB employing transmit beamforming vector wt , the INR is calculated as follows in (5.6): INRj = Pi + 10 log10 |wtH hj |2 + G/T − La − 10 log10 (W · kB ),

(5.6)

where G/T is the satellite antenna gain-to-noise-temperature ratio, La accounts for atmospheric and scintillation losses [13], and kB is the Boltzmann constant. Finally, the optimization problem is ultimately formulated in (5.7a)–(5.7d): maximize Ui (Pi )

(5.7a)

Pi

subject to

Ri (Pi ) ≥ ϵRimax ,

(5.7b)

INRj (Pi ) ≤ INRmax ,

(5.7c)

Pimin ≤ Pi ≤ Pimax , 133

∀i ∈ I,

(5.7d)

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS and pertains to the selection of the power budget at the gNB side that satisfies the operator’s throughput constraints, while providing maximum interference reduction to the satellites, according to ITU’s requirements. Therefore, for each link i ∈ I, the gNB-UE link is allocated the full transmit power budget of the gNB, denoted as Pi , and which satisfies the constraints defined in (5.7b)-(5.7d). Ui (Pi ) is the gNB’s utility function and Ri (Pi ) is the achievable rate for power level Pi . Finally, Pmin and Pmax are the minimum and maximum transmit power level of the gNB respectively, INRmax is the maximum interference level leaked toward the satellite as permitted by the ITU (e.g., −6 dB [259]), ϵ ∈ (0, 1] is the throughput threshold ratio (e.g., ϵ = 0.9 for maintaining 90% of the maximum theoretical throughput Ri (Pi ) attained at Pi = Pmax ).

5.3.3

Joint Interference Nulling and QoS-Aware Power Control The algorithm formulated in (5.1) captures the trade-off between ensuring QoS for the ter-

restrial network and minimizing interference leakage toward the satellites through the regularization parameter λ, while the formulation in (5.7a)-(5.7d) ensures that only power budgets satisfying the throughput constraint are considered, and among those, the one providing maximum interference reduction to the satellite is ultimately selected. The formulation for the joint selection of the power level at the gNB and the beamforming vectors is given in Algorithm 2 and summarized in Fig. 5.2.

134

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS Algorithm 2 Joint Interference Nulling and QoS-Aware Power Control sat Require: Channel matrix Hter for scheduled UE k, satellite channels {hj }N j=1 , regularization

parameter λ ≥ 0, utility parameters α, M , throughput threshold ϵ, INR cap INRmax , power bounds [Pmin , Pmax ], interference-plus-noise Iiter , bandwidth W opt

opt

opt

Ensure: Beamforming vectors wt , wr , and power Pi 1: Phase 1: Beamforming Vector Selection 2: Normalize: H̃ter ← Hter /∥Hter ∥F ,

h̃j ← hj /∥hj ∥2 , ∀j ∈ J 3: Compute SVD: H̃ter = U ΣV H opt

4: Assign the optimal receive beamformer as the dominant left singular vector: wr

← u1

H w opt (w opt )H H̃ − λ PNsat h̃ h̃H 5: Form M ← H̃ter r r ter j=1 j j

6: Compute eigendecomposition: M = QΛQH opt

7: Set the transmit beamformer to the principal eigenvector: wt

← q1

8: Phase 2: Power Control Optimization 9: Compute the terrestrial channel gain: opt

opt

H 2 Gter i ← |(wr ) Hter wt | ter 10: Compute Rimax ← W log2 (1 + Pmax Gter i /Ii ),

Rimin ← ϵRimax 11: Compute the satellite channel gain: opt

H 2 Gsat j ← |(wt ) hj | , ∀j ∈ J

12: Define the Utility function: ter

ter

f (Pi ) ← −W (1 − e−αPi Gi /Ii )M /Pi 13: Set Throughput constraint: ter c1 (Pi ) ← Rimin − W log2 (1 + Pi Gter i /Ii )

14: Set INR constraint: INRmax /10 , ∀j ∈ J c2 (Pi ) ← (Pi Gsat j /Nsat ) − 10

15: Solve minPi f (Pi ) subject to c1 (Pi ) ≤ 0, c2 (Pi ) ≤ 0 and Pmin ≤ Pi ≤ Pmax via Sequential

Quadratic Programming (SQP) opt

16: return wt

opt

opt

, wr , Pi

135

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

Start gNB determines terrestrial beamformers 𝒘! , 𝒘" Repeat for all gNB-UE links:

gNB determines the optimal DL TX power level, 𝑃∗ , through Sequential Quadratic Programming satisfying the throughput and INR constraints End

Figure 5.2: End-to-end resource management framework overview.

5.4

Simulation Setup

5.4.1

Terrestrial Channel Model

(a) Terrestrial LOS Path Gain as given by QuaDRiGa’s

(b) Complete topology overview of the terrestrial and NTN

power map with the 3GPP 38.901 RMa LOS channel.

network from gNB’s perspective in QuaDRiGa.

Figure 5.3: Simulation topology for terrestrial-satellite spectrum sharing. Without loss of generality, we assume that all UEs are within the coverage of a single gNB sector, as shown in Fig. 5.3a, and that all satellites are visible from the gNB’s sector for most of their trajectory segments. In order to create the terrestrial links, we leverage QuaDRiGa [200], a quasi-deterministic geometry-based stochastic channel model simulator compliant with 3GPP TR 38.901 specifications [260]. A detailed tutorial on channel modeling with QuaDRiGa can be found in [27, 199], but full details are omitted in this chapter. For the creation of the terrestrial links, 136

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS we use the parameters defined in Table 5.1. It is noted that the geographical area of the Earth station considered in our simulation is located in central Spain, with geographical coordinates given as 38◦ 51′ 00.0′′ N, 5◦ 00′ 00.0′′ W.

5.4.2

NTN Channel Model For the creation of the non-terrestrial links, we leverage QuaDRiGa’s satellite channel

modeling approach as it has been discussed in [261]. While a descriptive tutorial on satellite channel modeling is omitted, we provide an example of the configuration used in our work to define the satellite’s trajectories as follows in Listing 5.1, while the configuration of the NTN network is provided in Table 5.1. In Fig. 5.4, we show example satellite orbits created using QuaDRiGa’s qd satellite constructor, initialized with different Keplerian orbital elements as defined in Listing 5.1. In our work, we consider the satellite constellation corresponding to Fig. 5.4b. For each of the Nsat satellites, and in order to ensure distinct trajectories, we vary their inclination (i.e., Cin of Listing 5.1) by an offset parameter ∆, as depicted in Fig. 5.3b. Finally, the satellite trajectory, given the initialized parameters, is determined using QuaDRiGa’s orbit predictor.

137

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

(a) Horsham Township, PA (Cin = 95.4, Din = −70.57, Ein = 0, Fin = 40.42)

(b) Central Spain, EU (Cin = 63.4, Din = −28.8, Ein = 44.55, Fin = 0)

Figure 5.4: Satellite overview with QuaDRiGa’s visualize lotlan in various geographic regions. The trajectories shown correspond to different parameterizations of the orbital elements defined in Listing 5.1.

1

h_qd_sat = qd_satellite(constellation, Ain, Bin, Cin, Din, Ein, Fin);

2

% Ain: Semimajor axis [km]

3

% Bin: Orbital eccentricity [0-1]

4

% Cin: Orbital inclination [deg]

5

% Din: Ascending node longitude [deg]

6

% Ein: Argument of periapsis [deg]

7

% Fin: True anomaly [deg]

Listing 5.1: LEO satellite constellation definition in QuaDRiGa.

138

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

5.4.3

Considered Constellation ·10−2 Mean: 33.0◦ , Median: 28.8◦

PDF

4

Std: 14.8◦ , Min: 13.1◦ , Max: 88.6◦

2 0

20

40

60

80

Elevation angle (θ) [◦ ] Figure 5.5: Probability Density Function (PDF) of the satellite elevation angles for the constellation of Nsat = 40 satellites considered in this work. To study the interference leakage from the gNB toward the satellite UL, we focus on lower elevation angles, as these correspond to scenarios with considerable terrestrial-to-satellite interference. Previous studies [14, 262] have shown that such angles may lead to increased interference due to the higher antenna gain of the gNB in the direction of the satellite UL. In addition, the probability of encountering satellites at lower elevation angles is higher, which represents a realistic use-case scenario. The PDF of the satellite elevation angles observed from the considered constellation is shown in Fig. 5.5.

139

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

Table 5.1: Simulation Parameters Terrestrial Network Carrier frequency

7.125 GHz

UE noise figure

7 dB

Cell radius & Number of UEs

100 m & 30

Antenna array size

8 × 8 (gNB) & 2 × 1 (UE)

Antenna Configuration

3gpp-3d, 12◦ downtilt (gNB)

Antenna height

50 m (gNB) & 1.6 m (UE)

Channel model

3GPP 38.901 Rural Macro LOS Satellite Network

Number of satellites (Nsat )

40

G/T

13 dB/K

Satellite Antenna Gain & Altitude

32 dBi & 600 km

Satellite Antenna Configuration

parabolic, 0.25 m aperture radius

Trajectory duration (Nslots )

150 (satellite visible to gNB throughout)

Channel model

QuaDRiGa NTN Rural LOS

Power Control Parameters

5.5

Bandwidth (W)

30 MHz

INRmax (target INR)

−6 dB

Inter-cell Interference (Iiter )

−73 dBm

α & M (sigmoid utility parameters)

10−3 & 3

gNB min & max TX power

10 & 33 dBm

Throughput thresholds (ϵ)

ϵ ∈ {0.85, 0.98}

Key Performance Evaluation Metrics We introduce the KPMs used to evaluate the performance of the algorithms. Recall that

the goal of this work is to suppress interference toward the satellites, while maintaining acceptable QoS in the terrestrial network. We evaluate the INR at the satellite, as defined in (5.6), to measure the effectiveness of the interference mitigation methods. For the terrestrial network, we consider the following KPMs. Terrestrial Received Signal Strength (RSS) degradation. We define the terrestrial RSS loss at UE k

140

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS as: ∆RSSk = 10 log10

Pmax · |wkoptH Hter wt,λ=0 |2 b t,λ |2 P̂i · |wkoptH Hter w

!

,

(5.8)

b t,λ are the UE k’s and the gNB beamforming vectors, respectively, obtained where wkopt and w

as outlined in Section 5.3.1 with the corresponding λ value. Pmax and P̂i < Pmax denote the maximum available and the selected transmit power of the gNB, respectively. The numerator of (5.8) corresponds to the best performance case for the terrestrial network, i.e., when no interference mitigation measure is adopted, while the denominator quantifies the impact of the proposed methods (interference nulling and power control) on the terrestrial network QoS. Specifically, • when evaluating the interference nulling algorithm, without power control, we set P̂i = Pmax • when evaluating the power control optimization, we consider λ = 0, so ŵt,λ = wt,λ=0 

and (5.8) reduces to ∆RSSk = 10 log10 Pmax /P̂i



• when power control and interference nulling are jointly applied, ŵt,λ (for λ > 0) is determined by interference nulling and P̂i by the power control algorithm ˜ k represents the mean RSS degradation for UE k, averaged across all Nslots time slots of the ∆RSS satellites’ trajectories. Jain’s Fairness Index (JFI). To quantify the fairness of RSS degradation across the terrestrial UEs, we employ the Jain’s Fairness Index (JFI) [263]. For a set of K UEs with average ˜ 1 , ∆RSS ˜ 2 , . . . , ∆RSS ˜ K }, the index is defined as: RSS degradation {∆RSS  K ˜ k 2 ∆RSS k=1 J= . PK ˜ k2 K · k=1 ∆RSS P

(5.9)

The index ranges from 1/K (worst-case fairness, where degradation is concentrated on a single UE) to 1 (perfect fairness, where all UEs experience identical degradation). A value closer to unity indicates a more equitable distribution of RSS degradation. This allows us to compare the fairness characteristics of different interference management methods, including interference nulling with various regularization parameters λ and power control strategies.

5.6

Performance Evaluation In this section, we proceed by evaluating and comparing the optimization methods pre-

sented in Section 5.3, identifying their key advantages and limitations, and investigating how they 141

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS can complement each other. It is noted that our simulation findings have been averaged over multiple repetitions. The parameters considered in this chapter for the system-level experimental analysis are given in Table 5.1, while the optimization considers the scheduling decisions of a single gNB, given the interference from nearby cells. Multi-cell scheduling interactions are not considered within the scope of this work. Finally, the constrained optimization problem defined in (5.7a)–(5.7d) was solved by leveraging MATLAB’s [264] fmincon with SQP.

Impact of Interference Nulling on Beamforming Gain

6=0

6=0.1 80 10

40 20

0 -10

0 -20

-20 -30

-40 18.0 dB

-60

Satellites Peak Gain

60

Elevation angle (3) [°]

Satellites Peak Gain

Beamforming Gain [dB]

Elevation angle (3) [°]

60

0

40 20

-10

0 -20

-20 -30

-40 18.0 dB

-60

-40

-80

-40

-80 -60

-40

-20 0 20 Azimuth angle (?) [°]

40

60

-50

-60

-40

(a)

-20 0 20 Azimuth angle (?) [°]

40

6=1

-50

6=10 80

20

-10

0 -20

-20 -30

-40 18.0 dB

-40

-80

Elevation angle (3) [°]

60 0

40

Satellites Peak Gain

10

Beamforming Gain [dB]

Satellites Peak Gain

60

-60

60

(b)

80

Elevation angle (3) [°]

10

Beamforming Gain [dB]

80

10 0

40 20

-10

0 -20

-20 -30

-40 17.9 dB

-60

Beamforming Gain [dB]

5.6.1

-40

-80 -60

-40

-20 0 20 Azimuth angle (?) [°]

40

60

-50

-60

(c)

-40

-20 0 20 Azimuth angle (?) [°]

40

60

-50

(d)

Figure 5.6: Beamforming gain of the considered gNB with a 8 × 8 Uniform Planar Array (UPA) and λ ∈ {0, 0.1, 1, 10} for time instance (tl ) along the satellites’ trajectories. As λ increases, a greater priority is given to the interference nulling than to the terrestrial network QoS, leading to deeper nulls toward the satellites and lower gain in the UE direction. However, the degradation is minimal: 18 dB at λ = 0 (Fig. 5.6a), to 17.9 dB at λ = 10 (Fig. 5.6d), close to the theoretical maximum of 18.06 dB.

142

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS We begin by investigating and visually interpreting the Interference Nulling algorithm, examining the beamforming gain patterns. To show a more intuitive correspondence between the nulls and lobes and the physical locations of the satellites and the UEs, respectively, in this subsection we run the algorithm considering a simplified LOS-only channel instead of the QuaDRiGa channel matrix by replacing h̃j with e(θj , ϕj ), where (θj , ϕj ) are the elevation and azimuth angles to the j-th victim satellite, and e(θj , ϕj ) is the spatial signature of the gNB antenna array in that direction. In addition, since satellite trajectories can be derived from publicly available ephemeris data, this approach can reduce the real-time complexity associated with multi-path channel estimation, while still achieving interference suppression close to that of full multi-path nulling, as reported in [14]. Fig. 5.6 illustrates the resulting transmit beamforming gains |eH (θ, ϕ)wt |2 across the angular space of a gNB equipped with an 8 × 8 UPA, as obtained by solving the interference nulling algorithm in (5.1) for different values of λ and a given satellite configuration. Specifically, the Nsat = 40 satellite locations are marked with red crosses, while the main beam direction (i.e., the peak gain) toward the terrestrial UE is indicated by a black circle. The results qualitatively show that nulls can be effectively placed at all satellite locations while maintaining a high beamforming gain toward the desired terrestrial UE. As λ increases from 0 to 10, interference nulling is prioritized over terrestrial QoS, and deeper nulls appear at the satellite locations, providing stronger interference suppression. This comes at a minimal cost to the terrestrial UE gain, which decreases from 18 dB at λ = 0 (Fig. 5.6a) to 17.9 dB at λ = 10 (Fig. 5.6d). Finally, the beamforming gain of 17.9 dB remains close to the theoretical maximum of 10 log10 (64) = 18.06 dB, indicating minimal impact on terrestrial link quality.

5.6.2

QoS-Aware Power Control for Risk-Averse Scenarios As explained in Section 5.3.2, the goal of the power control algorithm is to strike a balance

between the interference toward the satellites and the terrestrial network QoS. INR Reduction. In Figs. 5.7 and 5.8, we compare the simulation results obtained using the algorithm defined in (5.1) with those obtained through the power control formulation defined in (5.7a)–(5.7d). It is noted that the terms Interference Nulling (λ = 0) and No Nulling can be used interchangeably. They both refer to the case where the trade-off between terrestrial QoS maximization and interference suppression—represented by the tunable parameter λ—is not considered, as explained in Section 5.3.1. In Fig. 5.7, we observe the INR at the satellites for the simulation setup described in Table 5.1 and a throughput threshold of ϵ = 0.85. We observe that Interference Nulling with λ = 10

143

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS achieves the most interference reduction toward the satellites with a median INR of ∼ 40 dB, while Interference Nulling with λ = 0.1 performs almost identically to the case with Interference Nulling with λ = 0 (equivalent to No Nulling). Similarly, Interference Nulling with λ = 1 yields similar performance to Power Control (λ = 0). Finally, all methods successfully meet the ITU constraint by keeping the INR at the satellites below −6 dB on average, with Interference Nulling for λ ∈ {1, 10} and the combination of No Nulling and Power Control performing best. 1

CDF

0.8 0.6 0.4 0.2 0 -80

-60

-40

-20

-6

0

10

INR [dB]

Figure 5.7: CDF of the INR as defined in (5.6): Power Control & Standard Beamforming vs.

No Null., Power Control

Interf. Nulling (λ = 1)

Interf. Nulling (λ = 10)

Interf. Nulling (λ = 0.1)

0

0.5

10 0

Perfect Fairness

0

1

2

3

Std. Dev. of RSS Interf. Nulling (λ = 0) Interf. Nulling (λ = 0.1) Interf. Nulling (λ = 1) Interf. Nulling (λ = 10) No Null. (λ=0), Power Control

(b)

(a)

0 Interf. Nulling (λ = 0) Interf. Nulling (λ = 0.1) Interf. Nulling (λ = 1) Interf. Nulling (λ = 10) No Null., Power Control

10

1

20 JFI

Worst Case RSS [dB]

20

Interf. Nulling (λ = 0)

RSS Degradation [dB]

Interference Nulling.

(c)

Figure 5.8: Worst-case evaluation and fairness analysis for the terrestrial network topology described in Table 5.1.

144

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS RSS Degradation. In Fig. 5.8, we analyze the UE RSS degradation. In Fig. 5.8(a), the simulation results indicate that although all Interference Nulling methods—regardless of the λ parameter—achieve near-zero median RSS degradation on the terrestrial links, they exhibit extreme outliers, particularly as λ increases. Conversely, Power Control combined with No Nulling results in a higher median RSS degradation, due to Pi < Pmax , but avoids the extreme outliers observed with λ = 1 and λ = 10. This approach maintains the worst-case degradation for all terrestrial links below 7 dB, in contrast to the Interference Nulling methods, where the degradation can reach up to ∼ 20 dB (Fig. 5.8(b)). This degradation indicates that UEs at certain locations cannot be effectively served by the gNB when using the interference nulling algorithm. To quantify this effect, we resort to the JFI, reported in Fig. 5.8(c). We observe that Perfect Fairness is achieved only for Interference Nulling with λ = 0, whether performed standalone or combined with Power Control. In the former case, No Nulling represents the scenario where no QoS compromise is performed on the terrestrial network, resulting in a JFI equal to unity. Similarly, the JFI for the combined method is also equal to unity, as all terrestrial links are degraded equally. This can be attributed to the fact that Power Control reduces the transmit power across all links. In contrast, Interference Nulling compromises the channel quality of each gNB–UE link individually, depending on its specific channel state. Consequently, certain UEs experience more severe QoS degradation, particularly as λ increases to values of 1 and 10. Also, as λ increases, a greater number of links are affected and experience similar levels of RSS degradation. Consequently, the JFI also increases with λ; for instance, λ = 10 results in more links with comparable and high QoS degradation compared to the degradation observed when λ = 0.1 is used in the optimization. Therefore, except for the case when a simple SVD is performed toward the UE (λ = 0), the JFI of the interference nulling method remains below 0.6. The aforementioned simulation results indicate that Interference Nulling with λ = {1, 10}, as well as No Nulling combined with Power Control, can keep the INR at the satellites well under the ITU thresholds on average, thereby successfully minimizing interference at the satellites. However, Power Control is more suitable for risk-averse scenarios, since it manages to maintain an equal degradation for all links while avoiding extreme outliers for specific UEs. Such an approach is wellsuited for tactical or public safety applications, where the goal is to guarantee a reliable link for all UEs, whereas Interference Nulling maintains high overall performance at the cost of compromising certain links to achieve a high average.

145

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

6

3

4

#10

1

0 10

0 10

(a)

30

33

4

#10

7

3

2

2 15 20 25 DL TX Power [dBm]

8

Utility

4 Utility

Utility

8

8 #10

2 1

15 20 25 DL TX Power [dBm]

(b)

30

33

0 10

15 20 25 DL TX Power [dBm]

30

33

(c)

Figure 5.9: Utility function analysis for the terrestrial network topology described in Table 5.1. In Fig. 5.9, the plots show the quasi-concavity, across the power levels considered, of the bell-shaped utility function formulated in (5.7a). Fig. 5.9a showcases the optimal DL transmit power chosen by the gNB for each gNB–UE link, given the time-division-based scheduling decisions made by the gNB scheduler for a power budget [Pmin , Pmax ] = [10, 33] dBm. Additionally, Fig. 5.9a displays the achieved utility per UE, depicting the various individual QoS satisfaction levels. Indeed, the varying utility curves indicate that different power levels are selected for each active link, resulting in distinct utility peaks. Figs. 5.9b and 5.9c demonstrate the selected power level versus the achieved utility at two distinct time instances. Both figures showcase, with a blue circle, the optimal power level that achieves the highest utility and QoS satisfaction (wrt. terrestrial SINR) without unnecessary energy expenditures, as explained in detail in Section 5.3.2. The shaded areas represent the power levels that satisfy the INR and throughput constraints defined in (5.7b)–(5.7c), while the solid black square indicates the minimum power level that satisfies the throughput requirements while also meeting the ITU interference allowance constraints as selected by the power control formulation defined in (5.7a)–(5.7d). It is noted that for the time instance represented by Fig. 5.9c, the optimal DL transmit power level chosen by the gNB closely matches the peak of the utility function. Conversely, for the time instance shown in Fig. 5.9b, the power level selected by the optimization is pushed to higher values to satisfy both the throughput and INR constraints. Therefore, the utility function in this work inherently captures the trade-off between QoS satisfaction and unnecessary power expenditure. This allows the system to shift from energy-efficient operations to more QoS-driven regimes while still maintaining a balance in total energy consumption.

146

8 24

Utility Peak

Utility Peak Power [dBm]

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

21 18 15

6 4 2 0

−66 −64 −62 −60 −58 −56

Beamformed Channel Gain [dB] (a)

·108

−66 −64 −62 −60 −58 −56

Beamformed Channel Gain [dB] (b)

Figure 5.10: Achieved utility and transmit power selection for the gNB DL under varying terrestrial channel conditions. Fig. 5.10 shows the gNB’s DL transmit power and the associated achieved utility relative to the beamformed channel gain (i.e., |wrH H̃ter wt |2 ) of the gNB–UE links. These results are obtained for the case where only SVD is employed on Hter , which is equivalent to interference nulling with λ = 0. We observe that as the beamformed channel gain improves, the selected transmit power for the gNB–UE link decreases (Fig. 5.10a), while the achieved utility—and consequently the QoS satisfaction of the link—increases (Fig. 5.10b). Therefore, the power control framework proposed in this work allows the gNB to select lower transmit power levels for terrestrial links with higher channel gains on the DL, thereby avoiding excess interference toward the incumbents.

5.6.3

Joint QoS-Aware Power Control and Interference Nulling In this section, we present the simulation results when both Interference Nulling and Power

Control are jointly considered and optimized to select the terrestrial beamforming weights and the gNB’s DL transmit power as summarized in Fig. 5.2. We aim to explore how Interference Nulling with a less aggressive parameterization (i.e., lower λ values) can be combined with Power Control to effectively minimize the INR at the satellites while maintaining an acceptable terrestrial QoS.

147

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS 1

0.8

CDF

0.6

0.4

0.2

0 -80

-60

-40

-20

-6

0

10

INR [dB]

Figure 5.11: CDF of the INR as defined in (5.6) for all optimization strategies. INR Reduction. In Fig. 5.11, we present the simulation results and the effect on the INR at the satellites for all the methods considered in this work, namely Interference Nulling, No Nulling with Power Control, and the case of joint optimization through Interference Nulling and QoS-Aware Power Control. We observe that the most significant interference reduction towards the satellites is achieved through Interference Nulling with λ = 10 combined with Power Control at a median INR of ∼ 46 dB, while the least interference suppression is observed with Interference Nulling for λ ∈ {0, 0.1}. The most aggressive nulling technique observed with Interference Nulling (λ = 10) yields similar performance to Interference Nulling with λ = 1 combined with Power Control at a median INR of ∼ 38 dB. Similarly, No Nulling combined with Power Control results in identical performance to Interference Nulling with λ = 1. Therefore, when both power and the terrestrial beamformers are optimized jointly, the interference reduction at the satellites can be achieved with less aggressive nulling techniques (i.e., lower λ values) given the proper adjustment of the gNB’s DL transmit power. Finally, Interf. Nulling (λ = 0.1) combined with Power Control provides a balanced trade-off, striking a middle ground between the less and more aggressive interference nulling techniques, by successfully keeping the interference leakage towards the satellites below the ITU interference levels.

148

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

Worst Case RSS [dB]

25 20 Interf. Nulling (λ = 0) Interf. Nulling (λ = 0.1) Interf. Nulling (λ = 1) Interf. Nulling (λ = 10) Interf. Nulling (λ = 0) & Power Control Interf. Nulling (λ = 0.1) & Power Control Interf. Nulling (λ = 1) & Power Control Interf. Nulling (λ = 10) & Power Control

15 10 5 0 0

0.5

1 1.5 2 2.5 Standard Deviation of RSS [dB]

3

Figure 5.12: Worst-case RSS degradation as defined in (5.8) for all optimization strategies. RSS Degradation. In Fig. 5.12, we present the worst-case analysis results for all the methods. It is observed that Interference Nulling with λ ∈ {1, 10}, both standalone and when combined with Power Control, always results in the worst RSS values reported in our work; however, the joint optimization of the beamformers and the power results in more uniform RSS degradation across the terrestrial links (e.g., lower standard deviation). Additionally, Interference Nulling with λ ∈ {0, 0.1} results in minimal, near-zero RSS degradation. Finally, No Nulling combined with Power Control achieves the same worst-case RSS as Interference Nulling with λ = 0.1 also combined with Power Control, with the latter, however, achieving the best interference reduction among the two (Fig. 5.11).

149

0

(a)

RSS degradation, (c) power savings.

150

20 18 16 14 12 10 8 6 4 2 0

Power Percentage Decrease [%]

Interf. Nulling (λ = 0.1) & Power Control (ϵ=0.98)

10

Interf. Nulling (λ = 0) & Power Control (ϵ=0.85)

20

Interf. Nulling (λ = 1)

-25.15

Interf. Nulling (λ = 0.1)

-28.05

RSS Degradation [dB]

-31.81 -31.84

Interf. Nulling (λ = 0)

-27.16

Interf. Nulling (λ=0.1) & Power Control (ϵ=0.98)

Interf. Nulling (λ=0) & Power Control (ϵ=0.85)

Interf. Nulling (λ=1)

30

Interf. Nulling (λ=0.1)

Interf. Nulling (λ=0)

Median INR [dB]

0 Interf. Nulling (λ=0.1) & Power Control (ϵ=0.98)

Interf. Nulling (λ=0) & Power Control (ϵ=0.85)

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

5.6.4 Combined QoS-Aware Power Control and Interference Nulling for Energy Efficiency Optimization

80

60

40

20

(b) (c)

Figure 5.13: Performance evaluation of joint interference nulling and power control: (a) INR, (b)

CDF

1 0.8 0.6 0.4 0.2 00

Nt = 4 · 4 Nt = 8 · 8 Nt = 16 · 16 Nt = 32 · 32

0.25 0.5 0.75

1

1.25 1.5 1.75

2

2.25

1 0.8 0.6 0.4 0.2 00

Nt = 4 · 4 Nt = 8 · 8 Nt = 16 · 16 Nt = 32 · 32

1

2

3

4

5

6

7

RSS Degradation [dB]

RSS Degradation [dB]

(a) Nsat = 2

(b) Nsat = 10

CDF

CDF

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

1 0.8 0.6 0.4 0.2 00

8

9

10

Nt = 4 · 4 Nt = 8 · 8 Nt = 16 · 16 Nt = 32 · 32

2

4

6

8 10 12 14 16 18 20 22 24 26 28 RSS Degradation [dB]

(c) Nsat = 40

25 20 15

27 191 9 13 20 26 23 18 12 15 24 24 10

20

40 Azimuth angle (ϕ) [◦]

60

10 5 0

2.5

Satellites UEs gNB

2 1.5 1

29 21 5 25

27 191 9 13 20 26 23 18 12 15 24 24 10

20

40 Azimuth angle (ϕ) [◦]

60

0.5 0

Elevation angle (θ) [◦]

80 60 40 20 0 −20 6 14 7 22 16 11 2817 8 −40 3 30 −60 −80 −60 −40 −20 0

6 5 4 3

29 21 5 25

27 191 9 13 20 26 23 18 12 15 24

2 1

24 10

20

40 Azimuth angle (ϕ) [◦]

0

60

(b) 8 × 8 array

RSS Degradation [dB]

Elevation angle (θ) [◦]

(a) 4 × 4 array

7

Satellites UEs gNB

(c) 16 × 16 array

80 60 40 20 0 −20 6 14 7 22 16 11 2817 8 −40 3 30 −60 −80 −60 −40 −20 0

0.5

Satellites UEs gNB

0.4 0.3 0.2

29 21 5 25

27 191 9 13 20 26 23 18 12 15 24 24 10

20

40 Azimuth angle (ϕ) [◦]

60

0.1 0

RSS Degradation [dB]

29 21 5 25

80 60 40 20 0 −20 6 14 7 22 16 11 2817 8 −40 3 30 −60 −80 −60 −40 −20 0

RSS Degradation [dB]

Satellites UEs gNB

Elevation angle (θ) [◦]

80 60 40 20 0 −20 6 14 7 22 16 11 2817 8 −40 3 30 −60 −80 −60 −40 −20 0

RSS Degradation [dB]

Elevation angle (θ) [◦]

Figure 5.14: CDF of RSS degradation for λ = 1 with varying number of satellites.

(d) 32 × 32 array

Figure 5.15: Angular positions for the complete topology for different antenna sizes with Nsat = 40 satellites and Interference Nulling with λ = 1.

151

-170

40 35 30 25 20 15 10 5

-180 -190 -200 -210

6=0

6=0.1

6=1

-220

6=10

Avg. Channel Gain [dB]

Satellite Index

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS

(a) Avg. Satellite Channel Gain

20 30

80 Snapshot Index λ=1

140

0

UE Index

20 20

15 10

10

5 20

80 Snapshot Index

140

0

15 10

10

5 20

30

80 Snapshot Index λ = 10

140

0

20 20

15 10

10

5 20

80 Snapshot Index

140

0

RSS Degradation [dB]

5

20

RSS Degradation [dB]

10 10

20 UE Index

15

λ = 0.1

30

UE Index

20

RSS Degradation [dB]

UE Index

20

RSS Degradation [dB]

λ=0

30

(b) Terrestrial UE RSS Degradation

Figure 5.16: Channel analysis: (a) Average signal leakage towards all satellites, and (b) UE RSS degradation over satellite movement as defined in (5.8) for all Interference Nulling techniques. INR Reduction, RSS Degradation and Power Savings. In Fig. 5.13, we present the simulation results obtained when both Interference Nulling and Power Control are applied. We focus on the case with λ = 1, which was previously identified as providing the best trade-off between interference mitigation and acceptable QoS in the terrestrial network when only Interference Nulling was considered. We additionally consider two different throughput thresholds (i.e., ϵ = {0.85, 0.98}), as given in Table 5.1, which represent different thresholds for the QoS requirements in the terrestrial network, as explained in Section 5.3.2. In Fig. 5.13(a), we observe that Interference Nulling with 152

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS λ = 1 yields identical performance to Interference Nulling with λ = 0 when power control is applied with ϵ = 0.85, providing an ∼ 7 dB decrease in the median interference leakage at the satellite compared to the baseline (i.e., only Interference Nulling with λ = 0). Additionally, Interference Nulling with λ = 0.1 combined with power control with ϵ = 0.98 (i.e., only a 2% loss in the DL throughput) manages to reduce the INR at the satellites by approximately 3 dB, compared to Interference Nulling alone (i.e., λ = 0.1), for which the corresponding interference reduction compared to the baseline (i.e., λ = 0) is about 2 dB. Likewise, as Fig. 5.13(b) demonstrates, Interference Nulling combined with power control manages to balance the degradation in the QoS across the UEs. In particular, the worst RSS degradation is reported to be less than 8 dB when power control with ϵ = 0.85 is considered, compared to worst-case values of about 20 dB obtained with Interference Nulling with λ = 1. In contrast, power control with ϵ = 0.98 combined with Interference Nulling with λ = 0.1 manages to maintain an almost uniform degradation of approximately 1 dB across all UEs, with the worst RSS degradation reported at ∼ 2 dB. Therefore, power control with a dedicated throughput threshold can be combined with lower λ values to balance the QoS degradation among all UEs, while suppressing the interference leakage toward the satellites and additionally providing energy savings (Fig. 5.13(c)). Indeed, Fig. 5.13(c) demonstrates the energy savings achieved with both of the aforementioned power control techniques. Intuitively, more aggressive power control (i.e., ϵ = 0.85) manages to save up to 80% of the energy at the gNB, at the cost of a 6.6 dB median degradation across all terrestrial UEs. Interference Nulling with λ = 0.1 combined with less aggressive power control (ϵ = 0.98), on the other hand, also manages to save up to 20% of the energy at the terrestrial gNB, providing the best trade-off among all considered techniques in terms of terrestrial QoS and interference suppression.

5.6.5

Case for Power Control: Limitations of Interference Nulling In Fig. 5.14, we present the CDF of the RSS degradation for various antenna array sizes

and number of satellites, focusing on the Interference Nulling technique with λ = 1. We observe that for a small constellation size (e.g., Nsat = 2 in Fig. 5.14(a)), Interference Nulling can effectively keep the terrestrial QoS degradation low for all considered antenna sizes. Specifically, when larger antenna arrays are employed (i.e., Nt ∈ {256, 1024}), the degradation is maintained below 1 and 0.25 dB, respectively. However, as the number of satellites increases to 10 and 40 (Figs. 5.14(b) and 5.14(c)), larger antenna arrays are required to keep the RSS degradation at low levels. For instance, in the case of 40 satellites, an array size of Nt = 1024 elements is necessary to maintain the degradation below

153

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS 1 dB (Fig. 5.14(c)). The simulation results demonstrate that antenna arrays with a higher number of elements yield more numerous and precise beams. This improved terrestrial performance aligns with the fundamental relationship Degrees of Freedom (DOF) = Nt − 1, wherein increased element count provides additional DOF for null steering without compromising main lobe gain [265, 266]. Our analysis in Fig. 5.15 further reveals that the degradation in terrestrial UE performance due to Interference Nulling is primarily driven by the UE angular position with respect to the gNB UPA and the satellite position. Specifically, UEs at positive azimuth angles (e.g., ϕ > 0◦ ), symmetric to the satellite positions, experience the highest degradation. Indeed, most affected UEs are located at positive azimuth angles (e.g., ϕ = 20◦ to 60◦ ) and negative elevations (e.g., θ < −20◦ ), while satellites are located at negative azimuth angles (e.g., ϕ = −20◦ to −60◦ ) and positive elevations (e.g., θ > 20◦ ). This represents a fundamental geometric trade-off inherent to null steering: constraining the beamformer to create nulls toward satellites at positive elevations inherently reduces the achievable gain for UEs at extreme negative elevations, with the effect most pronounced for UEs in the opposite azimuth quadrant. Importantly, this effect persists even with larger antenna arrays; while increasing the array size from 4 × 4 to 32 × 32 significantly reduces the degradation magnitude (from 25 dB to 0.5 dB), the same UEs consistently exhibit the worst relative performance (e.g., θ < −40◦ and 40◦ < ϕ < 60◦ ). As further detailed in [258], the array response satisfies a(θaz , θel ) = a(−θaz , −θel ) since sin(−θaz ) sin(−θel ) = sin(θaz ) sin(θel ) and sat , θ sat ) simulcos(−θel ) = cos(θel ). This symmetry implies that steering nulls toward satellites at (θaz el sat , −θ sat ), limiting the DOF for terrestrial beamforming. taneously affects beamforming gain at (−θaz el

These findings demonstrate that null steering introduces an unavoidable geometric coupling between the nulled and served directions, which can only be partially mitigated through increased array dimensions. In Fig. 5.16, we present simulation results showing the effect of the Interference Nulling strategies (formulated in Section 5.3.1) on the channels for different λ values. Fig. 5.16(a) illustrates how the signal leakage towards the satellites (averaged across all snapshots of the satellite movement and, denoted as |h̃H j wt |), is mitigated—particularly as λ increases—demonstrating the effectiveness of the interference nulling techniques, especially for λ values of 1 and 10. Fig. 5.16(b) shows the effects of the interference nulling techniques on all UEs throughout the satellite movement. Notably, λ = 0.1 incurs minimal degradation on the terrestrial network across all satellite snapshot indices, whereas λ = 1, and particularly λ = 10, demonstrate a stronger impact on specific UEs, which can experience up to ∼ 20 dB RSS degradation. Therefore, Fig. 5.16 indicates that, although the average RSS degradation remains near zero, specific UEs can experience severe QoS loss at certain points 154

CHAPTER 5. ENABLING SPECTRUM SHARING IN FR3 FOR CELLULAR AND NTN SYSTEMS along the satellite trajectories, highlighting the limitations and significant trade-offs of Interference Nulling for satellite interference suppression while maintaining acceptable terrestrial QoS.

5.7

Conclusions We have performed a comparative analysis of two state-of-the-art techniques used for

interference suppression to satellites, namely QoS-aware Power Control and Interference Nulling, in spectrum sharing scenarios for the FR3 band. Our simulation results demonstrated the suitability of power control techniques for risk-averse scenarios, while pinpointing the specific geometric and array-size limitations of interference nulling. Specifically, we demonstrated that interference nulling can severely degrade UEs at certain angular positions; however, this degradation can be significantly mitigated by employing larger antenna array sizes. Finally, we proposed the joint application of interference nulling and power control techniques to reduce interference towards the incumbents while ensuring a more energy-efficient operation and fair QoS distribution in the terrestrial network. Our future work will focus on the on-demand and adaptive per-UE optimization of power and beamforming. Specifically, we aim to apply power control selectively only for UEs experiencing significant QoS degradation, thereby maintaining the RSS degradation at manageable levels while preserving system-wide performance. Another research focus will be the joint optimization of transmit power levels and beamforming for Multi-User (MU)-MIMO systems, where power allocation decisions can be made proportionally fair to the links’ channel quality.

155

Chapter 6

Conclusions This dissertation has focused on the algorithmic design of network management solutions at the system level for the optimization of the gNB’s scheduling operations, ranging from classical optimization and game theory to ML and AI on and for the RAN. In detail, we have focused on the design of DRL solutions for slicing and scheduling control, leveraged at the network’s edge. We demonstrated the feasibility of the deployment of AI/ML algorithms that can on-demand fine-tune the network’s resources and functionalities, enabling AI on and for the RAN. More importantly, we contributed to the research body by performing the largest evaluation of xApps (23) in the literature, spanning various design control choices and ML architectures, providing the research community, telco operators, and xApp developers a framework that automates DRL agent training, xApp on-boarding, and large-scale evaluation and testing with Open RAN within Colosseum, the world’s largest wireless network emulator with hardware-in-theloop. Our work demonstrated the benefits of leveraging AI solutions on the RAN that can coexist with minimal conflicts using realistic protocol stacks, yielding practical lessons and offering key insights for the potential deployment of AI on commercial RANs. Next, building on the principles that 6G research will emerge from simulation, we have designed an online RL solution for link adaptation. We have designed the algorithmic framework within a DT, showcasing how AI on the RAN, deployed at the gNB’s scheduler, can optimize performance by yielding higher spectral efficiency compared to industry standard and state-of-theart algorithmic designs. Finally, we bridged the connection between simulation and real-world deployment, demonstrating the suitability of RL for link adaptation on 5G data collected OTA from 24 h operating 5G networks deployed at Northeastern University. Apart from treating the wireless environment as a static entity, we also treated the RF prop156

CHAPTER 6. CONCLUSIONS agation as a quantity that can be optimized. Through intelligent beamforming via RISs, we proposed a hierarchical game-theoretic approach for an energy-efficient power control mechanism that can be deployed in both FR1 and FR2, providing enhanced energy efficiency without compromising QoS. Our work demonstrated the enhanced power saving for the latter band, pinpointing the necessity for the deployment of small panels, avoiding the complexity of larger panel arrays. Taking a step further, we deployed the RIS-enabled channels within the Colosseum channel emulator, demonstrating for the first time the emulation of realistic RIS-enabled channels within a massive channel emulator. Finally, this work is one of the first of its kind, discussing the bridging between Open RAN and RIS. By deploying an xApp at the Near-RT RIC within Colosseum, we demonstrated the decreased levels of latency introduced by RISs for the URLLC slice, indicating that both a RIS and the proper fine-tuning of the scheduling algorithm within the gNB can enhance network performance in terms of spectral efficiency and utilization within resource-constrained environments for UEs allocated to various network slices. Finally, we have discussed the coexistence of cellular and NTNs, a critical issue alongside RIS and AI-RAN for 6G research. We evaluated state-of-the-art beamforming and power control, and proposed a joint approach combining both methods that strikes a balance between interference reduction towards the satellites while not compromising terrestrial QoS. Our work, in contrast to the plethora of research, focuses on dense satellite deployments, providing for the first time realistic insights for the design of optimization solutions for enhanced cellular-satellite coexistence. These insights are potentially valuable for the deployment of commercial NextG networks that coexist with incumbents. Overall, this work has focused on the design of novel algorithmic frameworks for advanced gNB control, focusing on 6G and beyond-ready networks encompassing AI, RIS, and coexistence mechanisms. This work tries to bridge the demands of both industry and academia for novel algorithmic designs and their successful deployment on realistic platforms for advanced wireless research, paving the way for commercial deployment by providing insights and lessons learned.

157

Bibliography [1] M. Polese, L. Bonati, S. D’Oro, S. Basagni, and T. Melodia, “Understanding O-RAN: Architecture, Interfaces, Algorithms, Security, and Research Challenges,” IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 1376–1411, Second quarter 2023. [2] P. Brach del Prever, S. D’Oro, L. Bonati, M. Polese, M. Tsampazi, H. Lehmann, and T. Melodia, “PACIFISTA: Conflict Evaluation and Management in Open RAN,” IEEE Transactions on Mobile Computing, vol. 24, no. 10, pp. 10 590–10 605, 2025. [3] 3GPP TS 22.261, “3GPP TS 22.261: Service requirements for the 5G system,” 3rd Generation Partnership Project (3GPP), Technical Specification, 2022, release 17, v17.11.00. [4] 3GPP TS 23.501, “3GPP TS 23.501: System architecture for the 5G system (5GS),” 3rd Generation Partnership Project (3GPP), Technical Specification, 2023, release 17. [5] L. Bonati, S. D’Oro, S. Basagni, and T. Melodia, “SCOPE: An Open and Softwarized Prototyping Platform for NextG Systems,” in Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, 2021, pp. 415–426. [6] 3rd Generation Partnership Project, “NR; Physical layer procedures for data (3GPP TS 38.214),” 2023. [7] 3GPP TS 38.213, “3GPP TS 38.213: NR; Physical layer procedures for control,” 3rd Generation Partnership Project (3GPP), Technical Specification, 2023. [8] O-RAN Working Group 3, “O-RAN Near-RT RAN Intelligent Controller Near-RT RIC Architecture 2.00,” O-RAN.WG3.RICARCH-v02.00, March 2021. [9] J. Breen, A. Buffmire, J. Duerig, K. Dutt, E. Eide, M. Hibler, D. Johnson, S. Kumar Kasera, E. Lewis, D. Maas, A. Orange, N. Patwari, D. Reading, R. Ricci, D. Schurig, L. B. Stoller, 158

BIBLIOGRAPHY K. Van der Merwe, K. Webb, and G. Wong, “POWDER: Platform for Open Wireless Datadriven Experimental Research,” in Proceedings of ACM WiNTECH, London, United Kingdom, Sept. 2020. [10] L. Bonati, M. Polese, S. D’Oro, S. Basagni, and T. Melodia, “OpenRAN Gym: AI/ML development, data collection, and testing for O-RAN on PAWR platforms,” Computer Networks, vol. 220, p. 109502, 2023. [11] M. Polese, L. Bonati, S. D’Oro, S. Basagni, and T. Melodia, “ColO-RAN: Developing Machine Learning-Based xApps for Open RAN Closed-Loop Control on Programmable Experimental Platforms,” IEEE Transactions on Mobile Computing, vol. 22, no. 10, pp. 5787–5800, Oct 2023. [12] O-RAN Working Group 1, “O-RAN Use Cases Detailed Specification 6.0,” O-RAN.WG1.UseCases-Detailed-Specification-v06.00 Technical Specification, July 2021. [13] 3GPP TR 38.821, “Solutions for NR to support non-terrestrial networks (NTN),” 3GPP, Technical Report, Mar. 2023, release 16. [14] S. Kang, G. Geraci, M. Mezzavilla, and S. Rangan, “Terrestrial-satellite spectrum sharing in the upper mid-band with interference nulling,” in IEEE International Conference on Communications, 2024. [15] A. Agarwal, “Coexistence assessment and interference mitigation for 5G and Fixed Satellite Stations in C-band in India,” arXiv preprint arXiv:2312.16079, 2023. [16] AI-RAN Alliance, “AI-RAN Alliance - Vision and Mission White Paper,” [Available Online]: https://ai-ran.org/wp-content/uploads/2024/12/AI-RAN Alliance Whitepaper.pdf, December 2024. [17] M. D. Renzo, M. Debbah, D.-T. Phan-Huy, A. Zappone, M.-S. Alouini, C. Yuen, V. Sciancalepore, G. C. Alexandropoulos, J. Hoydis, H. Gacanin et al., “Smart radio environments empowered by reconfigurable AI meta-surfaces: An idea whose time has come,” EURASIP Journal on Wireless Communications and Networking, vol. 2019, no. 1, pp. 1–20, 2019. [18] ETSI, “Reconfigurable Intelligent Surfaces (RIS); Use Cases, Deployment Scenarios and Requirements,” ETSI, ETSI Group Report RIS 001 V1.1.1, April 2023, [Available Online]: https://www.etsi.org/deliver/etsi gr/RIS/001 099/001/01.01.01 60/gr RIS001v010101p.pdf. 159

BIBLIOGRAPHY [19] A. Bazzi, R. Bomfin, M. Mezzavilla, S. Rangan, T. Rappaport, and M. Chafii, “Upper midband spectrum for 6G: Vision, opportunity and challenges,” arXiv preprint arXiv:2502.17914, 2025. [20] Nokia, “Nokia launches Nokia RAN Digital Twin to turbo-charge AI-native 6G, powered by NVIDIA Aerial Omniverse Digital Twin,” Nokia Corporate Blog, Feb. 2026, [Available Online]: https://www.nokia.com/blog/nokia-launches-nokia-ran-digital-twin-to-turbo-chargeai-native-6g-powered-by-nvidia-aerial-omniverse-digital-twin/. [21] L. Bonati, P. Johari, M. Polese, S. D’Oro, S. Mohanti, M. Tehrani-Moayyed, D. Villa, S. Shrivastava, C. Tassie, K. Yoder et al., “Colosseum: Large-scale Wireless Experimentation through Hardware-in-the-Loop Network Emulation,” in IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN), 2021, pp. 105–113. [22] J. Kochan, “PPO: Platforms for Advanced Wireless Research (PAWR) Project Office,” NSF Award Number 1719547. Directorate for Computer and Information Science and Engineering, vol. 17, no. 1719547, p. 19547, 2017. [23] J. Hoydis, S. Cammerer, F. Ait Aoudia, M. Nimier-David, A. Keller, A. Vem, M. Stark, and T. O’Shea, “Sionna: An Open-Source Library for Next-Generation Physical Layer Research,” arXiv preprint arXiv:2203.11854, 2022. [24] M. Polese, L. Bonati, S. D’Oro, P. Johari, D. Villa, S. Velumani, R. Gangula, M. Tsampazi, C. Paul Robinson, G. Gemmi, A. Lacava, S. Maxenti, H. Cheng, and T. Melodia, “Colosseum: The Open RAN Digital Twin,” IEEE Open Journal of the Communications Society, vol. 5, pp. 5452–5466, 2024. [25] M. Tsampazi, S. D’Oro, M. Polese, L. Bonati, G. Poitau, M. Healy, M. Alavirad, and T. Melodia, “PandORA: Automated design and comprehensive evaluation of deep reinforcement learning agents for open RAN,” IEEE Transactions on Mobile Computing, vol. 24, no. 4, pp. 3223–3240, 2025. [26] M. Tsampazi, S. D’Oro, M. Polese, L. Bonati, G. Poitau, M. Healy, and T. Melodia, “A Comparative Analysis of Deep Reinforcement Learning-Based xApps in O-RAN,” in GLOBECOM 2023 - 2023 IEEE Global Communications Conference, 2023, pp. 1638–1643.

160

BIBLIOGRAPHY [27] M. Tsampazi and T. Melodia, “System-Level Experimental Evaluation of Reconfigurable Intelligent Surfaces for NextG Communication Systems,” IEEE Transactions on Vehicular Technology, vol. 74, no. 9, pp. 14 138–14 153, 2025. [28] M. Tsampazi, M. Polese, F. Dressler, and T. Melodia, “O-RIS-ing: Evaluating RIS-Assisted NextG Open RAN,” in IEEE INFOCOM 2025 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), May 2025, pp. 1–6. [29] M. Tsampazi, N. N. Santhi, N. Perrotta, F. Dressler, and T. Melodia, “ARIADNE: Ai-ran informed link adaptation in digital twin network environments,” 2026, european Wireless. [30] M. Tsampazi, P. Testolina, M. Polese, and T. Melodia, “Satellite-Terrestrial Spectrum Sharing in FR3 through QoS-Aware Power Control and Spatial Nulling,” IEEE International Symposium on Dynamic Spectrum Access (DySPAN), 2026. [31] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson, J. Rexford, S. Shenker, and J. Turner, “OpenFlow: Enabling Innovation in Campus Networks,” ACM SIGCOMM Computer Communication Review, vol. 38, no. 2, pp. 69–74, March 2008. [32] A. S. Abdalla, P. S. Upadhyaya, V. K. Shah, and V. Marojevic, “Toward Next Generation Open Radio Access Networks: What O-RAN Can and Cannot Do!” IEEE Network, vol. 36, no. 6, pp. 206–213, November 2022. [33] B. Balasubramanian, E. S. Daniels, M. Hiltunen, R. Jana, K. Joshi, R. Sivaraj, T. X. Tran, and C. Wang, “RIC: A RAN Intelligent Controller Platform for AI-Enabled Cellular Networks,” IEEE Internet Computing, vol. 25, no. 2, pp. 7–17, March 2021. [34] U. Challita, H. Ryden, and H. Tullberg, “When machine learning meets wireless cellular networks: Deployment, challenges, and applications,” IEEE Communications Magazine, vol. 58, no. 6, pp. 12–18, June 2020. [35] D. Villa, M. Tehrani-Moayyed, C. P. Robinson, L. Bonati, P. Johari, M. Polese, and T. Melodia, “Colosseum as a Digital Twin: Bridging Real-World Experimentation and Wireless Network Emulation,” IEEE Transactions on Mobile Computing, pp. 1–17, In press, 2024. [36] D. Raychaudhuri, I. Seskar, G. Zussman, T. Korakis, D. Kilper, T. Chen, J. Kolodziejski, M. Sherman, Z. Kostic, X. Gu, H. Krishnaswamy, S. Maheshwari, P. Skrimponis, and C. Gutterman, “Challenge: COSMOS: A city-scale programmable testbed for experimentation with 161

BIBLIOGRAPHY advanced wireless,” in Proceedings of ACM MobiCom, London, United Kingdom, September 2020. [37] V. Marojevic, I. Guvenc, R. Dutta, M. L. Sichitiu, and B. A. Floyd, “Advanced Wireless for Unmanned Aerial Systems: 5G Standardization, Research Challenges, and AERPAW Architecture,” IEEE Vehicular Technology Magazine, vol. 15, no. 2, pp. 22–30, June 2020. [38] H. Zhang, Y. Guan, A. Kamal, D. Qiao, M. Zheng, A. Arora, O. Boyraz, B. Cox, T. Daniels, M. Darr, D. Jacobson, A. Khokhar, S. Kim, J. Koltes, J. Liu, M. Luby, L. Nadolny, J. Peschel, P. Schnable, A. Sharma, A. Somani, and L. Tang, “ARA: A Wireless Living Lab Vision for Smart and Connected Rural Communities,” in Proceedings of ACM WiNTECH, New Orleans, LA, USA, October 2021. [39] S. Pongratz, “Open RAN Market Opportunity and Risks,” Dell’Oro Group report, 2021, [Available Online]: https://www.delloro.com/wp-content/uploads/2021/11/DellOro-GroupArticle-Open-RAN-Market-Opportunity-and-Risks.pdf. [40] F. Kaltenberger, A. P. Silva, A. Gosain, L. Wang, and T.-T. Nguyen, “OpenAirInterface: Democratizing innovation in the 5G era,” Computer Networks, no. 107284, May 2020. [41] I. Gomez-Miguelez, A. Garcia-Saavedra, P. D. Sutton, P. Serrano, C. Cano, and D. J. Leith, “srsLTE: An Open-Source Platform for LTE Evolution and Experimentation,” in Proceedings of the Tenth ACM International Workshop on Wireless Network Testbeds, Experimental Evaluation, and Characterization, 2016, pp. 25–32. [42] J. Jagannath, K. Ramezanpour, and A. Jagannath, “Digital Twin Virtualization with Machine Learning for IoT and Beyond 5G Networks: Research Directions for Security and Optimal Control,” in Proceedings of the 2022 ACM Workshop on Wireless Security and Machine Learning, ser. WiseML ’22.

New York, NY, USA: Association for Computing Machinery,

2022, p. 81–86, [Available Online]: https://doi.org/10.1145/3522783.3529519. [43] A. Masaracchia, V. Sharma, M. Fahim, O. A. Dobre, and T. Q. Duong, “Digital Twin for Open RAN: Toward Intelligent and Resilient 6G Radio Access Networks,” IEEE Communications Magazine, vol. 61, no. 11, pp. 112–118, November 2023. [44] M. Corici and T. Magedanz, “Digital Twin for 5G Networks,” in The Digital Twin. 2023, pp. 433–446. 162

Springer,

BIBLIOGRAPHY [45] H. X. Nguyen, R. Trestian, D. To, and M. Tatipamula, “Digital Twin for 5G and Beyond,” IEEE Communications Magazine, vol. 59, no. 2, pp. 10–15, February 2021. [46] X. Lin, L. Kundu, C. Dick, E. Obiodu, T. Mostak, and M. Flaxman, “6G Digital Twin Networks: From Theory to Practice,” IEEE Communications Magazine, vol. 61, no. 11, pp. 72–78, November 2023. [47] A. Alkhateeb, S. Jiang, and G. Charan, “Real-Time Digital Twins: Vision and Research Directions for 6G and Beyond,” IEEE Communications Magazine, vol. 61, no. 11, pp. 128– 134, November 2023. [48] M. McManus, Y. Cui, J. Z. Zhang, J. Hu, S. K. Moorthy, N. Mastronarde, E. S. Bentley, M. Medley, and Z. Guan, “Digital twin-enabled domain adaptation for zero-touch UAV networks: Survey and challenges,” Computer Networks, vol. 236, p. 110000, 2023, [Available Online]: https://www.sciencedirect.com/science/article/pii/S1389128623004450. [49] J. Mirzaei, I. Abualhaol, and G. Poitau, “Network Digital Twin for Open RAN: The Key Enablers, Standardization, and Use Cases,” arXiv preprint arXiv:2308.02644, 2023. [50] I. Vilà, O. Sallent, and J. Pérez-Romero, “On the design of a network digital twin for the radio access network in 5g and beyond,” Sensors, vol. 23, no. 3, p. 1197, 2023. [51] P. Li, J. Thomas, X. Wang, A. Khalil, A. Ahmad, R. Inacio, S. Kapoor, A. Parekh, A. Doufexi, A. Shojaeifard, and R. J. Piechocki, “RLOps: Development Life-Cycle of Reinforcement Learning Aided Open RAN,” IEEE Access, vol. 10, pp. 113 808–113 826, 2022. [52] A. Ndikumana, K. K. Nguyen, and M. Cheriet, “Digital Twin Assisted Closed-Loops for Energy-Efficient Open RAN-Based Fixed Wireless Access Provisioning in Rural Areas,” in IEEE Global Communications Conference (GLOBECOM), 2023, pp. 6285–6290. [53] J. Deng, Q. Zheng, G. Liu, J. Bai, K. Tian, C. Sun, Y. Yan, and Y. Liu, “A Digital Twin Approach for Self-optimization of Mobile Networks,” in IEEE Wireless Communications and Networking Conference Workshops (WCNCW), March 2021, pp. 1–6. [54] Y. Ren, S. Guo, B. Cao, and X. Qiu, “End-to-End Network SLA Quality Assurance for C-RAN: A Closed-Loop Management Method Based on Digital Twin Network,” IEEE Transactions on Mobile Computing, pp. 1–18, 2023.

163

BIBLIOGRAPHY [55] S. K. Moorthy, A. Harindranath, M. McManus, Z. Guan, N. Mastronarde, E. S. Bentley, and M. Medley, “A Middleware for Digital Twin-Enabled Flying Network Simulations Using UBSim and UB-ANC,” in 18th International Conference on Distributed Computing in Sensor Systems (DCOSS), May 2022, pp. 322–327. [56] J. Hu, Z. Zhao, M. McManus, S. K. Moorthy, Y. Cui, N. Mastronarde, E. S. Bentley, M. Medley, and Z. Guan, “NeXT: Architecture, prototyping and measurement of a software-defined testing framework for integrated RF network simulation, experimentation and optimization,” Computer Communications, vol. 210, pp. 342–355, 2023, [Available Online]: https://www. sciencedirect.com/science/article/pii/S014036642300302X. [57] J. Baranda, J. Mangues-Bafalluy, E. Zeydan, C. Casetti, C. F. Chiasserini, M. Malinverno, C. Puligheddu, M. Groshev, C. Guimarães, K. Tomakh, D. Kucherenko, and O. Kolodiazhnyi, “Demo: AIML-as-a-Service for SLA management of a Digital Twin Virtual Network Service,” in IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), May 2021, pp. 1–2. [58] P. Bahl, M. Balkwill, X. Foukas, A. Kalia, D. Kim, M. Kotaru, Z. Lai, S. Mehrotra, B. Radunovic, S. Saroiu, C. Settle, A. Verma, A. Wolman, F. Y. Yan, and Y. Zhang, Accelerating Open RAN Research Through an Enterprise-scale 5G Testbed. Association for Computing Machinery, 2023. [59] D. Villa, I. Khan, F. Kaltenberger, N. Hedberg, R. S. da Silva, S. Maxenti, L. Bonati, A. Kelkar, C. Dick, E. Baena et al., “X5G: An open, programmable, multi-vendor, end-to-end, private 5G O-RAN testbed with NVIDIA ARC and OpenAirInterface,” IEEE Transactions on Mobile Computing, 2025. [60] P. S. Upadhyaya, N. Tripathi, J. Gaeddert, and J. H. Reed, “Open AI Cellular (OAIC): An Open Source 5G O-RAN Testbed for Design and Testing of AI-Based RAN Management Algorithms,” IEEE Network, vol. 37, no. 5, pp. 7–15, Sep. 2023. [61] P. S. Upadhyaya, A. S. Abdalla, V. Marojevic, J. H. Reed, and V. K. Shah, “Prototyping next-generation O-RAN research testbeds with SDRs,” arXiv preprint arXiv:2205.13178, 2022. [62] H. Koumaras, D. Tsolkas, G. Gardikis, P. M. Gomez, V. Frascolla, D. Triantafyllopoulou, M. Emmelmann, V. Koumaras, M. L. G. Osma, D. Munaretto, E. Atxutegi, J. S. d. Puga, 164

BIBLIOGRAPHY O. Alay, A. Brunstrom, and A. M. C. Bosneag, “5GENESIS: The Genesis of a Flexible 5G Facility,” in Proceedings of IEEE CAMAD, Barcelona, Spain, September 2018. [63] A. Chaudhari and M. Braun, “A Scalable FPGA Architecture for Flexible, Large-Scale, RealTime RF Channel Emulation,” in 2018 13th International Symposium on Reconfigurable Communication-centric Systems-on-Chip (ReCoSoC), July 2018, pp. 1–8. [64] Colosseum Website, “Colosseum Software,” accessed on March 12, 2024. [Available Online]: https://www.northeastern.edu/colosseum/colosseum-software/. [65] D. Villa, M. Tehrani-Moayyed, P. Johari, S. Basagni, and T. Melodia, “CaST: a toolchain for creating and characterizing realistic wireless network emulation scenarios,” in Proceedings of the 16th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization, 2022, pp. 45–52. [66] Eric Blossom and GNU Radio Project. GNU Radio. [Available Online]: https://www.gnuradio. org. Accessed February 2023. [67] R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction.

MIT press, 2018.

[68] X. Wang, J. D. Thomas, R. J. Piechocki, S. Kapoor, R. Santos-Rodrı́guez, and A. Parekh, “Self-play learning strategies for resource assignment in Open-RAN networks,” Computer Networks, vol. 206, p. 108682, 2022. [69] D. Johnson, D. Maas, and J. Van Der Merwe, “NexRAN: Closed-loop RAN slicing in POWDER-A top-to-bottom open-source open-RAN use case,” in Proceedings of the 15th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization, 2022, pp. 17–23. [70] A. Kak, V.-Q. Pham, H.-T. Thieu, and N. Choi, “HexRAN: A Programmable Multi-RAT Platform for Network Slicing in the Open RAN Ecosystem,” arXiv preprint arXiv:2304.12560, 2023. [71] C.-C. Chen, C.-Y. Chang, and N. Nikaein, “Flexslice: Flexible and real-time programmable ran slicing framework,” in IEEE Global Communications Conference, Kuala Lumpur, 2023. [72] R. Wiebusch, N. A. Wagner, D. Overbeck, F. Kurtz, and C. Wietfeld, “Towards open 6g: Experimental o-ran framework for predictive uplink slicing,” in IEEE International Conference on Communications.

IEEE, 2023, pp. 4834–4839. 165

BIBLIOGRAPHY [73] S.-p. Yeh, S. Bhattacharya, R. Sharma, and H. Moustafa, “Deep Learning for Intelligent and Automated Network Slicing in 5G Open RAN (ORAN) Deployment,” Open Journal of the Communications Society, 2023. [74] M. Kouchaki and V. Marojevic, “Actor-Critic Network for O-RAN Resource Allocation: xApp Design, Deployment, and Analysis,” in IEEE Globecom Workshops (GC Wkshps), 2022, pp. 968–973. [75] A. Filali, B. Nour, S. Cherkaoui, and A. Kobbane, “Communication and computation O-RAN resource slicing for URLLC services using deep reinforcement learning,” IEEE Communications Standards Magazine, vol. 7, no. 1, pp. 66–73, 2023. [76] M. Zangooei, M. Golkarifard, M. Rouili, N. Saha, and R. Boutaba, “Flexible RAN Slicing in Open RAN With Constrained Multi-Agent Reinforcement Learning,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 2, pp. 280–294, 2024. [77] P. Iturria Rivera, H. Zhang, H. Zhou, S. Mollahasani, and M. Erol Kantarci, “Multi-agent team learning in virtualized open radio access networks (o-ran),” Sensors, vol. 22, p. 5375, 07 2022. [78] H. Zhang, H. Zhou, and M. Erol-Kantarci, “Team learning-based resource allocation for open radio access network (O-RAN),” in IEEE International Conference on Communications. IEEE, 2022, pp. 4938–4943. [79] H. Zhang, H. Zhou, and M. Erol-Kantarci, “Federated deep reinforcement learning for resource allocation in O-RAN slicing,” in IEEE Global Communications Conference (GLOBECOM), 2022, pp. 958–963. [80] F. Rezazadeh, L. Zanzi, F. Devoti, H. Chergui, X. Costa-Pérez, and C. Verikoukis, “On the Specialization of FDRL Agents for Scalable and Distributed 6G RAN Slicing Orchestration,” IEEE Transactions on Vehicular Technology, vol. 72, no. 3, pp. 3473–3487, 2023. [81] N. C. Luong, D. T. Hoang, S. Gong, D. Niyato, P. Wang, Y.-C. Liang, and D. I. Kim, “Applications of deep reinforcement learning in communications and networking: A survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 4, pp. 3133–3174, 2019. [82] A. Alwarafy, M. Abdallah, B. S. Ciftler, A. Al-Fuqaha, and M. Hamdi, “Deep reinforcement learning for radio resource allocation and management in next generation heterogeneous wireless networks: A survey,” arXiv preprint arXiv:2106.00574, 2021. 166

BIBLIOGRAPHY [83] K. Suh, S. Kim, Y. Ahn, S. Kim, H. Ju, and B. Shim, “Deep reinforcement learning-based network slicing for beyond 5G,” IEEE Access, vol. 10, pp. 7384–7395, 2022. [84] Q. Liu, T. Han, N. Zhang, and Y. Wang, “DeepSlicing: Deep reinforcement learning assisted resource allocation for network slicing,” in GLOBECOM 2020-2020 IEEE Global Communications Conference.

IEEE, 2020, pp. 1–6.

[85] G. Sun, Z. T. Gebrekidan, G. O. Boateng, D. Ayepah-Mensah, and W. Jiang, “Dynamic reservation and deep reinforcement learning based autonomous resource slicing for virtualized radio access networks,” Ieee Access, vol. 7, pp. 45 758–45 772, 2019. [86] D. Yan, B. K. Ng, W. Ke, and C.-T. Lam, “Deep Reinforcement Learning Based Resource Allocation for Network Slicing with Massive MIMO,” IEEE Access, 2023. [87] N. Naderializadeh, J. J. Sydir, M. Simsek, and H. Nikopour, “Resource management in wireless networks via multi-agent deep reinforcement learning,” IEEE Transactions on Wireless Communications, vol. 20, no. 6, pp. 3507–3523, 2021. [88] Y. Azimi, S. Yousefi, H. Kalbkhani, and T. Kunz, “Energy-efficient deep reinforcement learning assisted resource allocation for 5G-RAN slicing,” IEEE Transactions on Vehicular Technology, vol. 71, no. 1, pp. 856–871, 2021. [89] A. M. Rahimi, A. Ziaeddini, and S. Gonglee, “A novel approach to efficient resource allocation in load-balanced cellular networks using hierarchical DRL,” Journal of Ambient Intelligence and Humanized Computing, vol. 13, no. 5, pp. 2887–2901, 2022. [90] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems,” 2015, software available from tensorflow.org. [Available Online]: https://www.tensorflow.org/. [91] Q. Wang, Y. Liu, Y. Wang, X. Xiong, J. Zong, J. Wang, and P. Chen, “Resource allocation based on Radio Intelligence Controller for Open RAN towards 6G,” IEEE Access, 2023.

167

BIBLIOGRAPHY [92] B. Brik, K. Boutiba, and A. Ksentini, “Deep learning for B5G open radio access network: Evolution, survey, case studies, and challenges,” IEEE Open Journal of the Communications Society, vol. 3, pp. 228–250, 2022. [93] S. Guadarrama, A. Korattikara, O. Ramirez, P. Castro, E. Holly, S. Fishman, K. Wang, E. Gonina, N. Wu, E. Kokiopoulou, L. Sbaiz, J. Smith, G. Bartók, J. Berent, C. Harris, V. Vanhoucke, and E. Brevdo, “TF-Agents: A library for Reinforcement Learning in TensorFlow,” [Available Online]: https://github.com/tensorflow/agents, 2018. [94] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015. [95] A. Kaloxylos, A. Gavras, D. Camps, M. Ghoraishi, and H. Hrasnica, “AI and ML–Enablers for beyond 5G Networks,” [Available Online]: https://5g-ppp.eu/wp-content/uploads/2021/05/ AI-MLforNetworks-v1-0.pdf, 2021. [96] J. Yang, X. You, G. Wu, M. M. Hassan, A. Almogren, and J. Guna, “Application of reinforcement learning in UAV cluster task scheduling,” Future generation computer systems, vol. 95, pp. 140–148, 2019. [97] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” 2017, arXiv:1707.06347. [98] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013. [99] S. Kakade and J. Langford, “Approximately optimal approximate reinforcement learning,” in Proceedings of the Nineteenth International Conference on Machine Learning, 2002, pp. 267–274. [100] “U.S. Naval Research Laboratory, ”Multi-Generator (MGEN) Network Test Tool”,” [Available Online]: https://www.nrl.navy.mil/itd/ncs/products/mgen, 2019. [101] O. Alliance, “O-RAN Working Group 2:“O-RAN AI/ML workflow description and requirements 1.03,” O-RAN. WG2. AIML-v01. 03 Technical Specification, 2021.

168

BIBLIOGRAPHY [102] M. Kim, J.-S. Kim, M.-S. Choi, and J.-H. Park, “Adaptive Discount Factor for Deep Reinforcement Learning in Continuing Tasks with Uncertainty,” Sensors, vol. 22, no. 19, p. 7266, 2022. [103] C. Zhang, Q. Song, Z. Meng et al., “Minibatch Recursive Least Squares Q-Learning,” Computational Intelligence and Neuroscience, 2021. [104] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015. [105] M. Hessel, J. Modayil, H. Van Hasselt, T. Schaul, G. Ostrovski, W. Dabney, D. Horgan, B. Piot, M. Azar, and D. Silver, “Rainbow: Combining improvements in deep reinforcement learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018. [106] M. Fortunato, M. G. Azar, B. Piot, J. Menick, I. Osband, A. Graves, V. Mnih, R. Munos, D. Hassabis, O. Pietquin et al., “Noisy networks for exploration,” arXiv preprint arXiv:1706.10295, 2017. [107] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, “Trust region policy optimization,” in International conference on machine learning.

PMLR, 2015, pp. 1889–1897.

[108] S. Fujimoto, D. Meger, and D. Precup, “Off-policy deep reinforcement learning without exploration,” in International conference on machine learning. PMLR, 2019, pp. 2052–2062. [109] Y. Xiao, Q. Zhang, F. Liu, J. Wang, M. Zhao, Z. Zhang, and J. Zhang, “NFVdeep: Adaptive Online Service Function Chain Deployment with Deep Reinforcement Learning,” in IEEE/ACM International Symposium on Quality of Service, 2019, pp. 1–10. [110] R. Kozlica, S. Wegenkittl, and S. Hiränder, “Deep q-learning versus proximal policy optimization: Performance comparison in a material sorting task,” in IEEE 32nd International Symposium on Industrial Electronics (ISIE), 2023, pp. 1–6. [111] O-RAN Working Group 3, “Conflict Mitigation,” O-RAN.WG3.TR.ConMit-R004-v01.00 Technical Specification, October 2024. [112] C. Adamczyk, “Challenges for Conflict Mitigation in O-RAN ’s RAN Intelligent Controllers,” in Proceedings of IEEE SoftCOM, 2023. 169

BIBLIOGRAPHY [113] C. Adamczyk and A. Kliks, “Conflict Mitigation Framework and Conflict Detection in O-RAN Near-RT RIC,” IEEE Communications Magazine, 2023. [114] A. Wadud, F. Golpayegani, and N. Afraz, “QACM: QoS-Aware xApp Conflict Mitigation in Open RAN,” arXiv preprint arXiv:2405.07324, 2024. [115] A. Hazra and N. Gogtay, “Biostatistics series module 4: comparing groups–categorical variables,” Indian Journal of Dermatology, vol. 61, no. 4, pp. 385–392, 2016. [116] W. Chen, Y. Chen, Y. Wang, and R. Liu, “Static Standing Balance With Musculoskeletal Models Using PPO With Reward Shaping,” Procedia Computer Science, vol. 226, pp. 78–84, 2023. [117] H. Attar, H. Issa, J. Ababneh, M. Abbasi, A. A. Solyman, M. Khosravi, R. Said Agieb et al., “5G System Overview for Ongoing Smart Applications: Structure, Requirements, and Specifications,” Computational Intelligence and Neuroscience, vol. 2022, 2022. [118] M. J. Shehab, I. Kassem, A. A. Kutty, M. Kucukvar, N. Onat, and T. Khattab, “5G networks towards smart and sustainable cities: A review of recent developments, applications and future perspectives,” IEEE Access, vol. 10, pp. 2987–3006, 2021. [119] S. Sobhi-Givi, M. G. Shayesteh, and H. Kalbkhani, “Energy-efficient power allocation and user selection for mmWave-NOMA transmission in M2M communications underlaying cellular heterogeneous networks,” IEEE Transactions on Vehicular Technology, vol. 69, no. 9, pp. 9866–9881, 2020. [120] H. Zhang, N. Liu, X. Chu, K. Long, A.-H. Aghvami, and V. C. Leung, “Network slicing based 5G and future mobile networks: Mobility, resource management, and challenges,” IEEE communications magazine, vol. 55, no. 8, pp. 138–145, 2017. [121] C.-X. Wang, J. Bian, J. Sun, W. Zhang, and M. Zhang, “A survey of 5G channel measurements and models,” IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 3142–3168, 2018. [122] I. Lopes, T. Guarda, A. Fernandes, and M. I. Ribeiro, “How 5G Will Transform Smart Cities: A Literature Review,” in International Conference on Computational Science and Its Applications.

Springer, 2023, pp. 70–81.

170

BIBLIOGRAPHY [123] L. Liu, X. Guo, and C. Lee, “Promoting smart cities into the 5G era with multi-field Internet of Things (IoT) applications powered with advanced mechanical energy harvesters,” Nano Energy, vol. 88, p. 106304, 2021. [124] A. Sharif, J. Guo, J. Ouyang, S. Sun, K. Arshad, M. A. Imran, and Q. H. Abbasi, “Compact base station antenna based on image theory for UWB/5G RTLS embraced smart parking of driverless cars,” IEEE Access, vol. 7, pp. 180 898–180 909, 2019. [125] D. Marabissi, L. Mucchi, R. Fantacci, M. R. Spada, F. Massimiani, A. Fratini, G. Cau, J. Yunpeng, and L. Fedele, “A real case of implementation of the future 5G city,” Future Internet, vol. 11, no. 1, p. 4, 2018. [126] Y. Jiang, W. Xiao, R. Wang, and A. Barnawi, “Smart urban living: Enabling emotion-guided interaction with next generation sensing fabric,” IEEE Access, vol. 8, pp. 28 395–28 402, 2019. [127] X. Zhao, H. Askari, and J. Chen, “Nanogenerators for smart cities in the era of 5G and Internet of Things,” Joule, vol. 5, no. 6, pp. 1391–1431, 2021. [128] J. E. Sánchez-Cano, W. X. Garcı́a-Quilachamin, J. Pérez-Véliz, J. Herrera-Tapia, and K. A. Fuentes, “Review of methods to reduce energy consumption in a smart city based on IoT and 5G technology,” iJOE, vol. 17, no. 08, p. 5, 2021. [129] B. Rana, S.-S. Cho, and I.-P. Hong, “Review paper on hardware of reconfigurable intelligent surfaces,” IEEE Access, 2023. [130] P. Zivuku, S. Kisseleff, V.-D. Nguyen, K. Ntontin, W. A. Martins, S. Chatzinotas, and B. Ottersten, “Maximizing the Number of Served Users in a Smart City using Reconfigurable Intelligent Surfaces,” in IEEE Wireless Communications and Networking Conference (WCNC), 2022, pp. 494–499. [131] A. Al Amin, J. Hong, V.-H. Bui, and W. Su, “Emerging 6G/B6G wireless communication for the power infrastructure in smart cities: Innovations, challenges, and future perspectives,” Algorithms, vol. 16, no. 10, p. 474, 2023. [132] S. Kisseleff, W. A. Martins, H. Al-Hraishawi, S. Chatzinotas, and B. Ottersten, “Reconfigurable intelligent surfaces for smart cities: Research challenges and opportunities,” IEEE Open Journal of the Communications Society, vol. 1, pp. 1781–1797, 2020.

171

BIBLIOGRAPHY [133] R. Liu, M. Li, H. Luo, Q. Liu, and A. L. Swindlehurst, “Integrated sensing and communication with reconfigurable intelligent surfaces: Opportunities, applications, and future directions,” IEEE Wireless Communications, vol. 30, no. 1, pp. 50–57, 2023. [134] M. Salah, A. Pitsillides, and A. S. Mubarak, “Paving the way for economically accepted and technically pronounced smart radio environment,” China Communications, vol. 19, no. 8, pp. 247–266, 2022. [135] A. U. Makarfi, K. M. Rabie, O. Kaiwartya, K. Adhikari, X. Li, M. Quiroz-Castellanos, and R. Kharel, “Reconfigurable intelligent surfaces-enabled vehicular networks: A physical layer security perspective,” arXiv preprint arXiv:2004.11288, 2020. [136] M. Kamruzzaman, “Key technologies, applications and trends of internet of things for energyefficient 6G wireless communication in smart cities,” Energies, vol. 15, no. 15, p. 5608, 2022. [137] L. Bariah, H. Sari, and M. Debbah, “Digital twin-empowered smart cities: A new frontier of wireless networks,” Authorea Preprints, 2023. [138] T. Dagiuklas, “The journey from 5G towards 6G,” in 8th International Symposium on Electrical and Electronics Engineering (ISEEE).

IEEE, 2023, pp. 14–18.

[139] J. Li, “From Liquid Crystal on Silicon and Liquid Crystal Reflectarray to Reconfigurable Intelligent Surfaces for Post-5G Networks,” Applied Sciences, vol. 13, no. 13, p. 7407, 2023. [140] P. Mishra and G. Singh, “6G-IoT Framework for Sustainable Smart City: Vision and Challenges,” in Sustainable Smart Cities: Enabling Technologies, Energy Trends and Potential Applications.

Springer, 2023, pp. 97–117.

[141] C. Masouros, J. A. Zhang, F. Liu, L. Zheng, H. Wymeersch, and M. Di Renzo, “Guest Editorial: Integrated Sensing and Communications for 6G,” IEEE Wireless Communications, vol. 30, no. 1, pp. 14–15, 2023. [142] T. Hou, Y. Liu, Z. Song, X. Sun, Y. Chen, and L. Hanzo, “Reconfigurable intelligent surface aided NOMA networks,” IEEE Journal on Selected Areas in Communications, vol. 38, no. 11, pp. 2575–2588, 2020.

172

BIBLIOGRAPHY [143] Zhou, Shaoqing and Xu, Wei and Wang, Kezhi and Di Renzo, Marco and Alouini, MohamedSlim, “Spectral and energy efficiency of irs-assisted miso communication with hardware impairments,” IEEE Wireless Communications Letters, vol. 9, no. 9, pp. 1366–1369, 2020. [144] X. GUAN and Q. WU, “IRS-Enabled Spectrum Sharing: Interference Modeling, Channel Estimation and Robust Passive Beamforming,” ZTE Communications, vol. 20, no. 1, pp. 28–35, 2022. [145] Z. Ji, Z. Qin, and C. G. Parini, “Reconfigurable intelligent surface aided cellular networks with device-to-device users,” IEEE Transactions on Communications, vol. 70, no. 3, pp. 1808–1819, 2022. [146] C. Liaskos, L. Mamatas, A. Pourdamghani, A. Tsioliaridou, S. Ioannidis, A. Pitsillides, S. Schmid, and I. F. Akyildiz, “Software-defined reconfigurable intelligent surfaces: From theory to end-to-end implementation,” Proceedings of the IEEE, vol. 110, no. 9, pp. 1466–1493, 2022. [147] A. Banchs, M. Fiore, A. Garcia-Saavedra, and M. Gramaglia, “Network intelligence in 6G: Challenges and opportunities,” in Proceedings of the 16th ACM Workshop on Mobility in the Evolving Internet Architecture, 2021, pp. 7–12. [148] S. Basharat, M. Khan, M. Iqbal, U. S. Hashmi, S. A. R. Zaidi, and I. Robertson, “Exploring reconfigurable intelligent surfaces for 6G: State-of-the-art and the road ahead,” IET Communications, vol. 16, no. 13, pp. 1458–1474, 2022. [149] Y. Liu, X. Liu, X. Mu, T. Hou, J. Xu, M. Di Renzo, and N. Al-Dhahir, “Reconfigurable intelligent surfaces: Principles and opportunities,” IEEE communications surveys & tutorials, vol. 23, no. 3, pp. 1546–1577, 2021. [150] X. Mu, Y. Liu, L. Guo, J. Lin, and N. Al-Dhahir, “Capacity and optimal resource allocation for IRS-assisted multi-user communication systems,” IEEE Transactions on Communications, vol. 69, no. 6, pp. 3771–3786, 2021. [151] A. Elzanaty, A. Guerra, F. Guidi, and M.-S. Alouini, “Reconfigurable intelligent surfaces for localization: Position and orientation error bounds,” IEEE Transactions on Signal Processing, vol. 69, pp. 5386–5402, 2021.

173

BIBLIOGRAPHY [152] H. Yang, Z. Xiong, J. Zhao, D. Niyato, Q. Wu, H. V. Poor, and M. Tornatore, “Intelligent reflecting surface assisted anti-jamming communications: A fast reinforcement learning approach,” IEEE transactions on wireless communications, vol. 20, no. 3, pp. 1963–1974, 2020. [153] N. M. Salih, M. Aldababsa, and K. Yahya, “Enhancing UAV communication links with Reconfigurable intelligent surfaces,” AEU-International Journal of Electronics and Communications, vol. 171, p. 154933, 2023. [154] M. Diamanti, M. Tsampazi, E. E. Tsiropoulou, and S. Papavassiliou, “Energy efficient multi-user communications aided by reconfigurable intelligent surfaces and UAVs,” in IEEE International Conference on Smart Computing (SMARTCOMP), 2021, pp. 371–376. [155] S. Li, B. Duo, X. Yuan, Y.-C. Liang, and M. Di Renzo, “Reconfigurable intelligent surface assisted UAV communication: Joint trajectory design and passive beamforming,” Wireless Communications Letters, vol. 9, no. 5, pp. 716–720, 2020. [156] S. Basharat, S. A. Hassan, H. Pervaiz, A. Mahmood, Z. Ding, and M. Gidlund, “Reconfigurable intelligent surfaces: Potentials, applications, and challenges for 6G wireless networks,” Wireless Communications, vol. 28, no. 6, pp. 184–191, 2021. [157] Z. Ning, T. Li, Y. Wu, X. Wang, Q. Wu, F. R. Yu, and S. Guo, “Intelligent-Reflecting-SurfaceAssisted UAV Communications for 6G Networks,” arXiv preprint arXiv:2310.20242, 2023. [158] Y. Yu, X. Liu, and V. C. Leung, “Fair downlink communications for RIS-UAV enabled mobile vehicles,” Wireless Communications Letters, vol. 11, no. 5, pp. 1042–1046, 2022. [159] MATLAB R2021a, The Mathworks, Inc., Natick, Massachusetts, 2021. [160] QuaDRiGa Source Code GitHub repository. [Available Online]:

https://github.com/

fraunhoferhhi/QuaDRiGa. [161] E. Basar and I. Yildirim, “Reconfigurable intelligent surfaces for future wireless networks: A channel modeling perspective,” IEEE Wireless Communications, vol. 28, no. 3, pp. 108–114, 2021. [162] E. Basar, and I. Yildirim, “SimRIS channel simulator for reconfigurable intelligent surfaceempowered communication systems,” in Latin-American Conference on Communications (LATINCOM).

IEEE, 2020, pp. 1–6. 174

BIBLIOGRAPHY [163] B. Sihlbom, M. I. Poulakis, and M. Di Renzo, “Reconfigurable intelligent surfaces: Performance assessment through a system-level simulator,” Wireless Communications, 2022. [164] Y. Sun, C.-X. Wang, J. Huang, and J. Wang, “A 3D non-stationary channel model for 6G wireless systems employing intelligent reflecting surfaces with practical phase shifts,” IEEE Transactions on Cognitive Communications and Networking, vol. 7, no. 2, pp. 496–510, 2021. [165] J. Dang, S. Gao, Y. Zhu, R. Guo, H. Jiang, Z. Zhang, L. Wu, B. Zhu, and L. Wang, “A geometry-based stochastic channel model and its application for intelligent reflecting surface assisted wireless communication,” IET Communications, vol. 15, no. 3, pp. 421–434, 2021. [166] M. Dajer, Z. Ma, L. Piazzi, N. Prasad, X.-F. Qi, B. Sheen, J. Yang, and G. Yue, “Reconfigurable intelligent surface: Design the channel–A new opportunity for future wireless networks,” Digital Communications and Networks, vol. 8, no. 2, pp. 87–104, 2022. [167] H. Zhou, M. Erol-Kantarci, Y. Liu, and H. V. Poor, “A survey on model-based, heuristic, and machine learning optimization approaches in ris-aided wireless networks,” IEEE Communications Surveys & Tutorials, 2023. [168] L. Kong, J. He, Y. Ai, S. Chatzinotas, and B. Ottersten, “Channel modeling and analysis of reconfigurable intelligent surfaces assisted vehicular networks,” in International Conference on Communications Workshops (ICC Workshops).

IEEE, 2021, pp. 1–6.

[169] Z. Tian, Z. Chen, M. Wang, Y. Jia, and W. Wen, “Reconfigurable intelligent surface-aided spectrum sharing coexisting with multiple primary networks,” in IEEE Wireless Communications and Networking Conference (WCNC), 2023, pp. 1–6. [170] Y. Chen, Y. Wang, J. Zhang, and M. Di Renzo, “QoS-driven spectrum sharing for reconfigurable intelligent surfaces (RISs) aided vehicular networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 9, pp. 5969–5985, 2021. [171] A.-A. A. Boulogeorgos and A. Alexiou, “Coverage analysis of reconfigurable intelligent surface assisted THz wireless systems,” IEEE Open Journal of Vehicular Technology, vol. 2, pp. 94–110, 2021. [172] E. Björnson, H. Wymeersch, B. Matthiesen, P. Popovski, L. Sanguinetti, and E. de Carvalho, “Reconfigurable intelligent surfaces: A signal processing perspective with wireless applications,” IEEE Signal Processing Magazine, vol. 39, no. 2, pp. 135–158, 2022. 175

BIBLIOGRAPHY [173] X. Yu, V. Jamali, D. Xu, D. W. K. Ng, and R. Schober, “Smart and reconfigurable wireless communications: From irs modeling to algorithm design,” Wireless Communications, vol. 28, no. 6, pp. 118–125, 2021. [174] J. Xu, X. Mu, and Y. Liu, “Exploiting STAR-RISs in near-field communications,” IEEE Transactions on Wireless Communications, 2023. [175] W. Tang, X. Chen, M. Z. Chen, J. Y. Dai, Y. Han, S. Jin, Q. Cheng, G. Y. Li, and T. J. Cui, “On channel reciprocity in reconfigurable intelligent surface assisted wireless networks,” Wireless Communications, vol. 28, no. 6, pp. 94–101, 2021. [176] L. G. da Silva, Z. Chu, P. Xiao, and A. Cerqueira S Jr, “A varactor-based 1024-element RIS design for mm-waves,” Frontiers in Communications and Networks, vol. 4, p. 1086011, 2023. [177] Y. Liu, J. Dou, Y. Cui, Y. Chen, J. Yang, F. Qin, and Y. Wang, “Reconfigurable Intelligent Surface Physical Model in Channel Modeling,” Electronics, vol. 11, no. 17, p. 2798, 2022. [178] R. Liu, J. Dou, P. Li, J. Wu, and Y. Cui, “Simulation and field trial results of reconfigurable intelligent surfaces in 5g networks,” IEEE Access, vol. 10, pp. 122 786–122 795, 2022. [179] M. Rossanese, P. Mursia, A. Garcia-Saavedra, V. Sciancalepore, A. Asadi, and X. Costa-Perez, “Designing, building, and characterizing RF switch-based reconfigurable intelligent surfaces,” in Proceedings of the 16th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization, 2022, pp. 69–76. [180] C. Huang, A. Zappone, G. C. Alexandropoulos, M. Debbah, and C. Yuen, “Reconfigurable Intelligent Surfaces for Energy Efficiency in Wireless Communication,” IEEE Transactions on Wireless Communications, vol. 18, no. 8, pp. 4157–4170, 2019. [181] X. Yu, G. Wang, X. Huang, K. Wang, W. Xu, and Y. Rui, “Energy Efficient Resource Allocation for Uplink RIS-Aided Millimeter-Wave Networks With NOMA,” IEEE Transactions on Mobile Computing, vol. 23, no. 1, pp. 423–436, 2024. [182] A. A. Deshpande, C. J. Vaca-Rubio, S. Mohebi, D. Salami, E. de Carvalho, P. Popovski, S. Sigg, M. Zorzi, and A. Zanella, “Energy-efficient design for ris-assisted uav communications in beyond-5g networks,” in 2022 20th Mediterranean Communication and Computer Networking Conference (MedComNet), 2022, pp. 158–165.

176

BIBLIOGRAPHY [183] M. Ahsan, S. Jamil, M. T. Ejaz, and M. S. Abbas, “Energy efficiency maximization in risassisted wireless networks,” in 2021 International Conference on Computing, Electronic and Electrical Engineering (ICE Cube), 2021, pp. 1–6. [184] R. K. Fotock, A. Zappone, and M. D. Renzo, “Energy Efficiency Maximization in RIS-aided Networks with Global Reflection Constraints,” in ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023, pp. 1–5. [185] Y. Zhang, Y. Lu, R. Zhang, B. Ai, and D. Niyato, “Deep reinforcement learning for secrecy energy efficiency maximization in ris-assisted networks,” IEEE Transactions on Vehicular Technology, vol. 72, no. 9, pp. 12 413–12 418, 2023. [186] Y. Guo, F. Fang, D. Cai, and Z. Ding, “Energy-Efficient Design for a NOMA Assisted STAR-RIS Network With Deep Reinforcement Learning,” IEEE Transactions on Vehicular Technology, vol. 72, no. 4, pp. 5424–5428, 2023. [187] M. Diamanti, P. Charatsaris, E. E. Tsiropoulou, and S. Papavassiliou, “The prospect of reconfigurable intelligent surfaces in integrated access and backhaul networks,” IEEE Transactions on Green Communications and Networking, vol. 6, no. 2, pp. 859–872, 2021. [188] M. Tehrani-Moayyed, L. Bonati, P. Johari, T. Melodia, and S. Basagni, “Creating RF Scenarios for Large-Scale, Real-Time Wireless Channel Emulators,” in Proceedings of IEEE MedComNet, 2021. [189] R. Rusca, F. Raviglione, C. Casetti, P. Giaccone, and F. Restuccia, “Mobile RF Scenario Design for Massive-Scale Wireless Channel Emulators,” in Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit).

IEEE, 2023, pp. 675–680.

[190] S. Lasaulce, Y. Hayel, R. El Azouzi, and M. Debbah, “Introducing hierarchy in energy games,” IEEE Transactions on Wireless Communications, vol. 8, no. 7, pp. 3833–3843, 2009. [191] G. He, S. Lasaulce, and Y. Hayel, “Stackelberg games for energy-efficient power control in wireless networks,” in 2011 Proceedings IEEE INFOCOM.

IEEE, 2011, pp. 591–595.

[192] srsRAN website. [Available Online]: https://www.srslte.com/. Accessed September 2022. [193] “Google Maps,” [Available Online]: https://www.google.com/maps.

177

BIBLIOGRAPHY [194] S. Ju, O. Kanhere, Y. Xing, and T. S. Rappaport, “A millimeter-wave channel simulator NYUSIM with spatial consistency and human blockage,” in IEEE global communications conference (GLOBECOM), 2019, pp. 1–6. [195] P. Kyosti, J. Meinila, L. Hentila, X. Zhao, T. Jamsa, C. Schneider, M. Narandzic, M. Milojevic, A. Hong, J. Ylitalo et al., “IST-4-027756 WINNER II D1. 1.2 V1. 2: WINNER II channel models,” Available: www. ist-winner. org, 2007. [196] F. Ademaj, M. Taranetz, and M. Rupp, “3GPP 3D MIMO channel model: A holistic implementation guideline for open source simulation tools,” EURASIP Journal on Wireless Communications and Networking, no. 1, pp. 1–14, 2016. [197] L. Pang, J. Zhang, Y. Zhang, X. Huang, Y. Chen, and J. LI, “Investigation and Comparison of 5G Channel Models: From QuaDRiGa, NYUSIM, and MG5G Perspectives,” Chinese Journal of Electronics, vol. 31, no. 1, pp. 1–17, 2022. [198] B. Mondal, T. A. Thomas, E. Visotsky, F. W. Vook, A. Ghosh, Y.-H. Nam, Y. Li, J. Zhang, M. Zhang, Q. Luo et al., “3D channel model in 3GPP,” IEEE Communications Magazine, vol. 53, no. 3, pp. 16–23, 2015. [199] S. Jaeckel, L. Raschkowski, K. Boerner, and L. Thiele, “F. Burkhardt, E. Eberlein,“QuaDRiGaQuasi Deterministic Radio Channel Generator,” User Manual and Documentation,” Tech. Rep. v2. 2.0, Fraunhofer Heinrich Hertz Institute, Tech. Rep., 2019, [Available Online]: https: //github.com/fraunhoferhhi/QuaDRiGa/blob/main/quadriga documentation v2.8.1-0.pdf. [200] S. Jaeckel, L. Raschkowski, K. Börner, and L. Thiele, “QuaDRiGa: A 3-D multi-cell channel model with time evolution for enabling virtual field trials,” IEEE transactions on antennas and propagation, vol. 62, no. 6, pp. 3242–3256, 2014. [201] M. E. Ranjkesh and A. A. Khazaei, “The Optimized QuaDRiGa Wi-Fi Channel Model,” International Journal of Science and Engineering Investigations, vol. 4, no. 9, pp. 51–61, 2015. [202] Intelligent Transport Systems (ITS); Access Layer; Part 1: Channel Models for the 5,9 GHz frequency band. Sophia Antipolis Cedex, FRANCE, ETSI TR 103 257-1 V1.1.1, Tech. Rep., May, 2019. [Available Online]: https://www.etsi.org/deliver/etsi tr/103200 103299/10325701/ 01.01.01 60/tr 10325701v010101p.pdf. 178

BIBLIOGRAPHY [203] Y. He, Y. Zhang, J. Zhang, L. Pang, Y. Chen, and G. Ren, “Investigation and Comparison of QuaDRiGa, NYUSIM and MG5G Channel Models for 5G Wireless Communications,” in IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), 2020, pp. 1–5. [204] ETSI TR 138 901 V14.3.0 (2018-01). [Available Online]: https://www.etsi.org/deliver/etsi tr/ 138900 138999/138901/14.03.00 60/tr 138901v140300p.pdf. [205] C. Arunachalaperumal, S. Dhilipkumar, and G. Abija, “Enhanced 3D MIMO channel for urban macro environment,” International Journal of Pure and Applied Mathematics, Special Issue, vol. 118, no. 10, pp. 259–269, 2018. [206] M. Giordani, T. Shimizu, A. Zanella, T. Higuchi, O. Altintas, and M. Zorzi, “Path loss models for V2V mmWave communication: performance evaluation and open challenges,” in IEEE Connected and Automated Vehicles Symposium (CAVS).

IEEE, 2019, pp. 1–5.

[207] C. F. Mecklenbrauker, A. F. Molisch, J. Karedal, F. Tufvesson, A. Paier, L. Bernadó, T. Zemen, O. Klemp, and N. Czink, “Vehicular channel characterization and its implications for wireless system design and performance,” Proceedings of the IEEE, vol. 99, no. 7, pp. 1189–1212, 2011. [208] B. Bloessl, M. Segata, C. Sommer, and F. Dressler, “Performance assessment of IEEE 802.11p with an open source SDR-based prototype,” IEEE transactions on mobile computing, vol. 17, no. 5, pp. 1162–1175, 2017. [209] A. A. Khan, A. A. Laghari, M. Shafiq, S. A. Awan, and Z. Gu, “Vehicle to everything (V2X) and edge computing: A secure lifecycle for UAV-assisted vehicle network and offloading with blockchain,” Drones, vol. 6, no. 12, p. 377, 2022. [210] L. C. Paul, H. K. Saha, T. Rani, M. Z. Mahmud, T. K. Roy, W.-S. Lee et al., “An OmniDirectional Wideband Patch Antenna with Parasitic Elements for Sub-6 GHz Band Applications,” International Journal of Antennas and Propagation, 2022. [211] R. Azim, A. M. H. Meaze, A. Affandi, M. M. Alam, R. Aktar, M. S. Mia, T. Alam, M. Samsuzzaman, and M. T. Islam, “A multi-slotted antenna for LTE/5G Sub-6 GHz wireless communication applications,” International Journal of Microwave and Wireless Technologies, vol. 13, no. 5, pp. 486–496, 2021.

179

BIBLIOGRAPHY [212] T. Rani, S. C. Das, M. S. Hossen, L. C. Paul, and T. K. Roy, “Development of a Broadband Antenna for 5G Sub-6 GHz Cellular and IIoT Smart Automation Applications,” in 12th International Conference on Electrical and Computer Engineering (ICECE). IEEE, 2022, pp. 465–468. [213] M. S. Sharawi, M. Ikram, and A. Shamim, “A two concentric slot loop based connected array MIMO antenna system for 4G/5G terminals,” IEEE Transactions on antennas and propagation, vol. 65, no. 12, pp. 6679–6686, 2017. [214] B. De Beelde, E. Tanghe, M. Yusuf, D. Plets, and W. Joseph, “Radio channel modeling in a ship hull: Path loss at 868 MHz and 2.4, 5.25, and 60 GHz,” IEEE Antennas and Wireless Propagation Letters, vol. 20, no. 4, pp. 597–601, 2021. [215] A. Zaidi, W. A. Awan, N. Hussain, and A. Baghdad, “A Wide and Tri-band Flexible Antennas with Independently Controllable Notch Bands for Sub-6-GHz Communication System.” Radioengineering, vol. 29, no. 1, 2020. [216] C.-X. Mao, M. Khalily, P. Xiao, T. W. Brown, and S. Gao, “Planar sub-millimeter-wave array antenna with enhanced gain and reduced sidelobes for 5G broadcast applications,” IEEE Transactions on Antennas and Propagation, vol. 67, no. 1, pp. 160–168, 2018. [217] S. Ranvier, S. Dudorov, M. Kyro, C. Luxey, C. Icheln, R. Staraj, and P. Vainikainen, “Low-cost planar omnidirectional antenna for mm-wave applications,” IEEE Antennas and Wireless Propagation Letters, vol. 7, pp. 521–523, 2008. [218] K. Fan, Z.-C. Hao, Q. Yuan, J. Hu, G. Q. Luo, and W. Hong, “Wideband horizontally polarized omnidirectional antenna with a conical beam for millimeter-wave applications,” IEEE Transactions on Antennas and Propagation, vol. 66, no. 9, pp. 4437–4448, 2018. [219] G. R. Maccartney, T. S. Rappaport, M. K. Samimi, and S. Sun, “Millimeter-wave omnidirectional path loss data for small cell 5G channel modeling,” IEEE access, vol. 3, pp. 1573–1580, 2015. [220] M. N. Hasan, S. Bashir, and S. Chu, “Dual band omnidirectional millimeter wave antenna for 5G communications,” Journal of Electromagnetic Waves and Applications, vol. 33, no. 12, pp. 1581–1590, 2019.

180

BIBLIOGRAPHY [221] S. Sun, G. R. MacCartney, M. K. Samimi, and T. S. Rappaport, “Synthesizing omnidirectional antenna patterns, received power and path loss from directional antennas for 5G millimeterwave communications,” in IEEE Global Communications Conference (GLOBECOM). IEEE, 2015, pp. 1–7. [222] M. Abirami, “A review of patch antenna design for 5G,” in IEEE International Conference on Electrical, Instrumentation and Communication Engineering (ICEICE).

IEEE, 2017, pp.

1–3. [223] Y. Niu, Y. Li, D. Jin, L. Su, and A. V. Vasilakos, “A survey of millimeter wave communications (mmWave) for 5G: opportunities and challenges,” Wireless networks, vol. 21, pp. 2657–2676, 2015. [224] H. K. Pan, “Dual-polarized Mm-wave phased array antenna for multi-Gb/s 60GHz communication,” in IEEE International Symposium on Antennas and Propagation (APSURSI).

IEEE,

2011, pp. 3279–3282. [225] C. X. Mao, M. Khalily, L. Zhang, P. Xiao, Y. Sun, and D. H. Werner, “Compact patch antenna with vertical polarization and omnidirectional radiation characteristics,” IEEE Transactions on Antennas and Propagation, vol. 69, no. 2, pp. 1158–1161, 2020. [226] E. E. Tsiropoulou, P. Vamvakas, G. K. Katsinis, and S. Papavassiliou, “Combined power and rate allocation in self-optimized multi-service two-tier femtocell networks,” Computer Communications, vol. 72, 2015. [227] ETSI TS 138 101-1, “5G; NR; User Equipment (UE) radio transmission and reception; Part 1: Range 1 Standalone (3GPP TS 38.101-1 version 17.5.0 Release 17),” [Available Online]: https://www.etsi.org/deliver/etsi ts/138100 138199/13810101/17.05.00 60/ ts 13810101v170500p.pdf, 2022, eTSI TS 138 101-1 V17.5.0 (2022-05). [228] ETSI TS 138 101-2, “5G; NR; User Equipment (UE) radio transmission and reception; Part 2: Range 2 Standalone (3GPP TS 38.101-2 version 17.5.0 Release 17),” ETSI, Tech. Rep. ETSI TS 138 101-2 V17.5.0, 2022. [229] ETSI TS 136 213, “LTE; Evolved Universal Terrestrial Radio Access (E-UTRA); Physical layer procedures (3GPP TS 36.213 version 14.2.0 Release 14),” [Avail-

181

BIBLIOGRAPHY able Online]: https://www.etsi.org/deliver/etsi ts/136200 136299/136213/14.02.00 60/ts 136213v140200p.pdf, 2017, eTSI TS 136 213 V14.2.0 (2017-04). [230] V. Tapio, I. Hemadeh, A. Mourad, A. Shojaeifard, and M. Juntti, “Survey on reconfigurable intelligent surfaces below 10 GHz,” EURASIP Journal on Wireless Communications and Networking, pp. 1–18, 2021. [231] E. Björnson, Ö. Özdogan, and E. G. Larsson, “Reconfigurable intelligent surfaces: Three myths and two critical questions,” IEEE Communications Magazine, vol. 58, no. 12, pp. 90–96, 2020. [232] M. Z. Siddiqi and T. Mir, “Reconfigurable intelligent surface-aided wireless communications: An overview,” Intelligent and Converged Networks, vol. 3, no. 1, pp. 33–63, 2022. [233] D. Villa, D. Uvaydov, L. Bonati, P. Johari, J. M. Jornet, and T. Melodia, “Twinning Commercial Radio Waveforms in the Colosseum Wireless Network Emulator,” in Proceedings of the 17th ACM Workshop on Wireless Network Testbeds, Experimental evaluation & Characterization, 2023, pp. 33–40. [234] J. Angjo, A. Zubow, and F. Dressler, “Side Effects of IRS: On the Need for Coordination in 6G Multi-Operator IRS-assisted Networks,” in IEEE Global Communications Conference (GLOBECOM), 4th Workshop on Emerging Topics in 6G Communications (6GComm). IEEE, 12 2023, pp. 1380–1385. [235] B. Zheng, Q. Wu, and R. Zhang, “Intelligent Reflecting Surface-Assisted Multiple Access With User Pairing: NOMA or OMA?” IEEE Communications Letters, vol. 24, no. 4, pp. 753–757, 2020. [236] NVIDIA Corporation, “NVIDIA and Nokia to Pioneer the AI Platform for 6G — Powering America’s Return to Telecommunications Leadership,” NVIDIA Newsroom, Press Release, Oct. 2025, [Available Online]: https://nvidianews.nvidia.com/news/nvidia-nokia-aitelecommunications. [237] N. Khedhri and M. Najar, “Adaptive Modulation Selection in Wireless Communications: A Comparative Study of Reinforcement Learning, Deep Learning, Deep Reinforcement Learning, and Traditional Policies,” in International Wireless Communications and Mobile Computing (IWCMC).

IEEE, 2025, pp. 1622–1625. 182

BIBLIOGRAPHY [238] S. Peri, A. Russo, G. Fodor, and P. Soldati, “Offline reinforcement learning and sequence modeling for downlink link adaptation,” in International Conference on Machine Learning for Communication and Networking (ICMLCN).

IEEE, 2025, pp. 1–7.

[239] S. K. Pulliyakode and S. Kalyani, “Reinforcement learning techniques for outer loop link adaptation in 4G/5G systems,” arXiv preprint arXiv:1708.00994, 2017. [240] V. Saxena, J. Jaldén, J. E. Gonzalez, M. Bengtsson, H. Tullberg, and I. Stoica, “Contextual multi-armed bandits for link adaptation in cellular networks,” in Workshop on Network Meets AI & ML, 2019, pp. 44–49. [241] A. Zubow, S. Rösler, P. Gawłowicz, and F. Dressler, “GrGym: When GNU radio goes to (AI) gym,” in Proc. of the 22nd International Workshop on Mobile Computing Systems and Applications, 2021, pp. 8–14. [242] J. P. Leite, P. H. P. de Carvalho, and R. D. Vieira, “A flexible framework based on reinforcement learning for adaptive modulation and coding in OFDM wireless systems,” in Wireless Communications and Networking Conference (WCNC).

IEEE, 2012, pp. 809–814.

[243] K. I. Pedersen, G. Monghal, I. Z. Kovacs, T. E. Kolding, A. Pokhariyal, F. Frederiksen, and P. Mogensen, “Frequency domain scheduling for OFDMA with limited and noisy channel feedback,” in IEEE 66th Vehicular Technology Conference, 2007, pp. 1792–1796. [244] P. Kela, T. Höhne, T. Veijalainen, and H. Abdulrahman, “Reinforcement learning for delay sensitive uplink outer-loop link adaptation,” in Joint European Conference on Networks and Communications & 6G Summit.

IEEE, 2022, pp. 59–64.

[245] R. Wiesmayr, L. Maggi, S. Cammerer, J. Hoydis, F. A. Aoudia, and A. Keller, “SALAD: Self-adaptive link adaptation,” arXiv preprint arXiv:2510.05784, 2025. [246] L. Maggi, B. Bonev, R. Wiesmayr, S. Cammerer, and A. Keller, “SINR Estimation under Limited Feedback via Online Convex Optimization,” arXiv preprint arXiv:2603.02061, 2026. [247] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “StableBaselines3: Reliable Reinforcement Learning Implementations,” GitHub Repository, 2021, [Available Online]: https://github.com/DLR-RM/stable-baselines3.

183

BIBLIOGRAPHY [248] Farama Foundation, “Gymnasium: A Standard Interface for Reinforcement Learning Environments,” GitHub Repository, 2023, [Available Online]: https://github.com/Farama-Foundation/ Gymnasium. [249] J. R. Quinlan, “Induction of decision trees,” Machine Learning, vol. 1, no. 1, pp. 81–106, 1986. [250] R. E. Kalman, “A New Approach to Linear Filtering and Prediction Problems,” Transactions of the ASME–Journal of Basic Engineering, vol. 82, pp. 35–45, 1960. [251] L. Breiman, “Random Forests,” Machine Learning, vol. 45, 2001. [252] D. Ernst, P. Geurts, and L. Wehenkel, “Tree-based batch mode reinforcement learning,” Journal of Machine Learning Research, vol. 6, 2005. [253] N. Heydarishahreza, T. Han, and N. Ansari, “Spectrum sharing and interference management for 6G LEO satellite-terrestrial network integration,” Communications Surveys & Tutorials, 2024. [254] T.-S. R. Niloy, S. Kumar, A. Hore, Z. Hassan, C. Dietrich, E. W. Burger, J. H. Reed, and V. K. Shah, “ASCENT: A Context-Aware Spectrum Coexistence Design and Implementation Toolset for Policymakers in Satellite Bands,” in International Symposium on Dynamic Spectrum Access Networks (DySPAN).

IEEE, 2024, pp. 240–248.

[255] T.-S. R. Niloy, Z. Hassan, N. Stephenson, and V. K. Shah, “Interference Analysis of Coexisting 5G Networks and NGSO FSS Receivers in the 12-GHz Band,” IEEE Wireless Communications Letters, vol. 12, no. 9, pp. 1528–1532, 2023. [256] D. Wu, C. Qin, Y. Cui, P. He, and R. Wang, “Space-Ground Multicast Group Control for Multiuser LEO Satellite Networks,” IEEE Transactions on Wireless Communications, 2023. [257] T. Ma, B. Qian, X. Qin, X. Zhang, L. X. Cai, and H. Zhou, “Resource Scheduling for HighCapacity Multicast Service in Ultra-Dense LEO Satellite Networks,” IEEE Transactions on Vehicular Technology, 2023. [258] A. Wadaskar and D. Cabric, “Satellite-Terrestrial Coexistence in FR3 Band via Hybrid TrueTime-Delay Array-based Nulling,” in IEEE International Symposium on Dynamic Spectrum Access Networks, 2025. 184

[259] ITU-R RS.2017-0, “Performance and interference criteria for satellite passive remote sensing,” International Telecommunication Union, Radiocommunication Sector, Recommendation, 2012. [260] G. T. 38.901, “Study on channel model for frequencies from 0.5 to 100 GHz,” 3GPP, Technical Report, 2023. [261] S. Jaeckel, L. Raschkowski, and L. Thieley, “A 5G-NR satellite extension for the QuaDRiGa channel model,” in IEEE Joint European Conference on Networks and Communications & 6G Summit, 2022. [262] S.-Y. Li and C. Liu, “An analytical model to predict the probability density function of elevation angles for LEO satellite systems,” IEEE Communications Letters, vol. 6, no. 4, 2002. [263] R. K. Jain, D.-M. W. Chiu, W. R. Hawe et al., “A quantitative measure of fairness and discrimination,” Eastern Research Laboratory, Digital Equipment Corporation, Hudson, MA, vol. 21, no. 1, 1984. [264] MathWorks, “MATLAB (R2025a),” [Available Online]: https://www.mathworks.com, 2025. [265] C. A. Balanis, Antenna theory: analysis and design.

John Wiley & Sons, 2016.

[266] H. L. Van Trees, Optimum array processing: Part IV of detection, estimation, and modulation theory.

John Wiley & Sons, 2002.

185

Acknowledgments The completion of my Ph.D. brings back so many memories of hope, perseverance, joy, effort, and determination to succeed. The past five years have tremendously shaped my character, making this journey one of the most intense and transformative experiences of my life so far. First and foremost, I would like to sincerely thank my academic advisor, Prof. Tommaso Melodia, for offering me the opportunity to pursue a Ph.D. at one of the world’s leading academic institutions, Northeastern University in Boston. I am forever grateful to him for investing in me and giving me the opportunity to conduct cutting-edge research in the United States of America. Knowing that he believed in me was my greatest motivation to persevere and, most importantly, to strive to produce high-quality research. Alongside Prof. Tommaso Melodia, I would also like to thank the other members of my Ph.D. dissertation committee, Prof. Josep-Miquel Jornet and Prof. Athanasios D. Panagopoulos, for all the knowledge and support they have offered me. I am beyond grateful to have worked with all of you. I would also like to express my heartfelt thanks to the U.S. National Spectrum Consortium (NSC), and specifically to its Executive Director Joe Kochan, Chief Program Officer Mari Silbey, and Ericsson’s Chief Technology Officer and NSC board member Dr. Sheryl Genco, for recognizing me as one of the Women in Spectrum Scholars. I am deeply grateful for their support and belief in my abilities, as well as for offering me valuable opportunities, such as a work experience at Nokia Illinois. Furthermore, I would like to thank Nokia for inviting me as one of the panelists at the Brooklyn 6G Summit in 2026 and for nominating me to participate in the wireless game show (students vs. professors) and, ultimately, for providing me with a chance to brag that we, the students, beat Prof. Tommaso Melodia and the other top professors in the game show. I promised my academic advisor I would never let him forget it, and here I am. In addition, I would like to thank the Ph.D. students of the Institute for Intelligent Networked Systems for the countless hours of shared workspace, jokes, and laughter. I would also

186

like to thank Prof. Diomedes and Faye Logothetis, Chris and Georgia Lagadinos, Christos Kazilas and Aimilia Tsolia, Fr. Sotiri and Presvytera Stephanie Malamis, Fr. Anthony Tandilyan and Despoina Lioliou, Anthony Ladas, Anastasia Asimacopoulos, Dawn (my first Airbnb host in the United States), SooMi Lee, Samar Elmaadawy, Neagin Neasamoni Santhi, Duschia Bodet, Sherif Badran, Ravis Shirkhani, Ali Saeizadeh, Albert Diez Comas, Stefano Maxenti, and Matteo Bordin for their friendship and for offering a helping hand whenever I needed it. Last but not least, I would like to thank the most important people in my life so far, my family and closest friends. I want to thank Fr. Athanasios Polyzos for all his support and for endlessly listening to my worries over the phone, as well as my dearest friends, Athanasia Psorogianni and Maria Fragkou. Most importantly, I would like to thank my mother, Kostoula, for being my greatest strength and my driving force to keep moving forward throughout these years. I am beyond thankful to you, Mom, for all your sacrifices. Thank you for teaching me that the people who ultimately succeed are not those who get every response they hope for; instead, they are the ones who keep moving forward with their heads held high and with noble dignity. You inspire me to be as strong, yet as humble, as you are.

187

Related documents

• A two‐layer control strategy for fuel‐efficient connected vehicles#144836
• Physical-Layer Aspects of Repeater-Assisted MIMO#1077773
• Home Automation Routing Requirements in Low-Power and Lossy Networks#254536
• Primal-Dual Contextual Bayesian Optimization for Control System Online Optimization with Time-Average Constraints#1166577
• Stochastic MPC under Heavy-Tailed Disturbances: An Extreme Value Theory Approach#1035138