An O-RAN-Assisted MARL Approach for Dynamic Sidelink and Infrastructure Selection in V2X Communications

1

An O-RAN-Assisted MARL Approach for Dynamic Sidelink and Infrastructure Selection in V2X Communications

arXiv:2608.17210v1 [cs.NI] 17 Aug 2026

Maria Barbosa, Kelvin Lopes Dias

Abstract—Future applications in the 6G-based Internet of Vehicles (IoV) will leverage sidelink (SL) transmissions in Vehicleto-Everything (V2X) scenarios. However, SL-based direct communication can significantly increase interference among vehicles and between vehicles and other entities of the Intelligent Transportation System (ITS). Thus, both Vehicle-to-Vehicle (V2V) communications and Vulnerable Road Users (VRUs) uplink resources may be degraded or subject to starvation. Existing solutions primarily focus on improving resource allocation and pair selection. Nonetheless, they lack a comprehensive approach to tackle the communication modes and the entire network. To address these challenges, this paper leverages Open RAN to manage V2X communication and proposes a multi-agent reinforcement learning (MARL) resource-aware system. Open RAN provides control loops through a global view of the network and also an open interface-based framework for machine learning models applied to resource decision-making. Meanwhile, the MARL model aims to mitigate interference, optimize resource usage, and enhance quality of service by optimally selecting between sidelink and network (V2N) transmissions. To reduce system complexity, this work employs a clustering strategy. Each agent manages a group of pairs, rather than assigning one agent to each pair. The solution supports this design by adopting a centralized training with decentralized execution (CTDE) approach, empowered by Open RAN. The O-RAN Alliance specifications require pre-trained models. To comply, the strategy uses offline training and an off-policy approach, where each agent stores experiences for fine-tuning. For benchmarking, the baselines include a single-agent approach and two heuristics: a Signal-to-Interference-plus-Noise Ratio (SINR)-based scheme and the Best Channel Quality Indicator (CQI) policy. The MARL approach reduces average loss by 21% and latency by 19% in vehicle-only scenarios, and by 18% and 30% in VRU coexistence scenarios, respectively. Index Terms—Sidelink, Open RAN, reinforcement learning, multi-agent, V2X, internet of vehicles.

I. I NTRODUCTION

I

NTELLIGENT Transportation Systems (ITS) aim to enhance road safety and operational efficiency, driving the evolution of the connected vehicles paradigm and, consequently, the emergence of the Internet of Vehicles (IoV). In this context, vehicle-to-everything (V2X) communication plays a crucial role in meeting the stringent requirements of low latency, high reliability, and high data throughput [1]. V2X enables the deployment of emerging vehicular applications, such as remote, advanced, and cooperative driving, road warning, platooning, and see-through systems [2]. Copyright © 2026 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org. Maria Barbosa and Kelvin Lopes Dias are with the Centro de Informática, Universidade Federal de Pernambuco, Recife-PE, Brazil (e-mail: mksb@cin.ufpe.br; kld@cin.ufpe.br).

To enhance V2X communication, the 3rd Generation Partnership Project (3GPP) proposed Sidelink (SL) communication, which enables vehicles to communicate directly with each other. The 3GPP introduced the SL concept in LTE Release 12 and added support for V2X communication in Release 14. Nevertheless, only the 5G NR Release 16 expanded the support to include the aforementioned advanced use cases [3]. SL uses part of the uplink radio resources [4]. In this context, resource sharing among Vulnerable Road Users (VRUs), which in ITS refers to non-motorized road users such as pedestrians and cyclists, as well as vehicle-toinfrastructure (V2I), vehicle-to-vehicle (V2V), and vehicle-tonetwork (V2N) communications, leads to mutual interference and reduces overall system performance. Furthermore, the limited availability of uplink resources also constrains the system [5]. To manage resource allocation (RA), 3GPP defines two control modes for SL operation. In Mode 1, the gNB handles RA directly. However, this centralized approach suffers from limited visibility into link conditions and the specific traffic demands of each V2X communication. In Mode 2, the UE autonomously allocates SL resources, offering flexibility, but it struggles to resolve resource conflicts effectively, resulting in increased interference and degraded communication performance [6]. The literature has addressed these issues by proposing models for path estimation, such as the one in [7], which aims to optimize resource utilization by enabling vehicles to list their neighbors and next hops in the Infrastructure-Based Hybrid Road-Aware Routing Protocol (IARAR). They also collect vehicle speed to estimate path duration and potential communication time over the link. However, despite the flexible infrastructure, the authors neither consider the current network state, such as resource availability or interference levels, nor propose any balancing scheme for link utilization. In contrast, [8] presents the Uplink Resource Shared Interference Mitigation Scheme (URSIMS), which consists of two heuristic algorithms: one to handle resource sharing among different zones and another to define signal power control. The algorithm uses distance to calculate path loss and, together with fading, determines the best Signal-to-Interference-plusNoise Ratio (SINR) between D2D pairs and a regular UE. Thus, this problem can be approached as a minimization issue. Nevertheless, the solution does not benefit from either the flexibility or the radio information measurements. Recently, the Open Radio Access Network (Open RAN), introduced by the O-RAN Alliance in 2018 with the aim of reducing vendor lock-in and fostering innovation within the RAN ecosystem, has also been adopted as a promising archi-

2

tecture to address key challenges in V2X communications [9]. This paradigm promotes multi-vendor interoperability and programmability in mobile networks through a modular architecture based on open interfaces. The architecture defines two categories of RAN Intelligent Controller (RIC): Non-Real-Time (Non-RT RIC) and Near-Real-Time (Near-RT RIC). The NonRT RIC operates with control times exceeding one second, is part of the Service Management and Orchestration (SMO) framework, and hosts microservices, referred to as rApps. The Non-Real-Time is responsible for policy generation and the lifecycle management of Artificial Intelligence/Machine Learning (AI/ML) models. The Near-RT RIC acts in control times between 10 and 1000 ms and manages the interfaces with the E2 nodes (e.g., 5G gNodeBs), enabling microservices, called xApps, to monitor and control functionalities, such as handover management [10], load balancing [11] and traffic steering [12], thereby facilitating AI-driven decision-making and dynamic policy enforcement. In [13], the proposal leverages the orchestration capabilities of O-RAN elements using data exchange in the Below 6 GHz (sub-6 GHz) band to facilitate the reliable selection of relay nodes operating in mmWave. To illustrate the advantages of the solution, an xApp was designed for relay selection, incorporating Line of Sight (LoS) recognition and distance measurement. The authors in [14] aim to address the scarcity of shared resources between the V2V and V2I problems, highlighting the risk of interference when both entities reuse the same resources. To tackle this, the proposed solution uses a multi-agent reinforcement learning (MARL) model aiming to select the most suitable resource block groups to minimize interference and increase transmission rates. Nevertheless, the adopted approach in [14] assigns one agent per link, which becomes unfeasible in an IoV network since the number of vehicles can grow indefinitely, leading to processing overload. While [15] proposes an Open RANbased solution for radio resource management, considering connected and autonomous vehicles (CAVs) as E2 endpoints. The performance evaluation compared Mode 2 with the proposed solution. However, the authors do not indicate what happens if resources are exhausted. Existing solutions in the literature typically do not jointly address interference mitigation and resource allocation optimization in IoV networks, especially on sidelink communications, nor do they ensure minimum QoS requirements. In addition, the coexistence between vehicles and VRUs, competing for limited resources, is often neglected. Given this, this work aims to jointly mitigate interference and optimize resource allocation under dynamic, resource-constrained conditions. In this context, communication mode selection is used to assign the most suitable mode to each vehicle pair, distributing users across V2V and V2N. Since resources are limited, it is crucial to prioritize their allocation to pairs where sidelink usage is more efficient, while avoiding resource starvation for nonprioritized VRUs and for vehicles operating in dense areas or under conditions of interference and resource scarcity. To address these challenges, it is important to recognize that traditional optimization approaches may struggle to capture the dynamic and large-scale nature of this problem,

especially in high-mobility, dense settings and heterogeneous traffic conditions. Moreover, single-agent learning strategies suffer from generalization limitations, while fully decentralized approaches with one agent per link become impractical as network density increases. Therefore, this work models the problem as a MARL task and introduces a clustering strategy in which each agent is responsible for a group of vehicles, improving scalability while preserving coordination among decisions. To enable this solution, this work also adopts the Open RAN architecture, which, unlike traditional controllers, explicitly defines the lifecycle management of AI/ML models and specifies different deployment scenarios that distribute training and inference processes between the Non-RT RIC and the Near-RT RIC. The proposed solution adopted deployment scenario 1.2 among those defined by O-RAN, as it allows structuring the learning process according to the Centralized Training with Decentralized Execution (CTDE) paradigm. In this scenario, the Non-RT RIC serves as the training host and manages continuous operations, which are critical and computationally intensive, however, since it operates at timescales greater than 1s, it avoids impacting control latencies. While the NearRT RIC is responsible for inference and operates with lower latency, it is critical for the management and control of IoV networks. Finally, another challenging factor is that the ORAN architecture requires pre-trained models for inference, therefore, the present proposal introduces a methodology for offline MARL training. Additionally, an essential aspect of the Open RAN architecture is the modularization of control and monitoring into separate xApps, enabling a modular solution that meets control latency requirements. In this way, the present proposal leverages this characteristic by functionally separating the monitoring, inference, and control components. This decomposition allows reduces the coupling between data collection, inference, and action execution. Furthermore, the architecture enables, through standardized signaling and interfaces, especially the E2 interface, the implementation of control loops, decision-making, and continuous monitoring. This monitoring provides a structured, standardized view of the network state by collecting and storing user measurements, which are directly used to construct the state space of the MARL model. These data also enable continuous model refinement through fine-tuning based on accumulated operational experience. In this context, reinforcement learning has shown promise in Open RAN environments by enabling continuous policy adaptation based on real operational data. This adaptability is crucial in vehicular networks, where high mobility increases network dynamism. In short, the main contributions of the paper are: • To address interference and limited sidelink availability, the problem is modeled as a Markov Decision Process (MDP). Therefore, to optimize communication mode selection and resource allocation, ensuring minimum QoS requirements, we use a MARL-based strategy. This approach enables dynamic assignment of V2N and V2V links according to the suitability of each communication

3

The allocation of Sidelink resources involves managing the Physical Sidelink Control Channel (PSCCH), Physical Sidelink Shared Channel (PSSCH), and Physical Sidelink Feedback Channel (PSFCH), which are responsible for resource scheduling, D2D data transmission, and ACK/NACK feedback signaling, respectively. These resources may be centrally managed by the gNodeB (Mode 1) or autonomously by the UE (Mode 2), with both modes (pre-)configured by the Radio Resource Control (RRC) layer. Sidelink uses a shared bandwidth for transmission and reception, structured into timefrequency resource pools [2] [6]. In the time domain, the Sidelink bitmap (pre-)configured by the RRC periodically defines the available slots. This bitmap is combined with the TDD pattern in TDD systems to determine usable slots [2]. For example, a subcarrier spacing of 60 kHz with a 10 MHz bandwidth and a TDD pattern of [DDDFUUUUUU] and a bitmap of [111111000111] define the sidelink slots, as shown in Figure 1. The figure shows that SL operates using uplink (UL) resources, but only the available ones. The system groups the resources into contiguous subchannels in the frequency domain, each containing N resource blocks (RBs). For NR V2X, N may assume 10, 15, 20, 25, 50, 75, or 100 RBs. According to [4], the last four RBs are reserved and unavailable for Sidelink operation.

Frequency 14

10

Subchannel

pair. Additionally, the proposal evaluates system performance in two scenarios: a vehicle-only network and one with coexisting VRUs competing for resources. Results show that the approach achieves a more efficient balance between uplink and downlink usage in both cases. Specifically, in the vehicle-only scenario, approximately 80% of communication pairs achieved SINR values around 28 dB, outperforming the single-agent strategy. • Since the Open RAN architecture enables lifecycle management of ML models, supports the CTDE strategy, and, given that the O-RAN specification does not permit untrained models in production, the methodology includes an initial offline training phase using a conservative approach, followed by conservative fine-tuning updates that are better suited to offline environments. • To avoid over-provisioning intelligent control loops in the Open RAN architecture, the proposal groups users by assigning one per agent. During training, each agent stores its experiences in a retraining buffer, enabling cooperative learning. The solution leverages the QMIX strategy, which enables agents to share their experiences through a Mixer network during training/fine-tuning. The evaluation reveals that increasing the number of agents results in higher computational resource consumption, with a 14.94% increase in memory and a 9.94% decrease in CPU usage when comparing 50 agents to 75 agents. In the vehicle-only scenario, adding more agents did not necessarily improve QoS metrics, as evidenced by the 8.6% increase in mean latency from 50 to 75 agents. However, in the more complex and heterogeneous scenario involving both vehicles and VRUs, the configuration with the highest number of agents achieved an 8% reduction in mean latency compared with 50 agents, highlighting a clear trade-off between QoS and computational resource usage. The remainder of this paper follows this structure. Section II introduces the overview concepts about SL. In Section III, we model the system and formulate the problem. Section IV details the proposed solution and its architectural components. Section V presents the evaluation scenarios and results. Finally, Section VII concludes the paper and outlines future research directions.

RB

1

Time

Slot TDD pattern DL DL DL F UL UL UL UL UL UL DL DL DL F UL UL UL UL UL UL SL bitmap 1 1 1 1 1 1 0 0 0 1 1 1 Slot/RB not available for SL

Slot/RB available for SL

Fig. 1. Frame structure and sidelink bitmap.

II. A N OVERVIEW OF SIDELINK COMMUNICATIONS In V2X systems, the V2N communication mode provides vehicular communication to the network services through the Uu interface, while the direct V2V transmissions utilize the PC5 interface. Introduced in 3GPP Rel-12 to support Proximity Services (ProSe) services, PC5 enabled early D2D communication and is now extensively employed in V2X scenarios. It is well-suited for high-mobility environments and offers stable performance even under high node density. Additionally, PC5 ensures low latency and high reliability, though it has a shorter transmission range than Uu. Unlike V2N, V2V enables out-of-coverage communication, which enhances road safety by allowing data exchange independent of gNodeB connectivity.

III. S YSTEM M ODEL AND P ROBLEM F ORMULATION A. System Model Consider a V2X scenario of V vehicles, represented by the set V = {1, 2, . . . , v, . . . , V }, moving along a highway supported by a base station (BS). Definition 1 – Correlation between available and used radio resources: The BS has a total of F Physical Resource Blocks (PRBs) available in time, where: F = FUL + FDL ,

with FDL > FUL .

(1)

The sidelink communication (PC5) consumes a portion of the uplink resources: BS FUL = FUL + FSL ,

(2)

4

BS where FUL corresponds to uplink resources used for conventional V2N communication, and FSL is the portion allocated for sidelink (V2V). Each V2V pair (i, j) ∈ PV2V requires fi,j PRBs to maintain stable communication using the PC5 interface. Therefore, the total number of PRBs consumed by all active V2V pairs is: X ΓV2V = fi,j , (3)

pairs pairs VRUs ΓV2N = ΓUL, + ΓDL, + ΓDL, , V2N V2N V2N

(i,j)∈PV2V

and all this demand is drawn from FSL ⊂ FUL . On the other hand, V2N pairs (i, j) ∈ PV2N consume resources from both uplink and downlink: X X UL DL ΓUL fi,j , ΓDL fi,j , (4) V2N = V2N = (i,j)∈PV2N

(i,j)∈PV2N

with total V2N demand: (5)

pairs ΓDL, V2N

and refer to uplink and downlink resources consumed by vehicle pairs using the Uu (V2N) interface (PV2N ). If the sidelink demand exceeds the available portion, i.e., ΓV2V > FSL , the network experiences overload in the sidelink domain, which may result in packet losses and increased latency. This study assumes this condition holds and investigates mechanisms for dynamic balancing between the PC5 (V2V) and Uu (V2N) interfaces. Definition 2 – V2V Interference: Each V2V pair (i, j) experiences a Signal to Interference plus Noise Ratio (SINR) given by: SINRi,j ≈

(10)

VRUs where ΓDL, corresponds to downlink resources conV2N sumed by VRUs. Moreover, VRUs increase the overall interference level in the downlink, as expressed in Equation 6, by contributing to the total interference term Itotal .

B. Problem Formulation

DL ΓV2N = ΓUL V2N + ΓV2N . pairs ΓUL, V2N

require significantly more downlink than uplink resources, such as video/music streaming or real-time alerts. Therefore, their uplink demand is considered negligible. As a result, vehicles experience resource competition not only with other V2V pairs but also with VRUs for the V2N communication resources. Specifically, the total consumption of PRBs in the V2N domain becomes:

P Wi,j hi,j , Itotal + N0

(6)

where P Wi,j is the transmission power from vehicle i to vehicle j, hi,j is the channel gain between them. The denominator includes the noise power N0 and the total interference from all other pairs (m, n): X Itotal = P Wm,n hm,n (7) (m,n)̸=(i,j)

As the number of vehicles V increases, more V2V pairs share the same limited PRBs, raising the interference level. Furthermore, using the Uu interface in V2X communications becomes more challenging when users move to the edge of the in-coverage area. In such cases, the total received power (P Wr ) is: P Wr = P Wt − P L,

(8)

where Pt is the transmitted power, and P L is the path loss. Under line-of-sight conditions [22], the path loss is given by: P L = 22log(d3D ) + 20log(fc ) + 28.0

(9)

Thus, signal degradation naturally occurs as the distance between the vehicle v and the gNodeB increases. Definition 3 – Coexistence with Vulnerable Road Users (VRUs): This work assumes that VRUs run applications that

We aim to minimize the resource consumption problem, particularly the uplink PRBs usage, while maximizing the SINR, ensuring the loss and latency requirements by optimizing the communication mode selection by assigning the most suitable modes for each pair. To this end, we formulate the problem as a Markov Decision Process (MDP) for MARL applied to balancing in V2V and V2N communication pairs. The MDP tuple (S, A, R, p, γ) represents the environment, where S is the state space, A is the action space, R is the rewards, p is the transition probabilities, and γ is the discount factor. Most existing solutions rely on single-agent strategies, where a centralized model performs predictions and makes decisions. However, the Open RAN architecture enables centralized control over the entire network. Due to the dynamic nature of IoVs, single-agent strategies often face generalization challenges. In these strategies, an agent selects an action At for a given state St and receives a reward Rt . If v vehicles share similar states, the agent tends to select the same action At for each, aiming to maximize its reward. This behavior can lead to overloading one link while underutilizing others. Multi-agent architectures provide a more suitable solution by distributing tasks among agents and supporting collaborative learning. This study addresses balancing challenges using MARL within the O-RAN framework in V2X scenarios. The problem involves dynamic network topologies and stringent latency and reliability requirements in vehicular networks. High vehicle mobility on highways further complicates the management of the PC5 interface. Additionally, the high density of vehicles in V2X scenarios exacerbates scalability challenges. This work adopts a MARL approach to meet the minimum QoS requirements for vehicle applications. Markovian principles provide a suitable structure for modeling MARL in dynamic vehicular networks [14], [16]. The proposed agents implement the QMix algorithm [17] with a conservative strategy [20]. Let P be the set of communication pairs (i, j) existing in the network, with |P | > 1. Solutions that use one agent per pair tend to grow indefinitely, such that N → |P | leads to scalability and exploration issues and complicates agent coordination and communication [18]. The clustering strategy

5

subdivides P into N subsets, where N is the number of agents in the network. Each agent k manages a group Gpk , that represents the subset of pairs assigned to agent k, formed using a round-robin strategy that balances the number of pairs among agents. Group assignments remain fixed while a pair stays active in the network. Nevertheless, the composition of each group naturally evolves over time as pairs form and dissolve. The proposed solution limits the number of agents to minimize this scalability problem. So, the study defines three scenarios with 25, 50, and 75 agents for evaluation. This study empirically selects these values to analyze how scalability affects coordination and system performance. This setup enables the observation of how different agent distribution scales affect collaborative learning efficiency, even without a predefined optimal criterion. Several factors influence load-balancing decisions. From the user perspective, key metrics include path loss (PL), channel quality indicators (CQI), reference signal received power (RSRP), signal-to-interference-plus-noise ratio (SINR), packet loss, and latency. From the base station perspective, resource availability plays a crucial role. The model optimizes user distribution across available resources while minimizing latency and packet loss. The system considers seven key decision variables defined in the previous subsection. According to Definition 1, the variable {FSL } represents resource availability for the sidelink. Definition 2 includes interference and user distance-related factors {SIN R, RSRP, CQI, loss, latency}. Definition 3 introduces downlink resource availability {FDL }. The MARL solution defines the state space, action space, and reward function as follows: (a) State Space (S): Each agent operates in a zone A for the group Gpk and observes only a partial view of the global environment state, associated with Gpk . Each group includes n links, and the agent state is defined individually per link. Formally, we define the state at time t as: t Spk,n = {FSL , Si,j , Ri,BS , Ci,j , Li , Di,j , FDL }

(11)

(b) Action Space (A): The action space is binary, since the proposed MARL model only determines the communication mode assigned to each link. At each decision step t, agent k, responsible for group Gpk , independently selects one action for every link n associated with a pair (i, j) ∈ Gpk . Each action determines whether the corresponding link operates through V2N or V2V, as defined by: ( 0, use infrastructure (V2N) t Ak,n = 1, use sidelink (V2V) (c) Reward (R): The reward computation is performed in two stages. First, after each state transition (St , At , St+1 ), agent t k computes an individual reward rD for each pair (i, j) ∈ i,j Gpk , based on the delay Di,j experienced by that pair. These individual rewards are stored and later aggregated during the offline centralized training phase to compute the global reward, X t Rt = rD , i,j (i,j)∈P

which is used by the QMIX mixing network to optimize the joint action-value function. t To compute rD , only the communication delay is coni,j sidered. Preliminary experiments showed a strong correlation between delay and packet loss. Therefore, to avoid optimizing partially redundant metrics, the reward formulation considers only delay, while indirectly improving packet delivery performance. The resulting reward function is defined as:

t rD = i,j

 t Di,j > 30 ms,

2 −0.1 · Di,j − 10 · Di,j + 300

Dt ≤ 30 ms, i,j

10 · tanh(0.45 · Di,j + 13.6)

The model rewards the agent positively when it selects appropriate actions, with the equations tuned to maintain reward values around 10. In contrast, the equations for negative rewards apply steeper penalties. When an agent selects an inappropriate action, the model imposes a more substantial penalty, emphasizing the severity of incorrect decisions. IV. P ROPOSED M ULTI -AGENT- BASED O-RAN F RAMEWORK FOR C ONNECTIVITY M ODE S ELECTION This section presents the proposed solution and its architectural components. As shown in Figure 2, the architecture includes the AI/ML model manager and the computational resource monitor in the Non-RT RIC. Meanwhile, in the NearRT RIC, the components are modularized into three services: (i) monitoring, through the KPM Monitor xApp; (ii) inference, via the QoS Predictor; and (iii) decision-making, through the Sidelink Manager. These xApps interact with the RAN through the standardized E2 interface, which enables control and monitoring loops based on the functionalities exposed by the E2 nodes. In this context, the network state is constructed from metrics reported by the E2 nodes (e.g., SINR, RSRP, CQI), as made available by the E2 interface. Initially, the KPM Monitor subscribes to the desired metrics from the E2 node. The measurements reported by the vehicles are aggregated by the gNodeB, forwarded to the xApp via indication messages, and then stored for use in the inference and decision-making process. A. AI/ML Model Manager In a multi-agent approach, there could be situations in which an individual agent cannot select an appropriate action. QMix [17] addresses this issue by leveraging a neural network architecture that models both the agents’ individual and global value functions. Each agent has its Q-network, which estimates the value of actions based on local observations. Then, the Mixer network aggregates the individual estimations, taking the global state of the environment as input and producing a joint action value. The Mixer network enforces monotonicity concerning individual Q-values, ensuring that the joint action maximizing the global value maximizes each local action of an agent. This design enables CTDE, making QMix particularly well-suited for collaborative environments such as Open RAN. The training of QMix involves minimizing the following loss function, which measures the mean squared error between the

6

SMO

Usage Monitor

AI/ML Model Management

Non-RT RIC CPU

Memory

AI/ML Training

AI/ML Continuos Operation

Pre-trained Models Agent 1

Agent 2

...

Agent N

Catalog

Near-RT RIC

A1 Database Cell Metrics UE Metrics

Sidelink Management

KPI Monitor

QoS Predictor

xApps

Messaging Infrastructure

Routing Manager

Subscription Manager

E2 Termination

E2 Manager

xApp Manager

E2 V2N (Uu interface)

Video Stream

Sidelink (V2V) (PC5 interface)

V2X

UE (Uu interface)

Fig. 2. Overview of the proposed solution.

estimated total Q-value and the target value computed from the immediate reward and the Q-value of the next state: B

Lqmix =

2 1 X tot y i − Qtot(τ i , ui , si ; θ) B i=1

(12)

This approach, based on the CTDE paradigm, enables centralized training of the joint value function while preserving decentralized execution among the agents. Furthermore, no model can be deployed in the Open RAN architecture without prior offline training [19]. Therefore, the training and model update phases also incorporated a conservative MARL approach. This strategy penalizes action values that fall outside the support of the offline dataset, helping avoid overestimation. The conservative loss function for the multi-agent scenario, N

Lcql =

α X (Ea∼πi [Qi (s, a)] − Ea∼Di [Qi (s, a)]) N i=1

(13)

Letting N denote the number of agents, Qi(s, a) represents the value function for agent i, πi is its current policy, and Di corresponds to the offline dataset used to estimate action values, and α is the regularization strength. The conservative penalty term is normalized by N , allowing the total loss to scale proportionally with the number of agents while maintaining optimization stability. So, the final loss function of the Conservative QMix (CQMix) is: Lcqmix = Lqmix + Lcql

(14)

During action selection, to balance the conservative training approach with exploration, an ε-greedy policy is adopted: h i 1 ′ π(a | s) = (1 − ε) · I a = arg max Q(s, a ) +ε· (15) a′ |A| Here, ε is the exploration rate, I[·] is the indicator function, and A denotes the set of available actions. In this policy, agents select the action with the highest Q-value with probability 1 − ε and choose a random action with probability ε, enabling controlled exploration. This configuration offers a safe yet effective policy: it remains predominantly conservative (e.g., 95% greedy actions), while the exploration component helps avoid local optima and compensates for inaccuracies in the Q-value estimates. Moreover, CQL acts as a safeguard against actions outside the dataset distribution. Hence, this approach ensures training safety and stability in offline settings without sacrificing limited exploratory capabilities. The initial model was offline-trained using data collected during simulations involving only V2N and V2V links, separately, resulting in an initial policy π0 . However, due to the limited exploration, the resulting model may suffer from poor generalization. Therefore, the model is updated every n executions via finetuning, enabling the policy πn to improve based on newly stored experiences in the dataset derived from prior policies πk−1 [21]. Finally, pre-trained models are stored in a catalog and are made available for offloading to initiate the inference phase. B. QoS Predictor The QoS Predictor (QP xApp) is an xApp responsible for performing ML model inference. To do this, it queries the model catalog to select the most suitable model for the requested prediction. In this work, we evaluate the number of

7

agents in operation at 25, 50, and 75. Thus, the QP xApp sends a request to the AI/ML Manager specifying the desired number of agents and the prediction type (e.g., peer communication mode). The AI/ML Manager checks for a compatible model and, if available, sends it to the QP xApp along with the execution prerequisites, including the required libraries and their versions. And the formatting and features of the model input state and output vectors. Appendix A presents an example of the model description received by the QP xApp. During the inference phase, agents operate independently, relying solely on observations of the pairs for which they are responsible, thereby eliminating the need for communication between agents and real-time synchronization. The agents are deployed in the near-RT RIC as part of the QP xApp and are continuously updated during network operation. To initiate the prediction process, the QP xApp waits for a request from the Sidelink Manager. Upon receiving the request, the service identifies the vehicle pairs to evaluate and retrieves the respective state vectors for each pair. The QP xApp then pre-processes the data to fit the ML model input format, executes the inference, and returns the recommended actions to the requester. The QP xApp triggers a model update request every 100 processed inference requests per agent. To reduce synchronization costs, the pairs in the network are evenly distributed among agents so that all agents reach the required number of inferences to trigger the update, approximately, at the same time. When all agents reach this threshold, the update request is triggered. This request initiates the fine-tuning procedure, in which the agents share their experiences. This procedure causes a synchronization cost linearly proportional to the number of agents, since the collected experiences (e.g. states, actions and rewards) must be aggregated to compute the joint actionvalue function. However, this step occurs outside the critical real-time cycle. As a result, the proposed approach ensures low-latency inference while maintaining coordinated learning through centralized training. Subsequently, the QP xApp performs periodic checks to verify the availability of the updated model. Once the model is ready for offloading, the QP xApp performs the replacement and restarts the inference cycle. C. Sidelink Management Mechanism The sidelink management mechanism dynamically redirects traffic between the Uu and PC5 interfaces. This mechanism aims to mitigate interference between vehicle pairs and VRUs while optimizing the utilization of available resources to meet the QoS requirements of vehicular applications. Algorithm 1 illustrates the logical operation of the Sidelink Manager. Initially, the manager queries the database and loads the set P = {(Vi,j , ωi )}. Vi,j represents the vehicle pairs composed of a source vehicle (vi ), a destination vehicle (vj ), and ωi is the current communication mode (e.g., V2V or V2N), as well as the user and network state denoted by S. The list W contains the pairs currently awaiting the application of a control action. The manager removes from P the

Algorithm 1 Communication Mode Selection Require: P = {(Vi,j , ωi )}, S, W 1: for all (Vi,j , ωi ) ∈ P do 2: if vi ∈ W then 3: if W[vi ] = ωi then 4: Remove vi from W 5: else 6: Remove corresponding elements from P and S 7: end if 8: end if 9: end for 10: if S ̸= ∅ then 11: Send S to QP xApp and receive predicted modes A 12: for all (Vi,j , ωi ), ai ∈ P, A do 13: if ai ̸= ωi then 14: Query resource availability 15: RUL , RSL ← GetResources(Vi,j ) 16: if (ai = DM ∧ RSL > θ) or (ai = IM ∧ RUL > θ) then 17: Apply control: ApplySwitch(Vi,j , ai ) 18: Add vi → ai to W 19: end if 20: end if 21: end for 22: end if pairs whose vi are already in W and whose previously recommended actions have not yet been executed, keeping only those eligible for a new decision. If there are remaining valid states S, the manager sends to the QP xApp, which returns a list of actions A containing the recommended communication modes for each pair (Vi,j , ωi ). For each recommendation ai ∈ A that differs from the current mode ωi , the manager verifies the UL and SL resource availability for the pair Vi,j , obtaining RUL and RSL . The communication mode switch is applied only if the available resources meet a minimum threshold θ. If this condition holds, the manager executes the action by sending an RIC Control Action to the E2 node, which performs the communication switching via ApplySwitch(Vi,j , ai ). Moreover, it then adds the user to the waiting list W with the new mode ai . This approach ensures that only feasible decisions are executed, avoiding resource overload and redundant commands for users who have not completed previous transitions. V. P ROPOSAL E VALUATION This section evaluates the proposed solution by demonstrating how the strategy mitigates interference, optimizes resource utilization, and minimizes latency and packet loss. For benchmarking, Single-Agent Conservative Q-Learning and two heuristic strategies are adopted as baselines. The first heuristic baseline is an SINR-based strategy deployed on the xApp, which selects the mode (V2V or V2N) according to a predefined SINR threshold (e.g. 20 dB) and switches to the link with the higher instantaneous SINR, as higher SINR enables more efficient resource usage, as shown in [23]. The second baseline is the Best CQI strategy deployed by the

8

A. Evaluation Environment We conducted the evaluation using OMNeT++ [25] with the Simu5G library [26]. The O-RAN support used NearRT provided by O-RAN Software Community (OSC), more specifically Release I with the OMNeT++ integration proposed in [10].

Fig. 3. Simulation mobility scenario.

The simulation scenario is a highway in Germany, as depicted in Figure 3. Simulation of Urban MObility (SUMO) [27] was used to generate car mobility, simulating 200 vehicles. The network scenario includes a single base station deployed along a 1.5 km segment of the Autobahn highway. TABLE I S IMULATION PARAMETERS Network Configuration Central Frequency 6GHz Number of Cars 300 Number of UEs 10 OFDM Numerology µ 0 Car/UE Transmit power 23dBm Resource Blocks 50 Application Configuration Sampling time 2.5 ms (Constant) Protocol User Datagram Protocol (UDP) Packet size 1024B Transmission Time 20s

The scenario implements a see-through application, where each vehicle shares visual information about road conditions with its immediate neighbors. We divided the vehicles into three categories: 150 regular cars (with a maximum speed of 40 m/s), 25 trailers, and 25 coaches (with a maximum speed of 30 m/s for both). The simulation only considers one direction of the highway. Vehicles transmit 720p and 30fps video streams at approximately 3000 kbps using a constant bit rate (CBR) [28] application to ensure consistent data flow for the see-through use case while conserving base station resources. In a separate scenario to evaluate pedestrian impact, we introduced 10 static pedestrians, each running a CBR application that simulates music streaming using 80 kbps. B. Evaluation Methodology We structured the evaluation into two phases. The first phase investigates MARL’s impact in a pure vehicular IoV scenario without pedestrian presence. The second phase analyzes the

coexistence of vehicles and pedestrians, both competing for shared network resources. We collected the following metrics for both phases: • RB Usage Distribution and Interference Mitigation: Measures how agents balance communication between V2N and V2V links and mitigates the SINR. • Latency and Packet Loss: Captures QoS metrics and quantifies the influence of MARL strategies on performance. • Resource Consumption: Quantifies the computational load associated with model prediction and fine-tuning. C. Use Case 1: Vehicle-Only Scenario In this scenario, the goal is to analyze the isolated impact of vehicles with all network resources available for unrestricted use. 1) Model Evaluation: The evaluation begins by analyzing the agent training process using loss curves across multiple versions. Figure 4 illustrates this progression and demonstrates that each model produces four versions during network operation. As shown in the figure, the model with 25 agents exhibits the lowest and most stable loss across all versions. This behavior suggests that the policy converged to a more stable region during training, likely because coordinating fewer agents is simpler. Furthermore, a downward trend in loss is observed across versions, indicating a progressive improvement in the policy. Meanwhile, in configurations with 50 and 75 agents, the initial loss exceeds that of the 25-agent model, which is expected given the increased complexity of the joint action space. Across versions, both configurations display alternating periods of instability and convergence, with the 75-agent model exhibiting the most significant fluctuations. However, loss alone does not definitively indicate policy quality. The effectiveness of each model is further assessed during inference, as discussed in the following section. (a) Version 1 Total Loss (log10)

gNodeB. This strategy selects the mode based on the Adaptive Modulation and Coding (AMC)-estimated bits per resource block, choosing the option with the highest transmission efficiency [24].

(b) Version 2

7.2

6.8

6.9

6.4

6.6

6.0

(c) Version 3 7.2

7.2

6.6

6.9

6.0

6.6 0

300

600

900

Training Step

0

300

600

900

Training Step 25 Agents

50 Agents

(d) Version 4

0

300

600

900

Training Step

0

300

600

900

Training Step

75 Agents

Fig. 4. Training loss over versions of the models.

2) RB Usage Distribution and Interference Mitigation: In the uplink, the single-agent strategy demonstrated the highest resource usage, exhibiting significant instability as evidenced by the wide interquartile range shown in the boxplot of Figure 7. This reflects the model’s inefficiency in balancing resource allocation. In contrast, the strategy showed considerably lower resource usage for the downlink, as evidenced by the narrow boxplot, suggesting underutilization of infrastructure resources. On the other hand, multi-agent approaches exhibited more balanced behavior, reducing uplink RB consumption while

9

Uplink 35

8

PRB UL Usage (%)

PRB DL Usage (%)

10

6 4 2

30 25

1.0 0.8 0.6 0.4 0.2 0.0

20

1.0

Cumulative Probability

Downlink

best performance in terms of both packet loss rate and average latency. The corresponding CDF curves are more left-skewed, indicating a higher probability of achieving lower packet loss and latency values. The 75-agent strategy follows closely, confirming that distributing the decision-making process among agents improves the estimation of Q-values and leads to better action selection for At . Cumulative Probability

increasing downlink usage. This indicates a better distribution between V2V (PC5) and V2N (Uu) communications. The reduction in uplink usage suggests that even when using infrastructure resources, the PC5 interface remains constrained by predefined bitmaps, effectively regulating spectrum usage. In contrast, the heuristic baselines preferred V2N usage, leading to higher downlink resource consumption and reduced uplink activity. Such a bias may increase the load on the cellular network and decrease the efficiency of direct V2V link utilization.

0

25

50

Loss Rate (Number of Packets)

10

Single agent 25 agents

0.6 0.4 0.2 0.0

75 100 125 150 175 200

15

0.8

50 agents 75 agents

50

100

150

200

250

Mean Latency (ms)

300

350

SINR-based Best CQI

5

Single

I t Agen25 Agents50 Agents75 Agents INR-basedBest CQ S

Single

I t Agen25 Agents50 Agents75 Agents INR-basedBest CQ S

Fig. 5. RB usage distribution for uplink and downlink.

The importance of balanced communication between V2V and V2N is further highlighted in Figure 6, which shows improvements in SINR values as the number of agents increases. The single-agent strategy yields the worst SINR results, with only about 30% of cases achieving an SINR of ≥ 25 dB and a maximum probability of approximately 27 dB. In contrast, strategies with more agents, particularly those with 75 agents, show that approximately 80% of the cases reach a SINR of ≥ 25 dB. This demonstrates effective interference mitigation enabled by improved coordination and channel access decisions. Among the heuristic baselines, the SINR-based strategy achieves the highest SINR values, with approximately 80% of samples at 31.5 dB, as expected, since this method explicitly selects the link with the highest instantaneous SINR. In contrast, the Best CQI strategy demonstrates behavior more similar to the proposed approach with 75 agents, with about 80% of samples around 30 dB, reflecting its focus on transmission efficiency rather than solely maximizing SINR, which aligns with its greater reliance on V2N communication.

Cumulative Probability

1.0 0.8 0.6

Single agent 25 agents 50 agents 75 agents SINR-based Best CQI

0.4 0.2

22

24

26

28

30

SINR (dB)

32

34

36

38

Fig. 6. Cumulative distribution function (CDF) of SINR.

Among all strategies, the models with 75 and 50 agents stood out for their effectiveness in resource allocation and interference management. 3) Latency and Packet Loss: Regarding QoS metrics, Figure 7 indicates that the strategy with 50 agents achieves the

Fig. 7. CDF of packet loss and latency.

Moreover, the strategies with fewer agents present higher variability, as seen by the less steep CDF curves, suggesting generalization issues and reduced robustness in decisionmaking. The heuristic baselines exhibit lower packet loss, particularly for the SINR-based strategy. However, this comes at the cost of significantly higher latency, as the increased reliance on V2N communication introduces additional transmission delays due to the round-trip to the infrastructure. When comparing the overall performance of the SINRbased strategy with the 50-agent MARL approach, at the 80th percentile of the cumulative distributions, the SINR-based heuristic improves the SINR by approximately 10.34%, but at the cost of a latency around 89.65% higher than that of the 50-agent strategy. These results indicate that maximizing instantaneous SINR does not necessarily lead to better QoS. Instead, the proposed MARL strategy prioritizes lower communication latency by accepting a reduction in SINR, resulting in a better overall balance between radio link quality and application-level performance. These results reinforce the limitations of single-agent or low-agent-count strategies in complex communication scenarios. 4) Computational Cost Analysis: We compared resource usage during model prediction and fine-tuning to assess the computational costs associated with the decision-making process. The QP xApp is responsible for prediction during runtime, while the ML Management service handles model fine-tuning. As shown in Table II, CPU usage during prediction remains relatively stable regardless of the number of agents, indicating that the inference step is not significantly affected by agent count. However, memory consumption increases with the addition of more agents, reflecting the additional overhead required to manage multiple policies or decision branches. In contrast, the fine-tuning phase exhibits a substantial increase in both memory and CPU usage. Still, similar to the prediction step, CPU usage remains relatively comparable across different agent configurations, with memory being the most impacted resource as the number of agents grows. This suggests that while parallel or distributed learning improves

10

1.0

Phase

Agents

Memory (MB)

CPU (%)

0.8

Prediction (QP xApp)

25 Agents 50 Agents 75 Agents

493.32 691.82 795.29

0.3710 0.3977 0.3582

0.6

Fine-tuning (ML Mgmt)

25 Agents 50 Agents 75 Agents

858.85 1158.19 1456.01

43.83 45.69 44.23

Score

TABLE II M EAN R ESOURCE U SAGE D URING M ODEL P REDICTION AND U PDATE

0.4 0.2 0.0

performance, it imposes a greater memory burden on the system. Moreover, during fine-tuning, memory consumption increased by a comparable amount between 25 and 50 agents (about 299 MB) and between 50 and 75 agents (roughly 298 MB), consistent with the linear synchronization cost discussed in Section IV-A. During prediction, memory consumption also increases with the number of agents, but the increment shrinks from nearly 199 MB (25 to 50 agents) to 103 MB (50 to 75 agents), indicating a decreasing growth rate. CPU usage, in turn, remains stable across both phases. This suggests memory usage would continue to scale as the number of agents increases (e.g., 100, 200, and so on). To evaluate the trade-off between QoS improvements and computational resource costs and to ensure fair comparisons among metrics with different units and scales, such as packet loss, delay, CPU usage, and memory consumption, the analysis normalizes all values to the range [0, 1] using min-max normalization: X − Xmin Xmax − Xmin

Single agent 25 agents 50 agents 75 agents Number of Agents

Fig. 8. Trade-off between QoS efficiency and computational cost on IoV only scenario.

When comparing the configurations that delivered the best results in each indicator, 50 agents for QoS efficiency and a single agent for computational cost, the single-agent setup consumed approximately 97.7% fewer resources. This finding highlights the significant computational savings of the singleagent approach despite its notable loss in QoS efficiency. Regarding the cost of resource consumption, the 75-agent configuration incurred the highest overhead. In contrast, the configurations with 50 agents, 25 agents, and a single agent consumed approximately 10.3%, 38.8%, and 97.9% fewer resources, respectively. These results show that increasing the number of agents improves QoS efficiency up to a certain point but also leads to a substantial rise in resource usage. D. Use Case 2: Vehicular-Pedestrian Coexistence in IoV

(16)

After normalizing the values (Xnorm ), the analysis computes two key indicators: QoS efficiency and computational cost. For QoS efficiency, the process uses the normalized values of packet loss (Xnorm loss ) and latency (Xnorm latency ). Since lower values indicate better performance in these metrics, the analysis inverts the scale after normalization to maintain consistency. It calculates the mean of the normalized values and applies the transformation 1 − mean (Xnorm ), which keeps the result within the [0, 1] range and ensures that higher values represent better QoS performance. To assess computational cost, the analysis calculates the average of the normalized CPU usage (Xnorm cpu ) and memory usage (Xnorm memory ). In this case, higher values correspond to higher resource consumption and, therefore, greater computational cost. Figure 8 presents the results for the analyzed indicators. Regarding the QoS efficiency indicator, the configuration with 50 agents achieved the best performance, reaching a score of approximately 0.726. In comparison, the single-agent and 25agent configurations exhibited reductions of 56.6% and 39.8%, respectively. Increasing the number of agents to 75 reduced efficiency by around 14% compared to the peak value achieved with 50 agents, suggesting potential overhead or decreased coordination among agents.

In this scenario, we assess the impact of coexistence between pedestrians and vehicles in IoV environments, where both compete for network resources. 1) Model Evaluation: Figure 9 shows the training loss for Vehicular-Pedestrian Coexistence on the same infrastructure scenario. Compared to the scenario with only IoV, the introduction of pedestrians adds a new level of complexity to the coordination task, reflected in the loss dynamics across all agent configurations and versions. (a) Version 1

(b) Version 2

(c) Version 3

(d) Version 4

7.6

Total Loss (log10)

Xnorm =

QoS Efficiency Resource Consumption Cost

7.2

7.2

7.0

7.2 6.9

6.8

6.8

6.5

6.6

6.0

6.4

6.4 0

300

600

900

Training Step

0

300

600

900

Training Step 25 Agents

0

50 Agents

300

600

900

Training Step

0

300

600

900

Training Step

75 Agents

Fig. 9. Training loss over models versions.

Unlike the previous scenario, all three configurations exhibit greater variability in losses across versions. In the first model version, the 75-agent model stands out, showing lower losses than the 50-agent model, but with high instability. This suggests that the presence of pedestrians alters the previously observed initial convergence behavior, leading the policy to

11

70

90

65

85

60 55 50 45

PRB UL Usage (%)

PRB DL Usage (%)

80

Cumulative Probability

0.8 0.6

Single agent 25 agents 50 agents 75 agents SINR-based Best CQI

0.4 0.2

26

28

30

SINR (dB)

32

34

36

Fig. 11. CDF of SINR in vehicular-pedestrian coexistence. 1.0

Cumulative Probability

1.0 0.8 0.6 0.4 0.2 0.0

0

25

50

0.8 0.6 0.4 0.2 0.0

75 100 125 150 175 200

Loss Rate (Number of Packets) Single agent 25 agents

50 agents 75 agents

50

100

150

200

250

Mean Latency (ms)

300

350

SINR-based Best CQI

Fig. 12. CDF of packet loss and latency in vehicular-pedestrian coexistence.

75 70 65 60

40

55

35

50

I t agenagentsagentsagents -baseedst CQ Single 25 50 75 SINR B

1.0

Cumulative Probability

explore a wider region of the action-state space before stabilizing. In versions 2 and 3, the models show signs of stabilization, with the 25-agent configuration again showing the most consistent convergence. The 50 and 75-agent models continue to exhibit oscillatory behavior in later versions, suggesting greater difficulty in converging in an environment with more users competing for resources. As in the previous scenario, these loss curves characterize training dynamics but do not directly quantify policy quality. 2) RB Usage Distribution and Interference Mitigation: Figure 10 illustrates the RB usage distribution in the presence of both vehicles and pedestrians. Compared to the previous vehicular-only scenario, the overall resource consumption increased, particularly in the downlink, which is consistent with the additional communication demands introduced by pedestrian entities. However, the multi-agent strategies continue to show improved stability in resource allocation, with narrower boxplots and more balanced uplink and downlink utilization.

I t agenagentsagentsagents -baseedst CQ Single 25 50 75 SINR B

Fig. 10. RB usage distribution in vehicular-pedestrian coexistence.

Figure 11 presents the SINR distribution for this scenario. The single-agent configuration remains the least effective, with most samples having a level below 31 dB between the RL models. On the other hand, the 75-agent strategy significantly outperforms the others, with a large portion of samples above 32 dB, indicating successful interference mitigation through distributed coordination. The curves suggest that increasing the number of agents helps better manage interference in highdensity environments with heterogeneous actors. Nevertheless, the heuristic baselines exhibit the lowest SINR performance in this scenario. This behavior is mainly due to increased infrastructure load from pedestrian users, which intensifies interference levels. As a result, simple mode-selection strategies based solely on a single metric (e.g., SINR or CQI) are insufficient to properly balance resource usage and interference, highlighting the importance of more coordinated, adaptive approaches. 3) Latency and Packet Loss: The QoS analysis for the coexistence scenario is shown in Figure 12. Similar to the SINR results, the 50 and 75-agent configurations provide the best performance, with their CDF curves for both packet loss rate and mean latency shifted further to the left. This indicates a higher probability of achieving lower latency and fewer lost packets. On the other hand, SINR-based and Best CQI present the worst overall QoS performance. As shown in the right plot,

neither curve reaches a cumulative probability above 0.45 within the evaluated latency range, indicating that a significant portion of transmissions experience excessively high mean latency, exceeding 200 ms in most cases. Similarly, in the loss rate plot, both baselines accumulate probability mass at lower values early on but plateau well below 1.0, suggesting a long tail of high-loss events. This result reinforces the conclusion that increasing the number of agents improves communication efficiency and overall reliability, especially in more complex and competitive environments such as those involving vehicles and pedestrians. 4) Computational Cost Analysis: Table III summarizes the average memory and CPU consumption during the two main operational phases: model prediction and fine-tuning. TABLE III M EAN R ESOURCE U SAGE D URING M ODEL P REDICTION AND U PDATE Phase

Agents

Memory (MB)

CPU (%)

Prediction (QP xApp)

25 Agents 50 Agents 75 Agents

470.86 685.15 734.46

0.90 0.39 0.48

Fine-tuning (ML Mgmt)

25 Agents 50 Agents 75 Agents

884.14 1211.24 1577.82

48.85 55.10 50.67

During prediction, memory consumption increases with the number of agents, reaching a maximum of 734MB for 75 agents. CPU usage remains relatively low across all configurations, with the 25-agent setup consuming the most CPU, approximately 0.90%. This indicates that increasing the number of agents imposes a greater memory burden during runtime inference while maintaining modest CPU demands. In contrast, during fine-tuning, memory and CPU usage increase significantly with the number of agents. The 50-agent

12

configuration consumed the highest average CPU at 55.10%, followed by the 75-agent configuration at 50.67% and the 25agent configuration at 48.85%. Regarding memory usage, it also increased proportionally, from 884MB in 25 agents to 1578MB with 75 agents, reflecting the cost of maintaining multiple parallel learning processes. These findings highlight that the fine-tuning phase is considerably more resource-intensive than prediction, particularly in terms of CPU usage. While multi-agent learning enhances decision-making performance, it incurs a higher computational demand during the training phase. Furthermore, memory consumption increased by approximately 327 MB between 25 and 50 agents and 367 MB between 50 and 75 agents, reflecting a slightly higher growth rate than that observed in the vehicleonly scenario. During prediction, memory consumption also increases with the number of agents, but the increment shrinks from nearly 214 MB to around 49 MB, indicating a decreasing growth rate. CPU usage, in turn, does not follow a monotonic trend in either phase, unlike the vehicle-only scenario. This suggests memory usage would continue to scale as the number of agents increases (e.g., 100, 200, and so on). Finally, Figure 13 presents the obtained results of the trade-off between QoS efficiency and computational cost in a vehicular-pedestrian coexistence scenario. Regarding the QoS efficiency indicator, the configuration with 75 agents achieved the best performance, reaching a score of approximately 0.715. In comparison, the 50-agent, 25-agent, and single-agent configurations exhibited reductions of 22.94%, 26.26%, and 49.82%, respectively. In this use case, the results show that increasing the number of agents improves QoS efficiency, as this scenario is more complex due to heterogeneity than use case 1, which is more homogeneous. 1.0

QoS Efficiency Resource Consumption Cost

Score

0.8 0.6 0.4 0.2 0.0

Single agent 25 agents 50 agents 75 agents Number of Agents

Fig. 13. Trade-off between QoS efficiency and computational cost in vehicular-pedestrian coexistence.

Concerning the cost of resource consumption, the 75-agent configuration also incurred the highest overhead. In comparison, the configurations with 50 agents, 25 agents, and a single agent consumed approximately 17.76%, 36.56%, and 89.52% fewer resources than the 75-agent configuration, respectively. These results reinforce that increasing the number of agents improves QoS efficiency but also leads to a substantial rise in resource usage. Therefore, achieving an optimal trade-off between performance and cost requires careful tuning.

VI. O PEN C HALLENGES AND F UTURE R ESEARCH D IRECTIONS This section discusses open challenges and future research directions that build upon the proposed framework. While the presented approach provides a basis for intelligent V2X mode selection within the O-RAN paradigm, several aspects related to real-world deployment, dynamic network conditions, and large-scale operation remain to be further investigated. Addressing these aspects represents a natural continuation of this work and may enhance its applicability in other scenarios. A. Real-World Experimental Validation The present proposal leverages real-world containerized intelligent controllers from the OSC. This approach facilitates integration into experimental environments compatible with the O-RAN architecture. Although the proposed solution was validated through simulation-based experiments that demonstrated its effectiveness, real-world validation poses significant challenges. In this context, testbeds based on software-defined radios (SDRs) and open-source platforms provide promising alternatives. Nevertheless, existing open-source RAN platforms offer limited sidelink support, and the availability of compatible commercial devices is scarce, which restricts largescale experimentation. Additionally, experiments in real IoV settings are constrained by safety considerations and deployment complexity. These factors underscore the necessity for hybrid validation methodologies that integrate simulation with real-world experimentation. B. Channel Aging in Control Loops In high-mobility scenarios, rapid channel variations and frequent handovers can introduce significant channel aging. Consequently, the state observed by the Near-RT RIC may be outdated by the time a control action is applied, resulting in suboptimal decision-making. Since these control loops operate on timescales ranging from 10 ms to 1 s, this latency can result in a stale-state problem. Understanding these time constraints in high-mobility environments remains an open challenge. Future research should investigate these degradation scenarios and explore mitigation strategies, such as predictive state estimation and aging-aware reward shaping, to enhance the proposed solution. C. Out-of-Coverage Operation The proposed framework considered continuous network coverage for UE measurement reporting and RIC control actions. However, out-of-coverage and intermittent connectivity scenarios are critical for V2X safety applications and remain unresolved challenges. These scenarios may benefit from hybrid approaches, in which the RIC optimizes longterm policy while UEs retain autonomous decision-making for immediate safety in the absence of network connectivity. Future research should examine coordination between these layers, emphasizing policy consistency and system stability during transitions between centralized and distributed control modes.

13

D. Multi-Cell Coordination and Interference Management Our study focuses on a single-cell environment. Future work consists of extending the framework to multi-cell deployments, which introduces significant challenges in coordination and interference management. Sidelink transmissions may affect users in neighboring cells, especially with aggressive resource reuse. While recent 3GPP releases improved sidelink communication, efficient inter-cell coordination and groupbased resource management remain active research areas. This highlights the need to consider multi-cell O-RAN deployments and their interactions with handover mechanisms under dynamic, heterogeneous network conditions. Furthermore, future evaluations should include a larger number of agents to assess the performance of the proposed solution in more complex, large-scale scenarios. VII. C ONCLUSION This work presented a MARL-based solution for determining the communication mode between PC5 interfaces, using either Sidelink (V2V) or network-based communication (V2N) via the Uu interface. The objective of the proposed solution was to select the optimal communication mode to mitigate interference among peers and improve the utilization of available radio resources. We tested two scenarios: the first involving only vehicles utilizing all available resources, and the second considering coexistence with VRUs. The results indicate that MARL-based solutions effectively mitigate interference in both cases, optimize resource usage, and meet the QoS metric requirements. ACKNOWLEDGMENT This work was supported by the National Council for Scientific and Technological Development (CNPq) - Research Productivity Fellowship (Grant No. 313083/2023-1) and Pernambuco Research Foundation (FACEPE) (Grant No. IBPG0130-1.03/23). A PPENDIX M ODEL D ESCRIPTOR { "name": "cqmix", "version": "1.0", "runtime": { "name": "python", "encoding": "protobuf", "version": "3.10.12", "dependencies": { "Pip": { "requirements": [ { "name": "torch", "version": "2.5.1" }, { "name": "pandas", "version": "2.2.2" }, { "name": "numpy", "version": "1.26.4" } ] }

} }, "methods": [ { "choose_com_mode": { "input": "NetworkState", "output": "CommunicationModeType", "description": "Receives network and user status and sets communication mode" } } ] }

R EFERENCES [1] M. Harounabadi, D. M. Soleymani, S. Bhadauria, M. Leyh and E. RothMandutz, ”V2X in 3GPP Standardization: NR Sidelink in Release-16 and Beyond,” in IEEE Communications Standards Magazine, vol. 5, no. 1, pp. 12-21, March 2021, doi: 10.1109/MCOMSTD.001.2000070. [2] 3rd Generation Partnership Project (3GPP), Study on NR Vehicle-toEverything (V2X), 3GPP TR 38.885, versão 16.0.0, March 2019. [3] V. Todisco, S. Bartoletti, C. Campolo, A. Molinaro, A. O. Berthet and A. Bazzi, ”Performance Analysis of Sidelink 5G-V2X Mode 2 Through an Open-Source Simulator,” in IEEE Access, vol. 9, pp. 145648-145661, 2021, doi: 10.1109/ACCESS.2021.3121151. [4] 3GPP, ”NR; Radio Resource Control (RRC); Protocol specification,” 3rd Generation Partnership Project (3GPP), TS 38.331, versão 17.3.0. [5] Shengfeng Liu, Feipeng Dai, and Haixia Cui. 2024. Joint Resource Allocation and Interference Management for Vehicular Networks. In Proceedings of the 2023 International Conference on Power, Communication, Computing and Networking Technologies (PCCNT ’23). Association for Computing Machinery, New York, NY, USA, Article 7, 1–5. https://doi.org/10.1145/3630138.3630419 [6] 3GPP, ”NR; NR and NG-RAN Overall Description; Stage 2,” 3rd Generation Partnership Project (3GPP), TS 38.300, versão 18.3.0, outubro de 2024. [7] M. T. Abbas, A. Muhammad, e W.-C. Song, “Road-Aware Estimation Model for Path Duration in Internet of Vehicles (IoV)”, Wireless Pers Commun, vol. 109, no 2, p. 715–738, nov. 2019, doi: 10.1007/s11277019-06587-5. [8] K. Ashfaq e G. A. Safdar, “Uplink resource shared interference mitigation scheme for in-band D2D underlay 5G networks”, Physical Communication, vol. 64, p. 102369, jun. 2024, doi: 10.1016/j.phycom.2024.102369. [9] O-RAN Alliance, Use Cases Analysis Report, O-RAN.WG1. Use-CasesAnalysis-Report-v07.00, Technical Report, versão 07.00, 2022. [10] M. Santana and K. L. Dias, ”Open RAN-Enabled Deep LearningAssisted Mobility Management for Connected Vehicles,” 2025 International Conference on Information Networking (ICOIN), Chiang Mai, Thailand, 2025, pp. 378-383, doi: 10.1109/ICOIN63865.2025.10992821. [11] H. Zafar, M. Kasparick, S. Maghsudi and S. Stańczak, ”A Deep Reinforcement Learning Approach for Load Balancing in Open Radio Access Networks,” GLOBECOM 2023 - 2023 IEEE Global Communications Conference, Kuala Lumpur, Malaysia, 2023, pp. 3723-3728, doi: 10.1109/GLOBECOM54140.2023.10437848. [12] P. Sroka, Ł. Kułacz, S. Janji, M. Dryjański and A. Kliks, ”Policy-Based Traffic Steering and Load Balancing in O-RAN-Based Vehicle-to-Network Communications,” in IEEE Transactions on Vehicular Technology, vol. 73, no. 7, pp. 9356-9369, July 2024, doi: 10.1109/TVT.2024.3399924. [13] F. Linsalata, E. Moro, F. Gjeci, M. Magarini, U. Spagnolini, e A. Capone, “Addressing Control Challenges in Vehicular Networks Through O-RAN: A Novel Architecture and Simulation Framework”, IEEE Trans. Veh. Technol., vol. 73, no 7, p. 9344–9355, jul. 2024, doi: 10.1109/TVT.2024.3355202. [14] N. Hammami e K. K. Nguyen, “Multi-Agent Actor-Critic for Cooperative Resource Allocation in Vehicular Networks”, em 2022 14th IFIP Wireless and Mobile Networking Conference (WMNC), Sousse, Tunisia: IEEE, out. 2022, p. 93–100. doi: 10.23919/WMNC56391.2022.9954301. [15] F. Gjeci, E. Moro, F. Linsalata, I. Filippini, e A. Capone, “Towards Smarter Vehicular Communications: Leveraging Open RAN for Enhanced Vehicle-to-Vehicle Resources Management”, em 2024 IEEE 100th Vehicular Technology Conference (VTC2024-Fall), Washington, DC, USA: IEEE, out. 2024, p. 1–7. doi: 10.1109/VTC2024Fall63153.2024.10758024.

14

[16] M. Polese, L. Bonati, S. D’Oro, S. Basagni and T. Melodia, ”ColO-RAN: Developing Machine Learning-Based xApps for Open RAN Closed-Loop Control on Programmable Experimental Platforms,” in IEEE Transactions on Mobile Computing, vol. 22, no. 10, pp. 5787-5800, 1 Oct. 2023, doi: 10.1109/TMC.2022.3188013. [17] T. Rashid, M. Samvelyan, C. Schroeder, G. Farquhar, J. Foerster and S. Whiteson, ”Monotonic Value Function Factorisation for Deep MultiAgent Reinforcement Learning”, Journal of Machine Learning Research, 2020, vol. 21, no. 178, pp. 1-51. [18] D. Liu, F. Ren, J. Yan, G. Su, W. Gu and S. Kato, ”Scaling Up Multi-Agent Reinforcement Learning: An Extensive Survey on Scalability Issues,” in IEEE Access, vol. 12, pp. 94610-94631, 2024, doi: 10.1109/ACCESS.2024.3410318. [19] O-RAN Working Group 2, “O-RAN AI/ML workflow description and requirements 1.03,” O-RAN, Alfter, Germany, document ORAN.WG2.AIML-v01.03 Technical artigo do multi-agent Specification, Jul. 2021. [20] A. Kumar, A. Zhou, G. Tucker, and S. Levine, ”Conservative Q-learning for offline reinforcement learning”, In Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS ’20), 2020, Curran Associates Inc., Red Hook, NY, USA, Article 100, 1179–1191. [21] H. Zhong, X. Wang, Z. Li and L. Huang, ”Offline-to-Online Multi-Agent Reinforcement Learning with Offline Value Function Memory and Sequential Exploration”, 2024, arXiv, url: https://arxiv.org/abs/2410.19450. [22] 3GPP TR 138 901, 5G; Study on channel model for frequencies from 0.5 to 100 GHz (Release 16),v16.1.0 Nov. 2020. [23] V. Todisco, S. Bartoletti, C. Campolo, A. Molinaro, A. O. Berthet and A. Bazzi, ”Performance Analysis of Sidelink 5G-V2X Mode 2 Through an Open-Source Simulator,” in IEEE Access, vol. 9, pp. 145648-145661, 2021, doi: 10.1109/ACCESS.2021.3121151. [24] G. Nardini, A. Virdis, G. Stea, “Simulating device-to-device communications in OMNeT++ with SimuLTE: scenarios and configurations”, OMNeT++ Community Summit 2016, Brno, CZ, September 15-16, 2016. [25] OMNeT++,”Discrete Event Simulator.” https://omnetpp.org/ (accessed Oct. 25, 2024). [26] G. Nardini, D. Sabella, G. Stea, P. Thakkar and A. Virdis, ”Simu5G–An OMNeT++ Library for End-to-End Performance Evaluation of 5G Networks,” in IEEE Access, vol. 8, pp. 181176-181191, 2020. [27] Institute of Transportation Systems, “Simulation of Urban MObility (SUMO),” Accessed: May 17, 2025. [Online]. Available: https://www.eclipse.org/sumo/ [28] Twitch, “Broadcasting Guidelines,” Twitch Help, [Online]. Available: https://help.twitch.tv/s/article/broadcasting-guidelines?language=pt BR. [Accessed: May 15, 2025]

Maria Katarine Santana Barbosa received the B.Sc. degree in Electrical Engineering from University of Pernambuco (UPE), Recife-Brazil, in 2021. She received M.Sc in Computer Science from the Informatics Center of the Federal University of Pernambuco (CIn/UFPE), in 2023. She is currently pursuing the PhD degree in Computer Science in Universidade Federal de Pernambuco. Her current research interests are Optimization, Network Functions Virtualization, Software-defined Radio, 5G and 6G networks.

Kelvin Lopes Dias received a Ph.D. degree in Computer Science from Federal University of Pernambuco (UFPE), Recife-Brazil, in 2004. He is a full professor at the Center of Informatics (CIn) of the UFPE. His current research interests are Softwaredefined Networking, Network Functions Virtualization, Edge Computing, 6G networks, and Advanced & Intelligent Network Architectures.

Related documents

• Quasi-Single-Mode Transmission over Ultra-Low Loss Few-Mode Fibre for Data Centre Interconnects#609318
• Low-Power Wide-Area Network Over White Spaces#163684
• Performance Optimization of Short Reach Optical Interconnects based on Direct Detection#784885
• Performance Optimization of Short Reach Optical Interconnects based on Direct Detection#783295
• Seeing Through the Overlay: Unveiling Potential Bottlenecks in SD-WANs#171290