EvoRIC: Reinforcement Learning Fine-Tuned LLM-empowered RAN Intelligent Control Toward Autonomous O-RAN

1

EvoRIC: Reinforcement Learning Fine-Tuned LLM-empowered RAN Intelligent Control Toward Autonomous O-RAN

arXiv:2608.06789v1 [cs.NI] 7 Aug 2026

Lingyan Bao, Graduate Student Member, IEEE, Jemin Lee, Senior Member, IEEE, and Tony Q.S. Quek, Fellow, IEEE

Abstract—Despite recent advances in applying artificial intelligence (AI) techniques to radio access network (RAN), critical challenges remain: traditional machine learning (ML) algorithms suffer from limited generalization across varying network topologies, whereas general-purpose large language models (LLMs) face high computational demands and lack domain-specific knowledge. To address these gaps, this article introduces the evolving RAN intelligent controller (RIC) (EvoRIC) framework, a hierarchical architecture that enables continuous evolution by leveraging a non-real-time RIC (non-RT RIC) for global model updates and a near-real-time RIC (near-RT RIC) for local execution, dynamically empowering LLMs with domain-specific decision-making capabilities. Within this framework, we employ a reinforcement learning-based fine-tuning (RLFT) mechanism where an LLM operates as an actor within a proximal policy optimization (PPO) agent. By leveraging the interaction tuples collected from the wireless environment, the LLM’s parameters are iteratively updated to align semantic reasoning with rigorous network performance objectives. We evaluate the generalization and efficacy of the proposed EvoRIC framework within integrated access and backhaul (IAB) networks, and finally, discuss the open challenges and future directions of the EvoRIC framework toward realizing autonomous O-RAN. Index Terms—Open radio access network, large language model, reinforcement learning, fine-tuning, RAN intelligent control

I. I NTRODUCTION The rapidly advancing landscape of 6G has positioned artificial intelligence (AI) algorithms as a foundational element for driving innovations in advanced radio access network (RAN). Within this context, the concept of AI-RAN, leveraging AI to enhance RAN performance, has gained significant traction. Specifically, the AI-for-RAN paradigm focuses on integrating intelligence directly into the network infrastructure to optimize critical metrics such as spectral efficiency and capacity. A key enabler of this vision is the open radio access network (O-RAN) architecture, which introduces the RAN intelligent controller (RIC) to decouple control functionalities from hardware. This architectural shift paves the way for Manuscript submitted 23 April 2026; revised 7 August 2026. Corresponding author is J. Lee. J. Lee and L. Bao are with the School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, South Korea (e-mail: jemin.lee@yonsei.ac.kr; lingyan@yonsei.ac.kr). T. Q. S. Quek is with the Information System Technology and Design, Singapore University of Technology and Design, Singapore 487372 (email:tonyquek@sutd.edu.sg).

autonomous O-RAN, a paradigm aiming for zero-touch, selfoptimizing network operations. In O-RAN, AI algorithms are deployed as microservices known as xApps (in near-realtime RIC (near-RT RIC)) and rApps (in non-real-time RIC (non-RT RIC)), enabling programmable network optimization across different control timescales [1]. Among various AI approaches, reinforcement learning (RL) has emerged as a dominant tool for solving complex resource allocation problems in wireless communications. To identify near-optimal policies, an RL agent interacts with the environment, optimizing network parameters to maximize long-term rewards. In [2], the RL agent is trained in the non-RT RIC and subsequently deployed in the near-RT RIC to perform resource management for network slicing. However, traditional RL algorithms struggle to adapt efficiently to the continuously varying action and state spaces inherent in dynamic wireless environments. This limitation often necessitates frequent retraining or the maintenance of multiple model versions for different network topologies, posing a significant challenge to scalable deployment in heterogeneous networks. Recently, large language models (LLMs) have demonstrated remarkable capabilities in natural language understanding, reasoning, and complex decision-making, driven by their extensive training on massive datasets. The wireless community has begun to explore the potential of LLMs for network optimization tasks, such as resource allocation [3], [4]. Early attempts utilized zero-shot learning or in-context learning, where the LLM infers policies based on provided examples (prompts) without parameter updates. Notably, hierarchical RIC architectures, i.e., the LLM-hRIC framework [5], deploy an LLM in the non-RT RIC to provide high-level policy guidance based on global network information, while a deep deterministic policy gradient (DDPG) agent in the near-RT RIC performs the resource optimization. More advanced approaches employ supervised fine-tuning (SFT) to adapt LLMs for specific tasks, ranging from channel estimation and reconstruction [6] to technical problems summarization [7]. Furthermore, LLMbased agents have been applied to packet analysis [8] and O-RAN knowledge retrieval [9]. Despite these advancements, the application of LLM-based techniques in wireless networks faces several critical challenges: 1) in-context learning struggles to guarantee consistent performance as providing high-quality and context-aware services in rapidly changing wireless channels is inherently difficult; 2) the widespread adoption of SFT remains constrained

2

by the scarcity of comprehensive and expert-labeled decisionmaking datasets for most wireless scenarios; 3) although general-purpose LLMs possess superior generalization capabilities compared to traditional AI, their high computational complexity hinders near-RT decision-making; and 4) without rigorous alignment, LLMs are prone to generating syntactically correct but physically unexecutable control actions. To overcome these critical challenges, recent research has started investigating the synergy between LLMs and RL, specifically formulating LLMs as adaptive agents capable of self-optimization via environmental feedback [10]. This convergence offers a unique opportunity to combine the generalization capabilities of foundation models with the autonomous optimization power of RL, thereby circumventing the need for expensive labeled datasets. Motivated by these opportunities and challenges, we propose a novel framework for O-RAN, termed evolving RIC (EvoRIC), which improves its control policy through closedloop interaction with the wireless environments. In this framework, a compact LLM (or small language model (SLM)) operates as the RL actor within the non-RT RIC, where it is fine-tuned using a critic-guided policy gradient approach. Subsequently, the optimized models are deployed to the near-RT RIC to execute near-RT resource allocation in the wireless environment. The motivation for introducing an LLMbased control arises from supporting the structural variability of control problems. Across heterogeneous deployments, number of controlled network entities, length of the networkstate description, dimension of the control actions, as well as associated operational constraints, the network can have various. To facilitate continuous learning, interaction trajectories are aggregated from the near-RT RIC and fed back to the non-RT RIC via standard open interfaces (e.g., O1). The key contributions of this work are summarized as follows: • The proposed EvoRIC framework leverages the inherent reasoning and transfer capabilities of LLMs. This facilitates generalization across diverse network scenarios (e.g., varying numbers of BSs or users) without the need for architecture reconstruction. • We adopt an RL-driven fine-tuning mechanism that enables the LLM to optimize its policy directly through environmental interaction. This approach circumvents the data bottleneck associated with the scarcity of expertlabeled datasets in traditional supervised methods. • We evaluate the feasibility and performance of EvoRIC within integrated access and backhaul (IAB) networks to demonstrate the superiority of our fine-tuned compact model over massive general-purpose models. II. P ROPOSED E VO RIC F RAMEWORK The O-RAN separates RAN control into the non-RT RIC and near-RT RIC, which operate with different responsibilities and timescales [1]. The non-RT RIC is integrated with the service management and orchestration (SMO) framework and hosts rApps for long-term policy optimization and AI model lifecycle management, in a timescale of greater than 1 second. In contrast, the near-RT RIC hosts xApps that process nearreal-time (near-RT) measurements collected from RAN nodes

and generate control actions for resource management in a timescale of 10 ms to 1 s. These functions interact through open interfaces such as E2 for measurements and control messages exchange between the near-RT RIC and RAN nodes, A1 for policies and model information exchange between the non-RT RIC and near-RT RIC, and O1 for management and data exchange between the SMO and RAN nodes. In this section, we describe the proposed EvoRIC framework for decision making in O-RAN, as illustrated in Fig. 1. Architecturally, the framework decouples the control loop into two distinct and yet interconnected domains: 1) Non-RT RIC for LLM fine-tuning: functioning as a training domain that orchestrates the resource-intensive LLM fine-tuning (e.g., the proximal policy optimization (PPO) trainer) using aggregated historical data. 2) Near-RT RIC for LLM inference: serving as an inference domain, responsible for executing near-RT decisionmaking at a network edge. By aggregating telemetry from distributed near-RT RICs, the non-RT RIC establishes a closed data loop between centralized fine-tuning and edge execution. At each near-RT RIC, the deployed LLM processes a prompt containing the intent, network state, and constraints to generate a corresponding control policy. The resulting interaction tuple, consisting of the input prompt, generated policy, and environmental feedback, is returned to the non-RT RIC for subsequent fine-tuning. Learning from interaction tuples collected under diverse network conditions helps the LLM adapt to heterogeneous network scenarios. The updated LLM is then redeployed to the near-RT RICs. Repeating this collect-update-deploy cycle enables the LLM policy to evolve from new network experiences, without requiring manually labeled optimal actions. This feedbackdriven cycle is the core mechanism underlying the continuous evolution of EvoRIC. Detailed descriptions of each component are provided in the following subsections. A. Non-RT RIC for LLM Fine-Tuning The non-RT RIC leverages its global visibility to fine-tune the LLM using the data collected from near-RT RICs. The RL-based fine-tuning workflow proceeds as follows: 1) Data Aggregation and Rollout Collection: The non-RT RIC aggregates interaction logs from distributed near-RT RICs via the O1 interface. Each interaction record is structured as a transition tuple (st , at , rt ) and stored in a rollout buffer D, where st denotes the input prompt, at is the policy generated by the LLM, and rt is the reward returned by the environment. 2) RL-based LLM Fine-Tuning: To align the LLM’s semantic reasoning with network performance objectives, the non-RT RIC executes a RL training loop [10]. The detailed mechanism includes three key aspects: a. Shift from SFT to Autonomous Exploration: A critical innovation of our approach is the shift from supervised instruction following to autonomous exploration. Unlike SFT, which relies on scarce and costly expertlabeled data, the RL-driven mechanism enables the LLM to explore candidate actions, receive rewards based on

3

Fig. 1. Proposed EvoRIC framework: a closed-loop architecture integrating a non-RT RIC for global model fine-tuning and a near-RT RIC for near-real-time inference, together with an abbreviated prompt template and representative LLM output for power allocation.

their network performance, and update its policy toward higher-reward decisions. b. Composite Reward Function: To ensure performance and executability, we design a composite reward function that simultaneously optimizes objective adherence (e.g., throughput) and format compliance. This mechanism significantly reduces the hallucination rate (i.e., non-executable outputs), ensuring that even a fine-tuned compact model can generate reliable control actions. c. Dual-Model Mechanism for Stability: To improve training stability and prevent the model from deviating excessively from its pre-trained knowledge (i.e., catastrophic forgetting), a dual-model mechanism is employed. For each sampled prompt, the policy LLM generates an action sequence, while the reference model evaluates the same sequence and provides the token-level reference log-probabilities. The resulting Kullback–Leibler (KL)divergence penalty is incorporated into the PPO reward as a soft trust-region regularizer to discourage large policy deviations that may cause unstable updates [11]. 3) Model Update and Deployment: Periodically or upon the convergence of the training epoch, the updated model is deployed to near-RT RICs via the A1 interface. Capitalizing on the generalization capabilities of the LLM, a common model fine-tuned in the non-RT RIC can be

deployed across multiple near-RT RICs serving diverse network topologies. B. Near-RT RIC for LLM Inference The near-RT RIC operates as a distributed inference engine, executing the fine-tuned LLM for near-RT radio resource management. By acting as a tactical execution layer, it utilizes the LLM’s generalized reasoning capabilities to adaptively optimize network parameters in response to fluctuating traffic patterns and topological changes. The inference workflow proceeds as follows: 1) Intent-and-Policy Driven Prompting: The near-RT RIC aggregates network states from the next generation node B (gNB) via the E2 interface. Depending on the optimization task, these states may include, for example, channel conditions, buffer states, and resource availability. To bridge the modality gap between continuous signals and discrete tokens, these numerical metrics are serialized into a structured text and combined with the optimization objective. The resulting prompt further includes two control directives: 1) a high-level service intents (e.g., ‘maximizing network throughput’), which defines the optimization direction; and 2) explicit policy specifications (e.g., ‘output a list of N power levels’), which strictly define the required output format. This composite prompt

4

guides the LLM to generate policies that are not only optimal for the intent but also syntactically valid and structurally compliant with the network requirements. 2) Near-RT Inference: The fine-tuned LLM processes the prompt to generate a structured textual response that encapsulates the intended policy action. By utilizing a model with small size, we can make this inference process complete within a sub-second control window, satisfying the strict latency constraints of the near-RT RIC. 3) Actuation & Safety Verification Module: A deterministic parsing mechanism processes the generated textual response to extract the policy actions (e.g., power levels, beam configurations) from the textual response. Before execution, the extracted policy actions undergo a safety verification check to ensure compliance with physical constraints (e.g., the maximum power budget). If the generated policy action fails this validation, a default fallback strategy (e.g., a conservative baseline configuration) is triggered to maintain network stability. Only validated policy actions are encapsulated into E2 control messages and transmitted to the gNB for execution. 4) Feedback Collection: After execution, the input prompt, the generated action, and the corresponding environment or validation feedback are returned from near-RT RIC to the non-RT RIC through O1 and stored in the rollout buffer for subsequent fine-tuning. This feedback completes the collect-update-deploy loop between near-RT execution and non-real-time (non-RT) model evolution. III. E VO RIC F RAMEWORK FOR P OWER A LLOCATION In this section, we present a use case based on IAB networks to verify the effectiveness of the proposed EvoRIC framework. This scenario was selected as a foundational proof-of-concept, as resource allocation in multi-cell IAB networks is a recognized challenge requiring robust adaptability. A critical hurdle in IAB networks is the diverse deployment configurations of macro base station (MBS) and small base station (SBS) nodes. This variability renders traditional machine learning approaches ineffective, as they typically use the rigid models, trained separately for each specific network configuration. This problem effectively demonstrates the core strength of our framework: it leverages the inherent generalization capabilities of the LLM to adapt to varying network structures. Furthermore, the RL trainer automatically guides the LLM towards optimal decision-making, ensuring that the model’s generative capacity is strictly aligned with the rigorous performance objectives of the wireless network. A. System Model As illustrated in Fig. 1, the IAB network consists of multiple MBSs, distributed in the network. Each MBS connects to a cluster of N SBSs via wireless backhaul. A set of mobile users served by these SBSs follow a Gaussian-Markov mobility model. The SBSs allocate a fixed power to its connected users. However, interference exists among SBSs connected to different MBS that use the same sub-carrier. The channel model considers both line-of-sight (LoS) and non-line-of-sight

(NLoS) path loss characteristics, where the LoS probability decays exponentially with the distance, alongside independent Nakagami-m fading small-scale fading. In this use case, the total channel bandwidth W is assigned to the backhaul link W α and the access links W (1 − α) based on a proportion factor α ∈ [0, 1]. Consequently, the effective data rate for each user is constrained by the bottleneck formed between the backhaul capacity and the aggregate access demand. The system objective is to maximize the total network throughput, defined as the summation of these bottlenecklimited rates across all nodes. This is achieved by optimizing the power allocation from each MBS to its associated SBSs, strictly subject to the maximum power budget constraint of each individual MBS. B. EvoRIC for power allocation In this section, the proposed EvoRIC framework is deployed to solve the power allocation problem in IAB networks. The MBSs are associated with a near-RT RIC for near-RT inference and data collection, while a centralized non-RT RIC aggregates global information for RL-based fine-tuning. 1) MDP Formulation: To facilitate the LLM fine-tuning via RL, we model the power allocation problem as a Markov Decision Process (MDP) and define its state, action, and reward components, denoted by (S, A, R), as follows: • State Space (S): The state st denotes the prompt provided to the LLM at control interval t. As illustrated in Fig. 1, it comprises three components: 1) the optimization objective, i.e., maximizing total throughput; 2) the network context, including the numbers of MBSs and SBSs, average channel gains, user information, and interference sources; and 3) the action specification, which requires N normalized power-allocation values. The network measurements are inserted into this template to construct st . • Action Space (A): The action at corresponds to the power allocation vector pm = (pm,1 , · · ·, pm,N ), where pm,n ∈ [0, 1] represents the normalized power ratio allocated to the n-th SBS. • Reward Function (R): The reward rt = rth + rformat is designed to maximize throughput while enforcing strict format compliance. It consists of the normalized environment reward (throughput) and a penalty term. To ensure system operability, a substantial penalty (e.g., rformat = −5) is imposed if the LLM fails to generate a valid, parsable numerical list. 2) non-RT RIC for LLM Fine-tuning: The non-RT RIC iteratively fine-tunes the LLM using PPO. The PPO trainer computes the policy gradient loss from the interaction tuples (st , at , rt ) sampled from the rollout buffer D and updates the trainable LLM parameters to maximize the expected reward, while a KL-divergence penalty relative to the frozen reference model limits policy drift and improves stability. 3) Near-RT RIC for LLM Inference: In the inference domain, the fine-tuned LLM is deployed within the near-RT RIC to execute power allocation. To bridge the gap between numerical network states and the language model, a structured prompt template is utilized. A representative prompt example,

5

C. Experimental Results Our simulations are conducted on a high-performance workstation equipped with a single NVIDIA RTX 4090 GPU. We utilize the Llama-3.2-3B-instruct as the backbone model of EvoRIC [12]. During the fine-tuning process, we implement a linear temperature decay schedule annealing from 1.5 to 0.6 combined with top p of 0.95. The PPO algorithm is driven by the Adam optimizer with a learning rate of 10−5 and a batch size of 16. The model is loaded in bfloat16 precision to reduce memory usage. We evaluate the proposed EvoRIC with different fine-tuning depths by training the final L transformer blocks, denoted as EvoRIC-L1 (L = 1), EvoRIC-L2 (L = 2), and EvoRIC-L3 (L = 3). Compared with EvoRICL1, EvoRIC-L2 improves throughput by 11.52%, while the improvement from L = 2 to L = 3 is marginal (e.g., 1.20%). Therefore, EvoRIC-L2 is adopted in all subsequent simulations to balance training overhead and network performance. 1) Baselines: We compare our fine-tuned compact 3B model with massive cloud-based models (e.g., DeepSeek and Gemini). It is important to clarify that this comparison highlights the trade-off between general-purpose reasoning and domain-specific efficiency. While massive models possess vast knowledge, their computational requirements make deployment at the near-RT RIC impractical. Our goal is to demonstrate that a specialized compact model, feasible for edge deployment, can surpass the performance of cloudbased giants in specific vertical tasks, effectively validating the ‘Small Model with Fine-Tuning’ paradigm for O-RAN. The baselines are described as follows: • Llama-3.2-3B-Instruct without fine-tuning (Llama3BNoFT): The pre-trained model without any domain-

200 EvoRIC (With KL) EvoRIC (Without KL) 175 Total Throughput (Mbps)

together with the corresponding power-allocation output, is presented on the right-hand side of Fig. 1. 4) Actuation & Safety Verification Module: To deploy LLM safely within critical infrastructure, we implement a hybrid actuation mechanism that strictly isolates reasoning from execution. The workflow proceeds in three stages as follows: • Extraction: A deterministic parsing algorithm (e.g., regex-based extraction) processes the LLM’s textual response to isolate the numerical power allocation vector by discarding any extraneous linguistic tokens. • Verification: This extracted vector undergoes a safety verification check (“guardrail”) to validate compliance with physical constraints, Pspecifically checking if the total power budget satisfies pn ≤ Pmax and if the output dimensionality matches the network topology. • Fail-Safe Execution: If the validation fails due to model hallucination or format errors, the system triggers an failsafe protocol that reverts to an equal power allocation (EPA) strategy (i.e., the conservative baseline). Consequently, this mechanism guarantees that only physically valid and safe control actions, whether generated by the LLM or the fallback logic, are encapsulated into E2 control messages and transmitted to the gNB, ensuring system availability and stability. After execution, the interaction tuple (st , at , rt ) is stored in the rollout buffer for subsequent fine-tuning.

150

125

100

75

50 0

1000

2000

3000

Training Episodes

Fig. 2. Training curve for EvoRIC with and without KL when M = 3, N = max = 44dBm, and α = 0.3. 5, W = 100MHz, Pm

specific fine-tuning. This baseline serves to quantify the gain achieved by our RL-based fine-tuning mechanism. • General-Purpose Large Models (DeepSeek & Gemini): We evaluate two massive, cloud-based models: DeepSeek-V3 (671B parameters) [13] and Google Gemini 1.5 Flash [14], accessed via APIs. To ensure a rigorous comparison, we applied comprehensive prompt engineering (incorporating task context and strict format constraints) to maximize their zero-shot capabilities. This baseline represents the representative general-purpose LLM baselines without parameter updates. • EPA: A conventional heuristic scheme where power is equally distributed among all SBSs. This serves as a standard reference for traditional engineering approaches. • Successive Convex Approximation (SCA)-based Algorithm: A model-based numerical optimization method that iteratively approximates the original non-convex power-allocation problem using tractable convex subproblems implemented in CVXPY [15]. Figure 2 depicts the training dynamics of EvoRIC over 3,000 training episodes under a backhaul-access bandwidth partitioning ratio of α = 0.3. It compares the KL-regularized EvoRIC and EvoRIC trained without KL regularization. The KL-regularized configurations exhibit steady throughput improvement during the initial training stage and subsequently converge to stable performance. In contrast, EvoRIC without the KL penalty exhibits pronounced fluctuations, and excessive policy drift eventually causes numerical instability and interrupts further parameter updates. This comparison demonstrates the role of KL regularization in stabilizing PPO updates. Figure 3 illustrates the comparative throughput performance across three distinct network topologies: Scenario 1 (M = 3, N = 6), Scenario 2 (M = 3, N = 5), and Scenario 3 (M = 2, N = 3). The results reveal that the proposed EvoRIC-L2 consistently outperforms all baselines, including the conventional EPA scheme and the Llama3B-NoFT. The performance gain is small in Scenario 3 because the opti-

6

DeepSeek

Fig. 3. Average total throughput of the proposed EvoRIC and baseline methods according to different scenarios over 500 test examples. TABLE 1 P ERFORMANCE UNDER NON - NOMINAL OPERATING CONDITIONS IN S CENARIO 2 OVER 500 PAIRED EPISODES .

Evaluation condition SCA-based and reported metric EvoRIC Llama3B-NoFT algorithm Intent adaptation PS (Mb/s)

289.94

153.7

273.88

Telemetry robustness: Fault-free (Mb/s) 166.57 URC (Mb/s) 166.46

81.69 80.49

155.45 150.58

mization headroom is limited. With three SBSs per MBS, the solver improves upon EPA by only 16.6 %, compared with 63– 67 % in the other two scenarios, which means limited room for the learned policies to further outperform EPA. This may occur when the end-to-end throughput is less affected by the backhaul power-allocation strategy or the optimal allocation is close to uniform. From Fig. 3, we can also observe that EvoRIC achieves superior performance with only a fraction of the inference cost of general-purpose large models (i.e., DeepSeek and Gemini). The EvoRIC effectively adapts a compact, edge-deployable model to the specific characteristics of IAB power allocation, while large models, despite possessing broad general knowledge, lack the specific ‘sense’ for wireless resource constraints. Furthermore, EvoRIC completes inference in 0.56 s, smaller than 3.82 s of DeepSeek and 1.60 s of Gemini, demonstrating its practical advantage for sub-second near-real-time control. Table 1 presents a comparative analysis of intent adaptability and telemetry robustness in Scenario 2 over 500 paired episodes. For intent adaptability, the network topology, physical constraints, observed states, and action space remain unchanged. The same EvoRIC model fine-tuned for throughput maximization is evaluated without additional parameter updates, and only the intent description and evaluation metric are modified. For the priority-SBS (PS) intent, i.e., the

throughput of the fourth SBS associated with each MBS has a weight of five, while the remaining SBSs have unit weights. EvoRIC achieves the weighted throughput of 289.94 Mb/s, outperforming EPA and the SCA-based algorithm by 96.6% and 10.6%, respectively. For the telemetry robustness evaluation, the unmarked rate corruption (URC) is injected into the observations provided to the controllers, while the underlying physical environment remains unchanged. Specifically, the reported average rate of one SBS per MBS is multiplied by either 0.25 or 4 during the first two evaluated intervals. No fault indicator is provided, and EvoRIC is evaluated without fault-aware parameter updates. Each prompt additionally contains the three preceding interaction records including the network state, executed action, and achieved throughput. As shown in Table 1, URC changes the throughput of EvoRIC slightly, from 166.57 to 166.46 Mb/s, whereas the throughput of the SCA-based algorithm decreases from 155.45 to 150.58 Mb/s, i.e., 3.1% reduction. This result provides empirical evidence that EvoRIC can have better adaptability to different control objectives and be less sensitive to some information corruption, compared to the SCA-based algorithm. However, note that this does not establish general robustness to all telemetry faults. IV. F UTURE C HALLENGES In this section, we discuss the open challenges for the EvoRIC framework in wireless communications. As illustrated in the previous sections, while RL fine-tuning empowers LLMs for specific tasks, several challenges remain, including fine-grained real-time control, efficient exploration, multi-modal information processing, and automated lifecycle management. A. Fine-grained and Real-time Control of Wireless Communication Resources A challenge for EvoRIC is bridging the granularity mismatch between discrete language tokens and continuous physical signals. LLMs operate on discrete tokens, creating precision barriers for continuous parameters like power levels. Current serialization schemes often result in precision loss or excessive context lengths. Future research may investigate semantic-aware tokenization or hybrid neuro-symbolic architectures, where the LLM acts as a high-level planner while a lightweight controller handles fine-grained adjustments. Moreover, edge deployment also introduces strict latency and resource constraints. While compact models (3B) are efficient for large-timescale management, symbol-level control demands millisecond responsiveness on limited hardware. Model distillation, aggressive quantization (e.g., 4-bit), and novel architectures, such as large recurrent action models, may reduce the computational and memory overhead for deployment on COTS edge servers. B. Reasoning-Driven Exploration Strategies Currently, LLM exploration relies primarily on the stochasticity of token generation (e.g., temperature sampling). However, this probability distribution is heavily biased by pretrained linguistic knowledge, often leading to limited strategic

7

diversity and convergence to suboptimal, ‘textually probable’ solutions. Token-level randomness alone may therefore be insufficient for complex wireless state-action spaces. Therefore, future work should investigate reasoning-driven exploration strategies. This includes incorporating intrinsic motivation mechanisms, such as curiosity-driven exploration, to guide the LLM toward novel policy regions. Furthermore, reflection-based methods (e.g., verbal reinforcement learning) may further enable the agent to analyze historical trajectories and low-reward actions, thereby combining semantic reasoning with conventional stochastic exploration. C. Fine-Tuning with Multi-modal Input The current EvoRIC framework primarily processes serialized text-based inputs, which may discard the rich spatial and topological dependencies inherent in wireless environments (e.g., interference graphs, coverage maps). Consequently, text-only representations may limit the quality of resourceallocation decisions. A key future direction is extending the framework to support multi-modal fine-tuning, such as leveraging vision-language models. This involves developing robust encoding techniques to project diverse data types (vision, time-series, text) into a unified latent space, allowing the LLM to reason over the full environmental context. By integrating multi-modal inputs, the RL agent can better interpret complex spatial correlations and signal patterns that are difficult to describe via text alone, leading to more adaptive and informed control policies. D. Integration with Agentic AI for Automated Management Beyond optimizing the decision model, integrating agentic AI to automate the RIC lifecycle represents a critical frontier. Manual tasks like reward engineering and dataset curation create significant bottlenecks requiring deep domain expertise. Agentic AI frameworks can automate these processes by: 1) automated reward design: Synthesizing complex objective functions from high-level operator intents (e.g., ‘maximize fairness’) and dynamically adjusting weights based on deployment feedback; and 2) autonomous data curation: Orchestrating digital twins to simulate diverse edge cases and automatically label high-quality trajectories for fine-tuning. Automating these processes may reduce the manual effort required to deploy and continuously evolve EvoRIC across heterogeneous O-RAN environments. V. C ONCLUSIONS This paper proposed the EvoRIC framework, a novel architecture that integrates LLMs into O-RAN for intelligent resource management. By structurally decoupling the system into a non-RT RIC layer for RL-based fine-tuning and a near-RT RIC layer for near-RT inference, the framework effectively addresses the challenges of adapting generative models to dynamic wireless environments. We evaluated EvoRIC for power allocation in multi-cell IAB networks under different topologies and non-nominal operating conditions. Numerical results show that the compact fine-tuned model outperforms

EPA, Llama3B-NoFT, and the evaluated general-purpose models. The same model also adapts to changing control intents through prompt modification and exhibits limited degradation under transient telemetry corruption without additional parameter updates. These results demonstrate the potential of domain-specific fine-tuning for closed-loop and environmentdriven O-RAN control. Finally, we outlined critical future challenges in Section IV to guide subsequent research in this emerging domain. R EFERENCES [1] M. Polese, L. Bonati, S. D’oro, S. Basagni, and T. Melodia, “Understanding O-RAN: Architecture, interfaces, algorithms, security, and research challenges,” IEEE Commun. Surv. Tutor., vol. 25, no. 2, pp. 1376–1411, Jan. 2023. [2] K. Qiao, H. Wang, W. Zhang, D. Yang, Y. Zhang, and N. Zhang, “Resource allocation for network slicing in open RAN: A hierarchical learning approach,” IEEE Trans. Cogn. Commun. Netw., vol. 11, no. 4, pp. 2584–2600, 2025. [3] W. Lee and J. Park, “LLM-empowered resource allocation in wireless communications systems,” IEEE Access, vol. 14, pp. 15 260–15 272, 2026. [4] H. Zhou, C. Hu, D. Yuan, Y. Yuan, D. Wu, X. Liu et al., “Prompting wireless networks: Reinforced in-context learning for power control,” arXiv preprint arXiv:2506.06526, 2025. [5] L. Bao, S. Yun, J. Lee, and T. Q. Quek, “LLM-hRIC: LLM-empowered hierarchical RAN intelligent control for O-RAN,” IEEE Commun. Mag., pp. 1–7, 2026, early access. [6] X. Liu, S. Gao, B. Liu, X. Cheng, and L. Yang, “LLM4WM: Adapting LLM for wireless multi-tasking,” IEEE Trans. Mach. Learn. Commun. Netw., pp. 1–1, 2025. [7] Y. Lin, R. Zhang, W. Huang, K. Wang, Z. Ding, D. K. C. So, and D. Niyato, “Empowering large language models in wireless communication: A novel dataset and fine-tuning framework,” IEEE Trans. on Commun., vol. 73, no. 12, pp. 14 260–14 274, 2025. [8] K. B. Kan, H. Mun, G. Cao, and Y. Lee, “Mobile-LLaMA: Instruction fine-tuning open-source LLM for network analysis in 5G networks,” IEEE Netw., vol. 38, no. 5, pp. 76–83, Sep. 2024. [9] P. Gajjar and V. K. Shah, “ORAN-Bench-13K: An open source benchmark for assessing LLMs in open radio access networks,” in Proc. IEEE Consum. Commun. Netw. Conf. (CCNC), Las Vegas, NV, USA, Jan. 2025, pp. 1–4. [10] T. Schmied, J. Bornschein, J. Grau-Moya, M. Wulfmeier, and R. Pascanu, “LLMs are greedy agents: Effects of RL fine-tuning on decisionmaking abilities,” arXiv preprint arXiv:2504.16078, 2025. [11] K. Liu, J. K. Liu, M. Chen, and Y. Liu, “Rethinking KL regularization in RLHF: From value estimation to gradient optimization,” arXiv preprint arXiv:2510.01555, 2025. [12] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan et al., “The Llama 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024. [13] A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan et al., “Deepseek-v3 technical report,” arXiv preprint arXiv:2412.19437, 2024. [14] G. Team, P. Georgiev, V. I. Lei, R. Burnell, L. Bai, A. Gulati, G. Tanzer, D. Vincent, Z. Pan, S. Wang et al., “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” arXiv preprint arXiv:2403.05530, 2024. [15] S. Diamond and S. Boyd, “CVXPY: A Python-embedded modeling language for convex optimization,” Journal of Machine Learning Research, vol. 17, no. 83, pp. 1–5, 2016.

Related documents

• Massively Parallel Reinforcement Learning with a Chaotic Reconfigurable Clockless Chip#1131189
• The Stochastic Mirror: Reflecting on the Architectural and Energetic Limits of Weight-Based Artificial Intelligence#120741
• The Stochastic Mirror: Reflecting on the Architectural and Energetic Limits of Weight-Based Artificial Intelligence#120740
• Guidance and Control Strategy for the CALLISTO Flight Experiment#246333
• From RAN Control to Agentic Intelligence: Architecture and Vision for Energy Efficient AI-RAN#299867