QoS Assurance Mechanism for 5G Network Slicing Based on the Deep Reinforcement Learning PPO Algorithm Qingyang Li* Department of Computer Science University of California Los Angeles, USA [email protected]
Abstract—With the increasing diversity of 5G service types and the intensifying dynamic fluctuations of network load, achieve differentiated quality of service assurance in a network slicing environment has become a key issue in resource management. To address this problem, this paper proposes a deep reinforcement learning mechanism for 5G network slicing quality of service assurance based on the traditional proximal policy optimization actor–critic framework. First, the slicing resource allocation is modeled as a constrained Markov decision process, jointly considering the collaborative optimization of bandwidth, computing, and wireless resources. Meanwhile, a graph attention network and bidirectional long short-term memory are introduced to extract topological correlations and temporal service features, combined with an adaptive Lagrangian penalty and dynamic reward shaping mechanism, to comprehensively optimize delay, throughput, reliability, fairness, and slice isolation performance. Experimental results show that the proposed method outperforms existing baseline models in terms of quality of service satisfaction rate, delay control, resource utilization, and convergence stability. Keywords—5G Network Slicing, QoS Assurance, Deep Reinforcement Learning, Proximal Policy Optimization, Resource Allocation
I.
INTRODUCTION The transition from conventional mobile communication systems to 5G has shifted network design from best-effort connectivity toward service-oriented capability provisioning [1]. In this context, network slicing has emerged as one of the core enabling technologies of 5G, because it allows multiple logical networks with different performance characteristics to coexist on a shared physical infrastructure [2]. According to 3GPP, a network slice is a logical network that provides specific network capabilities and network characteristics for different service requirements, which makes it particularly suitable for vertical applications such as enhanced broadband access, industrial automation, and massive Internet of Things connectivity. As a result, how to guarantee differentiated Quality of Service (QoS) for multiple slices under shared and time-varying resources has become a central issue in 5G network management and orchestration. From the perspective of service requirements, 5G is expected to support three representative usage scenarios: enhanced Mobile Broadband (eMBB), Ultra-Reliable and Low-Latency Communications (URLLC), and massive
XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE
Machine-Type Communications (mMTC) [3]. These scenarios differ significantly in user-experienced data rate, latency, reliability, mobility support, and connection density. Because these service categories impose fundamentally different QoS objectives, a uniform or static resource allocation strategy can hardly satisfy all slice demands simultaneously in practical 5G environments [4]. The QoS problem in 5G network slicing is further complicated by the architecture of the 5G system itself [5]. In the 5G core, QoS is organized around the concept of QoS Flows, and flow-level markings and parameters are used to map service requirements to differentiated forwarding and scheduling behaviors across the network [6]. This means that QoS assurance is not merely a matter of reserving bandwidth. Instead, it requires coordinated control over multiple dimensions. Consequently, slice-level QoS assurance must be treated as a cross-layer and system-level optimization problem rather than a single-domain scheduling task. Although network slicing provides a flexible framework for service customization, achieving stable QoS guarantees in real deployments remains difficult [7]. On the one hand, wireless channels, user mobility, traffic bursts, and session arrivals continuously change the network state, which makes optimal resource decisions highly dynamic and time-dependent. On the other hand, the resources consumed by slices are tightly coupled across radio access, transport, and core/cloud domains, while different slices compete for limited capacity and may interfere with each other if isolation is insufficient. II.
RELATED WORK Recent studies on 5G network slicing have first established the broader research landscape from the perspectives of admission control, vertical applications, and intelligent orchestration. Khani et al. [8] presented a survey on slice admission control in 5G cloud radio access networks and argued that conventional SAC methods are insufficient for highly dynamic and heterogeneous wireless environments. In parallel, Hamdi et al. [9] provided a comprehensive survey of learning-based network slicing techniques for the Internet of Vehicles (IoV), covering terrestrial, aerial, and marine deployment domains, V2X datasets, implementation tools, and future opportunities for machine learning-enabled slicing in vehicular scenarios.
From a broader systems perspective, Abba Ari et al. [10] examined the evolution from IoT-5G to B5G/6G and reviewed how learning algorithms are being used for resource allocation and network slicing orchestration in next-generation intelligent networks.
one resource domain. For this reason, the slice-level allocation is defined as the aggregation of user-level decisions across multiple resource dimensions, shown in Equation 2.
Beyond survey work, several studies have proposed DRLdriven control mechanisms for specific QoS-sensitive network functions. Cheng et al. [11] developed a self-adaptive QoS management system for the 5G core network, in which user behavior and service-level agreements are exploited to generate dynamic QoS marking rules, and deep reinforcement learning is used to adapt these rules over time as traffic behavior evolves. In another line of work, Raeisi and Sesay [12] investigated uplink power control for 5G-connected vehicular networks in mmWave bands between connected autonomous vehicles and roadside units. Their method adopts a proximal policy optimization (PPO)-based modified actor-critic architecture with a deep neural network to minimize transmission power and co-channel interference while meeting target uplink capacity, and the authors reported better performance than 3GPP-based power control in dynamic road environments.
where is the amount of resource allocated to user in slice at slot , while is the corresponding slice-level budget. The symbols , , and represent radio, bandwidth, and computing resources, respectively. Function is the active user set of slice .
Yan et al. [13] proposed xSlice, in which QoS optimization is formulated as a regret-minimization problem subject to throughput, latency, and reliability requirements. Their actorcritic framework further integrates a graph convolutional network to encode RAN information and accommodate a dynamic number of traffic sessions. In a URLLC-oriented scenario, Yağcıoğlu [14] introduced the DQN-ASRA framework, which unifies slice prediction and dynamic resource allocation into a single decision-making process for predictive network slicing.
To capture latency-sensitive slice behavior, the delay model must include both queueing and service effects. A compact end-to-end latency expression is therefore adopted as Equation 4.
Huang et al. [15] investigated the problem of scalable and efficient resource allocation for 5G RAN slicing and proposed a hierarchical deep reinforcement learning framework to address the growing complexity of heterogeneous slice services and large-scale user populations. Experimental results showed that the proposed framework was able to learn stable and effective slicing policies, outperforming both random and ADMM-based benchmarks in terms of network utility and QoS assurance, while also exhibiting stronger robustness under varying traffic conditions and network congestion. III.
METHODOLOGIES
A. Constrained Markov Decision Process To ensure that slice-level QoS guarantees are explicitly enforced rather than treated as soft heuristics, the proposed mechanism formulates 5G slicing control as a constrained sequential decision problem. The objective in Equation 1 is to maximize long-term service utility while keeping resource consumption and QoS violations within admissible limits. where denotes the state space, the action space, and the statetransition kernel. The term is the immediate reward, is the -th constraint cost, is the discount factor, and is the admissible bound of the corresponding constraint. The first modeling step is to jointly describe radio, bandwidth, and computing resources, because end-to-end QoS in network slicing cannot be guaranteed by optimizing only
A bottleneck-based service model is then introduced in Equation 3 to characterize the fact that end-to-end throughput is jointly limited by wireless transmission, backhaul capacity, and computation availability. This is essential because a large wireless rate alone does not imply a high effective service rate when transport or computing becomes the bottleneck. where is the effective end-to-end service rate of user , and is the achievable wireless transmission rate. The variable denotes the allocated bandwidth resource, is the assigned computing resource, and is the per-bit computing demand of the corresponding service.
where is the service rate in resource domain , is the traffic arrival rate, and is the packet or task size. The constant is a numerical stabilizer. To improve scalability, the policy is factorized into an upper-level controller for slice admission and budget assignment, and a lower-level controller for fine-grained intraslice scheduling, shown in Equation 5. where is the upper-level policy and is the lower-level policy. The variable denotes the admission-and-budget action taken at coarse time scale , is the set of fine-grained slots inside that interval, and is the admitted slice set. The upper-level controller is responsible for deciding which slices should be admitted and how much resource budget each admitted slice should receive. Equation 6 and 7 separates longhorizon structural decisions from short-horizon scheduling responses, which improves both convergence stability and decision interpretability.
where is the binary admission action for slice , and is its admission probability predicted by the upper actor. The variable is the budget logit for resource type , is the normalized budget ratio, is the slice-level resource budget, and is the total available amount of that resource. Given the upper-level budget, the lower-level controller allocates resources among users inside each admitted slice. Equation 8 ensures that fine-grained actions remain feasible and consistent with the global slice-level policy.


where is the scheduling logit produced by the lower-level actor, and is the corresponding user-level allocation ratio. The quantity is the actual resource assigned to user , while is the upper-level budget inherited by slice . B. Spatio-Temporal Multi-Objective PPO Optimization After the relational structure is encoded, temporal evolution is incorporated by Equation 9 to load trends and delayed QoS degradation. The fused representation is obtained by combining graph features and bidirectional temporal features into a unified latent state.
B. Experimental Analysis Figure 1 illustrates the variation in QoS satisfaction rate achieved by different resource allocation and network slicing methods under different normalized traffic load levels. It can be observed that the QoS satisfaction rate of all methods gradually decreases as the traffic load increases, mainly due to intensified resource competition and growing network congestion.
where is the graph-attention embedding of slice , and is the temporal feature extracted from the BiLSTM over historical slice observations. The function is a nonlinear activation, is a fusion layer, and is the final spatio-temporal embedding used by the actor and critic networks. Because the proposed method targets multiple QoS dimensions simultaneously, the reward should adapt to the current bottleneck rather than use fixed weights throughout training. Therefore, a dynamic multi-objective reward is constructed by Equation 10. 1.
where denotes the utility term of objective , corresponding to delay , throughput , reliability , fairness , and slice isolation . The variable is the normalized violation score of that objective, is the adaptive weight, and controls sensitivity of weight adjustment. Dynamic reward shaping is further introduced to provide denser learning signals when the system gradually moves toward the QoS-feasible region. In parallel, an adaptive Lagrangian penalty is incorporated by Equation 11, is explicitly suppressed during policy optimization. where is the shaping potential function, and is the potentialbased shaping term. The quantity is the instantaneous cost of the -th constraint. IV.
Comparison of QoS Satisfaction Rate under Different Traffic Load
However, compared with Standard PPO, DQN, Greedy, and Static Slicing, the proposed HM-PPO method consistently maintains the highest QoS satisfaction rate across the entire load range. In particular, under heavy-load conditions, the proposed method still demonstrates stronger robustness and stability. Figure 2 presents the system throughput achieved by different methods under dynamically varying traffic conditions, where system throughput is defined as the total amount of data successfully transmitted by the network per unit time. It can be observed that all methods exhibit periodic fluctuations as the traffic demand changes over time, and a noticeable throughput degradation appears during the highlighted high-load period due to intensified resource contention and temporary congestion.
EXPERIMENTS
A. Experimental Setup The experiment uses the Milan Telecommunications Activity dataset from the Telecom Italia Big Data Challenge, which originates from call detail records (CDR) generated by the real cellular network in Milan, Italy. It has a clear realworld context and can well reflect the spatiotemporal fluctuation characteristics of urban mobile communication services. The data covers the period from November 1, 2013, to January 1, 2014, and aggregates the original communication activities at a 10-minute time granularity, making it suitable for constructing dynamic arrival processes and time-varying network load scenarios. Spatially, the data is mapped onto a regular grid of the city of Milan, with a single grid size of approximately 235 m × 235 m, thus being able to reflect both the spatial heterogeneity of service distribution and local hotspot effects.
2.
Comparison of System Throughput under Dynamic Traffic Conditions
Table 1 presents the utilization results of radio, bandwidth, and computing resources under different time slots, and further reports the overall resource usage efficiency. I.
COMPARISON OF SYSTEM RESOURCE UTILIZATION
Time Slot
Radio Utilization (%)
Bandwidth Utilization (%)
Computing Utilization (%)
Overall Utilization (%)
T1
82.46
79.38
76.25
79.36
T2
84.13
81.02
78.47
81.21
T3
86.75
83.56
80.14
83.48
T4
88.24
85.33
82.68
85.42
T5
89.67
86.91
84.05
86.88
T6
91.12
88.47
85.94
88.51
T7
92.38
89.75
87.21
89.78
T8
93.64
91.08
88.56
91.09
T9
94.27
92.14
89.83
92.08
T10
95.03
93.26
90.71
93.00
2.
3.
It can be observed that, as the traffic load gradually increases from low to very high levels, the utilization of all three resource types shows an overall upward trend, indicating that the system is able to adapt its resource allocation strategy according to changing service demands. CONCLUSION In conclusion, this study addresses the QoS assurance problem in 5G network slicing by developing a hierarchical multi-objective optimization mechanism based on deep reinforcement learning and PPO, achieving coordinated improvements in latency, throughput, reliability, fairness, and resource utilization. In future work, the framework can be further extended toward 6G-oriented scenarios by integrating federated learning, digital twins, and cross-domain collaborative control to enhance real-time adaptability, generalization capability, and practical deployability.
4.
5.
6. 7.
8.
9.
10.
V.
Notice: This work has been submitted to IEEE MLISE 2026 for possible publication. Copyright may be transferred without notice. REFERENCES
‐
‐
Serôdio, Carlos, et al. "The 6G ecosystem as support for IoE and private networks: Vision, requirements, and challenges." Future Internet 15.11 (2023): 348.
‐
1.
11.
12.
13.
14.
15.
Rafique, Wajid, et al. "A survey on beyond 5g network slicing for smart cities applications." IEEE Communications Surveys & Tutorials 27.1 (2024): 595-628. Sefati, Seyed Salar, and Simona Halunga. "Ultra reliability and low latency communications on the internet of things based on 5G network: literature review, classification, and future research view." Transactions on emerging telecommunications technologies 34.6 (2023): e4770. Saibharath, S., Sudeepta Mishra, and Chittaranjan Hota. "Joint QoS and energy-efficient resource allocation and scheduling in 5G Network Slicing." Computer Communications 202 (2023): 110-123. Hamdi, Wafa, et al. "Network slicing-based learning techniques for IoV in 5G and beyond networks." IEEE Communications Surveys & Tutorials 26.3 (2024): 1989-2047. Zhao, Kevin, et al. "Prediction-Guided Control in Data Center Networks." arXiv preprint arXiv:2601.03593 (2026). Rafique, Wajid, et al. "A survey on beyond 5g network slicing for smart cities applications." IEEE Communications Surveys & Tutorials 27.1 (2024): 595-628. Khani, Mohsen, et al. "Slice admission control in 5G cloud radio access network using deep reinforcement learning: A survey." International Journal of Communication Systems 37.13 (2024): e5857. Hamdi, Wafa, et al. "Network slicing-based learning techniques for IoV in 5G and beyond networks." IEEE Communications Surveys & Tutorials 26.3 (2024): 1989-2047. Abba Ari, Ado Adamou, et al. "IoT 5G and B5G/6G resource allocation and network slicing orchestration using learning algorithms." IET Networks 14.1 (2025): e70002. Cheng, Sheng-Tzong, et al. "On self-adaptive 5G network slice QoS management system: a deep reinforcement learning approach." Wireless Networks 29.3 (2023): 1269-1279. Raeisi, Mostafa, and Abu B. Sesay. "Power control of 5G-connected vehicular network using PPO-based deep reinforcement learning algorithm." IEEE Access 12 (2024): 96387-96403. Yan, Peihao, et al. "Near-Real-Time Resource Slicing for QoS Optimization in 5G O-RAN Using Deep Reinforcement Learning." IEEE Transactions on Networking 34 (2025): 1596-1611. Yağcıoğlu, Mert. "Enhanced Deep Reinforcement Learning-Driven Adaptive Network Slicing and Resource Allocation for URLLC in 5G Networks." Journal of Network and Systems Management 34.1 (2026): 33. Huang, Renlang, et al. "Toward scalable and efficient hierarchical deep reinforcement learning for 5G RAN slicing." IEEE Transactions on Green Communications and Networking 7.4 (2023): 2153-2162.