State-Augmented Learning for Post-Deployment Adaptation of Coexisting O-RAN Applications

IEEE TRANSACTIONS ON MOBILE COMPUTING

1

State-Augmented Learning for Post-Deployment Adaptation of Coexisting O-RAN Applications

arXiv:2611.01376v1 [cs.LG] 1 Nov 2026

Wonhyeok Ko, Graduate Student Member, IEEE, Yeonseo Jeong, Graduate Student Member, IEEE, and Songnam Hong, Senior Member, IEEE

Abstract—Fixed control policies in Open RAN can require retraining when service requirements, traffic loads, or interacting applications change. We propose ATTUNE, a state-augmented learning method for an energy saving application that coexists with a learned mobility load balancing application under a service constraint. ATTUNE conditions the energy saving policy on a Lagrange multiplier and updates this scalar during operation using measured service degradation, while keeping both applications’ network parameters fixed. To train a single policy across service requirements, ATTUNE samples target service levels uniformly and maps them to multipliers through an adaptive table. This procedure accounts for policy-dependent multiplier responses and improves energy savings compared with training over a widened fixed multiplier range. We characterize the induced distribution of service levels and the convergence and fluctuations of the online update. An exact finite-window identity establishes convergence of average measured degradation to the target when the projection corrections vanish on average. Simulations show that a single ATTUNE policy achieves energy savings comparable to policies trained separately for each service requirement. It adapts to changes in the target, load, and partner application without retraining, increases energy savings at light load, and maintains target tracking for separately trained application pairs. No statistically significant increase in cell switching is detected in the evaluated comparisons. Index Terms—O-RAN, energy saving, load balancing, constrained reinforcement learning, state augmentation, multi-agent reinforcement learning.

I. I NTRODUCTION

E

NERGY saving through cell deactivation and mobility load balancing are key control functions implemented by Open RAN (O-RAN) applications on RAN Intelligent Controllers (RICs) [1], [2]. An energy saving rApp (ESrApp) on the Non-Real-Time RIC controls cell activation, while a mobility load balancing xApp (MLB-xApp) on the Near-Real-Time RIC adjusts cell individual offsets (CIOs) to redistribute traffic. These applications may originate from different vendors and operate at different time scales [3], [4]. Their actions are coupled through cell loads: deactivation redistributes traffic, while CIO control changes the load and resource demand of the remaining active cells. Our previous work [5] established sufficient conditions for oscillations under myopic ES and MLB rules despite the existence of a feasible operating point. Their interaction was formulated as a W. Ko and Y. Jeong contributed equally to this work. W. Ko and S. Hong are with the Department of AI-Semiconductor, Hanyang University, Seoul, Korea (e-mail: blackhawk77@hanyang.ac.kr). Y. Jeong and S. Hong are with the Department of Electronic Engineering, Hanyang University, Seoul, Korea (e-mail: wjddustj225@hanyang.ac.kr; snhong@hanyang.ac.kr). Corresponding author: Songnam Hong.

constrained Markov game with asymmetric service coupling: MLB penalizes service degradation in its reward, whereas ES minimizes energy consumption subject to a service constraint. The resulting ACCORD algorithm trains the applications with separate rewards and critics, applying a Lagrange multiplier only to ES and requiring no exchange of control parameters. ACCORD deploys fixed policies and discards its training multiplier and critics. Consequently, each policy is calibrated to a prescribed service requirement and training environment. This creates three deployment limitations. First, different service requirements require separately trained policies. Second, under light traffic, service degradation can remain substantially below the permitted level, limiting achievable energy savings; under overload, a separate guard is needed. Third, replacing the MLB partner can violate the service constraint, requiring ES recalibration and validation before deployment. These limitations motivate a policy that can adapt its operating point after training. We propose ATTUNE for post-deployment adaptation of coexisting ES and MLB applications. ATTUNE preserves the asymmetric service coupling of ACCORD and conditions the ES policy on the service-constraint multiplier. During operation, the multiplier is updated using measured service degradation while the network parameters of both applications remain fixed. During training, service targets are sampled uniformly and mapped to multipliers through an adaptive table. Building on state-augmented constrained reinforcement learning [6], [7], ATTUNE addresses three design challenges: selecting training multipliers that cover the desired service levels despite a policy-dependent response; training the MLB partner under the activation patterns induced by these multipliers, without providing it with the multiplier input; and adapting the operating point while controlling cell switching under dwell-time constraints. A. Related Work Primal-dual methods jointly update a policy and a Lagrange multiplier to enforce constraints [8], [9]. Zero-duality-gap results establish the validity of dual optimization under the stated regularity conditions, with approximation guarantees for sufficiently expressive policy parametrizations [10]. ACCORD applies primal-dual learning to the ES agent within an asymmetrically coupled game and discards the multiplier after training [5]. State augmentation incorporates dual variables into the policy input to enable adaptation during execution. Calvo-Fullana

IEEE TRANSACTIONS ON MOBILE COMPUTING

2

TABLE I C OMPARISON OF POLICY INPUTS , TRAINING CONDITIONS , AND DEPLOYMENT UPDATES OF RELATED METHODS AND ATTUNE. Method

Policy input

Training conditions

Deployment update

ACCORD [5]

Observation

Multiplier updated for fixed Vmax

None; fixed policy

Calvo-Fullana et al. [6]

State, multipliers

Sampled multipliers; Online dual distribution not update prescribed

NaderiAlizadeh et al. [7]

State, dual variables

Fixed multiplier Periodic dual sampling distribution update

CCPO [12]

State, threshold

Selected training thresholds

None; prescribed threshold input

ATTUNE

Observation, Uniform service multiplier (ES targets mapped to only) multipliers via an adaptive table

Measurementbased multiplier update; fixed network parameters

et al. [6] establish limitations of fixed reward scalarization in constrained problems and develop multiplier-conditioned policies with online dual dynamics. NaderiAlizadeh et al. [7] apply state augmentation to radio resource management with long-term per-user constraints. Their policies receive network states and dual variables, which are sampled from a fixed distribution during training and updated periodically during execution. Agorio et al. [11] extend state augmentation to multi-agent assignment, using dual-variable dynamics to coordinate task allocation. These formulations do not address the combination of asymmetric ES–MLB coupling, cell switching costs, and minimum dwell times considered here. Yao et al. [12] propose constraint-conditioned policy optimization (CCPO), which conditions a policy on the constraint threshold to support multiple safety requirements. Training uses data from selected thresholds, thresholddependent value estimates, and associated multipliers. At deployment, the policy receives the prescribed threshold without measurement-based input correction. CCPO considers an episodic discounted-cost constraint, whereas our setting imposes a long-run average service constraint on coexisting ES and MLB applications. Table I summarizes differences in policy inputs, training conditions, and deployment adaptation. ATTUNE combines adaptive target-uniform multiplier sampling with ES-only state augmentation, while retaining the MLB agent’s separate reward and observation. This design addresses requirements arising from ES–MLB coexistence: the multiplier-to-service response varies across trained policies, and the multipliers sampled for ES affect the activation patterns encountered by the MLB agent, which does not observe the multiplier. In addition, online adaptation changes discrete activation decisions subject to switching costs and minimum dwell times. These coupled training and operational requirements are not jointly addressed by the related methods.

We develop a state-augmented framework for postdeployment adaptation of asymmetrically coupled ORAN applications. ATTUNE conditions only the ES policy on the service-constraint multiplier and updates this scalar from measured degradation during operation. The MLB agent retains its own reward and observation, and both applications operate without network-parameter updates or control-parameter exchange. • We propose target-uniform multiplier sampling using an adaptive target-to-multiplier table. The table accounts for policy-dependent service responses and allocates training samples across the desired service range. We characterize the service-level distribution induced by log-uniform sampling and experimentally show that the proposed distribution improves ES performance and the effectiveness of the learned MLB partner relative to a widened fixed range. • We analyze the operational multiplier update to characterize target tracking and guide the selection of the update step size and period. Under the stated response and noise assumptions, diminishing-step updates converge to the target operating point or a range boundary. For constantstep operation, a local analysis quantifies the trade-off between response time and multiplier fluctuations, with a small-step limit connecting these results to the nonlinear projected update. An exact finite-window identity expresses average tracking error through the net multiplier change and projection corrections, while a dwell-time bound limits switching independently of the multiplier trajectory. • We evaluate adaptation across service requirements, traffic loads, and separately trained partners, including sequential changes during operation. A single ATTUNE policy achieves energy savings comparable to three specialized policies and approximately seven percentage points higher savings at light load than ACCORD. All 20 cross-seed pairs trained under the same load track their targets within ±0.005 without retraining, and a target margin yields service-feasible operation for all these pairs. Comparisons with fixed and feedback-corrected policy inputs isolate the role of measurement feedback, and no statistically significant increase in switching is detected in the evaluated online-update comparisons. Section II summarizes the network model and the constrained Markov game. Section III formulates the stateaugmented problem, Section IV presents the training and operation procedures, and Section V analyzes them. Section VI evaluates the approach, and Section VII concludes the paper. •

II. S YSTEM M ODEL AND C OEXISTENCE G AME This section summarizes the network model and asymmetrically constrained Markov game developed in [5]. The game provides the mathematical foundation for ATTUNE. A. Network Model and Performance Metrics

B. Contributions Our contributions are as follows.

A cluster of B cells serves U user equipment (UE) devices in slots of duration T0 . Traffic follows a diurnal profile that

IEEE TRANSACTIONS ON MOBILE COMPUTING

3

repeats every cycle of T slots, compressed relative to a physical day, and one episode spans one cycle. The ES-rApp sets the activation vector st ∈ {0, 1}B once every K slots, subject to a minimum dwell time Tdwell > K between state transitions of a cell and to the condition that every sleeping cell retains an active neighbor. The MLB-xApp adjusts the CIO vector θ t every slot by at most one quantization step per cell. Each UE in coverage associates with the active cell that maximizes its CIO-biased received power. A UE u with traffic demand du and achievable rate ru,t per physical resource block (PRB) requires du /ru,t PRBs to satisfy its demand. The normalized load of cell b, with Nb available PRBs and associated UE set Ub,t , is 1 X du . (1) ρb,t = Nb ru,t u∈Ub,t

Each UE u served by cell b receives a rate of du / max{ρb,t , 1}, with its PRB demand proportionally reduced when the cell is overloaded. We adopt an affine load-dependent power consumption model with an additional cell-switching cost. The network power consumption under activation vector st is Xh
Et = sb,t Pbfix + ∆b min{ρb,t , 1}Pbtx b

i X + (1 − sb,t )Pbsleep + ζ |sb,t − sb,t−1 |.

(2)

b

The terms account for load-independent active power Pbfix , load-dependent power with slope ∆b relative to the transmit power Pbtx , sleep-state power Pbsleep , and switching power. The coefficient ζ is the transition energy divided by T0 , and Ēt denotes the network power consumption normalized by the all-active, fully loaded power without transitions. Service degradation is defined as the fraction of UEs that are in outage or receive less than a fraction β ∈ (0, 1] of their traffic demand: X   1  out |Ut | + 1 ρb⋆ (u),t > 1/β , Vt = (3) U out u∈U / t

where Utout denotes the set of UEs not covered by any active ⋆

cell and b (u) denotes the serving cell of UE u. The handover rate Ht is the fraction of UEs that change their serving cell in slot t.

the start of epoch k using observation oE k , whereas MLB acts in each slot t using observation oM t . Both observations use preceding-slot measurements. ES observes cell loads, demand histories, its activation vector, and dwell timers; MLB observes cell loads, demand histories, and its CIO vector. Neither agent observes the other application’s control configuration. The MLB agent receives the slot-level reward RtM = −Vt − wh Ht , where wh weights the handover penalty. P The ES agent 1 receives the epoch-level reward RkE = − K t∈Tk Ēt , where Tk = {kK, . . . , (k + 1)K − 1}. The operator imposes the long-run average service constraint ′

T −1 1 X π E [Vt ] ≤ Vmax T →∞ T ′ t=0

C(π) = lim ′

(4)

on the joint policy π = (π E , π M ). The coupling is asymmetric: MLB directly penalizes service degradation in its reward, whereas ES minimizes energy consumption subject to the service constraint. The desired interaction is characterized by a generalized Nash equilibrium: a service-feasible joint policy at which ES cannot improve its return through a servicefeasible unilateral deviation, while MLB cannot improve its return through any admissible unilateral deviation [5], [13]. To handle the service constraint in this game, ACCORD introduces a Lagrange multiplier λ ≥ 0 into the ES policy update and uses the penalized advantage  E  1 Ak − µ E AC k − µC (5) ÂE = − λ . k 1+λ σE σC C Here, AE k and Ak denote the energy-reward and service-cost advantages, µE and µC their respective means, and σE and σC their respective standard deviations used for normalization.

III. S TATE -AUGMENTED F ORMULATION ATTUNE extends the coexistence game of Section II-B by conditioning the ES policy on the service-constraint multiplier. In ACCORD, the multiplier is used only during training; in ATTUNE, it also serves as a policy input that is updated during operation while both agents’ network parameters remain fixed, as illustrated in Fig. 1. This enables a single trained policy pair to adapt to different service targets and changing operating conditions without retraining. A. Augmented Energy Saving Policy

B. Asymmetrically Constrained Markov Game Proposition 1 of [5] identifies a two-cell configuration in which load balancing reduces a cell’s load below the ES deactivation threshold, while subsequent deactivation overloads the remaining cell. The myopic rules then alternate between activation states despite a service-feasible balanced configuration. Proposition 2 of [5] further shows that identical pre-deactivation observations can lead to different postdeactivation service outcomes. These findings motivate accounting for the service consequences of the applications’ coupled actions. Following [5], we model ES–MLB coexistence as an asymmetrically constrained Markov game with agents E and M representing ES and MLB, respectively. ES acts at

The agents use the rewards defined in Section II-B. We augment the ES observation with the multiplier, yielding the policy π E (s | oE k , λ). For each λ, the ES agent aims to maximize the Lagrangian return given the MLB policy: E M E C LE λ (π ; π ) = J (π) − λJ (π),

where π = (π E , π M ) and ∞ hX i J E (π) = Eπ γEk RkE , k=0

J C (π) = Eπ

∞ hX

(6)

γ t Vt

i

(7)

t=0

are the discounted energy-reward and service-cost returns, accumulated over epochs and slots, respectively, with discount factors γE , γ ∈ (0, 1). The cost return is a discounted surrogate

IEEE TRANSACTIONS ON MOBILE COMPUTING

4

(a) Training: target-uniform sampling

(b) Operation: closed-loop multiplier update Operator target V

Non-RT RIC

Adaptive multiplier table

Target sampler

Perturb, fix per episode

λg

Vj ∼ U[Vlo , Vhi ]

2

λj = λ(Vj ) e σξj grid vg λg

3

MLB agent

actor, critics: (oE , log λj )

actor, critic: oM table update

R1: λ

ES-rApp

ES agent

no multiplier input

Parallel episodes, one cycle each

Near-RT RIC MLB-xApp

λ ← clipΛ (λe α(Vˆ − V )/V )

vg

λj

advantage weighted by λj

Dual update, every period P

no exchange ×

π E (s | o Ek , λ) every K slots

t every slot

no multiplier input

1 O1: Vˆ

closed loop

4 on/off st

CIO t

control

Both actors: fixed network parameters

measurement

mean V̄ of environments nearest to each vg RAN cluster: measured degradation Vt Fig. 1. Overview of ATTUNE. (a) Training: each environment uniformly samples a service target, maps it to a multiplier through an adaptive table, and perturbs the multiplier. The multiplier is held fixed throughout the episode, and its logarithm conditions the ES actor and critics; the MLB agent receives no multiplier input. Measured degradation updates the table. (b) Closed-loop operation: (1) measure service degradation, (2) update the multiplier from its deviation from the operator’s target, (3) supply the multiplier to the ES-rApp, and (4) apply activation decisions that affect subsequent degradation. Both actors retain fixed network parameters, and the applications exchange no control configurations.

for the long-run average degradation in (4), and the Lagrangian return expresses the trade-off between energy saving and service degradation. In ACCORD, the multiplier is a training-time optimization variable used to learn a policy for a fixed service requirement. ATTUNE additionally treats the multiplier as a conditioning variable, learning a family of ES control behaviors represented by a single policy network. Each sampled multiplier conditions the policy and weights the service-cost advantage in (5), enabling the operating point to be adjusted after training by changing the input rather than the network parameters. The ES value and cost critics are also conditioned on λ because the policy they evaluate depends on it. Conditioning the policy on λ allows its operating point to be adjusted, but does not by itself identify the multiplier needed to meet a given service target. The degradation attained at a given λ depends on the trained policy and operating conditions. Moreover, separate advantage normalization in (5) makes the relative cost weight in the proximal policy optimization (PPO) update [14] equal to λσE /σC . ATTUNE therefore uses measured degradation to determine suitable multiplier inputs. During training, the adaptive table in Fig. 1(a) learns a mapping from service targets to multipliers. Targets are sampled uniformly, and each mapped multiplier is perturbed and held fixed throughout the episode; the resulting degradation measurements update the table. During operation, the feedback loop in Fig. 1(b) corrects the multiplier input according to the deviation of measured degradation from the prescribed target, while both actors’ network parameters remain fixed. P∞ The MLB agent maximizes J M (π) = Eπ [ t=0 γ t RtM ] using its own observation, without access to λ. Both agents are trained concurrently with separate rewards, while multiplier

conditioning and multiplier-based penalization are applied only to ES. B. Online Multiplier Update We specify the measurement-based multiplier update used to adjust the operating point of the trained policy pair during deployment. For fixed operating
conditions and an MLB policy π M , define V̄ (λ) = C πλE , π M , where πλE = π E (· | ·, λ) denotes the trained ES policy with its multiplier input held fixed at λ. Since the response V̄ (·) depends on the trained ES policy, traffic load, and MLB partner, a multiplier calibrated under the training conditions may be unsuitable after deployment. Let V ⋆ ∈ (0, Vmax ] denote the service-degradation target specified by the operator. Assuming that V̄ (λ) decreases with λ, we seek a multiplier λ⋆ ∈ Λ satisfying V̄ (λ⋆ ) = V ⋆ , provided such a multiplier exists. To track this target without explicitly estimating V̄ (·), ATTUNE updates the multiplier using the measured deviation from the target: 
 λn+1 = clipΛ λn exp α (V̂n − V ⋆ )/V ⋆ , (8) where n indexes update periods, V̂n is the average degradation measured over the nth period, and α > 0 is the step size. The deviation is normalized by the target V ⋆ . The operator clipΛ projects onto the operational range Λ = [λmin , λmax ]. Before projection, the update increases λ for positive deviations, decreases it for negative deviations, and leaves it unchanged for zero deviation. The multiplicative update implements integral control in log λ, accommodating multiplier values that span more than an order of magnitude across service levels.

IEEE TRANSACTIONS ON MOBILE COMPUTING

The operational range Λ = [λmin , λmax ] is determined from the final training table {λg } as   −2σ λmin = max λ, e min λg , g   (9) λmax = min λ̄, e2σ max λg ,

5

We initialize the target-to-multiplier table using servicedegradation measurements collected during warm-up. In this phase, multipliers are sampled log-uniformly from a fixed range and used to train the agents. Using measurements from the final warm-up iterations, we fit degradation as a linear function of log λ by least squares:

g

where λg is the multiplier at grid point g in the final training table, λ and λ̄ are the training bounds, and σ is the prescribed standard deviation of the log-space multiplier perturbation used during training. The range extends the table’s logmultiplier range by 2σ at each end while respecting these bounds; Section IV-A describes the table and perturbation. The update can start from any λ0 ∈ Λ, with the table value corresponding to V ⋆ providing an initial estimate under the training conditions. Section VI specifies the initialization used in each experiment. C. Design Requirements State-augmented learning samples multipliers during training and holds each multiplier fixed within an episode [6], [7]. Applying this framework to ES–MLB coexistence requires addressing the following issues in training and operation. The training distribution of multipliers should provide adequate coverage of the service targets of interest. Because V̄ (λ) varies across trained policies, a fixed multiplier range can provide insufficient target coverage or allocate many samples outside the desired service range. Multiplier sampling also affects the learning of the MLB partner. Only ES observes the multiplier, but both agents experience the resulting activation patterns. Excessive sampling outside the desired service range can therefore cause a mismatch between the activation patterns encountered by MLB during training and deployment. The operational update must account for cell switching behavior. Measurement variability can cause the multiplier to fluctuate around its target operating point, changing activation decisions and inducing reversals even when the dwell constraint is satisfied. IV. ATTUNE: T RAINING AND O PERATION This section presents ATTUNE’s training and operation procedures to address the design requirements in Section III-C. We describe how uniformly sampled service targets are mapped to training multipliers, how the online update parameters are selected, and how overload is handled. A. Training ATTUNE jointly trains the ES and MLB agents using PPO, with an adaptive table that maps service targets to ES multipliers. After warm-up, each training iteration consists of target sampling and multiplier assignment, rollout collection and PPO updates, and a table update based on measured service degradation. The table {(vg , λg )}G g=1 is defined on a target grid Vlo = v1 < · · · < vG = Vhi , where [Vlo , Vhi ] covers the service levels of interest with a margin on each side.

Vbfit (λ) = q0 + q1 log λ,

(10)

where q0 and q1 are the fitted intercept and slope. If q1 < 0, the fit indicates that degradation decreases with the multiplier. We invert this fitted response to initialize the multiplier for each grid target vg :    vg − q 0 . (11) λg = clip[λ,λ̄] exp q1 If q1 ≥ 0, we instead initialize the table multipliers with uniform spacing in log space over the warm-up range, assigning larger multipliers to stricter targets and clipping all entries to [λ, λ̄]. Both cases provide the initial table for the subsequent adaptive updates. Section VI-A specifies the warmup duration, sampling range, and fitting window. After initialization, we use the current table to assign a training multiplier to each parallel environment. At each training iteration, environment j samples a target Vj⋆ uniformly from [Vlo , Vhi ]. The corresponding multiplier is obtained by linear interpolation of the table entries in log space: for v ∈ [vg , vg+1 ], log λ(v) =

v − vg vg+1 − v log λg + log λg+1 . vg+1 − vg vg+1 − vg

(12)

The interpolated multiplier is perturbed multiplicatively, λj = λ(Vj⋆ ) exp(σξj ),

ξj ∼ N (0, 1),

(13)

and then clipped to [λ, λ̄]. The perturbation provides training coverage around the multiplier associated with each target. We use σ = 0.5, chosen before training ATTUNE from the standard deviation of log λ measured during online operation of the policy trained log-uniformly over the warm-up range, with α = 0.5 and V ⋆ = 0.05 (0.52); the trained ATTUNE policy shows 0.41 under the same update. The resulting λj is held fixed throughout the environment’s episode. We use the assigned multipliers to train the ES policy and critics jointly with the MLB agent. The ES actor and both ES critics receive log λj as an additional input. An affine transformation maps the logarithms of the warm-up range endpoints to 0 and 1, without clipping the transformed input. For each environment j, the ES actor update uses the penalized advantage in (5) with the same λj used to collect its rollout. The ES value and cost critics learn the corresponding reward and service-cost returns. MLB is trained concurrently using its own observations and reward, without multiplier conditioning. The base network architectures and PPO update procedure follow ACCORD [5], with the additional ES inputs described above. We then update the table using measured degradation to improve the correspondence between each target and its assigned multiplier. After each training iteration, environments are grouped by the grid point nearest to their sampled targets.

IEEE TRANSACTIONS ON MOBILE COMPUTING

6

Algorithm 1 ATTUNE Training Require: Target range [Vlo , Vhi ], grid {vg }, noise scale σ, smoothing coefficient κ, table step size η, update limit δ, and bounds [λ, λ̄] 1: Initialize both agents’ actors and critics, with log λ as an additional input to the ES actor and critics 2: Initialize the multiplier table {λg } through warm-up 3: for each post-warm-up training iteration do 4: for each environment j do 5: Draw Vj⋆ ∼ Unif[Vlo , Vhi ] and sample λj using (13); clip it to [λ, λ̄] and hold it fixed throughout the episode 6: end for 7: Collect rollouts in all environments using π E (· | oE , λj ) and π M (· | oM ) 8: Compute ÂE using (5) with each environment’s λj ; update both agents using PPO 9: Update Ĉg and λg using (14) for grid points with assigned environments 10: Restore monotonicity: λg ← ming′ ≤g λg′ 11: end for Algorithm 2 ATTUNE Operation Require: Trained actors; requirement Vmax , margin m, step α, period P , range Λ, initial λ0 1: V ⋆ ← Vmax − m; λ ← clipΛ (λ0 ) 2: for each slot t do 3: ES-rApp: at the start of each epoch k, select the activation vector from π E (· | oE k , λ) with dwell mask and projection M M 4: MLB-xApp: select aM t from π (· | ot ) 5: Measure Vt 6: if t ends an update period of length P then 7: Update λ by (8) with V̂ averaged over the period 8: end if 9: end for

For each grid point g with assigned environments, their mean degradation Ṽg updates the smoothed estimate Ĉg ← (1 − κ)Ĉg + κṼg , where κ is the smoothing coefficient. The table multiplier is adjusted according to the relative deviation from the grid target:  h  Ĉ − v i g g λg ← Π[λ,λ̄] λg exp clip[−δ,δ] η , vg

(14)

where η is the step size, δ limits the log-multiplier adjustment per iteration, and Π[λ,λ̄] projects onto the table bounds. Grid points without assigned environments skip this measurementbased update. After all grid-point updates, monotonicity is restored by a running minimum from the strictest target, λg ← ming′ ≤g λg′ , allowing ties without imposing minimum spacing. The table thus adapts to the service response of the policies being trained, subject to the perturbation, grid resolution, and multiplier bounds. Algorithm 1 summarizes the training procedure.

B. Operation During operation, both actors retain their trained network parameters. Only the scalar multiplier is updated using (8). The Non-RT RIC maintains the multiplier and passes it to the ES-rApp through the R1 interface. The MLB-xApp retains its original input and policy. Algorithm 2 summarizes the procedure. The following guidelines specify the update step size and period, the interpretation of range saturation, and the target margin used to accommodate finite-horizon variability. Update parameters: The step size α and update period P jointly determine response time and multiplier variability. Let u = log λ, F (u) = V̄ (eu ), and u† denote an interior target operating point. Near this point, Proposition 3 gives an approximate time constant τ ≈ P/(αa), where a = −F ′ (u† )/V ⋆ > 0 is the local response sensitivity. Adjacent entries of the trained table with distinct multipliers provide a finite-difference estimate of a. For a desired τ shorter than the time scale of operating-condition changes, select α = P/(aτ ) subject to αa < 2. If the averaging period is long compared with the correlation time of Vt , the measurement-noise variance satisfies ω 2 ≈ ν 2 /P for a constant ν 2 . Substitution into (21) gives Var(u) ≈ ν 2 /[a2 (V ⋆ )2 τ (2 − P/τ )]. Thus, P should allow sufficient averaging while remaining short relative to τ . The predicted standard deviation should also be compared with the distance from u† to the nearer boundary of U = [log λmin , log λmax ], particularly at stringent targets. These local relations do not fully describe large changes in operating conditions. Before projection, each update changes u by α(V̂n − V ⋆ )/V ⋆ , which is bounded below by −α. A decrease of ∆ in u therefore requires at least ⌈∆/α⌉ updates, whereas an increase greater than α in one update requires V̂n > 2V ⋆ . Section VI-A specifies the selected parameters, and Sections VI-C and VI-E assess the resulting response. Operational range: The multiplier is projected onto the range Λ defined in Section III-B. Persistent boundary saturation together with a sustained deviation from the service target indicates a possible mismatch between the operating conditions and the response range of the trained policy. Target margin: The operator sets V ⋆ = Vmax − m, with 0 ≤ m < Vmax , to allow for finite-horizon variability and projection effects. The margin is selected using degradation measurements over the intended evaluation horizon. When projection is inactive, Proposition 5 expresses the average tracking error through an endpoint term that vanishes as the horizon increases. Deployment validation therefore checks the margin and initial adaptation transient for the intended operating conditions and policy pairing. Remark 1 (Operation under overload): When traffic demand increases beyond the training conditions, the service target may become unattainable by the trained policy over Λ. Persistent saturation at λmax with degradation above the target indicates a possible need for recalibration or retraining. A traffic-based guard can provide a faster response to sudden overload by suspending cell deactivation when total traffic exceeds a selected percentile of its training distribution [5]. Section VI-D evaluates the guard and multiplier adaptation separately.

IEEE TRANSACTIONS ON MOBILE COMPUTING

7

V. A NALYSIS

on [F (umax ), F (umin )]. For any service interval [va , vb ] within this range, the fraction of samples assigned to that interval is

This section establishes how multiplier sampling affects service-target coverage and how the online update regulates service degradation. We characterize target tracking and range saturation, quantify the local trade-off between response time and multiplier fluctuations, and derive an exact identity for finite-window average tracking errors. We also bound cell switching under the dwell-time constraint. These results support the sampling design and the selection of operational parameters. We consider a fixed trained ES–MLB policy pair under fixed traffic conditions. Let u = log λ, and let F (u) denote the mean service degradation when the multiplier is held at λ = eu under the operating protocol considered. For the cyclebased evaluation used in this paper, each cycle starts from a prescribed initial state and spans one update period, giving # " T −1 1 X prescribed initialization, . (15) Vt F (u) = E λ = eu T t=0

With independent randomness across cycles, this expectation also equals the long-run average under the same initialization protocol, so F (u) = V̄ (eu ) with V̄ defined in Section III-B. For continuous operation without reinitialization, F (u) instead denotes the long-run average response; the expected degradation over an individual update period can also depend on its initial network state. Let U = [umin , umax ] = [log λmin , log λmax ]. The response-based results use the following assumptions, as specified in each proposition. (A1) F is continuously differentiable on U with F ′ (u) < 0. (A2) The degradation measured over update period n satisfies V̂n = F (un ) + εn , where E[εn | u0 , {V̂i }n−1 i=0 ] = 0 and E[ε2n ] ≤ ω 2 < ∞. Assumption (A1) specifies a differentiable, decreasing service response, providing the negative-feedback structure of the multiplier update and the response slope used in the samplingdensity and local-dynamics analyses. Assumption (A2) models the period-average degradation as a conditionally unbiased response measurement with bounded variance. Under the cycle-based evaluation protocol, prescribed initialization and independent randomness make the conditional mean equal to F (un ), while 0 ≤ Vt ≤ 1 ensures bounded variance. Without cycle resets, (A2) is an approximation whose accuracy depends on the effects of initial-state transients and temporal correlations over the update period. The finite-window tracking identity and dwell-time switching bound derived below do not require (A1)–(A2). A. Service-Target Coverage The following result characterizes the service-level distribution induced by log-uniform multiplier sampling and provides the rationale for ATTUNE’s target-uniform sampling. Proposition 1 (Service-level distribution): Under (A1), let u be drawn uniformly from U and let V = F (u) denote the corresponding mean service degradation. Then V has density pV (v) =

1
(umax − umin ) F ′ F −1 (v)

(16)

F −1 (va ) − F −1 (vb ) . (17) umax − umin is strictly decreasing, for v ∈

Pr{V ∈ [va , vb ]} = Proof: Since F [F (umax ), F (umin )],

Pr{V ≤ v} = Pr{u ≥ F −1 (v)} =

umax − F −1 (v) . umax − umin

Differentiation using (F −1 )′ (v) = 1/F ′ (F −1 (v)) gives (16). Taking the difference of the distribution function at vb and va gives (17). Equation (16) shows that the density of sampled mean service levels is inversely proportional to the response slope magnitude. Thus, service intervals of the same small width receive more samples where the response slope has small magnitude and fewer where it has large magnitude. In the evaluated policies, flat regions occur near stringent service levels, where most cells remain active. Moreover, because the response differs across trained policies, the same fixed multiplier range can allocate different fractions of samples to the desired service interval, as quantified by (17). Merely widening the multiplier range therefore does not ensure balanced coverage. ATTUNE addresses this imbalance by sampling V ⋆ uniformly and adapting the table that maps each target to a multiplier. With an exact inverse mapping and without perturbation or clipping, u = F −1 (V ⋆ ) gives F (u) = V ⋆ , yielding uniform coverage of mean service levels over the attainable target interval. In practice, table approximation, policy updates during training, perturbation, and clipping modify this ideal distribution. Away from the multiplier bounds, a small log-space perturbation with standard deviation σ produces a local service-level standard deviation of approximately σ|F ′ (F −1 (V ⋆ ))|. Thus, the table directs samples toward the service range of interest, while the perturbation provides coverage around the mapped operating points. B. Target Tracking and Range Saturation The following result establishes target tracking with diminishing steps when the prescribed service level is attainable within the operational range and characterizes boundary saturation otherwise. Writing (8) in u = log λ and allowing a diminishing step size gives   αn (18) un+1 = ΠU un + ⋆ (V̂n − V ⋆ ) , V where ΠU projects onto U . Since the exponential function is increasing, this projection is equivalent to clipping the multiplier to Λ. Proposition 2 (Target tracking and range saturation): Under (A1) 0 ∈ U and let the step sizes satisfy αn > 0, P and (A2), let uP 2 † α = ∞, and n n n αn < ∞. Then un → u almost surely, where   V ⋆ ≥ F (umin ), umin , u† = F −1 (V ⋆ ), F (umax ) ≤ V ⋆ < F (umin ), (19)   umax , V ⋆ < F (umax ).

IEEE TRANSACTIONS ON MOBILE COMPUTING

Consequently, F (un ) → F (u† ) almost surely, and the limiting degradation equals the target whenever V ⋆ ∈ [F (umax ), F (umin )]. Proof: Let h(u) = (F (u) − V ⋆ )/V ⋆ and Yn = (V̂n − ⋆ V )/V ⋆ . Then (18) becomes un+1 = ΠU (un + αn Yn ), a projected stochastic approximation of the form in [15, (5.1.3)]. We verify the conditions of [15, Thm. 5.2.1]. The step-size assumptions imply αn → 0 and satisfy the required stepsize conditions. By (A2), E[Yn | u0 , Yi , i < n] = h(un ), since Yi is an invertible affine transformation of V̂i . Thus, the conditional measurement bias is zero. By (A1), h is continuous and bounded on the compact interval U, and E[Yn2 ] ≤ 2 max h(u)2 + u∈U

2ω 2 < ∞. (V ⋆ )2

8

F is differentiable at u† with a = −F ′ (u† )/V ⋆ > 0. Consider the unprojected update in (18) with constant step α > 0 and the local response model V̂n = V ⋆ − V ⋆ a(un − u† ) + εn , where E[εn | Fn ] = 0, E[ε2n ] = ω 2 > 0, and Fn is generated by u0 and the preceding measurements. If E[u20 ] < ∞, the deviation en = un − u† satisfies α (20) en+1 = (1 − αa)en + ⋆ εn . V For 0 < αa < 2, its mean converges to zero as E[en ] = (1 − αa)n E[e0 ], and its variance converges to lim Var(en ) =

n→∞

These properties establish the continuity, moment, and noise conditions of the theorem. For the associated projected ordinary differential equation, an interior point is stationary when h(u) = 0, the lower boundary is stationary when h(umin ) ≤ 0, and the upper boundary is stationary when h(umax ) ≥ 0. Since F is strictly decreasing, these conditions give the unique stationary point u† in (19). Moreover, h(u) > 0 for every u < u† in U and h(u) < 0 for every u R> u† in U , so the drift points toward u u† . Define f (u) = − umin h(v) dv. Then f is continuously differentiable, h = −f ′ , and f is constant on the singleton stationary set {u† }, satisfying [15, (A5.2.6)]. The projected stochastic approximation theorem therefore yields un → u† almost surely. Continuity of F gives F (un ) → F (u† ), which equals V ⋆ whenever the target lies in [F (umax ), F (umin )]. Proposition 2 characterizes target attainment within the response range of the trained policy pair. If V ⋆ ∈ [F (umax ), F (umin )], the diminishing-step update converges to a multiplier that achieves the target. A target below this range leads to upper-bound saturation with degradation exceeding the target by F (umax ) − V ⋆ , whereas a target above this range leads to lower-bound saturation with degradation below the target. Under upper-bound saturation and with V ⋆ = Vmax − m, the limiting degradation exceeds the service requirement if and only if F (umax ) − V ⋆ > m. These outcomes describe the adjustment available through the multiplier for the fixed trained pair and operational range. ATTUNE uses a constant step size to remain responsive to changing operating conditions. The next subsection analyzes the resulting response time and fluctuations around an interior target. Proposition 5 subsequently characterizes finite-window average tracking, including the effect of projection.

We analyze the trade-off between response time and multiplier fluctuations under a constant step size through two complementary results. Proposition 3 derives explicit formulas for a local affine approximation without projection. Proposition 4 then establishes the corresponding leading-order behavior of the nonlinear projected update as the step size tends to zero. Proposition 3 (Local response time and multiplier fluctuations): Let u† = F −1 (V ⋆ ) lie in the interior of U, and suppose

.

(21)

For 0 < αa < 2 with αa ̸= 1, the decay envelope of the mean has time constant −1/ log |1 − αa| updates, approximately 1/(αa) for small αa. For αa = 1, the mean becomes zero after one update. Proof: Substituting the local response model into the unprojected update gives (20). Let ϕ = 1 − αa. Since en is Fn -measurable and the noise has zero conditional mean, E[εn ] = 0 and E[en εn ] = 0. Therefore, E[en+1 ] = ϕ E[en ], Var(en+1 ) = ϕ2 Var(en ) +

α2 ω 2 . (V ⋆ )2

When |ϕ| < 1, equivalently 0 < αa < 2, the mean converges to zero and the variance converges to α2 ω 2 /[(V ⋆ )2 (1 − ϕ2 )], yielding (21). For 0 < |ϕ| < 1, the mean envelope decays as |ϕ|n = exp(n log |ϕ|), giving the stated time constant. The approximation follows from − log(1 − αa) = αa + O((αa)2 ) for small αa. If ϕ = 0, the mean is zero for all n ≥ 1. Proposition 4 (Small-step limit of the projected update): Consider (18) with constant step α > 0 and u0 ∈ U. Suppose (A1)–(A2) hold with a fixed response F and a variance bound ω 2 independent of α. Let u† = F −1 (V ⋆ ) lie in the interior of U. Assume that the measurement noise is uniformly bounded over n and α and satisfies E[ε2n | Fn ] = ς 2 (un ), where Fn is generated by u0 and the preceding measurements, and ς 2 is continuous, independent of α, and positive at u† . Let a = −F ′ (u† )/V ⋆ > 0, and choose deterministic integer indices qα such that αqα /| log α| → ∞ as α → 0. Then Z α (t) =

C. Response Time and Multiplier Fluctuations

αω 2 (V ⋆ )2 (2a − αa2 )

uqα +⌊t/α⌋ − u† √ α

converges weakly on each bounded time interval to the stationary Ornstein–Uhlenbeck (OU) process ς(u† ) dW (t), V⋆ where W is a standard Wiener process. In particular,   uqα − u† ς 2 (u† ) √ ⇒ N 0, . 2a(V ⋆ )2 α dZ(t) = −aZ(t) dt +

(22)

(23)

IEEE TRANSACTIONS ON MOBILE COMPUTING

9

Proof: Let h(u) = (F (u) − V ⋆ )/V ⋆ and en = un − u† . The update is  α  un+1 = ΠU un + αh(un ) + ⋆ εn . V

(A10.1.4)–(A10.1.6), with Hurwitz matrix A = −a. For the shifted martingale difference δ M̃nα = εpα +n /V ⋆ , h i ς 2 (ũα ) n . E (δ M̃nα )2 | Fpα +n = (V ⋆ )2

We apply the constant-step diffusion theorem in [15, Thm. 10.1.1] after shifting the recursion beyond its initial transient. By (A1) and compactness of U , amin = minu∈U |F ′ (u)|/V ⋆ > 0. Since h(u† ) = 0, the mean value theorem gives en h(un ) ≤ −amin e2n . Projection does not increase the distance to u† , and the conditional mean of εn is zero. Consequently,

† The uniform moment bound implies ũα n → u in probability uniformly over n ≥ 0 as α → 0. Continuity of ς 2 therefore gives convergence of these conditional variances to ς 2 (u† )/(V ⋆ )2 . The same convergence holds with the neighborhood indicator required in (A10.1.7), since the probability of lying outside that neighborhood tends to zero. Finally, let q̃α = qα − pα . Since αpα = O(| log α|), the assumed condition on qα gives αq̃α → ∞. Applying [15, Thm. 10.1.1] to the shifted recursion at indices q̃α yields weak convergence of

E[e2n+1 ] ≤ (1 − 2αamin )E[e2n ] + α2 Γ, ω2 . Γ = max h(u)2 + u∈U (V ⋆ )2 For sufficiently small α, iteration yields E[e2n ] ≤ e−2αamin n (umax − umin )2 +

αΓ . 2amin

Define  | log α| , pα = 2αamin 

ũα n = upα +n .

Then, for all sufficiently small α, † 2 E[(ũα Γ n −u ) ] ≤ (umax − umin )2 + . α 2a min n≥0

sup

Thus, the normalized errors of the shifted recursion form a tight family, verifying (A10.1.3). We next verify concentration of the shifted interpolated process, as required by (A10.1.2). Fix a neighborhood of u† whose closure lies inside U, and stop the shifted recursion on leaving this neighborhood. Since h and the noise are bounded, individual proposed increments are O(α), so projection is inactive before exit for sufficiently small α. Over any fixed rescaled time interval [0, H], the martingale maximal inequality bounds the expected squared maximum of the accumulated noise by O(α). Together with the preceding initial-error bound, Lipschitz continuity of h, and a discrete Gronwall inequality, this gives an O(α) bound on the expected squared maximum of the stopped deviation. The exit probability therefore tends to zero, and sup 0≤t≤H

† ũα → 0. ⌊t/α⌋ − u − P

Hence the shifted process converges to the constant interior process u† , verifying (A10.1.2). For this shifted recursion, the delay indices in (A10.1.2) and (A10.1.3) can both be taken as zero. The remaining conditions follow from the drift and noise assumptions. Boundedness of h and the noise gives the uniform integrability in (A10.1.1) and the higher-moment bound in (A10.1.7). The conditional mean drift is h, which is continuously differentiable and independent of n and α, with h(u† ) = 0 and h′ (u† ) = −a < 0. These properties verify

ũq̃αα +⌊t/α⌋ − u† √ = Z α (t) α to the stationary solution
of (22). Its stationary distribution is N 0, ς 2 (u† )/[2a(V ⋆ )2 ] . Evaluation at t = 0 gives (23). Proposition 3 gives exact response-time and variance formulas for the affine model. For small αa and a fixed update period, the response time scales √ as 1/α, while the standard deviation of log λ scales as α. With an update period of P slots, the approximate time constant is τ = P/(αa), so the local sensitivity a determines how a given step size translates into adaptation speed. In this model, the mean approaches the target without alternating signs when 0 < αa < 1 and alternates around it when 1 < αa < 2. Proposition 4 connects these formulas to the nonlinear projected update. Under its assumptions, the normalized error process converges weakly to the stationary Ornstein– Uhlenbeck process in (22) as α → 0 after a sufficiently long initial transient. The limiting distribution gives the variance approximation ας 2 (u† )/[2a(V ⋆ )2 ], matching the leading term of (21) when ω 2 = ς 2 (u† ). The limiting process has time constant 1/a in the rescaled time t = αn, corresponding to approximately 1/(αa) updates. Thus, the small-step analysis establishes that the response-time and fluctuation scaling derived from the affine model also arises in the nonlinear projected update. For parameter selection, (21) provides a local approximation when the response is approximately affine and projection events are negligible. Its predicted standard deviation can be compared with the distance d = min{u† −umin , umax −u† } to assess the relevance of the multiplier bounds. Near a boundary, projection can substantially affect both fluctuations and average tracking error. The condition 0 < αa < 2 characterizes stability of the unprojected affine model; the small-step result does not extend this condition to arbitrary step sizes in the nonlinear update. These results inform the parameter-selection procedure in Section IV-B, and Section VI-C evaluates the approximation at the implemented step size. D. Finite-Window Service Tracking The following result gives an exact identity for the average tracking error of the nonlinear projected update over a finite

IEEE TRANSACTIONS ON MOBILE COMPUTING

10

window. Unlike the preceding response-based analyses, it holds for every measurement sequence without assumptions (A1)–(A2). Proposition 5 (Finite-window service tracking): Consider (18) with constant step α > 0, fixed target V ⋆ > 0, and u0 ∈ U. Define the projection correction at update n as α zn = un+1 − un − ⋆ (V̂n − V ⋆ ). V For every n0 ≥ 0 and N ≥ 1, ! n +N −1 n0 +N ⋆ X−1 1 0X V V̂n − V ⋆ = un0 +N − un0 − zn . N n=n αN n=n0 0 (24) The endpoint difference satisfies |un0 +N − un0 | ≤ umax − umin . Upper-bound clipping gives zn < 0, lower-bound clipping gives zn > 0, and zn = 0 when the proposed update lies in U. If projection is inactive throughout the window, then 1 N

n0 +N X−1 n=n0

V̂n − V ⋆ ≤

V ⋆ (umax − umin ) . αN

(25)

Proof: By the definition of zn , α un+1 − un = ⋆ (V̂n − V ⋆ ) + zn . V Summing over the window and rearranging gives (24). Both endpoints lie in U, so their difference is bounded in magnitude by umax − umin . Projection reduces a proposed update above umax , giving zn < 0, and increases one below umin , giving zn > 0. Otherwise, zn = 0. With no projection corrections, bounding the endpoint difference yields (25). Equation (24) separates the average tracking error into an endpoint contribution and a projection contribution. The endpoint contribution is O(1/N ) because the multiplier range is bounded. Consequently, over growing windows without projection, the average measured degradation approaches the target even if individual measurements and multipliers continue P to fluctuate. The same conclusion holds whenever n0 +N −1 zn → 0. Upper-bound corrections contribute N −1 n=n 0 positively to the average tracking error, whereas lower-bound corrections contribute negatively, explaining how range saturation can sustain a deviation from the target. For update periods of equal duration, the average of V̂n equals the slot-average degradation over the same window. With V ⋆ = Vmax − m and no projection, (25) gives the sufficient condition m ≥ V ⋆ (umax −umin )/(αN ) for this measured average to satisfy the service requirement. This worstcase condition uses the full operational range, whereas (24) gives the exact error from the realized endpoint change and projection corrections. Under (A2), the martingale-difference measurement errors have uniformly bounded second moments, so their averages converge to zero almost surely. Thus, for any fixed starting index n0 , the averages of V̂n and F (un ) have the same asymptotic behavior. E. Switching under the Dwell Constraint The following result bounds cell switching independently of the multiplier trajectory. It requires only the enforced dwell constraint and does not rely on assumptions (A1)–(A2).

Proposition 6 (Dwell-time switching bound): Suppose successive state transitions of each cell are separated by at least Tdwell slots. In any interval of L ≥ 1 consecutive slots, each cell undergoes at most   L (26) Ntr (L) = Tdwell state transitions and at most 

Ntr (L) Noff (L) = 2

 (27)

deactivations, regardless of the multiplier sequence. Proof: Partition the interval into ⌈L/Tdwell ⌉ consecutive blocks, each containing at most Tdwell slots. Two transitions within the same block would be separated by fewer than Tdwell slots, contradicting the dwell constraint. Each block therefore contains at most one transition, giving (26). Since transitions alternate between deactivation and reactivation, at most ⌈Ntr (L)/2⌉ can be deactivations. Proposition 6 gives a deterministic switching bound that remains valid under online multiplier adaptation. Over a cycle of T slots, the cluster undergoes at most BNtr (T ) transitions and BNoff (T ) deactivations. The dwell constraint limits switching frequency but still permits reversals after the dwell time expires. Whether multiplier adaptation increases switching within these bounds is evaluated in Section VI-C by comparing fixed-multiplier and online operation at matched degradation. VI. P ERFORMANCE E VALUATION We evaluate ATTUNE in terms of service-target tracking and energy savings under different operating conditions. Baseline comparisons and ablation studies assess the effectiveness of its training and deployment mechanisms. A. Setup Scenario: We evaluate a cluster of B = 7 small cells serving 100 UEs, with 70% initially located in two hotspots at nonadjacent outer cells. The hotspot demands peak half a cycle apart. A cycle contains T = 3600 slots of duration T0 = 1 s and represents a compressed diurnal profile. ES decisions occur every K = 60 slots, with a minimum dwell time of Tdwell = 300 slots. Service degradation is the fraction of UEs in outage or receiving less than β = 0.8 of their demand. We consider Vmax ∈ {0.03, 0.05, 0.10} and adopt the nominal traffic load and remaining network parameters of [5]. Training: We train for 1000 iterations using 16 parallel environments and five training seeds. The discount factors are γ = 0.99 and γE = 0.98, and the generalized advantage estimation parameter is λGAE = 0.95, following [5]. The effective discounted cost horizon is 1/(1 − γ) = 100 slots, whereas the table and operational updates use cycleaverage degradation. For ATTUNE, targets are sampled uniformly from [Vlo , Vhi ] = [0.025, 0.13]. The target grid is {0.025, 0.04, 0.06, 0.08, 0.10, 0.13}, with table-update parameters η = 0.05, δ = 0.05, and κ = 0.1, multiplier bounds [λ, λ̄] = [0.05, 200], and log-space perturbation standard

IEEE TRANSACTIONS ON MOBILE COMPUTING

11

TABLE II E NERGY- SAVING DIFFERENCE FROM SPECIALIZED ACCORD AT MATCHED DEGRADATION [ PERCENTAGE POINTS ], WITH 95% CONFIDENCE INTERVALS AND THE NUMBER OF SEEDS COVERING THE REFERENCE DEGRADATION . T HE FINAL ROW REPORTS SPECIALIZED ACCORD ENERGY SAVINGS [%]. Vmax = 0.03 (V̄ = 0.031) Training distribution

Difference

Fixed range [0.4, 15] +0.92 [−0.50, +2.34] Widened range [0.1, 50] −6.77 [−7.86, −5.81] Adaptive range −0.36 [−1.56, +0.80] Target-uniform (ATTUNE) +0.07 [−1.27, +1.45] Specialized ACCORD

21.1 [20.0, 22.1]

Vmax = 0.05 (V̄ = 0.045)

Vmax = 0.10 (V̄ = 0.101)

Seeds

Difference

Seeds

Difference

Seeds

2/5 4/5 5/5 5/5

+0.53 [0.00, +1.06] −8.46 [−9.10, −7.59] −1.13 [−1.94, −0.38] −0.32 [−1.11, +0.52]

4/5 4/5 5/5 5/5

−0.24 [−1.69, +0.91] −9.21 [−10.94, −7.52] −0.44 [−2.06, +0.97] +0.23 [−1.30, +1.59]

1/5 5/5 5/5 5/5

5/5

25.4 [24.9, 25.8]

5/5

32.0 [30.8, 33.3]

5/5

Reference degradation levels are the five-seed means of the specialized policies; differences include only seeds whose response curves cover the corresponding reference, as counted in the Seeds columns.

deviation σ = 0.5. During the first 200 iterations, multipliers are drawn log-uniformly from [0.4, 15]. Measurements from the final 50 warm-up iterations are used for the least-squares initialization described in Section IV-A. Evaluation: Fixed-multiplier response curves are evaluated over 150 cycles using the protocol of [5]. Online evaluation uses five parallel trajectories of 30 cycles each. Network states are reinitialized at the start of each cycle, while multipliers are retained between cycles. Unless otherwise specified, reported averages use the final 15 cycles of each trajectory. The default operational settings are an update period of one cycle, α = 0.5, and λ0 = clipΛ (1). The step size was selected beforehand using the fixed-range policy. We use m = 0, so that V ⋆ = Vmax , unless a target margin is explicitly specified. Open- versus closed-loop comparisons and cross-seed pairing experiments instead initialize the multiplier at the table value corresponding to V ⋆ . The observed initial transient in these fixed-condition evaluations lasts 2–9 cycles. For pairs trained under different loads, ATTUNE is additionally trained jointly at U = 75 with five seeds, using the same demand-input normalization as at U = 100. Policies compared: We construct baselines that incorporate the state-augmentation and threshold-conditioning principles summarized in Table I into a common ES–MLB setting. The methods share the base network architecture, PPO training procedure, MLB agent design, and common hyperparameters of [5]. Differences in policy inputs, training distributions, and deployment updates are specified below. Specialized: ACCORD policies trained separately for each service requirement Vmax [5], providing a reference for the energy savings achieved through target-specific training. • Fixed range: multiplier-conditioned policies based on the state-augmentation framework of [6], [7], using fixed-distribution multiplier sampling as in [7]. Multipliers are drawn log-uniformly from [0.4, 15] or, for the widened variant, [0.1, 50], and held fixed within each episode. These baselines assess fixed multiplier distributions within the common ES–MLB training setup. • Adaptive range: multiplier-conditioned policies whose log-uniform sampling range is adjusted during training so that its endpoints correspond to prescribed service levels. This baseline compares range adaptation with ATTUNE’s target-uniform sampling.

•

ATTUNE: the target-uniform training procedure of Section IV-A with the online multiplier update of Section IV-B. • Threshold-conditioned: a baseline following the threshold-conditioning principle of CCPO [12]. The ES policy receives a service threshold instead of λ, while retaining ATTUNE’s base architecture, target distribution, and training multiplier table. We evaluate both a fixed input V ⋆ and a feedback-corrected input initialized at V ⋆ and updated once
per cycle as in Vn+1 = clip[Vlo ,Vhi ] Vnin − kv (V̂n − V ⋆ ) . These variants distinguish the effect of the conditioning variable from that of deployment feedback. We use kv = 0.156, selected for a time constant of approximately 8 cycles based on the fixed-range policy; ATTUNE’s multiplier update has a time constant of 5–6 cycles. For separately trained application pairs, we also compare ATTUNE with ACCORD’s recalibration and validation procedure [5]. Under overload, we include ACCORD’s traffic-based guard as a reference. Metrics: Energy saving is measured relative to the all-active configuration, and V̄ denotes average degradation over the reported cycles. A pair is classified as tracking its target if |V̄ − V ⋆ | ≤ 0.005 and as service-feasible if V̄ ≤ Vmax . Switching is measured by the numbers of deactivations and reversals per cycle, where a reversal is a deactivation followed by reactivation of the same cell within Tdwell + K slots [5]. Energy savings at matched degradation are obtained by interpolating each policy’s fixed-multiplier trade-off curve at the nominal load. •

B. Service-Target Coverage We evaluate whether a single ATTUNE policy can cover multiple service levels with energy savings comparable to target-specific policies, and how the training distribution affects coverage and efficiency. Comparisons with specialized ACCORD use its five-seed mean degradation as the reference, with energy savings obtained by interpolating each augmented policy’s fixed-multiplier response curve. ATTUNE covers all three reference service levels for every training seed while achieving energy savings close to specialized ACCORD (Table II). The differences are +0.07, −0.32, and +0.23 percentage points for the references associated with

IEEE TRANSACTIONS ON MOBILE COMPUTING

12

(b) Efficiency at V¯ = 0.045

(a) Sample allocation

(c) Coverage at Vmax 0.03 0.05 0.10

88%

Fixed [0.4, 15] Widened [0.1, 50]

2/5 4/5 1/5

28%

4/5 4/5 5/5

Adaptive

63%

5/5 5/5 5/5

ATTUNE

67%

5/5 5/5 5/5

0 50 100 Training samples [%] < 0.025

∆ saving vs. specialized [pp]

Service level V [0.025, 0.03)

seeds/5

8 4 0

[0.03, 0.10]

(0.10, 0.13]

> 0.13

Fig. 2. Effect of the multiplier training distribution. (a) Share of training samples grouped by the mean degradation attained by the final policy at the sampled multiplier; the operational service range [0.03, 0.10] is highlighted. (b) Energy-saving difference from specialized ACCORD at V̄ = 0.045, with 95% confidence intervals. (c) Number of seeds whose response curves cover the specialized-policy reference degradation associated with each Vmax in Table II.

Vmax = 0.03, 0.05, and 0.10, respectively, with all confidence intervals containing zero. Thus, each trained ATTUNE policy supports the three evaluated service levels without separate training for each requirement. A fixed multiplier range presents a trade-off between service-level coverage and energy efficiency. The narrow range [0.4, 15] concentrates 88% of samples within the operational service range [0.03, 0.10] and achieves energy savings close to specialized ACCORD among seeds covering each reference. However, only four of five seeds cover V̄ = 0.045, and only one or two cover the other reference levels (Fig. 2(c)). Widening the range to [0.1, 50] improves coverage but allocates 51% of samples above V = 0.13 and only 28% within the operational service range. Its energy savings are 6.8–9.2 percentage points below specialized ACCORD across the three references, including a difference of −8.46 percentage points at V̄ = 0.045 (Fig. 2(b)). Thus, widening the multiplier range alone does not preserve energy efficiency. Adaptive-range and target-uniform sampling both cover all three reference levels for every seed. ATTUNE’s mean energy savings are closer to those of specialized ACCORD at all three references. At V̄ = 0.045, adaptive-range sampling gives a difference of −1.13 percentage points with confidence interval [−1.94, −0.38], whereas ATTUNE gives −0.32 with interval [−1.11, +0.52]. These intervals compare each method with specialized ACCORD; they do not establish a statistically significant difference between ATTUNE and adaptive-range sampling. The distinction between the two adaptive methods lies in sample allocation within the multiplier range. Adaptiverange sampling adjusts the endpoints but retains a log-uniform distribution, whose induced service-level density depends on the response slope as characterized by Proposition 1. ATTUNE instead samples service targets uniformly and learns their multiplier mapping. Fig. 2(a), which evaluates sampled multipliers

using the final policies, shows that 66.6% of ATTUNE samples fall within [0.03, 0.10], close to the 66.7% target-sampling probability implied by [0.025, 0.13]; adaptive-range sampling allocates approximately 63% to this interval. The achieved distribution remains subject to table approximation, perturbation, and clipping. Beyond sampling, ATTUNE’s learned table provides target-specific initial multipliers and responsesensitivity estimates for the operational parameter selection in Section IV-B. The training distribution also affects the MLB partner, even though MLB does not observe the multiplier. With the narrow-range ES actor held fixed, replacing its MLB partner with the widened-range MLB actor reduces energy saving at V̄ = 0.045 by 3.0 percentage points, with confidence interval [−4.35, −1.88]. The target-uniform MLB actor yields a smaller difference of −0.57 percentage points, with confidence interval [−1.46, +0.29]. These results support training the pair on activation patterns relevant to the intended service range. C. Closed-Loop Tracking and Switching The online update tracks all three service targets across the five training seeds. Starting from λ0 = clipΛ (1), every seed achieves average degradation within ±0.005 of each target. The means across seeds are 0.030, 0.049, and 0.098 for targets of 0.03, 0.05, and 0.10, respectively. The remaining average tracking error is accounted for by the endpoint and projection terms in Proposition 5. In windows without projection, the endpoint change over the final 14 updates is approximately 10% of the log-multiplier range, accounting for trajectory-average deviations of up to approximately 10% of V ⋆ . The endpoint contribution is bounded by a term that decreases as 1/N , and numerical residuals of the identity are below 10−16 in every run. Separate load-variation runs initialized at λ0 = clipΛ (1), rather than at the table value used in Fig. 3, illustrate the projection contribution: at U = 50, lower-bound saturation occurs in 24 of 25 trajectories and degradation remains below the target; at U = 125 and 150, upper-bound saturation accompanies degradation above the target. The accumulated projection corrections account for these sustained deviations. The measured multiplier fluctuations support the step-size trade-off characterized by Proposition 3. For the fixed-range policy trained on [0.4, 15], an eightfold step-size increase produces an approximately 3.5-fold increase in log-multiplier fluctuations, compared with 3.2 predicted by (21), while the local approximation overestimates their magnitude by 15– 30%. For ATTUNE at V ⋆ = 0.05, adjacent entries of the trained table give a median sensitivity of a = 0.33 across seeds, approximately consistent with the fixed-multiplier response curves. With α = 0.5 and a one-cycle update period, the table-based estimate gives a time constant of approximately 6 cycles; the response curves give approximately 5 cycles and a median αa ≈ 0.2. The predicted stationary standard deviation of log λ is 0.52; over 15- and 30-cycle windows, the predictions are 0.39 and 0.45, against measured values of 0.41 and 0.46. These comparisons assess the local approximation

IEEE TRANSACTIONS ON MOBILE COMPUTING

13

TABLE III C ELL SWITCHING : DEACTIVATIONS AND REVERSALS PER CYCLE , AND REVERSAL RATIOS . Vmax = 0.03 (V̄ = 0.031) Method

Vmax = 0.05 (V̄ = 0.045)

Vmax = 0.10 (V̄ = 0.101)

Off

Rev.

Share [%]

Off

Rev.

Share [%]

Off

Rev.

Share [%]

Rule-based pair [5] 6.66 Specialized ACCORD 7.88 ATTUNE, fixed λ 9.46 ATTUNE, online update 9.30

4.47 2.82 3.83 3.84

67 [52, 72] 36 [33, 39] 41 [38, 44] 41 [39, 44]

11.41 8.42 8.86 8.71

9.08 2.71 3.04 2.90

80 [76, 82] 32 [28, 36] 34 [32, 37] 33 [30, 36]

17.68 9.02 6.84 7.02

15.78 2.63 1.56 1.75

89 [88, 90] 29 [28, 31] 23 [20, 25] 25 [21, 29]

Off: deactivations per cycle; Rev.: deactivations followed by reactivation within Tdwell + K slots; Share: reversal-to-deactivation ratio, with 95% confidence intervals. The V̄ in each header is the specialized-policy reference. The rule-based pair and fixed-multiplier ATTUNE are interpolated at this degradation along the threshold and multiplier, respectively. Online ATTUNE uses V ⋆ = Vmax and attains V̄ = 0.030, 0.049, and 0.098 over the final 15 of 30 cycles.

D. Adaptation across Operating Conditions We evaluate whether deployment feedback maintains target tracking when traffic load or the MLB partner differs from the training conditions, and compare multiplier and threshold conditioning. All ATTUNE network parameters remain fixed. Table IV compares fixed inputs with feedback-adjusted inputs. Open-loop operation uses the table multiplier λ(V ⋆ ) or the prescribed threshold V ⋆ , while closed-loop operation adjusts the corresponding input using measured degradation. Measurement feedback improves target tracking under both nominal and changed operating conditions. For the original policy pairs, fixed multiplier and threshold inputs satisfy the tracking tolerance in 6/15 and 8/15 cases, respectively, compared with 15/15 for ATTUNE. Under load variation over U ∈ {50, 75, 100}, both open-loop variants satisfy the

Energy saving [%]

(a)

V¯

at the implemented step size, while Proposition 4 establishes its leading-order connection to the nonlinear update as α → 0. The nearer operational boundary lies 4.6–8.9 predicted standard deviations from the target operating point at V ⋆ = 0.05, compared with a median of 2.5 at V ⋆ = 0.03, highlighting the relevance of boundary proximity at stringent targets. Online multiplier adaptation produces no statistically significant increase in switching in the matched-degradation comparisons. Table III reports switching near the specializedpolicy reference levels: the rule-based pair reverses 67%– 89% of deactivations, compared with 41%, 33%, and 25% for online ATTUNE. In a separate seed-wise comparison at the degradation attained online, rather than at the reference levels of Table III, differences between online and fixed-multiplier ATTUNE in deactivations and reversals range from −0.19 to +0.15 per cycle, with confidence intervals containing zero. Proposition 6 gives a deterministic upper bound on switching, while these comparisons assess the additional switching associated with multiplier adaptation. The higher reversal count at the strictest target is observed with both fixed and updated multipliers. At Vmax = 0.03, ATTUNE reverses approximately one additional deactivation per cycle relative to the specialized policy, despite comparable energy savings. Handover rates remain close: 0.0123 [0.0117, 0.0129] for ATTUNE and 0.0126 [0.0122, 0.0130] for the specialized policy. Similar reversal counts with fixed and updated multipliers suggest that the additional reversals are associated with the learned policy rather than multiplier adaptation.

overload

40 ATTUNE (closed loop) Fixed λ (open loop) ACCORD ACCORD + guard

20 0

0.4 0.2 0.1 0.05 0.02 0.01

(b)

V = Vmax = 0.05

50

75

100 125 Number of UEs U

150

Fig. 3. Energy saving (a) and degradation (b) under load variation, with V ⋆ = Vmax = 0.05; shaded loads indicate overload. ATTUNE and the fixed-multiplier baseline start from λ(0.05) and report the final 15 of 30 cycles. ACCORD and its traffic-based guard are from [5]. Error bars show 95% confidence intervals.

tolerance in 3/15 cases, compared with 9/15 for ATTUNE. At U = 50 and 75, open-loop degradation remains between 0.012 and 0.016, substantially below the target of 0.05. The energy savings in Table IV are measured at the attained degradation and therefore do not represent matched-degradation efficiency comparisons. Under lighter traffic, ATTUNE uses the available service margin to increase energy savings. Fig. 3 evaluates policies trained at U = 100 under different UE counts, with perUE demand unchanged. At U = 50 and 75, ATTUNE saves 46.9% and 40.7% at degradation levels of 0.041 and 0.046, respectively. Corresponding savings are 42.4% and 34.8% with a fixed multiplier, and 40.1% and 33.5% with ACCORD; both approaches maintain degradation below 0.016. At U = 50, lower-bound saturation limits further adjustment toward the target. Under overload, multiplier adaptation reduces degradation but does not restore the service target. All evaluated methods, including the all-active configuration, exceed the requirement. ATTUNE attains degradation of 0.107 and 0.276 at

IEEE TRANSACTIONS ON MOBILE COMPUTING

14

TABLE IV O PEN - AND CLOSED - LOOP PERFORMANCE : TARGET TRACKING , SERVICE FEASIBILITY, DEVIATIONS EXCEEDING 0.005, AND MEAN ENERGY SAVING . Condition

Metric

Multiplier input

Threshold input

Closed (ATTUNE)

Open

Open (CCPO-style)

Closed

Same pairs (V ⋆ ∈ {0.03, 0.05, 0.10})

Within ±0.005 V̄ ≤ Vmax Above / below Saving [%]

15/15 14/15 0/0 26.2

6/15 11/15 2/7 25.4

8/15 9/15 3/4 25.5

14/15 8/15 0/1 25.8

Load (U ∈ {50, 75, 100})

Within ±0.005 V̄ ≤ Vmax Above / below Saving [%]

9/15 15/15 0/6 37.9

3/15 13/15 1/11 34.3

3/15 15/15 0/12 33.6

7/15 14/15 0/8 37.3

Cross pairs

Within ±0.005 V̄ ≤ Vmax Above / below Saving [%]

20/20 17/20 0/0 25.2

6/20 10/20 6/8 25.8

11/20 17/20 0/9 24.3

20/20 17/20 0/0 24.9

CCPO-style denotes a threshold-conditioned baseline adapted to the common ES–MLB training setup [12]. Multiplier inputs are initialized from each ES actor’s table at V ⋆ ; threshold inputs start at V ⋆ . Above / below counts cases with V̄ > V ⋆ + 0.005 or V̄ < V ⋆ − 0.005, respectively. Savings are averaged at the attained degradation, without matching V̄ across methods.

V¯ per pair

0.07

sd 0.0081

sd 0.0009

sd 0.0027

0.06 0.05

Vmax target ±0.005

0.04 Fixed λ (open loop)

ATTUNE (closed loop)

ACCORD recal. + validation

Fig. 4. Average degradation of 20 cross-seed application pairs with V ⋆ = Vmax = 0.05. The shaded band indicates the tracking tolerance of ±0.005. ACCORD uses pair-specific recalibration and validation [5]. TABLE V PAIRS TRAINED UNDER DIFFERENT LOADS , OPERATED AT U = 100 WITH V ⋆ = Vmax = 0.05 (20 CROSS - SEED PAIRS EACH ). Closed loop (ATTUNE) Pair (training load)

±0.005

V̄

Same load (100/100) ES 100, MLB 75 ES 75, MLB 100

20/20 18/20 12/20

0.049 25.2% 0.052 22.6% 0.055 22.4%

Open loop

Saving Rev. share ±0.005 34% 40% 51%

6/20 0/20 0/20

V̄ 0.051 0.077 0.135

U = 125 and 150, compared with 0.158 and 0.328 for a fixed multiplier, with correspondingly lower energy savings. ACCORD’s traffic-based guard attains degradation of 0.092 and 0.243 by suspending deactivation. The guard provides a separate mitigation mechanism, while persistent upper-bound saturation indicates that further multiplier increases within Λ are unavailable to correct the remaining degradation excess. Feedback also maintains target tracking when separately trained ES and MLB agents are paired. The fixed multiplier violates the service requirement in 10/20 cross-seed pairs, including six with degradation above V ⋆ + 0.005. ATTUNE keeps all 20 pairs within the tracking tolerance without retraining (Fig. 4). The across-pair standard deviation of V̄ is

0.0009, compared with 0.0027 after ACCORD recalibration and 0.0081 with fixed multipliers. Three ATTUNE pairs exceed the requirement by at most 0.0007 when m = 0; setting V ⋆ = 0.048 yields service-feasible operation for all 20 pairs. These results distinguish target tracking from strict service feasibility and demonstrate the role of the target margin. Recalibrated ACCORD achieves 26.0% energy saving at average degradation 0.045, whereas ATTUNE achieves 25.2% with target 0.05. Partner-specific recalibration therefore achieves higher savings and lower degradation in this comparison, while ATTUNE adapts without updating either actor’s network parameters. Measurement feedback improves tracking with either conditioning variable, but the resulting policies differ in their service response and energy efficiency. Feedback-corrected threshold conditioning meets the tracking tolerance in 14/15 originalpair cases and all 20 cross-seed cases (Table IV). Under overload, however, it reaches its most conservative input and attains degradation of 0.126 and 0.292 at U = 125 and 150, compared with 0.107 and 0.276 for ATTUNE. At matched degradation, it also achieves approximately 0.5 percentage points lower energy saving. Thus, both conditioning approaches benefit from deployment feedback, while the evaluated multiplierconditioned policy achieves lower degradation under overload and higher energy savings at matched degradation. Applications may also be trained under different loads. Pairing applications trained at U = 75 with those trained at U = 100 and operating them at U = 100, the fixed multiplier misses the tolerance in all pairs in both directions, with degradation of 0.077 and 0.135 (Table V). The online update brings 18 and 12 of 20 pairs within the tolerance, at about 2.7 percentage points lower saving than same-load pairs and with more reversals. When the ES application is trained under the lighter load, its operating range does not cover the multiplier required at the heavier deployment load for two of five seeds, which saturate at the upper bound as in Proposition 2. These results highlight the importance of covering anticipated deployment loads during ES training. Expanding

IEEE TRANSACTIONS ON MOBILE COMPUTING

V

0.10

(a)

Target

Base

Target

0.03

0.05

15

U = 75

U = 100

Partner swap

0.05 ATTUNE (closed loop)

0.00

Fixed λ (open loop)

Target V

(b) log λ

4 2 0

Saving [%]

40

(c)

30 20 0

40

80

120 Cycle

160

200

240

10-cycle windows containing only post-change measurements, so the earliest reporting time is 10 cycles after a change. The median time to satisfy the tolerance is 11 cycles after partner replacement and 18–19 cycles after load changes and target restoration, which require log-multiplier changes of approximately 2. These longer transients are qualitatively consistent with the finite update increments discussed in Section IV-B, although the reported tracking times also include the evaluation window. In an otherwise identical experiment with 20-cycle segments, only two of five seeds satisfy the tracking tolerance over the final 10 cycles after target restoration and after load reduction. Both segments therefore fall below the criterion of at least four successful seeds, with substantial transients remaining in the evaluation windows. These results highlight the need to select the update period and step size according to the time scale of operating-condition changes. If one compressed cycle is interpreted as one physical day, the observed 18–19-cycle tracking time corresponds to approximately three weeks. This illustrative conversion depends on the cycle mapping; the operational parameters must therefore be selected for the intended response time and measurement variability. VII. C ONCLUSION

Fig. 5. Adaptation to sequential changes at 40-cycle intervals: (a) degradation as a five-cycle moving average and the prescribed target; (b) log-multiplier; (c) energy saving. Curves show five-seed means; shading shows the ATTUNE seed range.

the operational multiplier range alone does not guarantee target attainment, because the policy response outside the training range has not been established. E. Adaptation to Sequential Changes We evaluate whether ATTUNE maintains target tracking through successive changes in the service target, traffic load, and MLB partner. Each training seed is evaluated over a 240cycle run comprising six 40-cycle segments. Following the nominal segment, the target is reduced to 0.03 and restored to 0.05, the load is reduced to U = 75 and restored, and the MLB agent is replaced by a cross-seed partner. The multiplier is retained across segment boundaries, while network states are reinitialized each cycle as in the preceding evaluations. The fixed-multiplier baseline changes its table input only when the prescribed target changes. ATTUNE adapts to successive changes without resetting the multiplier or retraining the policies. Over the final 10 cycles of each segment, at least four of five seeds satisfy the tracking tolerance (Fig. 5). The fixed-multiplier baseline adjusts its input for prescribed target changes but provides no feedback correction for load or partner changes. After the load reduction, its degradation remains at 0.019, with no seed within tolerance, whereas ATTUNE achieves approximately 41% energy saving, compared with approximately 35% for the baseline. After partner replacement, all five ATTUNE seeds satisfy the tolerance, compared with one seed for the baseline. Adaptation takes longer when the new operating point requires a larger multiplier adjustment. Tracking is assessed over

We proposed ATTUNE to adapt a trained ES–MLB policy pair to different service targets and operating conditions without retraining. ATTUNE combines target-uniform multiplier sampling with measurement-based feedback that adjusts the ES policy input while keeping both actors’ parameters fixed. The analysis characterizes service-target coverage and tracking dynamics, providing guidance for selecting the update step size and period to balance adaptation speed and multiplier fluctuations. Simulations show energy savings comparable to target-specific policies and approximately seven percentage points greater savings than ACCORD under light load. All 20 cross-seed pairs trained under the same load track their targets within ±0.005, and a target margin yields servicefeasible operation for all these pairs. These results support multiplier feedback as a practical mechanism for adapting coexisting applications without policy retraining. Future work will improve generalization across applications trained under different traffic conditions, investigate service guarantees, and extend ATTUNE to multiple interacting applications. R EFERENCES [1] M. Polese, L. Bonati, S. D’Oro, S. Basagni, and T. Melodia, “Understanding O-RAN: Architecture, interfaces, algorithms, security, and research challenges,” IEEE Communications Surveys & Tutorials, vol. 25, no. 2, pp. 1376–1411, 2023. [2] O-RAN Alliance, “O-RAN working group 1: Use cases detailed specification,” O-RAN Alliance, Technical Specification O-RAN.WG1.UseCases, 2025. [3] A. Akman, P. Oliver, M. Jones, P. Tehrani, M. Hoffmann, and J. Li, “Energy saving and traffic steering use case and testing by O-RAN RIC xApp/rApp multi-vendor interoperability,” in 2024 IEEE 100th Vehicular Technology Conference (VTC2024-Fall). IEEE, 2024, pp. 1–6. [4] M. Hoffmann, S. Janji, A. Samorzewski, Ł. Kułacz, C. Adamczyk, M. Dryjański, P. Kryszkiewicz, A. Kliks, and H. Bogucka, “Open RAN xApps design and evaluation: Lessons learnt and identified challenges,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 2, pp. 473–486, 2024.

IEEE TRANSACTIONS ON MOBILE COMPUTING

[5] Y. Jeong, W. Ko, and S. Hong, “Learning to coexist: Asymmetrically constrained multi-agent reinforcement learning for conflicting applications in Open RAN,” submitted to IEEE Transactions on Communications, 2026. [6] M. Calvo-Fullana, S. Paternain, L. F. O. Chamon, and A. Ribeiro, “State augmented constrained reinforcement learning: Overcoming the limitations of learning with rewards,” IEEE Transactions on Automatic Control, vol. 69, no. 7, pp. 4275–4290, 2024. [7] N. NaderiAlizadeh, M. Eisen, and A. Ribeiro, “State-augmented learnable algorithms for resource management in wireless networks,” IEEE Transactions on Signal Processing, vol. 70, pp. 5898–5912, 2022. [8] C. Tessler, D. J. Mankowitz, and S. Mannor, “Reward constrained policy optimization,” in Proc. International Conference on Learning Representations (ICLR), 2019. [9] A. Ray, J. Achiam, and D. Amodei, “Benchmarking safe exploration in deep reinforcement learning,” OpenAI, Tech. Rep., 2019. [10] S. Paternain, L. F. O. Chamon, M. Calvo-Fullana, and A. Ribeiro, “Constrained reinforcement learning has zero duality gap,” in Advances in Neural Information Processing Systems (NeurIPS), 2019. [11] L. Agorio, S. Van Alen, M. Calvo-Fullana, S. Paternain, and J. A. Bazerque, “Multi-agent assignment via state augmented reinforcement learning,” in Proc. 6th Annual Learning for Dynamics and Control Conference (L4DC), ser. Proceedings of Machine Learning Research, vol. 242, 2024. [12] Y. Yao, Z. Liu, Z. Cen, J. Zhu, W. Yu, T. Zhang, and D. Zhao, “Constraint-conditioned policy optimization for versatile safe reinforcement learning,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2023, pp. 12 555–12 568. [13] F. Facchinei and C. Kanzow, “Generalized Nash equilibrium problems,” Annals of Operations Research, vol. 175, no. 1, pp. 177–211, 2010. [14] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. [15] H. J. Kushner and G. G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, 2nd ed. New York: Springer, 2003.

16

Related documents

• Adaptive Policy Backbone via Shared Network#1327266
• Converter-Grid Interaction Stability Guaranteed Safe Deep Reinforcement Learning for Energy Storage Systems in Grid Frequency Support#1122813
• MMLA: Memory-Mediated Learning Architecture for Predictive Dual-State Adaptation#1109089
• RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms#1448486
• RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms#1442336