Queue-Aware ISAC Power Allocation Under Imperfect CSI: Integer-Service Analysis and Anchored Lyapunov Look-Ahead

1

Queue-Aware ISAC Power Allocation Under Imperfect CSI: Integer-Service Analysis and Anchored Lyapunov Look-Ahead

arXiv:2610.25307v1 [cs.NI] 18 Oct 2026

Hanyi Sun, Yifan Zhou, Yixiao Gu, Member, IEEE

Abstract—We study power allocation for a multiuser integrated sensing and communication (ISAC) downlink with bursty queues and imperfect channel state information (CSI). Starting from a long-run utility–backlog objective, we derive its common task reward and online score. Integer departures are represented by threshold events of indefinite Gaussian quadratic forms, accounting for estimation error in both signal and interference. Their moments give the conditional one-slot task reward and characterize the error of an efficient plug-in score. An anchored quadratic Lyapunov look-ahead corrects its reward-greedy action with a bounded nominal loss. Across five CSI-error levels, the rule reduces mean queue exceedance by 27.4–43.9% (0.49–0.99 percentage points) relative to the plug-in maximizer, with mean reward loss below 0.01 per slot. The rule achieves higher reward and lower exceedance than DPP, SAC and PPO; its advantage over freshly retrained learning baselines extends to five configurations with 9–23 power-allocation variables using fixed candidate budgets. For the seven-variable system, a fused implementation reproduces all 25 confirmation trajectories and achieves a one-core p99 decision time of 0.274–0.311 ms. Index Terms—Integrated sensing and communication, imperfect CSI, integer service, Lyapunov optimization, queue-aware power allocation.

the conditional-service approach. DRL supports ISAC allocation [10]–[12]; Lyapunov-guided learning uses DPP decisions as targets [13] or its negative objective as the SAC reward [14]. Our objective combines communication–sensing utility with aggregate backlog cost. Integer-service moments give its conditional reward; an efficient plug-in score then provides the anchor for a queue-distribution correction. The contributions are: 1) Full-SINR integer-service analysis maps CSI uncertainty to departure moments and conditional task reward, with Cantelli bounds and a convergent spectral probability series. 2) An anchored quadratic backlog correction refines the rewardaligned scheduler. Its nominal loss is at most ρ; the conditional loss adds the score-error oscillation, following the h-myopic principle [15]. 3) Common-stream experiments cover CSI errors, traffic variations and 9–23 allocation variables, with freshly retrained learning baselines and a trajectory-verified CPU implementation. II. S YSTEM M ODEL AND C ONTROL O BJECTIVE

I. I NTRODUCTION

An N -antenna BS serves M single-antenna users and K Integrated sensing and communication (ISAC) shares spec- sensing targets in slots of length ∆t . It transmits x = Wc sc + trum, power and hardware [1]. With bursty traffic and imperfect Ws ss with total power at most Pmax . channel estimates, powers chosen from estimated channels produce uncertain integer packet departures. A favorable A. Communication Service aggregate reward can also leave individual queues congested. User m has channel hm = ĥm +em , where ĥm is the MMSE These two effects motivate a reward-aligned rule with explicit estimate and em ∼ CN (0, σe2 INp ) is independent of it and of control of its queue correction. other users’ errors. Here σe = σe2 and σe2 = 1/(1 + Tp γ̄p ) Beamforming optimizes communication and sensing metfor Tp pilots at SNR γ̄p . The realized SINR is rics [2], [3], while Lyapunov DPP [4] addresses long-term 2 constraints. Zakeri et al. [5] consider single-user, single-target |hH m wc,m | P γm = P . (1) H 2 H 2 2 precoding with an average-SINR virtual queue. Yang et al. [6] j̸=m |hm wc,j | + k |hm ws,k | + σc allocate power among vehicular V2I, V2V and sensing beams, using traffic queues and sensing mutual information. We Replacing hm by ĥm gives γ̂m . With ζ = B∆t /(Lp ln 2), consider a multiuser BS downlink where bursty communication offered service is µcm = ⌊ζ ln(1 + γm )⌋ packets of Lp bits; and sensing-task queues share power under imperfect CSI. Here, the BS predicts Jm = ⌊ζ ln(1 + γ̂m )⌋. This is capacity-type the same channel error perturbs signal and all interference, service under realized SINR, with estimated-CSI decisions; changing integer departures and the next-slot backlog penalty. decoding errors and retransmissions are not modeled. Expected-service scheduling [7], OFDM delay–power–CSIT analysis [8] and imperfect-CSI rate adaptation [9] establish B. Sensing Service and Traffic Queues The authors are with the School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China (e-mail: sunhanyi05@shu.edu.cn; zhouyifan@shu.edu.cn; yxgu@shu.edu.cn). (Corresponding author: Yixiao Gu.)

Target k has response Gk = ςk αk aR (θk )aH T (θk ), with independent per-slot αk ∼ CN (0, 1). After communicationclutter suppression, the BS knows ηk = ∥Gk Ws ∥2F /σs2 , and serves µsk = µ0 1{ηk ≥ ηmin } tasks.

2

All M + K queues have unbounded buffers and update as where both sums run over j = 1, . . . , Qi . For communicaQi (t + 1) = Qi (t) − Di (t) + Ai (t), where Di = min{Qi , µi }. tion, pmj has a spectral series with a computable truncaEach user has an independent hidden two-state Markov mode tion remainder; for sensing it is deterministic. At σe2 = 0, setting its Poisson arrival rate (MMPP); sensing arrivals are pmj = 1{j ≤ Jm }. independent Poisson(λs ). Arrivals are independent of current Rearranging the full SINR proves (4); integer tail sums give (6). channel errors. Exceedance is the fraction of slots in which Appendix A derives (5), the Cantelli error bound and the any queue exceeds Qth . spectral series [16]. C. Objective, Reward and Power Allocation et + βSt combine communication and sensing Let Ut = αR utility, and let C(q) = ν1 q + ν2 q 2 price backlog. We seek a causal policy π maximizing T −1

1 X Eπ [V Ut − δZt − C(q̄t+1 )], T →∞ T t=0

max lim inf π∈Π

(2)

B. Conditional Reward and Online Score P Define λP i = E[Ai | Ft ], φ = i ϖi (Qi + λi − E[Di ]), P 2 σA = Var( i ϖi Ai | Ft ) and vD = i ϖi2 Var Di . Proposition 1 (Exact conditional reward): The reward in (3) satisfies 2 G(a) = V U − δZ − ν1 φ − ν2 (φ2 + σA + vD ).

(7)

independent departures give Eq̄ = φ and where Π uses pre-service queues, estimated channels, sensing Conditionally 2 Var q̄ = σ + v , establishing (7). The MMPP arrival mean λi D A responses and past arrivals, denoted Ft . The per-slot contribufollows a causal forward filter. To avoid evaluating all service tion defines the common task reward probabilities online, use stationary means λ̄i and predicted 2 rt = V Ut − δZt − ν1 q̄t+1 − ν2 q̄t+1 . (3) departures in P −1 Here q̄t+1 = for J0 (a) = V U − δZ − ν1 φ̂ − ν2 φ̂2 , i ϖi Qi (t + 1), with ϖi = (2M Qth ) X −1 e (8) users and (2KQth ) for targets. The normalized rate is R = φ̂ = ϖi (Qi + λ̄i − Jˆi ). P −6 log (1 + ω γ̂ )/(M log (1 + N ) + 10 ), where ω = m m m 2 2 m i ∥ĥm ∥2 /(∥ĥm ∥2 + N σe2 + 10−12 ). The sensing success fraction Thus J0 approximates the objective’s one-slot contribution. For is S; Z is the mean normalized shortfall (1 − ηk /ηmin )+ . e = G − J0 and dφ = φ − φ̂, MRT directions for users and unit-norm steering directions 2 e = −[ν1 + ν2 (φ + φ̂)]dφ − ν2 (σA + vD ). (9) for targetsPare fixed. An action a ∈ [−1, 1]M +K sets pi = ai aℓ Pmax e / ℓ e . At fixed power fractions, every SINR and Appendix B bounds these service errors using Theorem 1. The sensing SNR is nondecreasing in total power; service and same probabilities also give next-slot exceedance: reward increase or stay unchanged. All methods therefore use full power. Online control uses the conditional one-slot reward Pi (a) = F̄i (Qth − Qi ) Qi G(a) = E[rt | Ft , a] derived below to construct an efficient X (10) score and a bounded queue correction. − bi (Qth − Qi + j)pij (a), j=1

III. C ONDITIONAL S ERVICE AND R EWARD

where bi (k) = Pr(Ai = k | Ft ) and F̄i (k) = Pr(A Q i > k | Ft ). Independence gives any-queue risk P = 1 − i (1 − Pi ); its P H derivation is in Appendix B. For a fixed action, set Bm = u̸=m wc,u wc,u + P H 2 j/ζ − 1 and Hmj = k ws,k ws,k , n = σc > 0, τj = e H IV. A NCHORED LYAPUNOV L OOK -A HEAD wc,m wc,m − τj Bm . Write pij = Pr(µi ≥ j | Ft , a) and s ˆ A. Reward Anchor and Queue Correction Ji = min{Qi , Ji }, with Ji = µi for targets. The current errors are independent of Ft . All moments below condition on this The aggregate penalty in (3) can assign equal costs to information and the action. different queue distributions. To distinguish concentrated Theorem 1 (Integer departures under imperfect CSI): Com- backlog, define post-decision queues [17] yi (a) = Qi − Jˆi (a) P 2 munication service obeys the full-SINR threshold equivalence and potential h(y) = 1 i (yi /Qth ) . Following h-myopic 2 control [15], the potential guides a correction to the reward{µcm ≥ j} = {Xmj ≥ 0}, Xmj = hH m Hmj hm − τj n. (4) greedy anchor a0 ∈ arg maxa∈A J0 (a):
Its mean and variance are dρ (a) = clip χ[h(y(a0 )) − h(y(a))], −ρ, ρ , H 2 (11) x̄mj = ĥm Hmj ĥm + σe tr Hmj − τj n, a1 ∈ arg max{J0 (a) + dρ (a)}. (5) a∈A 2 2 2 4 2 smj = 2σe ∥Hmj ĥm ∥ + σe tr(Hmj ). Here clip(x, l, u) = min{u, max{l, x}}; both maximizations For every queue, the actual-departure moments are select the first maximizer. The parameters χ ≥ 0 and ρ ≥ 0 X X E[Di ] = pij , E[Di2 ] = (2j − 1)pij , (6) control the correction weight and nominal loss budget. The reward objective supplies J0 , while h redistributes service j j toward larger residual queues. A. Integer-Service Thresholds

3

B. Loss Control and Candidate Selection

Anchored

Proposition 2 (Anchored one-slot loss): On the same finite candidate set, let δ0 = J0 (a0 ) − J0 (a1 ) and osc(e) = max e − min e. Then

V. P ERFORMANCE E VALUATION A. Setup and Comparison Protocol The base setting is N = 8, (M, K) = (4, 3), B = 100 MHz, Pmax = 1 W, Lp = 1000 bits, ∆t = 0.5 ms, σc2 = σs2 = 10−10 W, ηmin = 10 dB and µ0 = 5. Reward parameters are (α, β, δ) = (0.5, 1, 10), V = 10, (ν1 , ν2 ) = (5, 20) and Qth = 100. Rayleigh channels have hm ∼ CN (0, IN ) and ĥm ∼ CN (0, (1 − σe2 )IN ); the normalized downlink is predominantly interference-limited. Targets at (−30, 0, 25)◦ have ςk2 = (1, 1.2, 0.9)10−7 . Channels and target gains renew each slot. MMPP rates are 30/65 packets/user/slot, with normalto-burst/reverse probabilities 0.05/0.1; mean arrivals are 125/3 per user and λs = 3 per target. Each CSI level uses three validation and five 10000-slot confirmation streams from empty queues. All methods share channels, errors, target gains and arrivals. DPP maximizes P Q J + κd (V U − δZ), selecting κd = 104 among i i i 0 5 10 , . . . , 10 by validation reward. Our original RLD-SAC design uses SAC [18]; clipped-surrogate PPO follows [19]. Both use three 256-unit Gaussian–tanh actor layers, estimated channels, one column per Gk , Qi /101 and σe2 . Five seeds per CSI level train 200000 steps in 200-slot terminal episodes; final checkpoints are evaluated. Historical four-candidate searches used two seeds and 50000 steps under power back-off, selecting SAC rate 3 × 10−4 /batch 256 and PPO rate 10−4 /clip 0.1/four epochs/batch 256. SAC adapts entropy; PPO uses none. SE uses realized SINR. FIFO residence uses completed communication packets; the five-CSI runs leave < 0.026% unfinished, and packet conservation holds. B. Reward–Queue Trade-off Table I and Fig. 1 place Anchored between π0 and unclipped h-myopic control. Anchored lowers exceedance on all 25 paired streams relative to π0 ; the paired reward losses in Fig. 1(b) average 0.0028–0.0089 per slot. Clipping preserves sensing success while accepting a smaller reduction in exceedance than the unbounded correction. The growth check (mean

RLD-SAC

PPO

(a) Exceedance across CSI levels

1 0.5 0.1 0.05 0

0.01

0

0.01

0.05

0.1

0.2

0.05

0.1

0.2

(b) Mean reward loss relative to π0 0.04 0.03 0.02 0.01 0.00

CSI-error variance σe2

Fig. 1. (a) Exceedance (log scale); SDs follow Table I. (b) Reward loss relative to π0 ; SDs use five paired-stream differences. CSI levels use equally spaced positions.

communication backlog > 100, or last-2000-slot mean > 2 times the first-2000-slot mean) passes on all 25 anchored confirmation streams. DPP’s lowest validation exceedance across six weights is 15.4%; all 50 learned checkpoints are outperformed by π0 in reward and exceedance. We retain V = 10 as a common utility–queue compromise: relative to V = 5, Anchored gains 0.087 bit/s/Hz in SE for a 0.118-percentage-point exceedance increase (0.814% to 0.932%); at V = 20, exceedance reaches 2.612% (Fig. 2). The sensitivity check fixes χ = 99 and ρ = 0.5. π0 6

(a) Queue exceedance

4 2 0

Anchored 7.3

SE (bit/s/Hz)

Since dρ (a0 ) = 0, maximization gives J0 (a1 ) + dρ (a1 ) ≥ J0 (a0 ) and hence δ0 ≤ ρ. Substituting G = J0 + e proves the conditional bound. At ρ = 0 the rule is π0 ; ρ = ∞ gives arg max(J0 − χh). The base system uses A = {−1, 0, 1}7 ; larger systems use a prescribed pool C. Relative to the complete grid, nominal loss is at most εC + ρ, where εC = maxA J0 − maxC J0 . Before validation, we fixed ρ = 0.5 and χ ∈ {0.99, 9.9, 99, 990}. Validation selects a common χ minimizing exceedance subject to mean reward loss ≤ 0.01 at every CSI level and the growth check; ties use reward then smaller χ. It selects 99: peak mean losses are 0.00924 for 99 and 0.01909 for 990.

DPP

10 5

Exceedance (%)

a

(12)

Reward loss / slot

max G(a) − G(a1 ) ≤ δ0 + osc(e).

Exceedance (%)

0 ≤ δ0 ≤ ρ,

50

h-myopic

π0

5

10

Utility weight V

(b) Spectral efficiency

7.1 6.9

20

5

10

Utility weight V

20

Fig. 2. Sensitivity to V at σe2 = 0.05: (a) exceedance and (b) SE. Both rules use five common streams; error bars show SDs. Anchored keeps χ = 99 and ρ = 0.5 fixed.

C. Traffic and System-Scale Variation At σe2 = 0.2, scaling MMPP rates by L = 0.8, 1, 1.2 gives exceedance reductions of 56.8/26.9/9.2% with unchanged rule weights; absolute reductions grow from 0.150 to 1.932 percentage points. Unequal user-rate multipliers (1 + d, 1 + d, 1 − d, 1 − d), d = 0, 0.25, 0.5, preserve total mean traffic and give reductions of 26.6/10.0/3.6%. Thus relative gains decrease with load and imbalance, while Anchored lowers exceedance on every paired stream (15 per experiment). At d = 0.5, the

4

TABLE I C ONFIRMATION PERFORMANCE AT σe2 = 0.2. E XCEEDANCE IS MEAN±SD; OTHER METRICS ARE MEANS . SD USES FIVE STREAMS FOR RULES AND FIVE TRAINING - SEED MEANS FOR LEARNED POLICIES . PPO

RLD-SAC

DPP

π0

h-myopic

Anchored

7.076 19.88 ± 9.16 6.012 83.90 0.571

7.514 11.43 ± 1.49 5.517 86.74 0.553

9.277 27.86 ± 0.66 6.535 98.15 0.625

9.795 3.62 ± 0.17 6.330 98.15 0.533

9.762 1.51 ± 0.12 6.269 97.78 0.523

9.793 2.63 ± 0.06 6.263 98.15 0.527

Metric Task reward Exceedance (%) SE (bit/s/Hz) Sensing success (%) FIFO residence (ms)

TABLE II S CALE - EXPANSION EXCEEDANCE (%), σe2 = 0.2, 2187 CANDIDATES . RULES AVERAGE FIVE STREAMS ; LEARNING AVERAGES THREE FIVE - STREAM SEED MEANS . (M, K) M + K (6, 3) (8, 3) (4, 5) (8, 7) (12, 11)

9 11 9 15 23

π0 RLD-SAC 0.212 0.016 9.470 0.522 0.920

4.269 10.980 12.925 21.835 39.461

PPO Anchored 26.344 42.612 36.599 48.161 50.091

0.036 0.000 7.058 0.072 0.004

heavy-group mean/p99 backlog falls from 69.9/180 to 68.5/169 packets. Table II fixes N = 8, total power and total mean arrivals (500/3 packets and nine tasks per slot), scaling user/target rates by 4/M and 3/K. Rules share nested pools of 2187/8192 ternary-logit candidates. Each scale retrains three seeds per learning method for 200000 steps, using full-power selected settings. Five common streams yield 5×2×3×5 = 150 policy– stream pairs. Anchored with 2187 candidates has higher reward and lower exceedance in every pair; the minimum reward gain is 1.492 per slot. These pairs share streams within each configuration. At 23 variables, expanding to 8192 candidates raises anchored reward from 8.971 to 9.078.

VI. C ONCLUSION The utility–backlog objective yields a task reward whose conditional value follows from integer-service moments. Anchored look-ahead refines its plug-in maximizer, reducing five-CSI exceedance by 27.4–43.9% with mean reward loss below 0.01. Fixed-budget tests outperform freshly retrained learning baselines through 23 variables; the seven-variable implementation achieves p99 below 0.312 ms. A PPENDIX A I NTEGER -S ERVICE P ROBABILITIES A. Gaussian Moments and Departure Errors For H = Hmj , centering the threshold form gives H 2 Xmj − x̄mj = 2 Re(eH m Hĥm ) + em Hem − σe tr H. (13)

The linear and centered quadratic terms have variances 2σe2 ∥Hĥm ∥2 and σe4 tr(H2 ). Their covariance vanishes by the zero third centered Gaussian moments, proving (5). With ϵij = pij − 1{j ≤ Jˆi }, integer tail sums also give E[Di − Jˆi ] = E[(Di − Jˆi )2 ] =

Qi X j=1 Qi X

ϵij ,

(14)

|2j − 2Jˆi − 1| |ϵij |.

(15) D. Retuning and Score Accuracy j=1 A full-power search at σe2 = 0.2 uses four configurations per Indeed, splitting at d = Jˆi yields P (2d − 2j + 1)(1 − j≤d learning method, two seeds and 200000 steps per run. Three pij ) + P (2j − 2d − 1)pij . Cantelli bounds |ϵmj | ≤ ψmj , j>d −3 validation streams select SAC rate 10 /batch 512 and PPO rate with ψ 2 2 2 ˆ mj = smj /(smj + x̄mj ) when x̄mj > 0, j ≤ Jm , or 10−4 /clip 0.05, at the rate/batch upper and clip lower endpoints. ˆ x̄mj < 0, j > Jm ; otherwise ψmj = 1. For sensing, ψij = 0. Five fresh seeds and five new common streams retain the reward/exceedance ordering: 7.520/8.41% (SAC), 7.378/26.18% B. Spectral Series and Truncation (PPO), 9.800/3.74% (π0 ), 9.798/2.73% (Anchored). On x ⊥ wc,m , xH Hmj x = −τj xH Bm x ≤ 0, so Hmj Per CSI level, 95 trajectory states, 2000 common error draws and a causal MMPP filter estimate a mean conditional gap of has at most one positive eigenvalue. If it has none, pmj = 0 0.017–0.029 for Anchored versus 0.0006–0.014 for π0 . The since τj n > 0. Otherwise, for σe2 > 0, let λ+ be its positive estimated ρ + osc(e) averages 0.64–1.03. Equal draw splits for eigenvalue and ν = UH ĥm the estimate in its eigenbasis. For negative eigenvalues λℓ , define q+ = |ν+ |2 /σe2 , qℓ = |νℓ |2 /σe2 , selection and evaluation change mean gaps by < 10−4 . βℓ = −λℓ /λ+ , θℓ = βℓ /(1 + βℓ ) and β0 = τj n/(σe2 λ+ ). Failure is T+ < Y , where E. Verified CPU Implementation X Seven-variable fused C preserves candidate order with referT+ = |z+ + ν+ /σe |2 , Y = β0 + βℓ |zℓ + νℓ /σe |2 , (16) ence fallback. On one Xeon Gold 6148 core (2.40 GHz), 200 ℓ warm-ups and 2000 calls per CSI level give median/p99 0.239– with independent standard complex Gaussian z’s. The noncen0.245/0.274–0.311 ms, including input processing and action tral exponential T+ is a Poisson(q+ ) mixture of Gamma(k + output. All 10000 timed actions match (maximum 0.388 ms), 1, 1) laws, giving as do all 25 complete trajectories (250000 slots; five fallbacks). k X X qk Same-round actor p99 is 0.222–0.248 ms. Compilation, CSI 1 − pmj = e−q+ + fk , fk = 1 − rl , (17) k! acquisition and control transmission are excluded. k≥0

l=0

5

with rl = E[e−Y Y l /l!] and uℓ (z) = 1 + βℓ (1 − z), the generating function is X R(z) = E[e−(1−z)Y ] = rl z l , l≥0

Y e−qℓ βℓ (1−z)/uℓ (z)

= e−β0 (1−z)

uℓ (z)

ℓ

(18) .

Writing log R(z) = log r0 + s≥1 cs z s gives Y P r0 = e−β0 − ℓ θℓ qℓ (1 + βℓ )−1 , P

ℓ

cs = β0 1{s = 1} +

X

θℓs [1/s + (1 − θℓ )qℓ ].

(19)

ℓ

Equating coefficients in R′ = R(log R)′ yields l

1X scs rl−s , l ≥ 1. (20) l s=1 P Here cs ≥ 0, rl ≥ 0 and l rl = R(1) = 1, so 0 ≤ fk+1 = fk − rk+1 ≤ fk ≤ 1. Truncation after L obeys X qk e−q+ + fk ≤ fL Pr{Poisson(q+ ) > L}. 0≤ (21) k! rl =

k>L

At zero error variance, service thresholds are deterministic as stated in Theorem 1. A PPENDIX B S CORE E RROR , E XCEEDANCE AND D RIFT A. Conditional Reward and Score Error Each communication departure depends only on its user’s current error; sensing departures are known. Conditional independence and the queue update give Eq̄ = φ and 2 Eq̄ 2 = φ2 +σA +vD . Substitution into (3) proves Proposition 1; 2 σA includes conditional arrival covariances. Subtracting (8) gives (9), where X X dφ = ϖi (λi − λ̄i ) − ϖi E[Di − Jˆi ]. (22) i

i

P The departure identities imply |E[Di − Jˆi ]| ≤ j ψij and P Var Di ≤ j |2j − 2Jˆi − 1|ψij . The spectral series evaluates 2 the moments exactly. The action-independent σA cancels from osc(e). B. Exceedance and Drift Identities on Di = d and using F̄i (k) − F̄i (k + d) = PConditioning d b (k + j) proves (10). Set bi (k) = 0 for k < 0; the i j=1 sum is empty at Qi = 0. Independent user errors and traffic processes give thePproduct for any-queue risk. For L(Q) = 12 i Q2i , expanding Q′ = Q − D + At gives X ∆L (a) = Ct (a) + Qi (λi − E[Di ]), ≤ Ct (a) +

i X

Qi λi − Jˆi +

i 1 2 2 E[∥At − D∥

X

ψij ,

(23)

j≤Jˆi

where Ct (a) = | Ft , a] retains its action dependence. The inequality drops excess service and bounds predicted-departure shortfall [4], [7].

R EFERENCES [1] F. Liu, Y. Cui, C. Masouros, J. Xu, T. X. Han, Y. C. Eldar, and S. Buzzi, “Integrated sensing and communications: Toward dual-functional wireless networks for 6G and beyond,” IEEE J. Sel. Areas Commun., vol. 40, no. 6, pp. 1728–1767, Jun. 2022. [2] H. Hua, J. Xu, and T. X. Han, “Optimal transmit beamforming for integrated sensing and communication,” IEEE Trans. Veh. Technol., vol. 72, no. 8, pp. 10588–10603, Aug. 2023. [3] F. Liu, Y.-F. Liu, A. Li, C. Masouros, and Y. C. Eldar, “Cramér-Rao bound optimization for joint radar-communication beamforming,” IEEE Trans. Signal Process., vol. 70, pp. 240–253, 2022. [4] M. J. Neely, Stochastic Network Optimization with Application to Communication and Queueing Systems. San Rafael, CA, USA: Morgan & Claypool, 2010. [5] A. Zakeri, N. T. Nguyen, A. Alkhateeb, and M. Juntti, “Dynamic joint communications and sensing precoding design: A Lyapunov approach,” IEEE Wireless Commun. Lett., vol. 14, no. 11, pp. 3779–3783, Nov. 2025. [6] H. Yang, L. Wang, Z. Feng, Z. Wei, J. Peng, X. Yuan, T. Q. S. Quek, and P. Zhang, “Dynamic power allocation for integrated sensing and communication-enabled vehicular networks,” IEEE Trans. Wireless Commun., vol. 23, no. 9, pp. 12313–12330, Sep. 2024. [7] W. Ouyang, S. Murugesan, A. Eryilmaz, and N. B. Shroff, “Scheduling with rate adaptation under incomplete knowledge of channel/estimator statistics,” in Proc. 48th Annu. Allerton Conf. Commun., Control, Comput., Monticello, IL, USA, Sep.–Oct. 2010, pp. 670–677. [8] V. K. N. Lau and C. H. Koh, “Tradeoff analysis of delay-power-CSIT quality of dynamic backpressure algorithm for energy efficient OFDM systems,” IEEE Trans. Signal Process., vol. 60, no. 8, pp. 4254–4263, Aug. 2012. [9] Y. Hu and A. Ribeiro, “Optimal wireless communications with imperfect channel state information,” IEEE Trans. Signal Process., vol. 61, no. 11, pp. 2751–2766, Jun. 2013. [10] L. Yang, Y. Wei, Z. Feng, Q. Zhang, and Z. Han, “Deep reinforcement learning-based resource allocation for integrated sensing, communication, and computation in vehicular network,” IEEE Trans. Wireless Commun., vol. 23, no. 12, pp. 18608–18622, Dec. 2024. [11] X. Ye, X. Song, Y. Wu, and L. Fu, “Intelligent omni-surface-aided multiobjective ISAC: A meta hybrid deep reinforcement learning approach,” IEEE Trans. Mobile Comput., vol. 25, no. 3, pp. 3760–3775, Mar. 2026. [12] Y. Qin, Z. Zhang, X. Li, W. Huangfu, and H. Zhang, “Deep reinforcement learning based resource allocation and trajectory planning in integrated sensing and communications UAV network,” IEEE Trans. Wireless Commun., vol. 22, no. 11, pp. 8158–8169, Nov. 2023. [13] S. Bi, L. Huang, H. Wang, and Y.-J. A. Zhang, “Lyapunov-guided deep reinforcement learning for stable online computation offloading in mobile-edge computing networks,” IEEE Trans. Wireless Commun., vol. 20, no. 11, pp. 7519–7537, Nov. 2021. [14] Y. Liang, H. Tang, H. Wu, Y. Wang, and P. Jiao, “Lyapunov-guided offloading optimization based on soft actor-critic for ISAC-aided internet of vehicles,” IEEE Trans. Mobile Comput., vol. 23, no. 12, pp. 14708– 14721, Dec. 2024. [15] S. Meyn, “Stability and asymptotic optimality of generalized MaxWeight policies,” SIAM J. Control Optim., vol. 47, no. 6, pp. 3259–3294, 2009. [16] T. Y. Al-Naffouri and B. Hassibi, “On the distribution of indefinite quadratic forms in Gaussian random variables,” in Proc. IEEE Int. Symp. Inf. Theory (ISIT), Seoul, South Korea, Jun.–Jul. 2009, pp. 1744–1748. [17] W. B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality, 2nd ed. Hoboken, NJ, USA: Wiley, 2011. [18] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Offpolicy maximum entropy deep reinforcement learning with a stochastic actor,” in Proc. Int. Conf. Mach. Learn. (ICML), Jul. 2018, pp. 1861– 1870. [19] L. Engstrom, A. Ilyas, S. Santurkar, D. Tsipras, F. Janoos, L. Rudolph, and A. Madry, “Implementation matters in deep RL: A case study on PPO and TRPO,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2020.

Related documents

• Full-Frame OFDM-ISAC: Joint Constellation, Pilot, and Receive-Filter Design#1048389
• Movable Antenna-Enabled Multi-Static Cooperative Secure ISAC: Architecture, Mechanisms, and Enabling Technologies#1388981
• A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems#6601
• Temporal vs Spectral Resolution in CSI Reporting for Wi-Fi ISAC#1387697
• WirelessOpsAgent: A Benchmark and Agent Design for Action Assurance in Wireless Networks#441733