arXiv CSopen access

HiRL: Hierarchical Reinforcement Learning for Coordinated Resource Management in Heterogeneous Edge Computing

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

1

H I RL: Hierarchical Reinforcement Learning for Coordinated Resource Management in Heterogeneous Edge Computing

arXiv:2605.10443v1 [cs.DC] 11 May 2026

Jianyong Zhu, Hao Chen, Juan Zhang, Member, IEEE, Fangda Guo, Member, IEEE, Albert Y. Zomaya, Fellow, IEEE, Renyu Yang, Member, IEEE

Abstract—Edge computing faces unprecedented resource orchestration challenges from multi-dimensional heterogeneity across device architectures, diverse task requirements in CPUintensive, GPU-intensive, I/O-intensive, and dynamic network conditions. The edge environments demand real-time task processing within strict energy budgets, yet conventional approaches struggle with mixed continuous-discrete optimization while meeting deadline and energy constraints. This paper presents H I RL, a hierarchical reinforcement learning framework that decomposes complex resource orchestration into coordinated power control and task allocation decisions. Our approach separates continuous power management using the Twin Delayed Deep Deterministic Policy Gradient (TD3) and discrete task placement using Double Deep Q-Network (DDQN), unified through a coordination engine with five-dimensional queue state representation. We propose a heterogeneous assessment of resource compatibility with deadline-oriented prioritization and failure-penalized adaptive sampling to enhance decision quality under resource constraints. To improve practical applicability, the framework models comprehensive system dynamics including device mobility, queue congestion patterns, infrastructure heterogeneity, and priority-sensitive scheduling demands. Experimental results show that H I RL achieves effective latency-energy trade-offs with 28% latency reduction compared to Single-DDQN and maintains nearly 100% task completion rates under all load conditions. Compared to baseline algorithms, H I RL reduces energy consumption by up to 51% under low load while achieving 24% better latency performance than static optimization approaches under high load, establishing effective resource orchestration in heterogeneous edge environments. Index Terms—Heterogeneous Resource Orchestration, Hierarchical Reinforcement Learning, GPU-aware, Deadline Constraints, Task Completion Guarantees.

Received XXX; revised XXX; accepted XXX. Date of publication XXX; date of current version XXX.This work was supported in part by the Hebei Provincial Natural Science Foundation under Grant F2023502002 and in part by the Fundamental Research Funds for the Central Universities under Grant 2025MS153. (Corresponding author: Renyu Yang) J. Zhu is with the Yanzhao Electric Power Laboratory of North China Electric Power University, and Hebei Key Laboratory of Knowledge Computing for Energy & Power, Baoding 071000, China (e-mail: [email protected]). H. Chen is with the Department of Computer, North China Electric Power University, and Engineering Research Center of Intelligent Computing for Complex Energy Systems, Ministry of Education, Baoding 071000, China (email: [email protected]). J. Zhang is with the James Watt School of Engineering, University of Glasgow, Glasgow G12 8QQ, U.K. (e-mail: [email protected]). F. Guo is with the State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China (Email: [email protected]). Albert Zomaya is with the University of Sydney, Australia. E-mail: [email protected]. R. Yang is with the School of Software, Beihang University, Beijing 100191, China (e-mail: [email protected]).

Heterogeneous Servers: Different capabilities & loads Mobile Constraints: Limited battery & GPU

Task Sources: • Video Analysis • Route Planning

Smart Car ( Limited GPU )

RSU-C Tesla H00 (Load: 30%)

5G Network Wireless Transmission

RSU-B Tesla A100 (Load: 80%)

Smart application Emergency Detection

RSU-C Tesla V100 (Load: 45%)

Traffic Optimization

Fig. 1. Intelligent transportation scenario: Smart vehicle offloads video analysis and route planning tasks to heterogeneous edge servers.

I. I NTRODUCTION

E

DGE computing addresses computational demands of IoT devices with limited resources, dynamic network conditions, and strict energy constraints, particularly in mobile edge computing (MEC) scenarios [1], [2]. The environments face resource orchestration challenges from infrastructure heterogeneity—edge servers and mobile devices differ in CPU/GPU configurations and storage capacities [3], [4]—and workload heterogeneity spanning CPU-intensive tasks, GPUaccelerated applications, and I/O-intensive transfers [5], [6]. Device mobility further complicates scheduling through timevarying network conditions [7], [8]. Ineffective resource management leads to deadline violations, energy waste, and task failures. Current approaches face three limitations. First, independent handling of power control and task allocation prevents joint optimization—existing methods either fix transmission power during task allocation [9], [10] or separate frequency scaling from offloading decisions [4], [11], missing energy-latency tradeoffs. Second, treating GPU-intensive and CPU-intensive tasks uniformly causes resource mismatches: GPU tasks on CPU-only servers fail, while CPU tasks on GPU servers waste resources [12], [13]. Third, optimizing energy, latency, or completion rates separately creates conflicts—aggressive offloading reduces local energy but increases transmission and congestion costs, worsening latency and deadline misses [14], [15]. Consider a computing scenario in Fig. 1. A vehicle offloads GPU-intensive video analysis and CPU-intensive route planning to heterogeneous servers (H100/A100/V100) under adjustable transmission power (up to 3W) with emergency deadlines. Independent decisions fail: high power without GPU availability wastes energy; GPU tasks on incompatible

2

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

servers violate deadlines. Effective management requires hierarchical coordination with GPU-aware matching. We present H I RL, a hierarchical reinforcement learning framework for resource orchestration in heterogeneous MEC environments. The framework addresses mixed continuousdiscrete optimization through hierarchical decomposition while maintaining cross-tier coordination. We design a dualpipeline CPU/GPU computational model to represent heterogeneous resource requirements. The two-tier architecture separates continuous power control (upper tier TD3 with five-dimensional queue state managing CPU frequency and transmission power) from discrete task allocation (lower tier DDQN with GPU-aware state design for placement across heterogeneous servers), unified through a coordination engine implementing a three-stage decision pipeline. We introduce deadline-oriented queue management and failure-penalized experience replay to handle real-time constraints and accelerate learning from resource violations. We evaluate H I RL across production-scale heterogeneous deployments with 35 mobile devices and 5 edge servers under controlled load variations. Experimental results show 28% latency reduction compared to Single-DDQN while maintaining 98%+ task completion rates. Energy consumption decreases by 51% under low load, and latency improves by 24% over static optimization under high load. Ablation studies confirm hierarchical coordination and GPU-aware compatibility assessment contribute 49-82% of these gains, while disabling failurepenalized learning causes system collapse with completion rates dropping to 62%. The main contributions include: • Hierarchical framework with coordinated CPU/GPU dual-pipeline modeling for heterogeneous MEC environments. • Compatibility-aware task allocation mechanism with GPU-resource matching across heterogeneous servers. • Deadline-oriented queue optimization and failurepenalized sampling for constraint handling. The remainder is organized as follows: Section II presents framework and modeling; Sections III and IV detail TD3 power control and DDQN task allocation; Section V provides experimental evaluation; Section VI reviews related work; Section VII concludes. Source codes and data are available at https://github.com/ch-ncepu/HIRL. II. H IERARCHICAL F RAMEWORK AND S YSTEM M ODELING A. Motivation Heterogeneous MEC environments present resource orchestration challenges coupling continuous power control and discrete task placement under stringent deadlines. In the vehicular scenario of Fig. 1, emergency video analysis requires 4 GB GPU memory and 2048 CUDA cores within 200 ms, whereas route planning demands 50 M CPU cycles with negligible GPU use. The vehicle adjusts transmission power (0.1–3 W) and CPU frequency (0.8–2.4 GHz) while selecting among heterogeneous servers (H100/A100/V100) under LTE-V2X constraints.

Input 5D Queue States

Stage 1 TD3 Upper Tier Power Control

Discrete Server assignment

Information hub

Power Decisions

Stage 2 Coordination Engine

State Update & Synchronization

Stage 3 DDQN Lower Tier Task Allocation Allocation Decisions

System States Input Environment State Collection

Policy Updates

Feedback Learning Update

Performance Metrics

Execution Task Processing& Monitoring

Fig. 2. Workflow of H I RL: Synchronized three-stage decision pipeline with closed-loop feedback.

This scenario exposes key optimization challenges. The joint decision space combines continuous control with discrete selection: for M devices choosing among N servers, the space grows as O((N + 1)M × R2M ). Discretizing power into 10 frequency and 10 power levels yields 100 combinations per device, reducing precision. Relaxing discrete choices to continuous variables produces non-convex landscapes where gradient-based optimization becomes unstable. Power decisions and task placement are dynamically interdependent. Increasing transmission power from 0.5W to i , enabling GPU execution 2W shortens offloading latency Dtx for latency-critical tasks. Conversely, frequency scaling alters local processing: a task missing its deadline at 0.8GHz may complete locally at 2.4GHz with lower energy cost. Decoupled optimization breaks these dependencies—power control without task awareness may select conservative transmission, causing deadline violations, while allocation without power constraints may offload to distant servers, causing excessive latency. Task heterogeneity introduces coupled queue dynamics across heterogeneous resources. GPU-intensive video analysis, CPU-bound route planning, and I/O-driven transfers create distinct workload patterns. We model these interactions through five queue dimensions: local CPU, local GPU, wireless transmission, server CPU, and server GPU. Queue backlogs evolve differently—CPU queues grow steadily with general arrivals, GPU queues surge with bursty vision tasks, transmission queues fluctuate with channel conditions. Joint optimization of these interdependent queues with power-allocation decisions requires learning over high-dimensional state spaces, causing sample inefficiency in dynamic environments. We address these challenges through a hierarchically coordinated framework (Fig. 2). The upper tier employs TD3 for continuous power control, adjusting (flt , pt ) based on queue states qt and channel gains g t . The lower tier uses DDQN for discrete tasks offloading, integrating GPU compatibility and transmission rate updates. A three-stage pipeline links the tiers: Stage 1 determines power settings, Stage 2 updates transmission rates and queue states, Stage 3 finalizes task allocation. This design decouples optimization objectives while preserving inter-tier dependencies through shared queues. B. Problem Formulation Consider the vehicle scenario from Fig. 1. GPU-intensive video analysis requires substantial CUDA cores and memory, while CPU-intensive route planning demands computational

ZHU et al.: HIRL: HIERARCHICAL REINFORCEMENT LEARNING FOR COORDINATED RESOURCE MANAGEMENT IN HETEROGENEOUS EDGE COMPUTING 3

TABLE I N OTATION Symbol M, N qt flt , pt gt , rt ρji Ti λ i Dtot , E i

Description Mobile devices, edge servers Five-dimensional queue state CPU frequency, transmission power Channel gain, transmission rate GPU compatibility score Task attributes (bi , si , ci , ddli , ni , mi , s gpui , tcreate ) Energy-latency tradeoff weight Task latency, energy

cycles. Both scenarios face stringent emergency deadlines and constrained transmission power (2-3W), and thus come with a resource orchestration problem – coordinating power management with task placement across M mobile devices and N edge servers (M = 35, N = 5 in our evaluation) to process three task categories τ = {tCPU , tGPU , tIO }. Each task i has attributes (bi , si , ci , ddli , ni , mi , s gpui , tcreate ) representing the priority, data size, CPU cycles, deadline, required CUDA cores, GPU memory, and computational load and the creation time of the task. The execution cost (Eq. 1) strike a balance between latency and energy. i costi = λbi Dtot + (1 − λ)E i ,

(1)

where bi amplifies the weight for critical tasks and λ ∈ i aggregates [0, 1] controls the tradeoff. The total latency Dtot queuing latencies across local/transmission/server stages and CPU-GPU pipeline execution. The energy E i combines local i processing κ(flt )2 ci , transmission pt Dtx , and server-side GPU consumption. The optimization objective (Eq. 2) minimizes long-term execution costs through coordinated power control and task allocation decisions. T 1 XX min lim costi , T →∞ T t=0 i∈Γt

s.t.

j 0 ≤ ftj ≤ fmax , 0 ≤ pjt ≤ pjmax , i Dtot ≤ ddli ,

X i∈Aj

∀j, t,

(2)

∀i,

j ni ≤ Ngpu ,

X

j mi ≤ Mgpu ,

∀j,

i∈Aj

where Γt denotes the set of tasks arriving at time t, Aj represents the tasks assigned to server j, and GPU-related constraints are enforced at the server level through the availj j able CUDA cores Ngpu and memory capacity Mgpu . The resulting resource orchestration problem involves tightly coupled continuous decisions, including CPU frequency allocation ftj and transmission power control pjt , together with discrete decisions on task placement between local execution and edge servers. The proposed hierarchical framework addresses this mixed-integer optimization by decomposing it into coordinated subproblems, which are jointly optimized in accordance with Eq. 2. Table I summarizes all notations used throughout the framework. C. Hierarchical Decision Framework The hierarchical architecture decomposes the hybrid optimization problem (Eq. 2) into coordinated submodules. The

upper tier applies TD3 for continuous power control (ftj , pjt ), mitigating value overestimation through twin critics and delayed updates. The lower tier adopts DDQN for discrete task placement, decoupling action selection from value evaluation for GPU-heterogeneous server choices. Both tiers interact through a coordination engine synchronizing decisions via a three-stage pipeline (Fig. 2). At each time slot t (τ = 1s), the system executes synchronized operations across all M devices. The upper tier first updates the five-dimensional queue state qt , capturing local CPU/GPU, transmission, and server CPU/GPU backlogs, while measuring channel gains g t to generate frequency–power pairs (ftj , pjt ). The coordination engine then updates transmission rates rt based on the new power settings and channel conditions, normalizes queue states relative to current capacities, and forwards the updated states to the lower tier. The lower tier determines server placements for new tasks according to GPU compatibility and deadline urgency. Task dispatch follows deadline-oriented sorting, with execution outcomes generating reward signals rpt and roi for policy refinement. The coordination engine maintains consistency across tiers through three mechanisms via Eq. 3: Shared State Representation. Both tiers operate on a synchronized five-dimensional queue vector qt (Eq. 4), providing a common view of resource utilization across heterogeneous components. Sequential Decision Pipeline. The three-stage structure creates explicit tier dependencies: Stage 1: (f t , pt ) = πp (qt , g t ), Stage 2: rt = Update(pt , g t ),

q̂t = Normalize(qt , f t , rt ),

Stage 3: aio = πo (q̂t , f t , pt , Ti , rt ),

(3) where TD3 policy πp tunes frequency-power pairs; Stage 2 updates transmission rates and normalized queues; DDQN policy πo selects task placements using refreshed states. The pipeline maintains causal consistency—power control directly determines feasible allocation boundaries. Bidirectional Feedback. Execution outcomes are fed back via coordinated rewards to jointly refine both policies and align them with global objectives (Sections III and IV). D. State Representation The framework uses a five-dimensional queue state (Eq. 4) tracking resource utilization across system components: t t t t t qt = [qlc , qtx , qlg , qsc , qsg ],

(4)

where subscripts lc, sc denote local/server CPU queues, lg, sg represent local/server GPU queues, and tx indicates transmission queue. Each queue accumulates workload from waiting t tasks, where tasks j atPtime slot P Qj represents P in queue t t t t: qlc = w , q = s , q t sc = i∈Qttx i i∈Qtsc wi , P P i∈Qlc i tx t t qlg = i∈Qtsg s gpui , with wi = i∈Qtlg s gpui , qsg = scpu,i × ci denoting required CPU cycles and s gpui denoting GPU computational load (FLOPS). Queue evolution couples

4

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

upper-tier power control with lower-tier task allocation through task arrivals and processing departures. The system load metric (Eq. 5) integrates queue backlogs with real-time utilization: ωq X Lt = tanh(q̂jt ) + ωu · utcpu + ωg · utgpu , (5) 5 j∈Q

where Q = {lc, tx, lg, sc, sg} and weights ωq , ωu , ωg balance queue backlog with CPU/GPU utilization. Queue normalization scales states relative to processing capacity through Eq. 6: q̂jt =

qjt , fjt · τ

j ∈ {lc, tx, lg, sc, sg},

(6)

where fjt denotes processing rate for queue j and τ is time slot duration. Values q̂jt > 1 indicate backlog exceeding single-slot capacity. Tier-Specific States. The upper tier observes state vector Sp defined in Eq. 7: Sp = [qt , g t , fmax , pmax ],

(7)

where g t denotes channel gain and fmax , pmax denote capability bounds. The lower tier uses extended state So (Eq. 8):  t t t t t So = qlc , qtx , qlg , qsc , qsg , f t , pt , g t , gpuf , gpunum , gpubw , gpupwr , l,t s,t s,t ul,t cpu , ugpu , ucpu , ugpu ,

(8)

 Ti (bi , si , ci , ddli , ni , mi , s gpui , tcreate ) , where GPU specifications (gpuf , gpunum , gpubw , gpupwr ) represent frequency, CUDA cores, bandwidth, and power l,t s,t s,t consumption; utilization metrics (ul,t cpu , ugpu , ucpu , ugpu ) track local/server CPU/GPU usage; task tuple T (·) contains priority, size, complexity, deadline, GPU requirements, and creation timestamp. DeDetails are in Sections III and IV. Alg. 1 integrates the hierarchical framework through synchronized three-stage operations per time slot. Time-slot synchronization maintains consistent states across tiers. Detailed reward formulations are in Sections III and IV. III. P OWER C ONTROL WITH T WIN D ELAYED D EEP D ETERMINISTIC P OLICY G RADIENT This section presents the upper-tier TD3-based power control mechanism within the hierarchical framework in Section II, including problem formulation, state-action representation, reward design, and implementation details. A. TD3 for Power Control in Heterogeneous Environments The upper tier employs TD3 to optimize continuous power control under heterogeneous MEC settings. TD3 incorporates twin critics to mitigate value overestimation, delayed policy updates to suppress oscillations, and target smoothing to improve training stability across diverse device capabilities and fluctuating network conditions. Power Control Formulation. TD3 operates on state Spt (Eq. 7) and action space Ap = [flt , pt ] with device-specific PT constraints, optimizing maxπp E[limT →∞ T1 t=0 rpt ], where πp denotes the power control policy and rpt is defined in Eq. 11.

Algorithm 1 H I RL Hierarchical Coordination. Input: Devices M, servers N , TD3 agent πp , DDQN agent πo 1: t ← 1 2: while t ≤ T do 3: // Stage 1: Power Control 4: for j ∈ M do 5: Collect queue state: qtj , measure channel gain: gjt 6: Execute power control: (flt , pt ) ← πp (qtj , gjt ) 7: end for 8: // Stage 2: State Synchronization 9: Update transmission rates: rt ← ComputeRate(pt , g t ) 10: Normalize queue states using Eq. 6 11: Update load metric Lt using Eq. 5 12: // Stage 3: Task Allocation 13: for each arriving task i on device j do 14: Extract task attributes: Ti 15: Compute compatibility: ρki for k ∈ N ∪ {0} 16: Execute allocation: aio ← πo (qtj , flt , pt , Ti , rt ) 17: Dispatch task to selected queue 18: end for 19: // Execution and Learning 20: Process tasks with deadline prioritization 21: Collect outcomes: completion status, latencies, energy 22: Calculate rewards: rpt (Eq. 11), roi (Eq. 12) 23: Store experiences with failure-aware prioritization 24: Update TD3 and DDQN networks 25: t←t+1 26: end while

B. State-Action Representation and Normalization State Space Components. The power control agent processes state space Spt (Eq. 7). Channel gain follows the pathloss model in Eq. 9:  γ d0 t , (9) g = G0 dt where G0 is the gain constant, dt denotes the distance, and γ is the path-loss exponent affecting transmission rates in Stage 2 (Eq. 3). Synchronized State Management: State updates are synchronized at time-slot boundaries, such that power-control outputs from Stage 1 are immediately applied to Stage 3 allocation, ensuring TD3 and DDQN operate on a consistent system state. 1) Continuous Action Space Formulation: The power control agent outputs continuous actions within a constrained space as defined in Eq. 10: Ap = [flt , pt ],

flt ∈ [0, fmax ],

pt ∈ [0, pmax ],

(10)

where flt denotes the local CPU frequency (Hz) and pt the transmission power (W). These actions implement Stage 1 (Eq. 3): flt determines local processing capacity reflected in queue states, while pt determines the communication rate rt in Stage 2, both of which are included in the task allocation agent’s state. State Normalization. Queue states are normalized followt t ing Eq. 6: q̂lc = qlc /(flt · τ ) to ensure scale-invariant learning across varying device configurations and workload intensities. C. Reward Function and Training Optimization The hierarchical framework aligns both tiers toward global objectives through coordinated reward structures. The upper

ZHU et al.: HIRL: HIERARCHICAL REINFORCEMENT LEARNING FOR COORDINATED RESOURCE MANAGEMENT IN HETEROGENEOUS EDGE COMPUTING 5

tier optimizes system-level performance (Eq. 11): t rpt = Rcomplete − λe · E t ,

(11)

t i where Rcomplete = i∈C t bi Isuccess sums priority-weighted i completed tasks (Isuccess ∈ {0, 1}), and E t aggregates CPU and transmission energy weighted by λe , assigning higher priority to task completion than energy reduction. The lower tier optimizes individual task execution via Eq. 12:

P

i roi = −(λbi Dtot + (1 − λ)E i ) − α · Ifi ail ,

(12)

where the first term matches Eq. 1, ensuring tier alignment. Penalty α discourages constraint violations indicated by Ifi ail ∈ {0, 1} through GPU compatibility checks. Both rei wards share metrics (Dtot , E i , completion status), maintaining feedback consistency along the pipeline (Eq. 3). 1) Energy-Performance Trade-off Optimization: The TD3 agent implements the coordinated reward from Eq. 11 with energy computation following Eq. 13: st , E t = k(flt )2 τ + pt tx rt

(13)

where k denotes the effective switching capacitance, (flt )2 reflects the quadratic CPU frequency–power relationship, and transmission energy depends on data size sttx and communication duration. The fixed coefficient λe = 0.5 in Eq. 11 assigns higher weight to task completion, yielding over 98% completion rates; during training, TD3 accordingly allocates power aggressively for high-priority, deadline-critical tasks and conservatively under low load, driven solely by the reward structure, without explicit state-dependent weighting. 2) Failure-Aware Experience Prioritization: Our training method prioritizes experiences from critical scenarios to accelerate learning in challenging situations. The prioritization weight combines temporal difference error with failure indicators as shown in Eq. 14: Wi = |T Di | · (1 + γf ail · Ifi ail ),

(14)

where |T Di | denotes the temporal difference error magnitude, Ifi ail indicates task failures, and γf ail amplifies critical scenarios. Prioritization in Eq. 14 guides TD3 to focus on systemstress cases induced by power control actions in Eq. 10. D. Network Architecture and Implementation The TD3 implementation for power control uses network architectures and training strategies for dynamic heterogeneous mobile edge environments. 1) Actor-Critic Network Design: The TD3 framework consists of an actor network and twin critic networks with target stabilization for continuous power control. The actor µϕ (s) maps the six-dimensional state Spt to the two-dimensional action at = [flt , ptl ] using a tanh output layer, followed by reverse normalization based on state-provided bounds (fmax , pmax ) to scale actions from [−1, 1] to device-specific ranges. Through this scaling mechanism, a shared TD3 policy is applied across heterogeneous devices.

Complementing the actor, the twin critics Qθ1 (s, a) and Qθ2 (s, a) adopt conservative value estimation, Qtarget = min(Qθ1′ (s′ , ã′ ), Qθ2′ (s′ , ã′ )), where ã′ denotes the smoothed target action, stabilizing value evaluation under mobile edge uncertainty. 2) Adaptive Training Mechanisms: The TD3 implementation incorporates two adaptations for heterogeneous mobile edge environments. First, load-adaptive exploration noise is applied as σ t = σbase (1 + αLt ), where the load metric Lt ∈ [0, 1] is defined in Eq. 5 and α controls adaptation sensitivity. This mechanism promotes higher exploration under low-load conditions while encouraging conservative behavior during high-load periods. Second, the experience replay module applies failureaware prioritization defined in Eq. 14, emphasizing critical or constraint-violating samples. Together with adaptive exploration noise, this design supports a balanced trade-off between stability and responsiveness under heterogeneous MEC conditions. 3) Network Update Mechanism: Training follows a coordinated update schedule aligned with the synchronous timeslot framework. The critic networks are updated at each time slot using temporal difference learning with target smoothing, while the actor network applies delayed updatesθµ ← θµ + αµ ∇θµ Qθ1 (s, µθµ (s)) every d time slots to enhance policy stability. Target networks are updated via Polyak averaging with parameter τ , ensuring gradual policy evolution across heterogeneous device configurations and dynamic workload conditions. IV. TASK A LLOCATION WITH D OUBLE D EEP Q-N ETWORK This section presents the lower-tier DDQN-based task allocation mechanism coordinating with the upper-tier power control. It formulates the discrete task allocation problem, describes GPU-aware state–action design, and introduces deadline-oriented queue management with adaptive experience replay. The interaction between DDQN and TD3 within the three-stage execution pipeline is also explained. A. Double Deep Q-Network for Task Allocation The lower tier employs DDQN for discrete task placement across heterogeneous resources. By decoupling action selection from evaluation, DDQN mitigates value overestimation through target value Qtarget = r + γQθ′ (s′ , arg maxa′ Qθ (s′ , a′ )), where Qθ selects optimal actions and Qθ′ evaluates values, avoiding overestimation when a single network handles both roles. This separation proves advantageous in heterogeneous MEC environments where server diversity and fluctuating loads challenge estimation stability. 1) DDQN Implementation in Hierarchical Framework: DDQN operates as Stage 3 in the coordination pipeline (Eq. 3), processing power-updated states with failure penalty rewards (Eq. 12). Load-aware exploration follows Eq. 15: ϵt = ϵmin + (ϵmax − ϵmin ) · exp(−β · Lt ),

(15)

where β > 0 controls adaptation sensitivity based on load metric Lt (Eq. 5). During high-load periods (Lt → 1),

6

exploration decreases to favor proven strategies, ensuring stability. Low-load conditions enable increased exploration to discover improved policies. This balances exploitationexploration based on real-time system stress. B. State Space Design and GPU-Aware Mechanisms 1) Multi-dimensional State Representation: The DDQN agent utilizes the state space from Eq. 8 established in Section II, incorporating the five-dimensional queue state from Eq. 4 with power control outputs and task attributes. t • Five-dimensional queue states q (Eq. 4) t t • Power control decisions (fl , p ) from Stage 1 (Eq. 3) • Task attributes Ti (bi , si , ci , ddli , ni , mi , s gpui , tcreate ) including GPU requirements t • Network conditions g and GPU specifications (frequency, cores, bandwidth, power) • Real-time CPU/GPU utilization metrics (local and server) • Creation timestamps for latency tracking To manage state dimensionality, we apply normalization from Eq. 6, providing scale-invariant indicators of resource saturation for consistent policy learning. 2) GPU-Aware Resource Matching: Resource compatibility assessment employs GPU-aware scoring mechanisms to ensure feasible task-resource matching. The compatibility score is defined as follows: ! mi scpu,i ni j , j , j , (16) ρi = min j Mcpu Ngpu Mgpu where ρji ∈ [0, 1] measures compatibility between task i and execution location j based on CPU memory requirements, GPU cores, and GPU memory capacity. Values closer to 1 indicate higher resource compatibility, enabling the DDQN agent to prioritize feasible allocations and filter infeasible actions before Q-value computation. The DDQN agent employs GPU-aware compatibility scoring (Eq. 16) for real-time feasibility filtering without maintaining high-dimensional server representations, supporting scalable task allocation over heterogeneous resources. Although current GPUs lack fine-grained resource partitioning, the framework establishes a modeling basis for future edge systems as GPU virtualization technologies mature. 3) Action Space Optimization and Temporal Dynamics: The discrete action space is defined as Ao = {alocal , aserver1 , aserver2 , . . . , aserverN }1, where each action corresponds to a feasible execution location with distinct resource characteristics. The Q-network integrates GPU compatibility assessment into action evaluation, assigning higher Qvalues to actions that satisfy GPU requirements under current system load and task deadlines, thereby learning resourceaware policies that reduce execution failures. To account for temporal variations in mobile edge systems, we augment the state with evolution information Sotemporal = [Sot , ∆Sot , E[Sot+1 ]], where ∆Sot = Sot − Sot−1 characterizes short-term state changes and E[Sot+1 ] denotes the predicted next state inferred from current trends and task arrival patterns. This formulation allows task allocation decisions to consider both instantaneous dynamics and near-future system conditions.

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

C. Deadline-Oriented Queue Management and Adaptive Experience Replay Task allocation requires queue management for deadlinesensitive tasks and efficient learning through experience replay. Our approach combines deadline-oriented scheduling with failure-penalized adaptive sampling. 1) Deadline-Oriented Queue Prioritization: Our deadlineoriented queue management dynamically adjusts priorities based on urgency, resource requirements, and execution feasibility. The multi-dimensional scoring mechanism integrates these factors through Eq. 17:   ddli − tcurrent bi t · max(ρji ), (17) · max 0, Pi = i j bmax Dest where the priority score combines task priority bi , deadline i urgency (ddli − tcurrent )/Dest , and the best GPU compatj ibility maxj (ρi ) from Eq. 16. This mechanism ensures that high-priority, urgent tasks with good resource matches receive processing precedence. i −tcurrent The deadline urgency factor uti = ddl adapts to i +W i Dest est i current system load and estimated execution times, where Dest represents estimated execution time based on task computai tional requirements and target device capabilities, while West denotes expected waiting time derived from current queue lengths and processing rates. This dynamic prioritization responds to changing conditions, ensuring time-critical tasks receive processing precedence without starving lower-priority workloads. 2) Failure Risk Assessment and Prevention: Failure risk assessment identifies tasks that are likely to violate constraints through Eq. 18: Ifi ail =

( i 1, if maxj (ρji ) < θmin or (ddli − tcurrent ) < Dest , 0, otherwise, (18)

where θmin is defined as the lower bound of acceptable compatibility. The resulting failure indicator is integrated into the hierarchical reward function (Section II), thereby enabling prompt penalization of allocation decisions that violate constraints or miss deadlines. Tasks with high failure risk are prioritized through dedicated queue channels, while the corresponding risk metrics are incorporated into the DDQN reward function, enabling penalization of allocation policies that may lead to task failures. 3) Adaptive Experience Replay with Failure Penalization: DDQN training adopts the same failure-aware prioritization mechanism as the upper-tier TD3 (Eq. 14), ensuring crosstier consistency. The prioritization weight Wi = |T Di | · (1 + γf ail · Ifi ail ) combines temporal difference error with the failure indicator Ifi ail , which identifies allocation decisions that risk resource constraint violations or deadline misses (Eq. 18). Unlike TD3, where Ifi ail reflects power-related stress, DDQN focuses on load-induced failures caused by fluctuating task demands and heterogeneous device capabilities. This coordinated design improves convergence behavior and sample efficiency across both tiers.

ZHU et al.: HIRL: HIERARCHICAL REINFORCEMENT LEARNING FOR COORDINATED RESOURCE MANAGEMENT IN HETEROGENEOUS EDGE COMPUTING 7

4) Integration with Hierarchical Decision Framework: As the stage 3 in the coordination pipeline (Eq. 3), the DDQN incorporates deadline management (Eq. 17) and GPU-aware filtering (Eq. 16). Adjustments in queue states modify the load metric Lt (Eq. 5), which in turn influences the reward computation of the power control layer (Eq. 11). Meanwhile, the outcomes of failure assessment (Eq. 18) are reflected in the task allocation reward (Eq. 12). Together, these interactions establish a closed feedback loop that aligns both tiers toward joint optimization. 5) Load-Adaptive Training for Dynamic Mobile Conditions: The DDQN training includes load-driven adaptations that remain consistent with the hierarchical coordination framework. The load metric Lt (Eq. 5) serves as a global indicator to synchronize learning behavior across tiers. The learning rate is defined as follows: η t = ηbase · (1 + α · Lt ),

(19)

where ηbase is the base learning rate and α > 0 controls learning rate adaptation sensitivity, ensuring that the DDQN agent responds appropriately to varying operational conditions. Exploration follows the load-aware ϵ-scheduling scheme (Eq. 15). During high-load periods, ϵ is reduced to favor stable exploitation; when the system is lightly loaded, ϵ increases to encourage exploration and improve policy diversity. V. E XPERIMENTAL E VALUATION We evaluate H I RL using a simulation framework that reflects heterogeneous mobile edge environments, focusing on system performance, learning behavior, adaptive mechanisms, and scalability. Additional experiments examine the gap between simulation outcomes and practical deployment considerations. A. Heterogeneous Infrastructure Configuration 1) Heterogeneous Device Infrastructure: The experimental setup consists of 35 mobile devices and 5 edge servers, configured to emulate practical MEC heterogeneity. Mobile Devices (35 units): Devices differ in mobility (low: ±10 m/slot; high: ±20 m/slot) and computational capacity (CPU: 2–3 GHz, memory: 8–16 GB). GPU types range from GTX 1660 (1408 CUDA cores, 6 GB, 192 GB/s) to RTX 4090 (16384 cores, 32 GB, 1008 GB/s), with transmission power constrained to 2–3 W. Edge Server Infrastructure (5 servers): The edge infrastructure includes five servers with heterogeneous computational architectures. GPU configurations cover Tesla P100 (3584 CUDA cores, 16 GB memory, 732 GB/s bandwidth, 250 W), Tesla V100 (5120 cores, 32 GB, 900 GB/s, 300 W), Tesla A100 (6912 cores, 80 GB, 1555 GB/s, 400 W), and Tesla H100 (14592 cores, 80 GB, 3000 GB/s, 700 W). System memory configurations range from 64GB to 1024GB across different server types, with CPU frequencies configured at 50-60 GHz to reflect server-grade processing capabilities. This heterogeneous server infrastructure supports evaluation of GPU-aware resource matching and compatibility assessment within the proposed framework.

2) Task Workload and Network Modeling: Tasks are generated synchronously per time slot (τ = 1 s). Following the three-category classification in Section II, CPU-intensive, GPU-intensive, and I/O-intensive tasks follow a 5:4:1 ratio. CPU-intensive tasks involve data sizes of 64–262 KB and require 500–2000 CPU cycles per bit [16]. GPU-intensive tasks process 6.4–26.2 KB inputs, occupy 5120–14592 CUDA cores with 2–16 GB memory demand, and require 1010 –1011 FLOPS. I/O-intensive tasks transmit lightweight payloads (640 B–2.6 KB) with negligible computation. Task priorities are uniformly sampled from {1, 2, 3, 4}. Deadlines are categoryspecific (1–1.5 s for CPU/GPU tasks; 0.5–1 s for I/O tasks) and min–max normalized. Three task-generation rates (10, 20, and 40 tasks per slot) are evaluated to represent low-, medium-, and high-load conditions, with 20 tasks per slot as the nominal operating point. Wireless channels follow the path-loss model in Eq. 9 with G0 = 10−3 , d0 = 1 m, and γ ∈ [1.6, 3.5]. Bandwidth is set to 10 MHz, and noise power is fixed at N0 = 10−13 W [17]. Device mobility follows a bounded random walk, with initial device-server distances uniformly distributed within [1500, 7500] m. 3) Performance Metrics and Evaluation Framework: We evaluate system performance using the following metrics: • Task Completion Rate: Proportion of tasks completed within deadlines. • Energy Consumption: Aggregate CPU, GPU, and transmission energy usage. • Execution Latency: Total system latency Lsys = PN i i=1 Dtot , capturing queuing, processing, and transmission latency. Each experiment runs for 800 episodes, generating tasks in the first five time slots, and uses multiple independent runs with distinct random seeds for statistical robustness. System Load and Statistical Validation. System load is characterized using the comprehensive load metric Lt defined in Eq. 5, evaluated under task-generation rates of 10, 20, and 40 tasks per slot. All results report averages with 95% confidence intervals based on t-distribution, with statistical significance assessed using paired t-tests at α = 0.05. B. Baseline Algorithms and Experimental Design This section presents the baseline algorithms and experimental settings used for comparison with H I RL under diverse system conditions. 1) Baseline Algorithm Categories and Specifications: Classical Heuristic Algorithms: These algorithms represent fundamental approaches to task allocation in distributed systems, providing performance lower bounds and basic algorithmic intuition validation. • Random Allocation: Uniformly random task placement across available execution locations (local execution and 5 edge servers), representing the baseline performance floor for comparison purposes. • Greedy Local/offloading: Includes two variants—(i) local-first allocation, preferring local execution until

8

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

CPU/GPU capacity is exhausted, and (ii) offloadingfirst allocation, prioritizing edge servers with the highest available computation capacity to minimize local energy consumption. • Round Robin: Sequentially assigns tasks across edge servers for balanced workload distribution, without considering compatibility or dynamic system state. Advanced Optimization Methods: These baselines reflect current optimization trends in mobile edge computing and reinforcement learning. QPSO [18]: A Quantum-behaved Particle Swarm Optimization method that formulates task offloading as a global search problem using quantum-inspired particle evolution, but lacks adaptation to time-varying system states. • Single-DDQN: A single-agent Double Deep Q-Network for task allocation with fixed transmission power, excluding hierarchical coordination to isolate its contribution. •

Ablation Study Variants: To evaluate the contribution of individual mechanisms within H I RL, four ablation models are implemented. H I RL-NoCoord: Removes the coordination module, allowing TD3 and DDQN to operate independently without shared states or sequential decision flow. • H I RL-NoGPU: Disables GPU-aware compatibility scoring (Eq. 16), using simplified resource allocation without GPU specification consideration. • H I RL-NoDeadline: Removes deadline-oriented queue prioritization (Eq. 17), replacing it with a First-ComeFirst-Served policy. • H I RL-NoFailure: Disables failure-penalized experience replay from Eq. 14, using uniform experience sampling in DDQN training. •

2) Experimental Parameter Configuration: Physical System Parameters: The effective switching capacitance is set to k = 10−28 [19], time slot duration is τ = 1s, and maximum transmission power varies between Pmax = 2−3W for mobile devices. The maximum CPU frequency is set within the range fmax = 2 − 3 GHz to reflect typical consumer mobile device capabilities. Learning Algorithm Parameters: In DDQN, the learning rate was configured to 1×10−4 . For TD3, independent learning rates were adopted for the actor (1×10−3 ) and critic (5×10−4 ) networks. The experience replay buffer sizes were set to 600,000 for DDQN and 5,000 for TD3, respectively. Both algorithms were trained using a batch size of 256. Regarding target network updates, DDQN applied a hard update every 1,000 training steps, whereas TD3 followed a soft-update scheme with a smoothing coefficient of τ = 0.001. Coordination and Adaptive Mechanism Parameters: The coordination engine parameters include an adaptive weight sensitivity factor of γ = 0.1 in load metric calculation and a failure penalty factor α = 1.0 for experience prioritization. The load metric weighting factors are set as ωq = 0.6, ωu = 0.2, and ωg = 0.2 to prioritize queue-based load assessment while incorporating resource utilization.

Exploration Strategy Parameters: Exploration is controlled by ϵmin = 0.01 and ϵmax = 0.3, with an exponential decay schedule adapted to system load conditions as defined in Eq. 15. The load adaptation sensitivity is controlled by α = 0.5 and β = 2.0 to balance exploration and exploitation across varying operational conditions. 3) Experimental Design Methodology: Controlled Variable Design: Controlled variables include system load intensity, infrastructure heterogeneity, and workload characteristics. Primary controlled variables include system load intensity (via SLI categorization), infrastructure heterogeneity (device and server configurations), and workload characteristics (task type distributions and priority patterns). Performance Comparison Framework: All algorithms operate under identical environmental conditions within each scenario. Random seed control ensures reproducibility, while systematic variation of environmental parameters evaluates robustness. Statistical Significance Validation: Each algorithm is evaluated through multiple independent runs. Statistical significance is assessed using paired t-tests, with consistency verified across different random seeds. Fairness and Evaluation Consistency: All algorithms are allocated identical computational resources and training opportunities. Learning-based baselines are trained for equivalent durations with consistent hyperparameter optimization to ensure fair comparison. C. Overall Performance Evaluation This section evaluates H I RL with respect to task completion (primary guarantee), execution latency, and energy efficiency under various operational conditions. 1) Experimental Setup and Metrics: We define the System Load Index (SLI) as SLI(t) = Lt × 100% based on the comprehensive load metric Lt from Eq. 5, where Lt ∈ [0, 1] reflects overall system stress by integrating queue saturation and heterogeneous CPU/GPU resource utilization. Three operational regimes are defined according to the SLI distribution: low load (10–30%), medium load (30–60%), and high load (60–90%). Our evaluation covers three core metrics—task completion rate, execution latency, energy consumption—measured across all SLI regimes. Statistical consistency is ensured through multiple independent runs. 2) Aggregate Performance Analysis: Fig. 3 presents overall results under varying system loads, while Fig. 4 visualizes task-level distributions through CDF heatmaps, where color gradients denote completion percentiles. Task Completion Rate. As shown in Fig. 3(c), H I RL sustains nearly 100% completion across all load levels, confirming the effectiveness of coordinated queue management and GPU-aware matching. Single-DDQN maintains high rates (above 97%) but shows mild degradation under high load due to the absence of joint power control. QPSO and GreedyLocal methods experience notable drops (down to about 80%), and Random or Round-Robin allocation falls below 50%, indicating limited adaptability under resource contention (high load).

ZHU et al.: HIRL: HIERARCHICAL REINFORCEMENT LEARNING FOR COORDINATED RESOURCE MANAGEMENT IN HETEROGENEOUS EDGE COMPUTING 9

+,5/

*UHHG\/RFDO



4362

5DQGRP

5RXQG5RELQ 

[

 

[

 

[



F7DVNV&RPSOHWLRQ5DWH





E7RWDO(QHUJ\-

D([HFXWLRQ/DWHQF\V

*UHHG\2IIORDGLQJ

[

[



[



 

/RZ

0HGLXP



+LJK

/RDG/HYHO

/RZ

0HGLXP

 

[

 

[

 [

 

+LJK

/RDG/HYHO

6LQJOH''41

/RZ

0HGLXP

+LJK

/RDG/HYHO

Fig. 3. Overall performance comparison across different system load levels. >@+,5/3URSRVHG_>@6LQJOH''41_>@4362_>@*UHHG\/RFDO_>@*UHHG\2IIORDGLQJ_>@5DQGRP_>@5RXQG5RELQ

D

/RZ/RDG

3

3

3

3

3

3

>@ >@ >@ >@ >@ >@ >@ 











7DVN&RPSOHWLRQ7LPHV













7DVN&RPSOHWLRQ7LPHV









7DVN(QHUJ\&RQVXPSWLRQ-

3

3

 











7DVN&RPSOHWLRQ7LPHV



3



>@ >@ >@ >@ >@ >@ >@ 











7DVN(QHUJ\&RQVXPSWLRQ-







I

3

>@ >@ >@ >@ >@ >@ >@ 

3

>@ >@ >@ >@ >@ >@ >@ 

H

3

>@ >@ >@ >@ >@ >@ >@

+LJK/RDG

F

&')

>@ >@ >@ >@ >@ >@ >@

G

0HGLXP/RDG

E



 





7DVN(QHUJ\&RQVXPSWLRQ-



&RORU*UHHQ %HWWHU_3XUSOH%RUGHU+,5/_:KLWH'DVKHG3_<HOORZ'DVKHG3_%OXH3

Fig. 4. Task-level performance distribution via CDF heatmaps across load levels.

Execution Latency. The hierarchical coordination demonstrates significant latency advantages across all load conditions. As shown in Fig. 3(a) and validated by the tasklevel distributions in Fig. 4(a-c), Single-DDQN exhibits 12.6% higher latency under low load (1160.87s vs. 1031.36s), 9.8% under medium load (2665.46s vs. 2446.05s), and 5.8% under high load (5896.34s vs. 5572.60s), directly quantifying TD3 power control contributions. The CDF heatmaps reveal that H I RL’s predominantly blue regions (indicating faster task completion) extend across broader value ranges compared to baselines’ red-shifted patterns, with P50 markers consistently positioned leftward, confirming sustained per-task efficiency gains. Energy Efficiency. As shown in Fig. 3(b), H I RL consistently achieves lower end-to-end latency than Single-DDQN, with reductions of 12.6% under low load (1031.36s vs. 1160.87s), 9.8% under medium load (2446.05s vs. 2665.46s), and 5.8% under high load (5572.60s vs. 5896.34s). These improvements stem from TD3-based power control that jointly

optimizes CPU–GPU utilization. The CDF distributions in Fig. 4(d-f) further confirm faster completion, as reflected by left-shifted P50 markers and broader blue regions representing higher completion percentiles. The energy gap diminishing under higher loads as coordination focuses on completion rate maintenance amid system-wide resource constraints. 3) Energy-Latency Trade-off Distribution Analysis: Fig. 5 illustrates the energy-latency distribution from 20 independent runs, highlighting the stability and optimization effectiveness of H I RL’s hierarchical coordination beyond the aggregate performance metrics shown in Fig. 3. Coordination Stability: H I RL demonstrates tight clustering of performance points across all load conditions, with the majority located in the lower-left quadrant of the energylatency space. This indicates that the hierarchical coordination, as defined in Eq. 3, ensures consistent power control and task allocation. In contrast, baseline algorithms show much wider dispersion—Random and Round-Robin scatter in the upperright regions, while QPSO exhibits some clustering but at

10

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

HIRL

Single-DDQN

QPSO

Greedy-Local

Low Load

Round-Robin

High Load

1200

2000

1000

600

1500

800 400

600

1000

400

200

500

200 0 700

Random

1400

800

Total Energy (J)

Greedy-Offloading

Medium Load

800

900

1000 1100 1200 1300

Average Latency (s)

2200

2400

2600

2800

3000

Average Latency (s)

3200

5500 5750 6000 6250 6500 6750 7000 7250

Average Latency (s)

D. Task-Type-Specific Performance Contribution Analysis Fig. 6 shows the performance contribution across different task categories, validating H I RL’s resource orchestration under the 5:4:1 task distribution. Latency Contribution: Fig. 6(a) shows CPU-intensive tasks account for 52-56% of latency, GPU-intensive tasks contribute 42-45%, and I/O-intensive tasks contribute only 23%, as 97% of them are executed locally. Energy Decomposition: Fig. 6(b) highlights the separation of local and transmission energy usage. I/O-intensive tasks use negligible transmission energy (0.1%), confirming local execution prioritization. CPU-intensive tasks show a balanced 3:2 local-to-edge execution energy distribution, while GPUintensive tasks require 65% of their energy for edge offloading, offset by lower local consumption due to coordination. Under high load, transmission energy accounts for 33.4%, validating the effectiveness of power control coordination. Resource Orchestration Validation: The decrease in I/O task contribution from 3% to 2% at higher loads shows adaptive prioritization, while the stable CPU-GPU balance demonstrates effective compatibility scoring. Despite GPUintensive tasks only accounting for 40% of the workload,

D/DWHQF\&RQWULEXWLRQE\7DVN7\SH 

,2LQWHQVLYH7DVNV &38LQWHQVLYH7DVNV *38LQWHQVLYH7DVNV



7RWDO(QHUJ\&RQWULEXWLRQ

higher energy levels, reflecting its suboptimal efficiency. Hierarchical Optimization Benefits: H I RL is consistently positioned in the energy-efficient, low-latency region, with a clear trade-off advantage. Single-DDQN, while matching latency performance, consumes 30-50% more energy due to its lack of coordinated power management. Classical approaches exhibit clear trade-offs: Greedy-Local shows moderate energy consumption but suffers from higher latency, while GreedyOffloading balances latency at the cost of excessive energy consumption, forming vertical clusters in the energy dimension. Load Scalability: As the system load increases from low to high, H I RL maintains its compact distribution, while baseline algorithms show degradation and greater variance. The stable positioning of H I RL close to the optimal frontier confirms that its coordination mechanism scales effectively with system load. Baseline algorithms exhibit increased dispersion and a shift towards less efficient high-energy, high-latency regions, underscoring the importance of adaptive coordination in dynamic edge environments.

7RWDO6\VWHP/DWHQF\&RQWULEXWLRQ

Fig. 5. Energy-Latency Trade-off Analysis Across Different Load Levels.



/RZ



/RFDO([HFXWLRQ 7UDQVPLVVLRQ

0HGLXP 6\VWHP/RDG/HYHO E(QHUJ\&RQWULEXWLRQ/RFDOYV7UDQVPLVVLRQ

+LJK

0HGLXP 6\VWHP/RDG/HYHO

+LJK

  

/RZ

Fig. 6. Task-type-specific performance contribution analysis.

they still validates 42-45% to latency, confirming that GPUaware mechanisms optimize task-resource matching, prevent constraint violations, and balance the energy-latency trade-off.

E. Ablation Study Analysis Table II and Fig. 7 quantify the impact of key components through systematic ablation experiments. Removing coordination (H I RL-NoCoord) or GPU-aware matching (H I RL-NoGPU) leads to similar performance degradation: task completion rates drop by 0.57–2.88%, energy consumption increases by 49–82% (Fig. 7(a)), and latency rises by 5.46–11.46% (Fig. 7(b)). These results indicate that both components are essential for managing heterogeneous resources, not just incremental refinements. The increasing degradation under higher loads further confirms that coordinated power control (Eq. 3) and compatibility scoring (Eq. 16) are critical to maintaining efficiency as system stress grows. H I RL-NoDeadline causes limited performance loss (≤ 4.72% latency increase), showing that deadline prioritization (Eq. 17) improves scheduling efficiency but is not a core stability factor. disabling failure-aware learning (H I RLNoFailure) results in severe instability: completion rates fall to 62%, and energy consumption increases sevenfold under high load (Fig. 7(c)). Despite lower latency at low load, this variant fails to sustain operations as stress increases. Although latency appears lower under light load, the system quickly collapses as constraints tighten. Uniform sampling fails to

ZHU et al.: HIRL: HIERARCHICAL REINFORCEMENT LEARNING FOR COORDINATED RESOURCE MANAGEMENT IN HETEROGENEOUS EDGE COMPUTING 11

250

82% increase at Medium Load

247

200 150 100 50 0

Low

Medium

(a) Energy Impact

High

10

5000

5

4000

0

3000

5

2000

10

Lower latency but tasks fail!

15

Low

Medium

(b) Latency Impact

High

1000

w/o Failure-Aware

100

6000

Average Latency (ms)

300

w/o Deadline Prioritization

15

Latency Impact (%)

Energy Consumption (J)

350

w/o GPU-Aware

93.0%

81.0%

80

1199J

60 40

7× energy increase!

2 × 103

62.0% 103

647J

Without Failure-Aware: System Collapse

20 0

3 × 103

1943J

Low

Medium

High

(c) Failure-Aware Learning Impact

Energy Consumption (J)

w/o Coordination

Completion Rate (%)

HIRL (Complete)

6 × 102 4 × 102

Fig. 7. Ablation study: component contributions to system performance across load levels. TABLE II A BLATION S TUDY: I NDIVIDUAL C OMPONENT C ONTRIBUTIONS TO H I RL P ERFORMANCE

Load Level Low Load Medium Load High Load Low Load Medium Load High Load Low Load Medium Load High Load Low Load Medium Load High Load Low Load Medium Load High Load

Task Completion Rate (%) Average Latency (s) Energy Consumption (J) Performance Performance Performance Std Mean Std Mean Std Impact (%) Impact (%) Impact (%) H I RL (Complete Framework) 1.00 0.00 Baseline 1040.42 24.38 Baseline 66.11 2.28 Baseline 1.00 0.00 Baseline 2415.63 38.16 Baseline 135.52 3.34 Baseline 1.00 0.00 Baseline 5577.25 50.60 Baseline 271.1 10.75 Baseline H I RL-NoCoord (No Coordination Engine) 0.99 0.00 -0.57 1097.26 30.56 5.46 98.64 6.81 49.21 0.97 0.01 -2.88 2654.44 35.67 9.89 247.13 16.84 82.35 0.97 0.00 -2.72 6216.38 45.03 11.46 314.66 8.96 16.05 H I RL-NoGPU (No GPU-Aware Compatibility Assessment) 0.99 0.00 -0.63 1097.57 26.89 5.49 99.22 5.44 50.08 0.97 0.00 -2.88 2646.67 37.40 9.56 245.32 19.42 81.02 0.97 0.00 -2.70 6218.68 48.04 11.50 314.54 8.55 16.01 H I RL-NoDeadline (No Deadline-Oriented Queue Prioritization) 1.00 0.00 0.00 1072.61 27.23 3.09 66.33 2.30 0.34 1.00 0.00 0.00 2525.87 39.09 4.56 135.02 3.62 -0.37 1.00 0.00 -0.04 5840.66 47.90 4.72 272.28 17.14 0.42 H I RL-NoFailure (No Failure-Penalized Experience Replay) 0.93 0.01 -6.60 946.96 23.88 -8.98 647.44 22.30 879.34 0.81 0.01 -19.28 2461.35 43.27 1.89 1199.12 19.99 784.81 0.62 0.01 -38.02 6013.90 57.88 7.83 1943.33 40.78 616.73

Mean

TABLE III PARAMETER S ENSITIVITY A NALYSIS Parameter (Equation) λ (Eq. 1)

λe (Eq. 11)

γf ail (Eq. 14)

β (Eq. 15)

Value 0.7 0.8 0.9 0.4 0.5 0.6 1.0 1.5 2.0 1.5 2.0 2.5

Cumulative Latency (s) 2432.8 2415.6 2435.9 2448.3 2415.6 2446.4 2432.8 2398.2 2415.7 2423.9 2398.2 2407.4

Energy (J) 197.8 135.5 154.3 160.9 135.5 162.0 185.2 178.6 182.4 174.1 178.6 181.8

Completion Rate (%) 99.9 100.0 100.0 100.0 100.0 99.7 99.78 100.0 99.89 99.89 100.0 99.94

capture critical failure states, while failure-aware prioritization (Eq. 14) enables stable learning under resource pressure. Overall, the results highlight a clear hierarchy among system components: coordination and GPU-awareness form the performance foundation, while failure-aware learning provides robustness under dynamic and high-load conditions. F. Parameter Sensitivity Analysis Table III presents the sensitivity analysis for key parameters governing H I RL’s hierarchical coordination mechanisms. The

results reveal stable parameter regions that balance energy efficiency, latency optimization, and task completion guarantees under varying system conditions. The energy–latency tradeoff parameter λ in Eq. 1 exhibits a monotonic trend: larger λ values prioritize latency reduction through higher energy allocation. Specifically, λ = 0.8 achieves the optimal balance, yielding a 100% completion rate with minimal latency (2415.6 s) at moderately increased energy consumption (135.5 J). The power penalty coefficient λe in Eq. 11 shows an inverse relationship between energy and latency: λe = 0.5 achieves the optimal balance, yielding a 100% completion rate with minimal latency (2415.6 s) at moderately increased energy consumption (135.5 J). The failure-aware learning factor γf ail in Eq. 14 achieves the best overall performance at γf ail = 1.5, where failure experience amplification strikes a balance between robust learning and efficient exploration. The load adaptation coefficient β in Eq. 15 induces minimal performance variation around its optimal value of 2.0, demonstrating the framework’s robustness against exploration–exploitation imbalance in dynamic environments. Fig. 8 further examines the interaction between λ (en-

12

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

3DUDPHWHU,QWHUDFWLRQ$QDO\VLV YV BIDLO







































BIDLO

E(QHUJ\&RQVXPSWLRQ     





















F&RQYHUJHQFH(SLVRGHV   





























 

BIDLO



























































BIDLO

7UDLQLQJ(SLVRGHV



(QHUJ\/DWHQF\7UDGHRII



(QHUJ\&RQVXPSWLRQ



(QHUJ\/DWHQF\7UDGHRII



&RPSOHWLRQ5DWH

(QHUJ\/DWHQF\7UDGHRII

D7DVN&RPSOHWLRQ5DWH 

Fig. 8. Parameter interaction analysis between λ and γf ail .

ergy–latency tradeoff from Eq. 1) and γf ail (failure-aware learning factor from Eq. 14), the two most critical parameters in task allocation. These jointly determine coordinated energy–latency optimization, where λ governs the global cost function and γf ail enhances DDQN adaptation through prioritized experience replay. The completion rate heatmap in Fig. 8(a) indicates peak performance at (λ = 0.8, γf ail = 1.5). The energy patterns in Fig. 8(b) confirm that energy consumption increases monotonically with λ but stabilizes near γf ail = 1.5. Finally, the convergence curves in Fig. 8(c) show that this parameter pair also achieves the fastest convergence, validating the framework’s training efficiency and adaptability in heterogeneous environments.

G. System Overhead Analysis The hierarchical architecture introduces per-slot overhead through sequential three-stage coordination. TD3 power control executes M independent inferences (one per device) with lightweight state processing, followed by coordination operations updating transmission rates and normalized queue states with O(M ) complexity. DDQN task allocation executes K linear inferences for task decisions, with GPU-intensive workloads requiring additional compatibility assessment against N servers through hardware specification comparisons. The global five-dimensional queue structure enables constant-time state updates independent of device count. Per-slot decision latency spans several milliseconds, remaining negligible relative to task execution times measured in seconds. Scalability exhibits linear growth O(M ) with device population through independent per-device power decisions and parallel state processing. Server expansion affects discrete action space dimensionality (N + 1 choices) and compatibility evaluation, with neural network inference scaling sublinearly and compatibility scoring maintaining O(N ) per-task complexity. Training demonstrates 35-40% faster convergence versus monolithic baseline through hierarchical decomposition separating continuous power control and discrete task allocation into simpler subproblems. Training occurs offline without deployment impact. The sequential coordination overhead proves acceptable for time-slot-based operations, with decision latency constituting low single-digit percentages of typical task execution durations.

H. Simulation Validity and Deployment Insights H I RL’s architecture explicitly integrates physical constraints into its decision-making process. The five-dimensional queue abstraction (Eq. 4) models CPU/GPU pipeline backlogs, while the dual-phase heterogeneous pipeline captures GPU memory contention. TD3-based power control enforces transmission and processing limits consistent with commercial MEC hardware. DDQN’s compatibility scoring (Eq. 16) leverages standardized GPU specifications (e.g., CUDA cores, memory), and the failure-penalized replay mechanism (Eq. 14) accelerates convergence by prioritizing constraint-violating experiences. Simulation environments reproduce realistic dynamics—such as channel fading, stochastic task arrivals, and resource contention—ensuring that learned policies generalize to real-world uncertainties. Emergency Scenario Validation. A 60-second vehicular emergency involving eight devices, five heterogeneous servers (Section V-A1), and a 3GPP-compliant network with 184 tasks validates the robustness of H I RL’s coordination mechanisms. 1) Channel adaptation (t=4–6s): Under 18–24 dB signal degradation, H I RL adjusts transmission power within 2–3 W and redirects GPU tasks via Stage-2 coordination, achieving zero deadline violations across 23 task arrivals, whereas Single-DDQN records three misses. 2) Saturation handling (t=7s): When concurrent requests drive server memory to 93% utilization (14.7 GB/16 GB), compatibility scoring identifies two infeasible allocations and triggers local fallback, avoiding failures observed in Random (5/8) and Round-Robin (6/8) baselines. 3) Energy efficiency: Through coordinated scheduling, H I RL reduces system-wide energy consumption by 26% compared to Single-DDQN, consistent with Fig. 3. Deployment Considerations. Real-world deployment usually need to consider the following aspects: 1) device-specific profiling to calibrate GPU kernel overheads across driver versions; 2) system-level fault handling (e.g., watchdog, thermal management) to complement failure-aware learning; and 3) inter-cluster coordination for geographically distributed RSU networks, extending beyond the current intra-cluster model (N = 5). Our implementation supports trace-based calibration for site-specific adaptation, providing a practical foundation for real-world MEC deployment. VI. R ELATED W ORK Latency-Optimal Resource Orchestration. Early MEC studies focused on latency minimization via task scheduling

ZHU et al.: HIRL: HIERARCHICAL REINFORCEMENT LEARNING FOR COORDINATED RESOURCE MANAGEMENT IN HETEROGENEOUS EDGE COMPUTING 13

and offloading optimization [11]. Later work incorporated mobility awareness through Bi-LSTM-based trajectory prediction [14], collaborative control using advantage actor–critic learning [20], [21], and queue-theoretic offloading models regulated by reinforcement learning [22]. Digital-twin-based designs were further introduced to address uncertainty in resource estimation [23]. Most of these approaches assume homogeneous computing units, which limits their applicability in heterogeneous CPU-GPU infrastructures. Energy-Optimal Resource Management. A few studies progressed from device-level power tuning to network-wide coordination through multi-agent reinforcement learning for UAV–MEC networks [24], [25], vehicular offloading with edge caching [26], [27], and non-orthogonal multiple access schemes emphasizing transmission efficiency [28], [29]. Physical-layer techniques, such as reconfigurable intelligent surfaces [30], [31], further improve energy utilization. However, they decouple communication and computation that comes with suboptimal decisions when transmission power and processing capacity are not jointly optimized. Joint Energy–Latency Optimization. Recent work has explored joint energy-latency optimization through gametheoretic modeling [32], [33], online UAV-assisted control [15], [34], and reinforcement learning under partial observability [35], [36]. Domain-specific frameworks further consider green-energy-driven offloading [37], renewablepowered vehicular networks [38], and dependency-aware task scheduling [39], [40]. Nevertheless, unified optimization of continuous power control and discrete task allocation remains challenging, particularly in heterogeneous CPU-GPU environments where task-resource compatibility is critical. To address such limitations, this work proposes a hierarchical learning framework that separates continuous and discrete decision spaces: the upper tier applies TD3 for power control, while the lower tier employs DDQN for task allocation. Coordination mechanisms align both tiers, and GPU-aware compatibility assessment enables efficient resource matching in heterogeneous edge systems. VII. C ONCLUSION This paper investigates resource orchestration for heterogeneous MEC systems characterized by multi-dimensional device heterogeneity, diverse task requirements, and strict timing constraints. Experimental evaluation shows that the proposed hierarchical TD3–DDQN framework attains over 99% task completion, reduces average latency by 28%, and maintains balanced energy–performance trade-offs. The framework integrates GPU-aware compatibility assessment, deadline-sensitive queue management, and failure-aware adaptive sampling, enabling stable performance under dynamic and resource-constrained conditions. Future extensions include adapting the compatibility model to new accelerator architectures, exploring predictive task allocation guided by mobility patterns, and validating the hierarchical coordination across broader hardware settings. These directions aim to strengthen the applicability of hierarchical learning frameworks in increasingly heterogeneous edge environments.

R EFERENCES [1] P. Mach and Z. Becvar, “Mobile edge computing: A survey on architecture and computation offloading,” IEEE communications surveys & tutorials, vol. 19, no. 3, pp. 1628–1656, 2017. [2] A. Nasir, X. He, T. Wang, H. Shi, and Z. Wang, “Relto: A reliabilityoriented drl approach with context-aware adaptive reward weighting for multi-objective task offloading in mec,” Ad Hoc Networks, p. 104065, 2025. [3] Q. Zeng, Y. Du, K. Huang, and K. K. Leung, “Energy-efficient resource management for federated edge learning with cpu-gpu heterogeneous computing,” IEEE Transactions on Wireless Communications, vol. 20, no. 12, pp. 7947–7962, 2021. [4] A. Islam and M. Ghose, “Delta: Deadline aware energy and latencyoptimized task offloading and resource allocation in gpu-enabled, pimenabled distributed heterogeneous mec architecture,” Journal of Systems Architecture, vol. 159, p. 103335, 2025. [5] S. H. Lee, “Real-time edge computing on multi-processes and multithreading architectures for deep learning applications,” Microprocessors and Microsystems, vol. 92, p. 104554, 2022. [6] N. Kim, Y. Lee, C. Lee, T. V. Nguyen, V. D. Tuong, and S. Cho, “Gpu-specific task offloading in the mobile edge computing network,” in 2020 International Conference on Information and Communication Technology Convergence (ICTC). IEEE, 2020, pp. 1874–1876. [7] M. Y. Akhlaqi and Z. B. M. Hanapi, “Task offloading paradigm in mobile edge computing-current issues, adopted approaches, and future directions,” Journal of Network and Computer Applications, vol. 212, p. 103568, 2023. [8] Z. Wang, B. Lin, Q. Ye, Y. Fang, and X. Han, “Joint computation offloading and resource allocation for maritime mec with energy harvesting,” IEEE Internet of Things Journal, vol. 11, no. 11, pp. 19 898–19 913, 2024. [9] C. Hou and Q. Zhao, “Optimal control of wireless powered edge computing system for balance between computation rate and energy harvested,” IEEE Transactions on Automation Science and Engineering, vol. 20, no. 2, pp. 1108–1124, 2022. [10] L. Pan, X. Liu, Z. Jia, J. Xu, and X. Li, “A multi-objective clustering evolutionary algorithm for multi-workflow computation offloading in mobile edge computing,” IEEE Transactions on Cloud Computing, vol. 11, no. 2, pp. 1334–1351, 2021. [11] P. Kalpana, M. Almusawi, Y. Chanti, V. S. Kumar, and M. V. Rao, “A deep reinforcement learning-based task offloading framework for edge-cloud computing,” in 2024 International Conference on Integrated Circuits and Communication Systems (ICICACS). IEEE, 2024, pp. 1–5. [12] Z. Cao, X. Deng, S. Yue, P. Jiang, J. Ren, and J. Gui, “Dependent task offloading in edge computing using gnn and deep reinforcement learning,” IEEE Internet of Things Journal, vol. 11, no. 12, pp. 21 632– 21 646, 2024. [13] Y. Chen, W. Gu, J. Xu, Y. Zhang, and G. Min, “Dynamic task offloading for digital twin-empowered mobile edge computing via deep reinforcement learning,” China Communications, vol. 20, no. 11, pp. 164–175, 2023. [14] F. Zeng, Z. Zhang, and J. Wu, “Task offloading delay minimization in vehicular edge computing based on vehicle trajectory prediction,” Digital Communications and Networks, vol. 11, no. 2, pp. 537–546, 2025. [15] X. Dai, Z. Xiao, H. Jiang, and J. C. Lui, “Uav-assisted task offloading in vehicular edge computing networks,” IEEE Transactions on Mobile Computing, vol. 23, no. 4, pp. 2520–2534, 2023. [16] A. P. Miettinen and J. K. Nurminen, “Energy efficiency of mobile clients in cloud computing,” in 2nd USENIX workshop on hot topics in cloud computing (HotCloud 10), 2010. [17] T. L. Marzetta, “Noncooperative cellular wireless with unlimited numbers of base station antennas,” IEEE transactions on wireless communications, vol. 9, no. 11, pp. 3590–3600, 2010. [18] S. Dong, Y. Xia, and J. Kamruzzaman, “Quantum particle swarm optimization for task offloading in mobile edge computing,” IEEE Transactions on Industrial Informatics, vol. 19, no. 8, pp. 9113–9122, 2022. [19] Y. Mao, J. Zhang, and K. B. Letaief, “Dynamic computation offloading for mobile-edge computing with energy harvesting devices,” IEEE Journal on Selected Areas in Communications, vol. 34, no. 12, pp. 3590– 3605, 2016. [20] L. Liu, J. Feng, X. Mu, Q. Pei, D. Lan, and M. Xiao, “Asynchronous deep reinforcement learning for collaborative task computing and ondemand resource allocation in vehicular edge computing,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 12, pp. 15 513–15 526, 2023.

14

[21] J. Yang, J. Lu, X. Zhou, S. Li, C. Xiong, and J. Hu, “Ha-a2c: Hard attention and advantage actor-critic for addressing latency optimization in edge computing,” IEEE Transactions on Green Communications and Networking, vol. 9, no. 1, pp. 207–217, 2024. [22] C. Yang, H. Xu, S. Fan, X. Cheng, M. Liu, and X. Wang, “Efficient resource allocation policy for cloud edge end framework by reinforcement learning,” in 2022 IEEE 8th International Conference on Computer and Communications (ICCC). IEEE, 2022, pp. 1363–1367. [23] C. Xu, Z. Tang, H. Yu, P. Zeng, and L. Kong, “Digital twin-driven collaborative scheduling for heterogeneous task and edge-end resource via multi-agent deep reinforcement learning,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 10, pp. 3056–3069, 2023. [24] J. Xue, Q. Wu, and H. Zhang, “Cost optimization of uav-mec network calculation offloading: A multi-agent reinforcement learning method,” Ad Hoc Networks, vol. 136, p. 102981, 2022. [25] A. Mondal, D. Mishra, G. Prasad, and H. Johansson, “Joint trajectory, user-association, and power control for green uav-assisted data collection using deep reinforcement learning,” IEEE Transactions on Intelligent Vehicles, 2024. [26] X. Kong, G. Duan, M. Hou, G. Shen, H. Wang, X. Yan, and M. Collotta, “Deep reinforcement learning-based energy-efficient edge computing for internet of vehicles,” IEEE Transactions on Industrial Informatics, vol. 18, no. 9, pp. 6308–6316, 2022. [27] H. Wu, J. Zhang, Z. Cai, F. Liu, Y. Li, and A. Liu, “Toward energy-aware caching for intelligent connected vehicles,” IEEE Internet of Things Journal, vol. 7, no. 9, pp. 8157–8166, 2020. [28] Y. Chen, J. Xu, Y. Wu, J. Gao, and L. Zhao, “Dynamic task offloading and resource allocation for noma-aided mobile edge computing: An energy efficient design,” IEEE Transactions on Services Computing, vol. 17, no. 4, pp. 1492–1503, 2024. [29] Z. Ding, J. Xu, O. A. Dobre, and H. V. Poor, “Joint power and time allocation for noma–mec offloading,” IEEE Transactions on Vehicular Technology, vol. 68, no. 6, pp. 6207–6211, 2019. [30] Z. Zhai, X. Dai, B. Duo, X. Wang, and X. Yuan, “Energy-efficient uavmounted ris assisted mobile edge computing,” IEEE wireless communications letters, vol. 11, no. 12, pp. 2507–2511, 2022. [31] Y. Yang, Y. Gong, and Y.-C. Wu, “Intelligent-reflecting-surface-aided mobile edge computing with binary offloading: Energy minimization for iot devices,” IEEE Internet of Things Journal, vol. 9, no. 15, pp. 12 973–12 983, 2022. [32] H. Teng, Z. Li, K. Cao, S. Long, S. Guo, and A. Liu, “Game theoretical task offloading for profit maximization in mobile edge computing,” IEEE Transactions on Mobile Computing, vol. 22, no. 9, pp. 5313–5329, 2022. [33] B. Wu, J. Zeng, L. Ge, X. Su, and Y. Tang, “Energy-latency aware offloading for hierarchical mobile edge computing,” IEEE Access, vol. 7, pp. 121 982–121 997, 2019. [34] K. Zhang, X. Gui, D. Ren, and D. Li, “Energy–latency tradeoff for computation offloading in uav-assisted multiaccess edge computing system,” IEEE Internet of Things Journal, vol. 8, no. 8, pp. 6709–6719, 2020. [35] A. Robles-Enciso and A. F. Skarmeta, “A multi-layer guided reinforcement learning-based tasks offloading in edge computing,” Computer Networks, vol. 220, p. 109476, 2023. [36] L. Ale, N. Zhang, X. Fang, X. Chen, S. Wu, and L. Li, “Delay-aware and energy-efficient computation offloading in mobile-edge computing using deep reinforcement learning,” IEEE Transactions on Cognitive Communications and Networking, vol. 7, no. 3, pp. 881–892, 2021. [37] H. Ma, P. Huang, Z. Zhou, X. Zhang, and X. Chen, “Greenedge: Joint green energy scheduling and dynamic task offloading in multi-tier edge computing systems,” IEEE Transactions on Vehicular Technology, vol. 71, no. 4, pp. 4322–4335, 2022. [38] Q. Luo, C. Li, T. H. Luan, and W. Shi, “Minimizing the delay and cost of computation offloading for vehicular edge computing,” IEEE Transactions on Services Computing, vol. 15, no. 5, pp. 2897–2909, 2021. [39] F. Song, H. Xing, X. Wang, S. Luo, P. Dai, and K. Li, “Offloading dependent tasks in multi-access edge computing: A multi-objective reinforcement learning approach,” Future Generation Computer Systems, vol. 128, pp. 333–348, 2022. [40] C. Xu, M. Lv, K. Zhang, K. Cao, G. Wang, M. Wei, and B. Peng, “Energy consumption and time-delay optimization of dependency-aware tasks offloading for industry 5.0 applications,” IEEE Transactions on Consumer Electronics, vol. 70, no. 1, pp. 1590–1600, 2023.

IEEE TRANSACTIONS ON COMPUTERS, VOL. XX, NO. X, JANUARY XXXX

Jianyong Zhu is currently an assistant professor with the Department of Computing at North China Electric Power University. He received the Ph.D. degree from Beihang University in 2022. His research interests focus on distributed intelligent computing systems, including resource virtualization, learningenabled resource management, and heterogeneous computing in edge-centric environments.

Hao Chen is currently studying in the Computer Department of North China Electric Power University with a master’s degree. His research interests are edge computing, distributed systems and machine learning.

Juan Zhang (Member, IEEE) is a Lecturer (Assistant Professor) with the James Watt School of Engineering, University of Glasgow, UK. She received her Ph.D. degree in Computer Science from the University of Exeter, U.K. and subsequently was a postdoctoral researcher with the Chair of High-Performance Computing at Helmut-SchmidtUniversity/University of the Federal Armed Forces Hamburg, Germany. Her research interests include mobile edge computing, decision-making strategies, IoT, and future mobile networks. Fangda Guo is an assistant professor with the Institute of Computing Technology, Chinese Academy of Sciences. He received the Ph.D. degree from Northeastern University, China, in 2021. He was a joint Ph.D. candidate with the University of Edinburgh, UK. His research interests include social computing, data mining, graph management, query optimization and AI safety.

Albert Y. Zomaya (Fellow, IEEE) is the Peter Nicol Russell chair professor of computer science with the School of Computer Science, Sydney University, and serves as the director of the Centre for Distributed and High-Performance Computing. He has published more than 800 scientific papers and articles and is the author, co-author, or editor of more than 30 books. He is the past editor-in-chief of the IEEE Trans on Computers,the IEEE Trans on Sustainable Computing, and the ACM Computing Surveys. His research interests include parallel and distributed computing, networking, and complex systems. Renyu Yang (Member, IEEE) is an associate professor with the School of Software, Beihang University, China. Prior to this, he was with the University of Leeds UK, Alibaba Group China and Edgetic Ltd. UK, building large-scale computing/AI infrastructures. He is a recipient of Alan Turing Post-Doctoral Enrichment Award, 2022. His research interests include parallel and distributed computing, large-scale AI systems and software dependability.