1
Technical Report: A Hierarchical Dynamically Weighting Deep Reinforcement Learning Method for Multi-UAV Multi-Task Coordination
arXiv:2605.08623v1 [cs.NI] 9 May 2026
Xindi Wang, Haining Li, Tao Ding, Bolin Cai
Abstract—This paper investigates the multi-UAV multi-task coordination problem in infrastructure-less emergency scenarios, where UAVs collaboratively are required to jointly perform aerial image acquisition and ground-user communication. To tackle the challenge of balancing heterogeneous tasks within dynamic environments, we propose a hierarchical dynamic weighting Deep Reinforcement Learning (DRL) framework. Specifically, an episode-level module is introduced to capture global task preferences, while a step-level module adaptively adjusts the objective weights according to real-time system conditions. By integrating global and instantaneous weights, the proposed framework improves decision stability and responsiveness during task execution. Simulation results demonstrate that the proposed method achieves faster convergence, more stable training, and higher task completion efficiency than conventional works. Index Terms—unmanned aerial vehicles, multi-task coordination, deep reinforcement learning, dynamic weighting.
I. I NTRODUCTION With the development of cooperative multi-UAV systems for single-task execution, research interest has increasingly shifted toward intelligent multi-UAV multi-task systems. In such systems, multiple UAVs are required to collaboratively accomplish heterogeneous tasks under spatiotemporal constraints and limited onboard resources [1], [2]. However, due to the strong interdependence among subtasks and the potential conflicts between objectives, it remains challenging to simultaneously achieve all task requirements in dynamic environments [3]. Recently, Deep Reinforcement Learning (DRL) has demonstrated great potential in solving long-term sequential decisionmaking problems in dynamic environments [4]. Nevertheless, in DRL-based multi-UAV multi-task systems, the optimization of multi-UAV decision-making relies primarily on environmental reward feedback for their actions, which remains one of the core issues in current system design [5]. In general, the overall reward of multi-task is typically formulated as a weighted aggregation of the rewards of individual subtasks. In the context, some works design the weights in a static way, relying on manual tuning [6], [7] or static pre-configuration [8], [9], resulting in an inability to dynamically adjust to system features, and thus affecting overall performance. To overcome the limitations of static weighting schemes, recent works have explored dynamic reward weighting mechanisms [10]–[12] to improve adaptability to environmental variations. However, the existing methods generate weights mainly based on current scene and historical information, which causes significant fluctuations in the reward function. As a result, this reward inconsistency, induced by time-varying preferences, All authors are with the School of Internet, Anhui University, China; Corresponding author: Bolin Cai (email: [email protected]). This work was supported in part by NSFC under Grant 62202007.
can bias value estimation and degrade policy learning performance [12]. In summary, in the considered multi-UAV multitask dynamic systems, due to the drawbacks of weight design, existing DRL frameworks fail to efficiently adapt to dynamic environments where the importance of different subtasks may vary over time as resource availability and task completion status evolve. Consequently, reward design remains a major bottleneck in current DRL-based solutions. To address these challenge, a Hierarchical Dynamically Weighting mechanism based multi-agent multi-task DRL (HDWDRL) framework is proposed here, in which an episode-level Actor-Critic module is introduced at the weightlearning layer to capture global prior weight distributions, and a step-wise state-aware weighting network is used to analyze real-time system conditions. By combining the two above modules to construct a dynamic weighting framework, enabling balancing balance objectives among subtasks and adapt to the current environmental conditions. Simulation results demonstrate that, under the considered scenarios, the proposed framework exhibits better adaptability to dynamic environments, maintains stable training behavior, and improves execution efficiency in multi-task scenarios compared with conventional methods. II. S YSTEM M ODEL AND P ROBLEM F ORMULATION A. System Model This paper considers an emergency scenario where terrestrial communication infrastructure is completely unavailable. In such a situation, a multi-UAV system is deployed to simultaneously accomplish two heterogeneous tasks. The first task is aerial image acquisition for rapid disaster reconnaissance. The second task is to provide reliable emergency data links for densely distributed ground users (GUs). To support real-time coordination in the multi-UAV multi-task system, each UAV is equipped with an independent decision-making module, which determines its actions based on local observations and neighboring information exchanged through communication links. Moreover, UAVs make decisions sequentially according to their assigned priorities within each time slot. Through this information-sharing mechanism, UAVs can achieve conflictfree mission allocation in a decentralized and efficient manner.* In the aforementioned application scenarios where a set of UAVs (M = {1, . . . , m}) are deployed and a set of GUs (N = {1, . . . , n}.) are spatially distributed and move with velocity
* For clarity, the main notations used throughout this paper are summarized in Table. I.
2
TABLE I N OTATIONS TABLE
Notation
Meaning
M N vt θt α wv,t wθ,t h Ct hm,n (t) αm,n B Rm,n (t) ptx Dn (t) pprop m (t) p̄ γ wcov wcomm C(T ) R(T ) h Q(st , at ) αema xt wkep wtst wt sep k C̄k−1 R̄k−1 αk−1 t Pm t Um t Cm gt τ ε δt
The set of UAVs deployed in the scenario. The set of GUs deployed in the scenario. The velocity of the user at time slot t. The heading angle of the user at time slot t. Memory factor of the Gauss-Markov mobility model. Zero-mean Gaussian noise term for the velocity process at time slot t. Zero-mean Gaussian noise term for the heading angle process at time slot t. The fixed flight altitude of the UAV. The image acquisition state matrix. The channel coefficient between GU n and UAV m at time slot t. The channel power gain at the reference distance of Ltm − Ltn = 1 m. The total system bandwidth. The achievable communication rate between UAV m and GU n at time slot t. The transmit power of UAV. The data queue length of GU n at time slot t. The propulsion power of UAV m at time slot t. The energy budget of each UAV. The discount factor. The weighting coefficients associated with image acquisition. The weighting coefficients associated with communication. The completion ratio of image acquisition. The completion ratio of communication. The channel coefficient threshold for satisfying the QoS requirement. The action-value functions. The exponential moving average smoothing coefficient. The local state of UAV at time slot t. The episode-level weight. The step-level weight. The fused weight of the episode-level weight and the step-level weight. The global state at the k-th episode The exponential moving averages of the image acquisition completion rate at (k − 1)-th episode. The exponential moving averages of the communication completion rate at (k − 1)-th episode. The weight coefficient learned in the (k − 1)-th episode. The coordinates of UAV m. The coordinates of users performing communication uploads to UAV m. The local image acquisition state matrix centered on the current UAV m. The global context state. Softmax temperature in step-level weight net. The greedy coefficient in action selection of the UAV. The hierarchical weight-mixing coefficient.
vt and heading θt [13] as :
by ( t
vt = αvt−1 + (1 − α)v + wv,t ,
(1)
θt = αθt−1 + (1 − α)θ + wθ,t ,
(2)
where wv,t ∼ N 0, σv2 , wθ,t ∼ N 0, σθ2 . The target area is discretized into two-dimensional grids with size H × W [14]. ■ Image Capture Model: During flight, UAVs capture ground images via onboard cameras and extract useful information. The horizontal and vertical fields of view are given by HF OV = 2h tan φ2h and VF OV = 2h tan φ2v where h denotes the UAV altitude, and φh and φv are the horizontal and vertical viewing angles, respectively. The UAVs take photos as they fly to the center of the grids, in which the image acquisition state matrix C t ∈ RH×W at time slot t is given
C [i, j] =
1, 0,
if the cell (i, j) has been captured, otherwise.
(3)
■ Communication Model: The GU-to-UAV wireless channel is modeled as a Rician fading channel [15]. The channel coefficient between GU n and UAV m at time slot t is expressed as q hm,n (t) = βm,n (t) h̃m,n (t), (4) α
m,n where βm,n (t) = and h̃m,n (t) = K /2 (h2 +∥Ltm −Ltn ∥2 ) ps q q Ks 1 Ks +1 h̃L + Ks +1 h̃N with |h̃L | = 1 and h̃N ∼ CN (0, 1). Here, Kps and Ks denote the path loss exponent and the Rician factor, respectively. The parameter αm,n represents the channel power gain at a reference distance of ∥Ltm − Ltn ∥ =
3
1m. By adopting OFDMA technology [15], the UAV predivides the available communication resources into multiple equal and orthogonal resource blocks, and serves at most N GUs simultaneously. Given a total system bandwidth B equally partitioned into N sub-channels, the bandwidth alB . Accordingly, the located to each sub-channel is Bs = N achievable communication rate between UAV m and GU n at time slot t is expressed as |hm,n (t)|2 ptx , (5) Rm,n (t) = Bs log2 1 + σ2 where |hm,n (t)|2 denotes the channel power gain, ptx is the transmit power of UAV m, and σ 2 represents the additive white Gaussian noise (AWGN) power. Meanwhile, the data queue length of GU n at time slot t evolves as Dn (t) = Dn (t − 1) − Rm,n (t) tcomm where tcomm denotes the communication transmission duration of GU n within a single time slot. B. Problem Formulation Under the considered scenario, the objective is to derive a policy for the multi-UAV multi-task system that minimizes the total time T required to complete the tasks, formulated as P1 : π ∗ = arg min T π∈Π
T X
s.t.
pprop m (t) ≤ p̄, ∀m ∈ M
(a)
t=1
C(T ) ≥ ρc , R(T ) ≥ ρr , hti ≥ h, Pi (t) ̸= Pj (t), ∀i, j ∈ M, i ̸= j,
(b) (c) (d)
where π ∗ denotes the optimal policy and Π is the feasible policy space. In (a), pprop m (t) denotes the propulsion power of UAV m at time slot t, and p̄ is the energy budget of each UAV. Constraint (b) requires that the completion ratio of image acquisition P t satisfies threshold C(T ) ≥ ρc , where 1 C(t) = H·W C [i, j], and that the completion ratio of communication satisfies R(T ) ≥ ρr , where R(T ) = 1 − PN Dn (T ) Pn=1 . Constraint (c) ensures the QoS of communication N n=1 Dn (0) channel, and constraint (d) prevents collisions among UAVs. III. M ULTI -AGENT M ULTI -TASK DRL A LGORITHM Due to the fact that the optimization objective in P1 is intractable to solve directly, we reformulate it as a long-term reward maximization problem over T time slots under the same constraints. Specifically, the original objective of minimizing task completion time is transformed into maximizing the cumulative system performance, which jointly reflects the completion rates of image acquisition and communication. The resulting optimization problem can be expressed as "T −t # X ∗ i π = arg max E γ (wcov C(t + i) + wcomm R(t + i)) π
i=0
(6) where γ ∈ (0, 1] is the discount factor, and wcov and wcomm denote the weighting coefficients associated with image acquisition C(t) and communication R(t), respectively. To solve this problem efficiently, we develop a DRL-based framework that jointly models long-term reward estimation and dynamic multi-objective weighting. The proposed framework consists
of two main components: a multi-head Q-network for taskspecific value estimation, and a hierarchical dynamic weighting mechanism for adaptive objective balancing. The former captures the long-term impact of UAV actions under the current policy, while the latter adjusts the relative importance of different tasks according to both episode-level and step-level system states. A. DRL Framework To optimize the objective in (7), we design a DRL framework that decomposes the learning process into two coupled parts. The first part, shown in the yellow block of Fig. 1, models accumulated discounted returns of the system, i.e., PT −t the i i=0 γ (C(t + i) + R(t + i), which captures the long-term effect of UAV actions on task completion. The second part, shown in the blue blocks of Fig. 1, learns dynamic weighting coefficients wcov and wcomm , enabling the policy to adapt to changing task priorities in real time. ■ The first part: A multi-head Q-network is employed to estimate the long-term cumulative return associated with each action under the current policy π. However, to reduce gradient interference caused by shared parameters across tasks, we adopt a multi-head architecture with a shared backbone and task-specific output heads. Accordingly, the task-specific action-value functions are defined as "T −t # X i Qcov (st , at ) = E γ C(t + i)|st , at , (7) i=0
Qcomm (st , at ) = E
"T −t X
# i
γ R(t + i)|st , at .
(8)
i=0
In the proposed multi-head Q-network, the local state of UAV m at time slot t, denoted as xt , is first encoded by a shared multilayer perceptron (MLP) backbone into a latent feature representation ht . The resulting feature is then fed into multiple independent value heads, each consisting of taskspecific fully connected layers that map ht to the corresponding action-value estimate. In this work, two value heads are used to evaluate the image acquisition and communication objectives, respectively. The two branches are trained independently using task-specific Bellman updates, thereby decoupling value estimation and facilitating multi-task weight fusion. ■ The second part: After obtaining Qcov (st , at ) and Qcomm (st , at ), the remaining challenge is how to guide decision-making when the relative importance of subtasks varies across different stages in a dynamic environment. To address this issue, we propose a hierarchical dynamic weighting mechanism to model wcov and wcomm . The mechanism decomposes weight learning into an episode-level network and a step-level network, as illustrated in the blue block of Fig. 1, which avoids the instability and policy drift that may arise when weights are adjusted solely based on instantaneous observations. The episode-level weight captures the global importance of objectives over an entire episode, thereby establishing long-term preference and resource allocation. In contrast, the step-level weight adapts the objective importance at each time step, allowing the policy to respond to short-term state variations. By combining these two levels of weighting, the proposed framework improves both policy stability and environmental responsiveness. The episode-level weight wkep and the step-level weight wtst are fused as wt = Normalize((1 − δt )wkep + δt wtst ), where
4
Fig. 1. An illustration of the proposed HDWDRL framework
δt is designed according to the imbalance between the two subtasks. B. Input Layer Architecture Based on the proposed DRL framework, the network inputs are organized into three parts. Specifically, the local state is fed into the multi-head Q-network and, together with the global context state, serves as the input to the step-level weight network. The global state is used as the input to the episodelevel weight network. ■ Global State: To capture long-term performance trends and inter-task discrepancies for episode-level preference learning, the global state at the k-th episode is defined as: sep k = T C̄k−1 , R̄k−1 , ζk , αk−1 , where C̄k−1 and R̄k−1 denote the exponential moving averages (EMA [16]) of the image acquisition completion rate and communication completion rate in the previous (k−1)-th episode, respectively. † . ζk represents the normalized discrepancy between the two objectives, k−1 +1 defined as ζk = C̄k−1 −R̄ . The term αk−1 denotes the 2 weight coefficient learned in the (k − 1)-th episode. These components are concatenated in a fixed order to form a onedimensional feature vector, which is then used as an input to the episode-level weight network. The network takes the concatenated pair (sep k , αk ) as input and outputs its corresponding value estimate. ■ Local State: To support step-level decision-making, the local state incorporates the UAV’s own environment, its spatial relationships with neighboring UAVs, and its service associations with GUs. Specifically, it includes the coordinates of the current UAV, the coordinates of GUs uploading data to this UAV, the coordinates of other UAVs, the coordinates of GUs served by other UAVs, and the local image acquisition state matrix centered on the current UAV. For UAV m, the t t t t t T input is defined as xt = [Pm , Um , P−m , U−m , Cm ] where t t Pm denotes the position of the current UAV m, Um denotes t t the state of its associated GUs, P−m and U−m represent the t states of the other UAVs and their associated GUs, and Cm is the local 3 × 3 image acquisition state matrix. † The EMA, or its equivalent forms such as soft target updates, is widely adopted in DRL to stabilize training. It is defined as C̄k = (1−αema )C̄k−1 + αema Ck and R̄k = (1 − αema )R̄k−1 + αema Rk , where αema denotes the smoothing coefficient.
■ Global Context State: To bridge the episode-level preference and the real-time task progression, we define the global context state as gt = [C(t), R(t), 1 − C(t), 1 − R(t), αk , 1 − αk , C(t) − R(t)]T . This context vector encodes the completion rates of image acquisition and communication, their corresponding deficiencies, and the episode-level weight. The concatenated input [xTt , gtT ]T is then fed into the steplevel weight network. C. Hierarchical Dynamic Weight Networks ■ Episode-level Weight Network: As shown in the blue box at the top of Fig. 1, to provide a stable global baseline by capturing long-term task progression and mitigating oscillations caused by transient step-level variations, the global state sep k is fed into the Actor network fϕ (·) which is implemented as an MLP with ReLU activations in the hidden layers and a sigmoid output layer to produce a normalized scalar weight. The episode-level weight is obtained as αk = fϕ (sep k ), and the corresponding weight vector is defined as wkep = [αk , 1 − αk ]T . The Critic network then evaluates the state-weight pair ep (sep k , αk ) and outputs the value estimate Vψ (sk , αk ), which is used to guide network updates. ■ Step-level Weight Network: As shown in the blue box in the center of Fig. 1, to enable the network adaptively reflect the current task status while preserving consistency with the episode-level preference, the designed step-level weight network fξ (·) takes the local observation xt and the global context gt as input at each decision step t. The concatenated input vector [xTt , gtT ]T is processed by two fully connected layers with ReLU activations, and the network outputs a normalized step-level weight via a temperature-scaled softmax function wtst = Softmax(fξ ([xTt , gtT ]T )/τ ), where τ controls the sharpness of the output distribution. Then, the steplevel weight network is trained online using a self-supervised scheme. Specifically, a heuristic target weight is constructed as wttarget = Normalize(λ1 Gt + λ2 bt + λ3 wkep ) where Gt = Normalize([max(rcov , 0), max(rcomm , 0)]) represents immediate positive rewards and bt = Normalize([1−C(t), 1−R(t)]) captures task completion deficiencies. The coefficients λ1 , λ2 , and λ3 determine the relative contributions of these terms. The step-level weight network is then updated by minimizing target 2 st Lst ∥ . ξ = ∥wt − wt ■ Weight Fusion: To this end, as shown in the blue box
5
Fig. 2. Performance comparison of various methods in terms of (a) image acquisition completion rate, (b) communication completion rate, and (c) completion time.
on the left of Fig. 1, a mixing coefficient δt is introduced to fuse the two weight components, yielding the final decision weight wt = Normalize((1 − δt )wkep + δt wtst ), where δt = clip(δ0 + βdt , δmin , δmax ). Here, δ0 , β, δmin , and δmax are constant hyperparameters, and dt = |(1−C(t))−(1−R(t))| quantifies the imbalance between the progress of the two subtasks. A larger imbalance leads to a higher value of δt , thereby increasing the influence of the step-level weight and allowing more flexible adaptation to the current environment. Given the fused weight vector wt = [wcov (t), wcomm (t)]T , the scalarized Q-value used for action selection is obtained via an inner product between wt and the task-specific Q-values. Finally, the UAV selects its action according to an ε-greedy strategy , which balances exploration and exploitation, and executes the resulting optimal action a∗t . IV. N UMERICAL R ESULTS A. Simulation and Parameter Configuration The proposed method is evaluated via PyTorch simulations. The mission horizon consists of T discrete time slots. In the default training setup, we deploy M = 6 UAVs and N = 50 users over a 1 km × 1 km area, discretized into square cells of side length L = 100 m (i.e., a 10 × 10 grid). UAVs start from the initial point with full battery, fly at constant speed v = 20 m/s, and maintain a fixed altitude h = 50 m. We assume no collisions with static obstacles (e.g., buildings). An episode terminates when (i) full-area coverage and complete user-data upload are both achieved, or (ii) all UAVs exhaust their onboard energy. User mobility follows a Gauss–Markov process with mean speed v̄ = 0.6 m/s, mean turning angle θ̄ = π/2, and memory factor α = 0.9. The slot decomposition is tf = 5 s, td = 0.01 s, tcov = 0.1 s, and tcomm = 0.75 s. The propulsion power is set to pprop m (t) = 497.25 W, and the per-UAV energy budget is p̄ = 250 kJ. The field of view of the UAV camera is φh = φv = 90◦ , yielding Lh = 2h tan φ2h = 100 m, Lv = 2h tan φ2v = 100 m. Total bandwidth B = 16 MHz, number of subchannels N = 10, channel-gain threshold h = 2 × 10−5 , transmit power ptx = 0.18 W, path-loss exponent parameter Kps = 2, Rician factor Ks = 1, αm,n = 5 × 10−5 , and noise power σ 2 = 1 × 10−14 W. The initial data demand per user is Dn (0) = 100 Mbits. The completion-ratio thresholds are corrected to ρc = 0.8, ρr = 0.98. The hierarchical weight-mixing coefficient is δt with δ0 = 0.45, β = 0.20, δmin = 0.15, and δmax = 0.45. In all DRL-based methods, action selection follows an ε-greedy policy with a unified linear annealing t schedule defined as εt = 1− (1−0.0025). Here, Table. II 7500 shows the other parameter settings and network configurations.
TABLE II MAIN PARAMETERS IN HDWDRL FRAMEWORK Parameters Action dimension (|A|) Episode Softmax temperature in step-level weight net (τ ) Discount factor (γ) EMA smoothing coefficient (αema ) Episode weight actor/critic learning rate Step-level weight net learning rate DQN optimizer learning rate Replay buffer size Target-network update period
Values 4 300 0.5 0.9 0.25 1 × 10−3 5 × 10−4 1 × 10−3 8000 5 (steps)
B. Performance Evaluation and Analysis According to the objective of the system (see (7)), we conduct extensive experiments to comprehensively evaluate the effectiveness of the proposed HDWDRL framework. The overall performance of each method is assessed in terms of the completion rate of image acquisition C(T ), communication R(T ) and the task completion time T . The results of performance evaluation are shown in Fig. 2, in which HDWDRL w/o EAC denotes the proposed HDWDRL framework that removes the episode-level Actor-Critic module (see subsection. III-C) which is used to evaluate the effect on long-term policy stability, and HDWDRL w/o SWS denotes the proposed HDWDRL framework that eliminates the stepwise supervision mechanism (see subsection. III-C) which is used to assess the necessity of short-term weight adaptation. Furthermore, both the weight scalarization method [7] and dynamic weighting method [10] are also subjected to performance comparisons to verify the performance advantages of HDWDRL framework. First, we evaluate the overall performance of all schemes in the constructed baseline scenario, as illustrated in Fig. 2(a) and Fig. 2(b). Under the same constraints, the performance of different methods is compared in terms of the completion rate of image acquisition C(T ) and communication R(T ). It can be observed that, after 240 training episodes, the proposed HDWDRL approach successfully achieves all predefined task thresholds (i.e., C(T ) ≥ 0.8 and R(T ) ≥ 0.98) whereas the other baseline methods have not yet reached the requirements. Obviously, this advantage is primarily attributed to the hierarchical dynamic weighting mechanism in HDWDRL, which enables more efficient and stable policy learning in multi-objective settings. Further analysis reveals that the static weighting strategy in [7] exhibits a relatively slower growth rate in performance, indicating that fixed weights are insufficient to adapt to dynamically changing environments, thereby limiting convergence efficiency. In contrast, although
6
the dynamic weighting method in [10] introduces a certain degree of adaptability, its performance curve shows significant fluctuations, suggesting that it fails to fully address the instability issue in multi-objective dynamic environments. Moreover, the ablation results of HDWDRL demonstrate the effectiveness of its key components. Without the episode-level weighting mechanism, the algorithm struggles to maintain long-term policy consistency, while removing the step-level weighting mechanism weakens its ability to respond to instantaneous environmental variations. Subsequently, Fig. 2(c) compares the task completion time T required by each method to achieve all task thresholds (i.e., C(T ) ≥ 0.8 and R(T ) ≥ 0.98). It is evident that HDWDRL shows less time to complete the tasks and exhibits the lowest performance variance among all methods. At the 80-th episode of the training process, HDWDRL has almost reached the optimal performance level and gradually converges during the subsequent training process, which demonstrates that the steplevel weights enhance short-term responsiveness to dynamic environments, while the episode-level weights ensure longterm stability. Their synergistic effect not only accelerates convergence but also significantly improves the robustness of the learning process. V. C ONCLUSION This work proposes HDWDRL, a DRL framework for multi-agent collaborative environments. It integrates an episode-level Actor–Critic for global weight adaptation with a step-level lightweight network for short-term weight inference, achieving a convex trade-off between long-term stability and short-term responsiveness. Driven by distributed DQN agents, the framework mitigates resource imbalance inherent in static weighting and alleviates policy instability in dynamic weighting, while maintaining a balance between coverage and communication efficiency. Simulation results show that HDWDRL outperforms static and single-scale dynamic baselines in convergence stability, Pareto optimality, and adaptability to dynamic environments. R EFERENCES [1] T. Lei, C. Luo, T. Sellers, Y. Wang, and L. Liu, “Multitask allocation framework with spatial dislocation collision avoidance for multiple aerial robots,” IEEE Transactions on Aerospace and Electronic Systems, vol. 58, no. 6, pp. 5129–5140, 2022. [2] Y. Wang, H. Li, and Q. Shen, “A hierarchical multi-task and multi-agent assignment approach: Learning DQN strategy from execution,” IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 1471214722, 2025. [3] G. Sun, Y. Wang, Z. Sun, Q. Wu, J. Kang, D. Niyato, and V. C. M. Leung, “Multi-objective optimization for multi-UAV-assisted mobile edge computing,” IEEE Transactions on Mobile Computing, vol. 23, no. 12, pp. 14803–14820, 2024. [4] K. Wang and Z. Cheng, “Multi-UAV cooperative task scheduling and trajectory optimization system under communication constraints,” Physical Communication, vol. 76, pp. 103073, 2026. [5] Z. Lv, L. Xiao, Y. Du, G. Niu, C. Xing, and W. Xu, “Multi-agent reinforcement learning based UAV swarm communications against jamming,” IEEE Transactions on Wireless Communications, vol. 22, no. 12, pp. 9063–9075, 2023. [6] B. Li, R. Yang, L. Liu, J. Wang, N. Zhang, and M. Dong, “Robust computation offloading and trajectory optimization for multi-UAV-assisted MEC: A multiagent DRL approach,” IEEE Internet Things J., vol. 11, no. 3, pp. 4775–4786, 2024. [7] J. Pan, Y. Li, R. Chai, S. Xia, and L. Zuo, “Multiobjective trajectory planning for UAV-assisted IoT networks based on DRL approach,” IEEE Internet Things J., vol. 12, no. 11, pp. 15840–15852, 2025. [8] Y. Yu, J. Tang, J. Huang, X. Zhang, D. K. C. So, and K.-K. Wong, “Multi-objective optimization for UAV-assisted wireless powered IoT
networks based on extended DDPG algorithm,” IEEE Transactions on Communications, vol. 69, no. 9, pp. 6361–6374, 2021. [9] N. Mu, Y. Luan, and Q.-S. Jia, “Preference-based multi-objective reinforcement learning,” IEEE Transactions on Automation Science and Engineering, vol. 22, pp. 18737-18749, 2025. [10] F. Song, M. Deng, H. Xing, Y. Liu, F. Ye, and Z. Xiao, “Energy-efficient trajectory optimization with wireless charging in UAV-assisted MEC based on multi-objective reinforcement learning,” IEEE Transactions on Mobile Computing, vol. 23, no. 12, pp. 10867–10884, 2024. [11] Z. Gao, L. Yang, and Y. Dai, “MO-AVC: Deep-reinforcement-learningbased trajectory control and task offloading in multi-UAV-enabled MEC systems,” IEEE Internet of Things Journal, vol. 11, no. 7, pp. 11395– 11414, 2023. [12] H. Huang, Z.-Y. Chai, B.-S. Sun, H.-S. Kang, and Y.-J. Zhao, “Multiobjective deep reinforcement learning for computation offloading and trajectory control in UAV-base-station-assisted MEC,” IEEE Internet of Things Journal, vol. 11, no. 19, pp. 31805–31821, 2024. [13] Z. Sheng, H. Fu, Z. Huang, A. A. Nasir, Q. Wu, and D. Zeng, “Outageaware online prediction control for securing UAV-aided communication,” IEEE Transactions on Vehicular Technology, vol. 74, no. 7, pp. 1103911054, 2025. [14] D. Rizvi and D. Boyle, “Multi-agent reinforcement learning with action masking for UAV-enabled mobile communications,” IEEE Transactions on Machine Learning in Communications and Networking, vol. 3, pp. 117–132, 2024. [15] J. Wang, X. Wang, X. Liu, C.-T. Cheng, F. Xiao, and D. Liang, “Trajectory planning of UAV-enabled data uploading for large-scale dynamic networks: A trend prediction based learning approach,” IEEE Transactions on Vehicular Technology, vol. 72, no. 6, pp. 8272–8277, 2023. [16] C. H. Liu, X. Ma, X. Gao, and J. Tang, “Distributed energy-efficient multi-UAV navigation for long-term communication coverage by deep reinforcement learning,” IEEE Transactions on Mobile Computing, vol. 19, no. 6, pp. 1274–1285, 2019.