JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
1
Task-Oriented Sensing and Covert Transmissions for Collaborative Multi-AUV Systems
arXiv:2607.13880v1 [cs.LG] 15 Jul 2026
Xueyao Zhang, Chenyang Yan, Bo Yang, Xuelin Cao, Zhiwen Yu, Bin Guo, George C. Alexandropoulos, Mérouane Debbah, and Chau Yuen
Abstract—In underwater covert cooperative missions, autonomous underwater vehicles (AUVs) often cannot rely on active sonar to continuously obtain complete information, since active sensing and frequent communications increase the risk of exposure. As a result, AUVs primarily rely on passive observation, an approach that yields incomplete local perception and limited task efficiency. Although underwater acoustic communications can mitigate this limitation through information sharing, they are simultaneously constrained by long delays, severe interference, low reliability, and the risk of covert exposure. Existing communications-oriented multi-agent reinforcement learning (MARL) studies often model communication as an ideal information flow, whereas traditional communication optimization primarily focuses on link-level performance. However, both are insufficient to characterize the actual contribution of perceptual information to cooperative tasks under realistic conditions of covert physical communications. This paper proposes a Sensed Information Value Realization Multi-Agent Reinforcement Learning (SVR-MARL) framework that leverages practical information to characterize the utility of information for cooperative tasks and learns distributed cooperative policies under realistic communication and covert constraints. Through a case study of covert multi-AUV cooperative localization and tracking, the potential of the proposed framework to improve collaborative task efficiency while reducing unnecessary communication and exposure risks is demonstrated. Index Terms—Autonomous underwater vehicles, multi-agent reinforcement learning, covert transmission.
I. I NTRODUCTION S sixth-generation (6G) communications evolve from conventional data transmission toward an integrated network paradigm that deeply combines communication, sensing, computing, and intelligence, the application boundaries of network systems are continuously expanding from traditional
A
X. Zhang, C. Yan, B. Yang, and B. Guo are with the School of Computer Science, Northwestern Polytechnical University, Xi’an, Shaanxi, 710129, China (email: yang bo, [email protected]). X. Cao is with the School of Cyber Engineering, Xidian University, Xi’an, Shaanxi, 710071, China (email: [email protected]). Z. Yu is with the School of Computer Science, Northwestern Polytechnical University, Xi’an, Shaanxi, 710129, China, and Harbin Engineering University, Harbin, Heilongjiang, 150001, China (email: [email protected]). G. C. Alexandropoulos is with the Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, 16122 Athens, Greece (email: [email protected]). M. Debbah is with KU 6G Research Center, Department of Computer and Information Engineering, Khalifa University, Abu Dhabi 127788, UAE and also with CentraleSupelec, University Paris-Saclay, 91192 Gif-sur-Yvette, France (email: [email protected]) C. Yuen is with the School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore (email: [email protected]).
terrestrial spaces to more open and complex ocean environments [1]. Autonomous underwater vehicles (AUVs), as important platforms for marine environmental perception and autonomous operations, have been widely used for tasks such as ocean monitoring, underwater security, resource exploration, and target tracking [2]. However, in complex underwater environments, a single AUV is often unable to accomplish large-scale, long-duration, and highly dynamic tasks based solely on limited observations. Consequently, multi-AUV cooperation has gradually become an important approach for improving environmental perception capability and mission efficiency. Compared with terrestrial wireless environments, underwater spaces inherently suffer from severe propagation attenuation, long propagation delays, narrow bandwidth, and strong dynamics, which pose significant challenges to information exchange and cooperative decision-making. In complex underwater environments, multi-AUV systems usually rely on active sensing or information exchange to acquire richer environmental and target information. Underwater perception commonly relies on sonar systems. Among them, active sonar can obtain range and other target-related information by actively transmitting acoustic signals. However, its high-power acoustic radiation can be directly detected by distant devices, making it difficult to satisfy the survivability requirements of covert missions. In contrast, passive sonar offers greater concealment and is therefore more suitable as the primary sensing approach for covert underwater tasks. However, passive observations usually provide only limited local bearing information, making it difficult for an individual AUV to accurately recover the target state and potentially leading to accumulated perception uncertainty and unstable localization. To mitigate this ambiguity, multiple AUVs can exploit spatially complementary information from different locations and exchange local observations and cooperative states via limited underwater acoustic communication, thereby improving overall environmental perception and cooperative capabilities. Nevertheless, inappropriate communications may also introduce exposure risks. Therefore, a fundamental challenge is how to support cooperative information exchange under low-detectability constraints. Multi-Agent Deep Reinforcement Learning (MADRL) provides an end-to-end policy-learning paradigm for cooperative tasks, in which agents learn collaborative behaviors through environmental interactions under shared objectives. Since each agent can only make decisions based on local observations during distributed execution, communication mechanisms have been incorporated into MADRL to alleviate coordination
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
Local uncertainty
Single AUV
2
Collaboration demand
Local sensing
Communication helps
Covert risk
Message
Eavesdropper
High efficiency
Low efficiency
MARL Collaboration AUV 1
AUV i
Communication-centric Optimization
Our scenario
AUV N
... Ideal Communication
On
Oi
O1
The value of transmitted information for task collaboration is not considered.
Communication-centric Optimization End-to-end Optimization
Inputs
Higher Reward
Network Information
Policy network
... traj
vel
Q1: In covert collaborative scenarios, how can we determine whether a piece of local perceptual information truly contributes to the team task? • Q2: When messages must be transmitted through real underwater acoustic links, how can the effects of propagation, decoding, and covert requirements on their collaborative value be characterized? • Q3: Under distributed partial-observation settings, how can AUVs learn when to communicate and which information is worth transmitting, so as to avoid communication interference and unnecessary exposure? •
Collaboration demand
Resource Allocation
The Problem
Qos Constraint
Outputs Optimization Problem MINLP, Type: Multi-objective Methods
Alternating optimization, SCA...
Reliable Links High Throughput Low Delay
Joint actions
MARL-based collaboration focuses on task success, but commonly assumes ideal communication and ignores realistic transmission constraints.
Communication optimization focuses on ensuring reliable transmissions, regardless of whether the transmitted information is critical for collaboration.
Fig. 1. Task motivation and limitations of MARL-based collaboration and communication-centric optimization for covert multi-AUV collaboration.
difficulties arising from partial observability. Representative studies such as IC3Net [4] have demonstrated that interagent information exchange can improve cooperative decisionmaking. Specifically, these methods typically model communication as an information-exchange channel within the policy network, allowing agents to learn when to communicate and how to use the information they receive. However, the “communication” considered in these works is often highly idealized. Messages are modeled as abstract information flows, and practical constraints on transmission and decoding in realworld physical systems are rarely considered. Research on underwater wireless communications has primarily focused on ensuring information transmission performance over constrained acoustic links. Existing studies commonly formulate communication as a link-quality or resourceallocation problem and improve transmission reliability and communication efficiency through optimization or learningbased approaches. Covert communication studies [5] further incorporate low detectability into the design; however, their primary objective remains the improvement of data transmission rates under covert constraints. Overall, current research largely focuses on ensuring reliable communication, while paying limited attention to whether the transmitted information contributes to cooperative tasks in such harsh environments. In our covert collaboration scenario, this may cause redundant communication, inefficient resource utilization, and unnecessary exposure risks. The aforementioned studies characterize “information interaction” in cooperative communication and “link reliability” in physical communication, respectively. However, they remain insufficient for the more practical underwater multiagent covert collaboration scenario considered in this work. On the one hand, more realistic communication transmission processes need to be taken into account; on the other hand, in harsh, resource-constrained environments, communication with higher collaborative value should be prioritized. Motivated by these challenges, this work focuses on the following three questions:
II. OVERVIEW OF M ULTI - AGENTS U NDERWATER C OLLABORATION A. Information-Flow Communication in MARL Communication-enabled MARL has been widely studied to address partial observations in distributed multi-agent systems. Under the centralized training and decentralized execution (CTDE) paradigm, agents can learn collaborative behaviors toward shared task objectives. Representative methods include value decomposition methods (e.g., VDN and QMIX) and policy optimization methods (e.g., MADDPG and MAPPO), both of which have demonstrated strong cooperative capabilities in tasks such as cooperative navigation and target tracking. However, during distributed execution, each agent typically relies only on local observations and cannot directly access other agents’ states, intentions, or global environmental information. This limitation on local information becomes a key bottleneck for efficient collaboration. To alleviate collaboration difficulties arising from partial observations, communication mechanisms have been incorporated into MADRL, in which encoded messages are exchanged to supplement local observations and improve joint decisionmaking. These messages usually represent local observations, task intentions, or compact state summaries. Early works such as CommNet [6] modeled inter-agent communication as a continuous, differentiable process and learned communication protocols and cooperative policies end-to-end under idealized communication assumptions. IC3Net [4] introduced a gating mechanism for selective communication, enabling agents to learn when to transmit messages. TarMAC [7] further adopted attention-based targeted communication, allowing agents to decide what messages to send and whom to communicate with. These studies demonstrate that learned communication can mitigate incomplete local information and improve the efficiency of collaboration. However, the “communication” in the aforementioned methods essentially remains within the scope of information-flow communication. Their primary focus is on how messages are exchanged among policy networks, rather than on how information is physically transmitted between real-world entities. During training and execution, inter-agent messages are typically treated as directly accessible abstract information flows, implicitly assuming that messages can be delivered in a timely, reliable, and low-cost manner. Therefore, practical factors such as propagation delay, link interference, decoding reliability, and covert communication requirements are rarely
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
3
TABLE I C OMPARISON OF DIFFERENT COMMUNICATION APPROACHES . Block
Representative Works
Realistic Channel Constraints
MARL Collaboration with ideal communication
Sukhbaatar et al. [6]
×
×
✓
Singh et al. [4]
×
×
✓ ✓
Communicationcentric Optimization Our work
Covert Transmission
Task-oriented Value
Das et al. [7], Niu et al. [12]
×
×
Jiang [8]
✓
×
×
Liang et al. [9]
✓
×
×
Zhang et al. [13]
✓
✓
×
Chen et al. [11]
✓
✓
×
SVR-MARL
✓
✓
✓
considered. When these methods are applied to covert underwater collaboration, such idealized assumptions create a clear gap between learned cooperation and physical realizability. B. Communication Optimization for Covert Networks Research on underwater wireless communications has long focused on reliable transmission over complex acoustic links. Since acoustic propagation is strongly affected by transmission distance, signal frequency, ocean conditions, and node mobility, existing studies typically optimize link reliability, data transmission rate, and energy efficiency [8]. At the physical layer, techniques such as adaptive modulation and coding, channel estimation, and equalization are widely employed to adapt to time-varying channel conditions, mitigate intersymbol interference, and reduce bit error rates [9]. At the link layer, MAC scheduling and power control mitigate conflicts and resource competition over shared acoustic channels [10], thereby improving network throughput and energy utilization efficiency. In covert communications, research has further considered information transmission under low-detectability requirements. Existing works [11] commonly perform joint optimization over transmission power, energy consumption, AUV positioning/trajectory, and transmission strategies to improve communication reliability and transmission performance while satisfying covert constraints. Nevertheless, their objectives still primarily serve communication itself, i.e., reliable and covert data transmission. For multi-AUV covert collaboration tasks, link availability does not necessarily imply that a transmission is worthwhile, since some messages may contribute little or even detract from team collaboration. Blind transmissions may instead lead to unnecessary resource occupation, link conflicts, and the waste of valuable covert communication opportunities. C. The Missing Gap Existing studies have improved multi-agent collaboration and physical communications from the perspectives of collaborative learning and transmission optimization, respectively. However, these approaches remain insufficient for covert underwater multi-AUV collaboration. On the one hand, MARLbased message passing typically lacks realistic underwater acoustic modeling, including propagation delays, interference, decoding reliability, and covert constraints. On the other
hand, traditional communication optimization focuses on the physical transmission process and mainly aims to improve communication performance itself, without considering the task significance of the message content. Therefore, in covert collaboration scenarios, communication decisions should depend not only on link availability but also on whether the transmitted information can truly benefit team collaboration. Motivated by this gap, this work focuses on learning AUV cooperative policies distributively under covert constraints to maximize overall collaborative efficiency. We introduce the concept of “value of sensed information” to characterize the potential contribution of local observations to collaborative tasks, as illustrated in Fig. 1. Compared with conventional communication schemes centered on link performance, we place greater emphasis on the actual impact of local perceptual information on team collaboration under realistic physical communication conditions, while jointly considering information value, link availability, and covert requirements during communication decision-making. From this perspective, covert communication is no longer a purely data-transmission problem but an information-selection process that serves collaborative perception and cooperative task execution. III. SVR-MARL F RAMEWORK FOR C OVERT C OLLABORATION In covert multi-AUV collaboration, communication decisions should depend not only on the importance of local information or link availability, but also on the realized contribution of transmitted information to team tasks under physical and covert constraints. Motivated by this, this section introduces the concepts of value of sensed information and practical information value. As illustrated in Fig. 2, the value of sensed information characterizes the potential collaborative benefit inherent in local observations, such as complementary target observations, improved sensing geometry, or reduced team uncertainty. However, such local information must undergo dual attenuation due to realistic physical communication and covert communication constraints before ultimately becoming practical information value. Therefore, a high value of sensed information does not necessarily imply a high transmission priority. Some messages may still yield limited actual benefits due to poor link conditions, excessive resource consumption, or high exposure risk. Conversely, a message with moderate
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
4
B. Coupling Mechanisms Value of sensed information
Practical information value
COMMUNICATION physical transmission
Delay
TRADE OFF
Reachability Interference
Decoding
Real-world transformation
COVERT SAFETY Exposure constraints
Detection Trajectory
This value reflects how useful the sensed information would be for collaboration under ideal information sharing.
Transmit Power
Covert Budget
The information value is reshaped by realistic communication and covert constraints, reflecting its contribution to collaboration.
Perspective: The collaborative value of information is not determined solely by its importance, but also by whether it can be reliably, covertly, and timely delivered under realistic communication conditions.
Fig. 2. A perspective on realizing collaborative value under realistic communication and covert constraints.
sensed value may become more useful when it can be delivered reliably and covertly at a lower cost. Moreover, since each AUV makes decisions based only on local observations, the system can easily fall into a prisoner’s-dilemma-like Nash equilibrium, rather than producing actions that truly benefit global collaboration. Therefore, the core problem addressed in this section is: how can practical information value be utilized to guide distributed communication and task decision-making, enabling AUV swarms to achieve more efficient collaboration under realistic physical communication and covert communication requirements? A. Task Modeling and Action Decomposition In complex underwater environments, many tasks are characterized by long durations, wide operational areas, high dynamics, and strong uncertainty. A single AUV often struggles to complete such missions independently due to limited sensing capability and restricted individual mobility. For example, in target tracking, cooperative navigation, area search, and pipeline inspection, multiple AUVs can achieve complementary perception through spatially distributed deployment, and further improve task efficiency through limited information exchange for state sharing, intention coordination, and cooperative execution. Although different tasks may involve different objectives and execution actions, their decision-making processes can generally be divided into two basic aspects: how to exchange information that benefits collaboration, and how to perform task-oriented physical actions. Therefore, we extract their common structure and decompose each AUV’s policy into a communication phase and a task-execution phase, with the corresponding action expressed as ai = acomm , atask . i i comm Here, ai describes decisions related to information transmission, whose core role is to determine whether local sensing messages can be delivered over realistic underwater acoustic links in a reliable, timely, and covert manner. Depending on the specific underwater communication model, acomm i may include transmission gating, transmit power, bandwidth, frequency, and access time slots. The task action atask is i oriented toward the specific cooperative objective and can be represented by speed, heading, depth, or task-interaction actions, depending on the task type.
1) Covert-Constrained Task Coupling: For the covert communication problem considered here, we adopt probabilistic detection models for underwater covert communications to make covertness compatible with optimization and learning. Specifically, the eavesdropper’s listening behavior is modeled as a binary hypothesis-testing problem, and the lowdetectability requirement is transformed into a tractable covertness condition using the KL divergence. Meanwhile, the receiver should also meet a decoding reliability requirement to ensure that messages can be reliably received by teammates. Based on this modeling, covertness couples the communication and task phases. On the one hand, the transmission decision and transmit power in communication affect the received signal strength at the eavesdropper, thereby influencing the exposure risk. On the other hand, the distance and relative geometry between an AUV and the eavesdropper affect the channel gain, which means that task actions can also indirectly influence covertness. Therefore, the covertness requirement serves as a key coupling factor between the communication and task phases. In essence, the communication action determines “at what cost information is transmitted,” while the task action determines “under what geometry the task and communication are performed.” These two phases should therefore be jointly considered under a unified objective of covert collaboration, rather than being designed independently. 2) Closed-Loop Coupling in POMG: A Markov game can be viewed as the multi-agent extension of a Markov decision process. For underwater multi-AUV collaborative tasks, each AUV can only obtain local observations and cannot directly access the global state. Therefore, we formulate the problem as a partially observable Markov game (POMG). In this model, each AUV first forms its communication observation ocomm (t) based on its local observation, historical informai tion, and received messages, and then generates the communication action acomm (t). After passing through realistic i underwater acoustic links, the communication may result in different outcomes, such as delayed arrival, decoding failure, or successful reception. These outcomes further change the information available to the AUV and affect the task-stage observation otask (t). Subsequently, the AUV executes the i task action atask (t) based on the updated task observation, i thereby driving the environment state transition. This transition changes the spatial geometry, channel conditions, interference relationships, and covertness-related factors at the next time step, thereby further affecting the subsequent communication observation ocomm (t + 1). Therefore, the communication and i task phases form a closed-loop coupling in the POMG. C. Value Realization Mechanism of Sensed Information 1) Task-Aware Sensed-Information Message: This part mainly defines what should be included in the messages transmitted by the AUVs. We divide the message content into three categories. The first category is task observation information. For example, in a pursuit task, this may include the target bearing, target belief, or local estimation results; in a navigation task, it may include local maps, route risks,
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
Step III: Policy Learning Module
Target
Eve
Step I: Task Decomposition
o
Obs
comm i
move
Ocomm Otask
, oitask
5
1
Comm Actor
Encoder
Stage Ⅰ：Communication and Local Information
aicomm = {gi , Pi , Bi , f i , ri , li } g i :Gate
Bi :Bandwidth allocation
:Tx power
:Frequency
:rate
:Access slot
GRU
AUV3 AUV1
comm
AUV2
comm 4 6
vi :Velocity
ψi :Direction
zi :depth
κ i :Maneuver
2
3
Stage Ⅱ：Motion Planning and Action Execution
aimove = {vi , ψi , zi , κi }
SV-Est
Distribution
( ot , at , rt , ot +1 )
Bernoulli
... Beta
acomm atask PPO update
Reward
Critics /
Step II: Constrained POMG POMG
oicomm ( t )
Covert & reliability constraints
min/ max task comm task
a
i
, ai
Q ( f ( i ) ) max ,
aicomm ( t )
s.t.
oitask ( t )
( log (1 + ) − R ) ln 2 f =
t+1
aimove ( t )
D ( Q0 || Q1 ) 2 2 . ( i)
D ( Q0 || Q1 ) =
2
i
1 1 1 − M (1 + i )2
i ,e M ln 1 + i ,e − 2 1 + i ,e
( ot , at , rt , ot +1 )
rollout buffer
Comm Environment Link Realization
• • •
Role: Compute link gain and received power Input: Comm action, geometry, channel Output: Acoustic link state, received power
Delay Modeling
• • •
Role: Determine packet arrival timing Input: Distance, sound speed, packet length Output: Delay,arrival window
• Overlap • Interference •
Decoding Feedback
Role: Evaluate packet overlap Input: Arrival windows of multiple packets Output: Overlap ratio and interference
• Role: Check decoding result • Input: SINR, rate, packet delay • Output: timely / delayed / failed
Sample
5
Comm Critic
Task Critic PPO update
Encoder Concat
Task Actor
MLP
Distribution
Beta
...
Gaussian
Fig. 3. A general framework for covert multi-AUV collaboration.
or obstacle information. The second category is task intention information. When AUVs need to perform multi-target pursuit, multi-region search, or cooperative encirclement, the message can include information such as “where I intend to go,” “which target I am responsible for,” or “what role I am currently playing.” The third category is self-state and communication-context information, such as the AUV’s own position, historical actions, message timestamp, link state, and covert risk. This information can help break the isolatedobservation dilemma under fully distributed execution. 2) Temporal Message Context Encoder: Each AUV is equipped with a local communication context encoder. It encodes its own observations and actions, aggregates received messages, concatenates the current communication input with historical actions, and uses a GRU to extract temporal context, providing the basis for the communication policy to make transmission decisions. 3) Sensed Information Value Estimator: This module estimates the potential collaborative benefit that current-sensing information may provide to other AUVs. Such an estimate can be learned from trajectories collected through reinforcement learning interactions with the environment. When a message is received and fused, supervision signals are constructed based on changes in the receiver’s state or task-related loss to train the value estimation module, enabling the AUV to infer, from its own observations, how much collaborative contribution its message may provide if successfully decoded by teammates. This estimate is then used as input to the communication policy network, guiding the policy in identifying information with collaborative value. 4) Physics-Grounded Covert Communication Environment: Unlike many existing MARL frameworks built on Gym-based or simplified simulation environments, where the scenarios often lack realistic communication processes, our proposed framework treats the communication environment as the key realization layer that connects ”value of sensed information” and ”practical information value”. Specifically, each AUV first selects its communication action according to the policy. The environment then computes the received signal strength by jointly considering the inter-node distance, channel gain, and underwater acoustic path loss. Path attenuation is characterized
by the Thorp model [14], while the channel gain accounts for large- and small-scale fading as well as environmental noise. Based on this, the environment calculates the receiverside SINR, message arrival time, transmission duration, and decoding result, thereby determining whether the message can be reliably received by teammates. Meanwhile, our environment explicitly considers delay, concurrent interference, and covertness in underwater acoustic communication. Due to the low propagation speed of underwater sound, packets transmitted by different AUVs may arrive at the receiver at different times because of varying propagation distances and transmission durations. When the arrival intervals of multiple packets overlap, the environment calculates concurrent interference based on the overlap ratio and feeds it back into the SINR and decoding results. 5) Practical Information Value-Guided Communication Phase: As discussed in Section III-A, we have categorized the communication actions. In the communication phase, each AUV generates the corresponding communication action using the communication policy network, based on the historical message context and the estimated value of the sensed information. For different communication parameters, the policy network employs corresponding action-distribution heads. For example, the send/no-send decision can be modeled with a Bernoulli distribution, receiver or channel selection with a Categorical distribution, and continuous transmit power with a Beta distribution. The communication action is then executed in our designed communication environment, where its value is jointly affected by link attenuation, propagation timeliness, concurrent interference, and covert constraints. Based on this, the communication reward rcomm is designed to characterize the practical information value of a message. It measures the message’s actual utility for team collaboration by balancing its counterfactual benefit with realistic communication and covert costs. Furthermore, the framework adopts a Lagrangian primal-dual method to convert the above constraints into penalty terms in the reward, forming a new reward. Given that different AUVs contribute differently to constraint violations, we perform individual credit assignment and allocate penalties proportionally, rather than imposing a single collective penalty on all agents. 6) Task Phase: Based on the results of the communication phase, each AUV updates its local observation and uses the task network to execute physical actions. The specific action form depends on the task and can be represented by velocity, heading, and other variables. The resulting actions further alter the spatial geometry among AUVs, observation quality, channel conditions, and exposure risk, thereby affecting the realization of value from communication actions in the next time step. Therefore, in SVR-MARL, the communication and task phases are not independent but form a closed-loop collaboration process in which “communication improves task decisions, and tasks reshape the communication environment.” IV. C ASE S TUDY: C OVERT C OLLABORATIVE L OCALIZATION AND T RACKING In this section, we validate the proposed framework through a case study of underwater multi-AUV pursuit.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
6
A. Scenario and Framework Instantiation We consider a three-dimensional underwater collaborative pursuit scenario with a mission area of 1000m × 1000m × 200m, including four cooperative AUVs, one moving target, and one passive eavesdropper, Eve. The task of the AUVs is to cooperatively approach and capture the target within a limited time while avoiding exposure to frequent or high-power communication. Due to covert requirements, each AUV is equipped with passive sonar as the primary sensing modality. Each AUV can only obtain noisy azimuth and elevation angles of the target relative to itself, without directly measuring the range. Moreover, the angular observation error increases with target distance, leading to significant uncertainty in the targetstate estimate for a single AUV. We build a passive sonar observation model in which each AUV maintains a local belief about the target position, including the estimated position and its uncertainty. At each time slot, an AUV first obtains local angular observations from passive sonar and updates its local belief through an EKF [15]. Since a single bearing-only observation is unobservable in the range direction, observations from AUVs located at different spatial positions can provide complementary geometric information. Therefore, during the communication phase, an AUV can transmit its sensing message to other AUVs. Once the message is successfully decoded, the receiver updates its target localization estimate with the received information, thereby reducing localization ambiguity arising from singleagent passive observation. Each time slot follows the order of “communication phase–movement pursuit phase.” In the communication phase, each AUV needs to decide whether to broadcast its current sensing message and what transmit power to use. The selected message is then processed by our modeled communication environment to determine whether it arrives and can be decoded. Successfully decoded messages are used to update the receiver’s belief and then serve as input to the subsequent movement policy network, which outputs continuous actions to adjust the pursuit direction. The overall training uses a PPO structure, while during execution, each AUV makes distributed decisions based solely on local observations. B. Comparative Schemes and Performance Analysis In this subsection, we conduct simulation experiments for the designed case study. First, we consider four comparative schemes: True Obs scheme: As an upper-bound reference, AUVs can obtain dynamic real-time observations of the target without communication, which is used to evaluate the performance of the movement pursuit policy under ideal sensing conditions. • Full comm: Each AUV broadcasts its message at every time slot and transmits with a fixed transmit power that satisfies the covert requirement. • No comm: AUVs do not communicate with each other. Each AUV relies solely on its local observations to update its target belief and complete the movement pursuit task. •
Fig. 4. Capture rate and task steps under different communication strategies.
a1) Comm decoding heatmap (IC3Net)
b1) Transmit power heatmap (IC3Net)
c1) Covert status (IC3Net)
a2) Comm decoding heatmap (Ours)
b2) Transmit power heatmap (Ours)
c2) Covert status (Ours)
Fig. 5. Trajectory-based communication behaviors, including communication decoding, transmit power, and covert status, for IC3Net (a1, b1, c1) and the proposed framework (a2, b2, c2).
•
IC3Net [4]: Each AUV learns a binary communication gate from local observations in an end-to-end manner. When the gate is activated, the encoded message is broadcast to teammates through the underwater acoustic channel using a fixed transmit power.
In the experimental setup, each episode lasts up to 300 time slots, and each slot sequentially executes the communication and movement pursuit phases. The training process uses a two-stage PPO structure, and the evaluation metric is task efficiency, defined as the ratio of capture rate to task steps. Fig. 4 illustrates the task efficiency achieved by different schemes during evaluation. As the ideal upper bound, True Obs achieves the highest task efficiency by providing accurate target information. In contrast, No Comm performs the worst, indicating that bearing-only passive observations are insufficient for reliable cooperative tracking without information sharing. Full Comm improves task efficiency through AUV communication, but indiscriminate transmissions may introduce channel contention and decoding failures under realistic underwater acoustic links. IC3Net learns communication gating through end-to-end policy optimization, but decoding failures due to realistic acoustic and covert constraints can weaken information exchange and lead to fluctuations in efficiency. Notably, the proposed framework achieves a task
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021
7
efficiency of approximately 1.2×10−2 , outperforming IC3Net by about 20% and achieving the best performance among all practical communication schemes. Fig. 5 further illustrates the communication behaviors of different schemes. Compared with IC3Net, the proposed framework achieves more successful message decodings at critical stages of the task, indicating more effective information sharing among teammates. Meanwhile, under the same covert constraint, it adaptively allocates transmit power according to task requirements, increasing communication investment when target uncertainty is high and reducing unnecessary transmissions once the target is well localized. These results indicate that the proposed framework can utilize communication resources more efficiently, thereby improving cooperative tracking performance.
munication policy can control acoustic exposure risk, the system may still face security threats, such as malicious node attacks or the injection of false messages. An attacker may impersonate a legitimate AUV and send incorrect messages to disrupt team collaboration. Future work can further investigate secure authentication and trustworthy message evaluation mechanisms based on acoustic fingerprints, channel characteristics, motion behaviors, and historical communication patterns. These mechanisms can distinguish legitimate nodes from malicious ones and assess message credibility before fusion, thereby improving the robustness and communication security of underwater distributed cooperative systems in complex adversarial environments.
V. C ONCLUSION AND F UTURE D IRECTION
[1] T. Qiu, Z. Zhao, T. Zhang, C. Chen, and C. L. P. Chen, “Underwater Internet of Things in smart ocean: System architecture and open issues,” IEEE Transactions on Industrial Informatics, vol. 16, no. 7, pp. 4297– 4307, Jul. 2020. [2] Y. Yang, Y. Xiao, and T. Li, “A survey of autonomous underwater vehicle formation: Performance, formation control, and communication capability,” IEEE Communications Surveys & Tutorials, vol. 23, no. 2, pp. 815–841, Secondquarter 2021. [3] Z. Zhou, J. Liu and J. Yu, “A Survey of Underwater Multi-Robot Systems,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 1, pp. 1-18, January 2022. [4] A. Singh, T. Jain, and S. Sukhbaatar, “Learning when to communicate at scale in multiagent cooperative and competitive tasks,” Proc. Int. Conf. Learn. Represent. (ICLR), 2019. [5] S. Liu, M. A. Khan, M. Bilal, and H. H. Zuberi, “Low probability detection constrained underwater acoustic communication: A comprehensive review,” IEEE Communications Magazine, vol. 63, no. 2, pp. 21–30, Feb. 2025. [6] S. Sukhbaatar, A. Szlam, and R. Fergus, “Learning multiagent communication with backpropagation,” Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), pp. 2244–2252, 2016. [7] A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, and J. Pineau, “TarMAC: Targeted multi-agent communication,” Proc. Int. Conf. Mach. Learn. (ICML), pp. 1538–1546, 2019. [8] S. Jiang, “On Reliable Data Transfer in Underwater Acoustic Networks: A Survey From Networking Perspective,” IEEE Communications Surveys & Tutorials, vol. 20, no. 2, pp. 1036-1055, Secondquarter 2018. [9] Y. Liang, H. Yu, L. Xu, H. Zhao, F. Ji and S. Yan, “Joint Bayesian Channel Estimation and Data Detection for Underwater Acoustic Communications,” IEEE Transactions on Communications, vol. 72, no. 9, pp. 5868-5883, Sept. 2024. [10] T. Zhang, Y. Gou, J. Liu, S. Song, T. Yang and J. -H. Cui, “Joint Link Scheduling and Power Allocation in Imperfect and Energy-Constrained Underwater Wireless Sensor Networks,” IEEE Transactions on Mobile Computing, vol. 23, no. 10, pp. 9863-9880, Oct. 2024. [11] J. Chen, J. Wang, Z. Wei, Y. Ren, C. Masouros and Z. Han, “Joint Autonomous Underwater Vehicle Trajectory and Energy Optimization for Underwater Covert Communications,” IEEE Transactions on Communications, vol. 72, no. 11, pp. 7327-7341, Nov. 2024. [12] Y. Niu, R. R. Paleja, and M. C. Gombolay, “Multi-Agent GraphAttention Communication and Teaming,” in Proc. Int. Conf. Auton. Agents Multiagent Syst. (AAMAS), pp. 964–973, 2021. [13] X. Zhang, B. Yang, Z. Yu, X. Cao, G. C. Alexandropoulos, M. Debbah, and C. Yuen, “Cooperative target detection with AUVs: A dual-timescale hierarchical MADRL approach,” arXiv preprint arXiv:2509.13381, 2025. [14] Y. L. L. Brekhovskikh, Fundamentals of Ocean Acoustics. Springer, New York, 1990. [15] N. Modalavalasa, G. S. B. Rao, K. S. Prasad, L. Ganesh, and M. N. V. S. S. Kumar, “A new method of target tracking by EKF using bearing and elevation measurements for underwater environment,” Robotics and Autonomous Systems, vol. 74, pp. 221–228, Dec. 2015.
This article proposes a collaborative framework for covert multi-AUV tasks that transforms sensed information into practical information value. Unlike existing multi-agent communication methods that treat communication as an ideal information exchange, this article incorporates realistic underwater acoustic links, propagation delays, concurrent interference, decoding reliability, and Eve-side covert risk into the collaboration loop. In this way, agents can jointly decide whether to communicate and how to communicate, based on both task value and physical realizability. In the case study, we consider a covert multi-AUV collaborative pursuit task and verify that value-driven communication scheduling can effectively improve target localization and task completion efficiency under bearing-only passive observations. The results show that the proposed framework improves the performance of multi-AUV collaboration under realistic communication and covert constraints, offering a new design perspective for taskoriented optimization of underwater covert communication. 1) Teammate Intention Prediction for Distributed Coordination: Although we adopt a fully distributed execution architecture, each AUV makes communication and task decisions based on local observations and received messages. However, without global scheduling, an AUV may still struggle to accurately infer its teammates’ future communication behaviors, movement directions, and task roles. This may lead to repeated transmissions, overly conservative decisions, or conflicts in action in some cases. In future work, we will consider introducing a teammate-intention prediction module that enables each AUV to recursively estimate other AUVs’ communication tendencies and action intentions over the next few steps, based on historical messages and communication records. In this way, AUVs can better use communication messages to predict the team’s collaborative state based on local information and thus decide how to communicate more effectively. This direction is expected to further improve scheduling efficiency and task coordination in distributed covert collaboration. 2) Secure Agent Authentication and Trustworthy Communication: In real underwater environments, even if the com-
R EFERENCES