An Asynchronous Triggered MAC Protocol for Underwater Acoustic Networks

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

1

An Asynchronous Triggered MAC Protocol for Underwater Acoustic Networks

arXiv:2608.10533v1 [cs.NI] 11 Aug 2026

Bingwen Huangfu, Jiani Guo, Shanshan Song*, Member, IEEE, Nan Sun, Jun Liu

Abstract—Time Division Multiple Access (TDMA)-based Medium Access Control (MAC) protocols have proven their practicality through extensive field trials in Underwater Acoustic Networks (UANs), attributable to their hardware-agnostic and easily implementable properties. Most existing protocols rely on a synchronized and fixed-length slot paradigm to mitigate channel contention and facilitate orderly transmissions. However, this paradigm imposes significant clock synchronization overhead in UANs with low and variable sound speed and struggles to improve scheduling flexibility. Although some protocols attempt to refine this slot paradigm (adjust the slot length to improve channel reuse efficiency or scheduling frequency), they are still constrained by the trade-off between channel utilization and scheduling complexity. To this end, this paper advocates a paradigm shift in underwater MAC design, transitioning from synchronized slot to asynchronous scheduling, and realizes it through an Asynchronous Triggered MAC (AT-MAC). AT-MAC introduces a triggered slot paradigm without time synchronization, decoupling transmission scheduling from a rigid timeline and enabling asynchronous, variable-length slots to accommodate the long and diverse propagation delays. To power this slot paradigm, AT-MAC augments conventional Multi-Agent Deep Reinforcement Learning to handle asynchronous interaction, achieving efficient coordinated channel access under partial observations. It further devises a load-aware fairness guard mechanism to enable network-wide fairness status inference solely through local overhearing, thereby guiding adaptive scheduling correction to maintain fairness. Trace-based and on-board experiments validate the feasibility and computational practicality of AT-MAC. Extensive simulation results further demonstrate its superiority and adaptability across various scenarios and traffic conditions. Index Terms—Underwater acoustic networks, medium access control, deep reinforcement learning.

I. I NTRODUCTION

U

NDERWATER acoustic networks (UANs) have attracted significant research interest in recent decades due to their applications in the ocean exploration, environmental monitoring, disaster prevention, and military surveillance [1]– [4]. However, these applications hinge on the efficient and reliable data transmission. To achieve this, medium access control (MAC) protocols play a pivotal role in enhancing network throughput, orchestrating competing channel access, and managing multi-user fairness [5]–[7]. Specifically, the unique S. Song is the corresponding author. B. Huangfu, J. Guo, and S. Song are with the College of Computer Science and Technology, Jilin University, Changchun 130012, China (e-mail: hfbw24@mails.jlu.edu.cn, jnguo@jlu.edu.cn, songss@jlu.edu.cn). N. Sun is with the College of Software Engineering, Jilin University, Changchun 130012, China (e-mail: nsun23@mails.jlu.edu.cn). Jun Liu is with the School of Electronic and Information Engineering, Beihang University, Beijing 100191, China (e-mail: liujun2019@buaa.edu.cn).

characteristics of underwater acoustic communication, such as long propagation delay, high attenuation, narrow bandwidth, and half-duplex communication, impose formidable challenges to the MAC protocol design in UANs [8]. Due to the high compatibility with communication schemes and device models, Time Division Multiple Access (TDMA)based MAC protocols have been extensively adopted in underwater networks across various fields. Numerous offshore field trials indicate their practicality and ease of implementation [9]–[12]. These protocols rely on synchronized and fixedlength slots to enable nodes to access the channel orderly and avoid packet conflicts. However, this synchronized slot paradigm suffers from operational instability due to its reliance on strict time synchronization [13], [14]. The unavoidable estimation skew in underwater time synchronization methods leads to persistent clock offsets among nodes that accumulate over time. Moreover, maintaining time synchronization generally requires extra information exchange, which incurs control overhead in terms of channel and energy within resourceconstrained UANs. Even assuming perfect clock synchronization, the synchronized slot paradigm presents a fundamental conflict between high channel utilization and low scheduling complexity. 1) Dedicated slot allocation can readily avoid packet collisions but leaves substantial channel idleness, as propagation delays dominate the round-trip latency [15], [16]. Although exploiting differences in propagation delays allows for spatial slot reuse, its rigid topology-dependence renders transmission concurrency vulnerable [17]–[23]. 2) Shortening the slot length improves scheduling frequency, holding promise for dense and staggered channel access. However, this design may introduce across-slot packet collisions. This necessitates MAC protocols to predict future channel usage, significantly escalating scheduling complexity [24]–[27]. Furthermore, the synchronized and fixed-length slot paradigm makes it difficult to realize dynamic scheduling to accommodate varying traffic load conditions while ensuring high channel utilization [28], [29]. To introduce the necessary flexibility and efficiency, the integration of shortened time slots and adaptive scheduling methods like Deep Reinforcement Learning (DRL) may become a promising solution. However, constrained by the highlighted scheduling complexity, existing protocols struggle to achieve effective network-wide coordination, ultimately failing to maintain fairness. Motivated by the aforementioned limitations of networkwide synchronized time slots, we advocate a paradigm shift in underwater TDMA-based MAC design, transitioning from synchronized slot to asynchronous scheduling, thereby effi-

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

ciently leveraging the long and diverse propagation delays in UANs. To this end, we propose AT-MAC, an Asynchronous Triggered MAC protocol that aims to significantly improve channel utilization and fairness without time synchronization. Specifically, AT-MAC innovatively designs a triggered slot paradigm that decouples coordinated channel access from a rigid timeline, allowing flexible asynchronous scheduling. Based on this paradigm, AT-MAC enhances Multi-Agent Proximal Policy Optimization (MAPPO) to handle asynchronous interaction and incorporates a local fairness guard mechanism to facilitate network-wide fairness, thus achieving efficient adaptive channel access based on partial observations. The main contributions of this paper are as follows: • Design a triggered slot paradigm to enable asynchronous transmission scheduling without clock and slot synchronization. All nodes locally maintain slots, each dedicated to a single, complete DATA-ACK transaction. Asynchronous, variable-length slots grant nodes scheduling flexibility, allowing them to exploit long and diverse propagation delays to naturally stagger channel access. • Propose an asynchronous time-offset MAPPO algorithm adapted for the triggered slot paradigm. We formalize the asynchronous channel access as a time-offset decentralized partially observable Markov decision process (To-Dec-POMDP). To-Dec-POMDP introduces relative time offset to ensure causally consistent credit assignment and avoid estimation bias caused by asynchrony. By integrating To-Dec-POMDP with a centralized training and decentralized execution framework, our algorithm achieves stable convergence and efficient execution. • Present a low-overhead load-aware fairness guard mechanism that enables decentralized nodes to infer the network-wide fairness status and guides transmission correction to maintain fairness. By leveraging the inherent broadcast nature of acoustic signals, each node can overhear the attached information to infer the network’s actual and desired load status, which serves as the basis for transmission correction. • Conduct trace-based experiments, on-board benchmarking, extensive simulations, and ablation studies to comprehensively evaluate AT-MAC. The results validate ATMAC’s practical feasibility and computational practicality on resource-constrained UANs, while extensive simulation results demonstrate its superior performance and strong adaptability across diverse scenarios and traffic load conditions. Ablation studies further confirm the effectiveness of the fairness guard mechanism for maintaining load-aware fairness, as well as the robustness of the time-offset mechanism. II. R ELATED W ORK A. Conventional TDMA-based MAC: Trading Overhead for Efficiency To simplify implementation and mitigate collisions, most TDMA-based MAC protocols adopt the network-wide synchronized time slots that encompass a full transmissionfeedback cycle. The authors in [15] proposed a protocol

2

to avoid packet collisions by extending the slot duration to encompass the maximum propagation delay plus the transmission time of a control packet, thereby ensuring transmission isolation between slots. [12] adopted the same time slot setting but allowed nodes far apart to transmit data in the same slot to accommodate chain topology, thereby increasing network throughput. However, the dedicated synchronized slot design inevitably leads to channel wastage due to the long propagation delays of underwater acoustic signals. To reduce the channel wastage of long synchronized slots, later protocols exploit topology knowledge or handshaking to enable spatial reuse [5], [17]–[21]. Although effective in relatively static settings, these methods require collecting propagation or scheduling information through repeated exchanges, which is costly in large-scale and resource-constrained UANs. Moreover, the feasibility of slot reuse is inherently bounded by network topology, and modifying static scheduling schemes in dynamic environments entails sustained control overhead. B. RL-based MAC Suffering from Synchronized Slot Limitations Reinforcement Learning (RL) has emerged as a promising framework to address the overhead-efficiency trade-off by enabling nodes to learn effective transmission policies through local interaction with the environment. The work in [23] designed a RL-based protocol where all nodes in the network use basic Q-learning to independently find the most appropriate time slots to transmit, using the number of time slots per frame as the state space and updating the Q-values directly. [30]– [32] adopted the same slot design as [23] and incorporated additional considerations to enhance adaptability. The authors in [24] designed a probability-based MAC protocol, which optimizes the channel access probability of nodes within the next adjustable time slots to minimize collisions. All these RL-based MAC protocols enhance channel utilization by reusing either a single long-duration slot or a frame comprising multiple short slots. While the flexible transmission timing selection within a slot can gain additional opportunities for spatial reuse, its feasibility is dependent on the network topology. To make such reuse feasible, the slot duration must be sufficiently long, which further exacerbates channel underutilization when network traffic is insufficient. To further adaptively exploit the substantial unoccupied channel resources, several studies have proposed shortening the time slot duration to conduct fine-grained scheduling, aiming to achieve dense and staggered receptions at the sink node. [25]–[27] shorten the time slot length to the duration of data plus ACK transmission, and employ a DRL algorithm to enable nodes to autonomously decide whether to transmit packets based on local observations. [33] proposed a delayfluctuation-resistant MAC protocol based on Double DQN in the shortened-slot paradigm, which can adapt to the delay fluctuation to avoid the credit assignment bias and achieve better performance. [34] proposed a MAPPO-based MAC protocol that allows multiple intelligent nodes to cooperatively access the channel in the shortened-slot paradigm, thereby improving network efficiency and achieving better

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

network fairness. Such an approach breaks the transmission isolation between slots, significantly enhancing the flexibility of channel access scheduling, enabling intelligent nodes to exploit idle channel. [35] proposed DA-MARLA, which leverages MAPPO to facilitate cooperative channel access among multiple nodes, thereby improving network efficiency. DA-MARLA adopts a more radical design by setting the time slot duration equal to the data transmission time and omitting the use of ACKs. This design avoids the action timedependence caused by waiting for ACKs, reducing scheduling complexity and enabling cooperative channel access among multiple intelligent nodes. However, these DRL-based MAC protocols with shortened time slots overlook the heterogeneity of propagation delays, leading to issues such as credit assignment bias and self-action masking. This leads to their reliance on idealized environment assumptions, making it difficult to achieve stable learning and efficient execution in real acoustic propagation environments. C. Traffic Load Awareness and Fairness in UANs Recently, with the increasing number of nodes and the development of underwater applications, the fairness issue in UANs has become more prominent. Some research has begun to focus on how to achieve load awareness and fair scheduling in UANs. [36] proposed a heuristic link scheduling algorithm (TOTA) that considers traffic load and routing information to save energy and improve throughput. The base station collects the global information such as topology, traffic load on each link, to compute the optimal schedule. [37] allowed nodes to attach load information to control packets, achieving trafficload adaptation by transmitting variable length packets before the scheduled time slots. [38] proposed a load-based time slot allocation scheme to mitigate intra-network interference and improve channel utilization under fluctuating traffic loads. [39] proposed a traffic load-aware link scheduling scheme for UWSNs, where traffic load is evaluated based on the proportion of time the control channel remains busy during the observation period. Distributed nodes exchange control packets to update their channel resource lists. These strategies necessitate information exchanges between nodes to acquire either global or local traffic load information. However, such information exchange incurs additional overhead in terms of channel and energy consumption, which is detrimental in resource-constrained UANs. Therefore, a practical fairness mechanism should maintain network-wide fairness with minimal additional overhead. III. S YSTEM M ODEL AND P ROBLEM F ORMULATION A. System Model As shown in Fig. 1, we consider a one-hop underwater acoustic network (UAN) where N transmitters send data packets to a sink node over a shared acoustic channel. Such topology is typical for small-scale networks and serves as a fundamental building block for large UANs. We denote the set of transmitters as N = {1, 2, . . . , N }, with the sink node represented as m. Each transmitter i maintains a data queue with length qi and generates data at various rates λi . The

3

Fig. 1. Overview of the considered system. Transmitters send data packets to a central sink, which returns ACK packets.

transmit power pi is pre-configured based on the link distance and channel conditions, and includes an adequate margin to ensure reliable connectivity to the sink. All nodes operate in half-duplex mode, defined as Modei (t) ∈ {Tx, Rx, Idle}, where the tri-mode definition is mutually exclusive, enforcing half-duplex operation such that a node cannot transmit and receive at the same time. Let ui (t) ∈ {0, 1} denote the transmission decision for transmitter i at time t. When ui (t) = 1, transmitter i sends a data packet of size li (t) to sink m. The complete process consists of four phases: 1) Data transmission: at time t, transmitter i sends a data packet to sink m with a transmission delay of δltx = i δ pre + li /R, where δ pre is the preamble duration and R is the physical-layer transmission rate; 2) Data propagation and reception: the packet propagates prop over delay δi,m = di,m /c, where di,m is the distance between transmitter i and sink m, c is the sound speed. prop Sink m receives the packet at time t + δltx + δi,m ; i 3) ACK transmission: upon successfully receiving the data, sink m sends back an ACK packet with a transmission tx delay of δack = δ pre , since the acknowledgment information can be embedded in the preamble; 4) ACK propagation and reception: the ACK packet propagates back to transmitter i with the same one-way delay prop prop tx δi,m , and is received at time t′ = t+δltx +2δi,m +δack . i Upon receiving the ACK, transmitter i considers the data packet delivered and removes it from its queue. Therefore, we can define the data reception window WiD at the sink and the ACK reception window WiA at transmitter i as:  prop prop  WiD (t) = t + δi,m , t + δltx + δi,m , (1) i   prop prop tx WiA (t) = t + δltx + 2δi,m , t + δltx + 2δi,m + δack . (2) i i Constrained by the half-duplex operation and the shared narrowband broadcast channel, the transmission of the data packet pkt from transmitter i to sink m fails if any of the following events occurs: 1) Data reception collision (at sink). Another packet overlaps with WiD (t) at the sink, causing destructive interference and corrupting pkt;

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

TABLE I. Notation Description Notation Description N Transmitters set m Sink node λi Data generation rate li Packet size δltx Data transmission delay i di,m Euclidean distance tx δack ACK transmission delay lmax Maximum packet size h Fairness horizon Biavail Available data amount

Notation Description N Number of transmitters qi Data queue length ui Transmission decision δ pre Preamble duration prop δi,j Propagation delay c Sound speed fi Transmission feedback L Observation length ϵ Fairness deviation tolerance Bideliv Delivered data amount

2) ACK–data half-duplex collision (at sink). The sink is transmitting another ACK during WiD (t), its half-duplex operation precludes simultaneous reception of pkt; 3) ACK reception collision (at transmitter). another packet arrival overlaps with WiA (t) at transmitter i, corrupting the ACK decoding. Based on the above collision analysis, we define fi (t) ∈ {−1, 0, 1} as the feedback outcome for transmitter i’s decision at time t. Specifically, fi = 1 indicates that the packet is successfully delivered (an ACK is received), fi = −1 indicates a failed attempt (no ACK is received for any reason), and fi = 0 indicates that transmitter i remains idle (no transmission is scheduled). Table I summarizes the notations.

4

where π is the transmission scheduling policy, T is the network lifetime, and tki is the k-th decision epoch of transmitter i. 2) Load-Aware Fairness: Let h denote the fairness horizon. At time t, we define the sent-to-available ratio ρi (t) of transmitter i over [t − h, t] to quantify the proportion of successfully delivered data to the total available data within the horizon h, calculated as: P I{fi (tki ) = 1} · li (tki ) tk i ∈[t−h, t] . (5) ρi (t) = Rt qi (t − h) + t−h λi (τ ) dτ Thus, we quantify the network-wide load-aware fairness at time t using Jain’s index over {ρi (t)}i∈N :
2 P i∈N ρi (t) P . (6) F (t) = N · i∈N (ρi (t))2 By definition, F (t) ∈ [0, 1], and F (t) = 1 when all transmitters exhibit identical ρi (t). The long-term fairness objective is "Z # T

JF (π) = Eπ

F (t) dt .

(7)

0

3) Joint Optimization Problem: Combining the effective utilization and load-aware fairness, we formulate:   max JU (π), JF (π) (8a) π

s.t. B. Problem Formulation

ui (t) ∈ {0, 1},

∀i ∈ N , ∀t ∈ [0, T ],

0 ≤ li (t) ≤ min{lmax , qi (t)},

This paper aims to: (i) maximize effective channel utilization by increasing successful receptions and avoiding collisioninduced waste; and (ii) maintain load-aware fairness so that each transmitter obtains transmission opportunities proportional to its load. However, higher channel utilization requires more concurrent transmissions to exploit spatial reuse, and the feasibility of such concurrency is strongly shaped by topology-induced interference and heterogeneous propagation delays among transmitters. Specifically, aggressively maximizing channel utilization may enable a subset of transmitters with favorable concurrent transmission capability to dominate channel access over time, thereby causing unfairness. Given the inherent conflict between these two objectives, we formulate a joint optimization problem to enhance effective channel utilization while ensuring load-aware fairness. 1) Effective Utilization: We define the effective utilization utility Ui (t) to quantify the effective channel utilization contributed by a single transmission scheduling action of transmitter i at time t, which captures both throughput and collision: Ui (t) = fi (t) · li (t)/lmax , (3) where fi (t) is the feedback for the transmission decision at time t, and lmax is the maximum allowable packet size. Given the per-action effective utilization utility Ui (t), the long-term effective utilization of policy π is defined as   X X
JU (π) = Eπ  Ui tki  , (4) i∈N tk i ≤T

(8b)

∀i ∈ N , ∀t ∈ [0, T ]. (8c)

Constraint (8b) restricts the binary transmission decision of each transmitter. Constraint (8c) ensures that the packet size does not exceed the maximum allowable size or the current queue length. IV. AT-MAC P ROTOCOL To address the joint utilization–fairness formulation in Section III, we propose AT-MAC, an asynchronous triggered MAC protocol that improves channel utilization while maintaining fairness, without requiring time synchronization. As illustrated in Fig. 2, AT-MAC comprises three components: (i) a triggered slot paradigm that defines the protocol’s fundamental interaction framework; (ii) a time-offset asynchronous MAPPO that enables coordinated channel access based on local observations to maximize effective utilization; and (iii) a load-aware fairness guard mechanism that suppresses transmission decisions violating the fairness tolerance. AT-MAC decouples fairness from RL objective to avoid the non-stationary learning signals and degraded policy generalization caused by multi-objective reward coupling in longdelay underwater acoustic environments. Within this architecture, AT-MAC leverages MAPPO to prioritize opportunistic utilization subject to the lightweight deterministic fairness constraint, enabling the transmitters to exploit the released channel opportunities resulting from fairness guard mechanism. Sections IV-A–IV-C detail each component, and Section IV-D describes the execution and training workflow.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

5

Section IV-C Load-Aware Fairness Guard Mechanism

Section IV-B Time-offset Asynchronous MAPPO Training

Centralized Critic

State Value

State

Actor Execution

Time-offset Actor … Observation Encoding

1. Load-Unit Construction and Broadcast

2. Load-Unit Table Maintenance and Inference

Candidate Action

3. Tolerance-Based Fairness Guarding Action

Observation

Section IV-A Triggered Slot Paradigm

Interaction Framework

Environment

Available Channel Duration for Other Transmitters Sink

Data Pkt ACK Pkt Overhear One Slot Decision Time

Transmitter 1

…

Transmitter N

Asynchronous channel access & Triggered slot with variable duration

Fig. 2. Overview of the AT-MAC protocol. Built upon the triggered slot paradigm, AT-MAC employs a time-offset asynchronous MAPPO to achieve coordinated scheduling for improved channel utilization, and incorporates a load-aware fairness guard mechanism that corrects transmission decisions to maintain network-wide fairness.

𝑢𝑢 = 0

𝑢𝑢 = 0

Timer Startup

Timer Expiration

(a) u=0&f =0 to Sink

from Sink Timer Startup Timer Expiration Delay 𝜏𝜏 Pkt (size=𝑙𝑙) ACK to Sink from Sink Timer Startup Timer Expiration Timer Startup 𝑢𝑢 = 0 Triggered by ACK Arrival Delay 𝜏𝜏 Pkt (size=𝑙𝑙) ACK 𝑢𝑢 = 1 from SinkExpiration Timer Startup to Sink to Sink Timer from Sink Timer Startup Triggered by ACK Arrival Delay 𝜏𝜏 Pkt (size=𝑙𝑙) ACK 𝑢𝑢 = 1 to Sink from Sink Delay 𝜏𝜏 (size=𝑙𝑙) 𝑢𝑢 = 1 (b) Pkt uto= Sink1 & f = 1 fromACK Sink Timer Startup Triggered by ACKACK Arrival Delay 𝜏𝜏 Timer Pkt (size=𝑙𝑙) 𝑢𝑢 = 1 Startup Timer Expiration Delay 𝜏𝜏 ACK Pkt (size=𝑙𝑙) 𝑢𝑢 = 1 to Sink from Sink Timer Startup Triggered by ACK Arrival Current Slot Next SlotStartup Decision Time Timer Start Time Deadline Timer TimerTimer Expiration Sink from SinkACK Delay 𝜏𝜏 Pktto(size=𝑙𝑙) 𝑢𝑢 = 1 Current SlotDelayNext SlotStartup Decision Timer Deadline Timer Expiration 𝜏𝜏Timer ACK Pkt (size=𝑙𝑙)Time Timer Start Time 𝑢𝑢 = 1 u = 1 & f = −1 Timer(c) Startup Timer Expiration Current Slot Next Slot Decision Time Timer Start Time Timer Deadline

𝑢𝑢 = 0 𝑢𝑢 = 1

Current Slot

Next Slot

Decision Time

Timer Start Time

Timer Deadline

Fig. 3. Three state transitions of triggered slot paradigm: (a) no transmission; the transmitter remains idle until timer expiration and enters the next slot; (b) transmission with an ACK received before timer expiration, then enters the next slot; (c) transmission without an ACK, entering the next slot upon timer expiration. A. Triggered Slot Paradigm We propose a triggered slot paradigm that operates without clock synchronization, allowing transmitters to independently determine when and how to occupy the acoustic channel, and fully utilize the long propagation delay of acoustic signals to maximize channel utilization. At the initial stage of the network, transmitter i can make a decision ai (t) at time t, which marks the beginning of a triggered slot. The decision ai (t) is a tuple containing three elements, represented as:
ai (t) = ui , τi , li , (9) where ui ∈ {0, 1} indicates whether transmitter i chooses to transmit at time t, τi and li are only valid when ui = 1,

representing that transmitter i sends a data packet of size li at time t + τi . To trade off the transmitter’s decision flexibility and action space complexity, we define the selectable range of τi as [0, τmax ], where τmax is given by: tx τmax = δltx + δack +2· max

dmax , c

(10)

which represents the worst-case round-trip delay for transmitting a maximum-size packet. This includes the transmission delays for a maximum-sized data packet (lmax ) and its corresponding ACK, plus the two-way propagation delay over the maximum range dmax at a sound speed of c. In the triggered slot paradigm, the transmission decision ai and subsequent feedback fi jointly affect the slot’s status transition. 1) Non-Transmission (ui = 0): As illustrated in Fig. 3??, if transmitter i selects non-transmission at time t, it starts a timer and remains in the Idle state until the timer is triggered. The timer deadline is set as prop tx ddli (t) = t + δltx + δack + 2δi,m + δ guard , max

(11)

prop where δi,m is the propagation delay estimated from the last successful communication between transmitter i and sink m (continuously updated to reflect location fluctuations), and δ guard is a guard interval that accounts for propagation uncertainty. Since no transmission is scheduled in this slot, the feedback is set to fi (t) = 0, and transmitter i enters the next slot to make a new decision when the timer expires at ddli (t). 2) Transmission (ui = 1): If transmitter i selects transmission at time t, it first stays in the Idle state for a duration of τi and then transmits a data packet of size li . Meanwhile, it initiates a timer with deadline ddli (t) as (11). After sending the data packet, transmitter i returns to the Idle state and waits for the corresponding ACK. Transmitter i may obtain either of the following two feedback:

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

6

𝜁𝑖 =

𝑠𝑡𝑖 , 𝑜𝑡𝑖 , 𝑎𝑡𝑖 , 𝑟𝑡 , 𝑠𝑡’𝑖

𝑇

Time-Aware Discounted Reward

𝑡=0

Time Offset Weight w (weight&timeliness)

Information 0

Offset to Decision Time

Hybrid Actor Network

wdata wack waction

TD-error

Critic

Asynchronous Execution Candidate Action Transmit or Not Delay Time

Concatenate

Observation Decision-Making Agent

Actor

Update

Time-offset Observation Encoding

Data Ack Action

Gradient

Time

State

Arrival Time

Centralized Training

Centralized Critic Network

Non-Uniform Interval Trajectory 𝜻

Packet Size

Load-Aware Fairness Guard Mechanism Transmit

Overhear Data/Ack

Other Interacting Agents

Historical Action

…

Decision Time

Time

Delay Time Packet Size

Environment with Triggered Slot

Fig. 4. The flowchart of the proposed time-offset asynchronous MAPPO. Agents execute asynchronously under triggered slot paradigm and generate candidate actions, while centralized training leverages global states to update the policy.

ACK Reception: if the ACK arrives before ddli (t), transmitter i cancels the timer, sets fi (t) = 1, and immediately proceeds to the next slot. • ACK Loss: if no ACK is received by ddli (t), transmitter i sets fi (t) = −1 and enters the next slot when the timer expires. The triggered slot transitions under transmission are depicted in Fig. 3??–??. As illustrated in (11), the timer’s deadline is positively correlated with the propagation delay, allowing transmitters closer to the sink node to make decisions more frequently. This effectively takes advantage of the propagation delay differences between each transmitter and the sink node. Moreover, the triggered slot paradigm eliminates the limitation of synchronized slot with fixed duration to allow transmitters to freely choose when to transmit data, thereby improving the flexibility of channel utilization. •

B. Time-offset Asynchronous MAPPO To fully exploit our proposed triggered slot paradigm and maximize the long-term effective utilization objective JU (π) in (4), we leverage multi-agent reinforcement learning (MARL) to coordinate cooperative decision-making among decentralized transmitters. However, the introduction of the triggered slot breaks the conventional MARL assumption of synchronous decision-making across agents, creating a fundamental mismatch between protocol design and learning framework. To accommodate asynchronous decision-making enabled by the triggered slot, we model the multi-node cooperative channel access problem in UANs as a Time-offset Decentralized Partially Observable Markov Decision Process (To-Dec-POMDP), and develop a time-offset asynchronous MAPPO algorithm to solve it. The flowchart of the proposed time-offset asynchronous MAPPO is illustrated in Fig. 4. To-Dec-POMDP can be represented as a tuple < S, O, A, P, R, N, γ >, where s(t) ∈ S is the environment state at time t, oi (t) ∈ O is the local observation of agent i at time t. Agent i selects action ai (t) ∈ A based on observation

oti . The environment transits to the next state s(t′ ) according to the transition probability P and returns a reward r = R(s, ai ). Async Hybrid Action Space A. All transmitters in our proposed multi-node network share the same action space to select actions asynchronously, thus Ai = A. As described in the triggered slot, the action of a transmitter is defined as a tuple consisting of three elements: whether to transmit, the delay before transmission, and the data packet size. Accordingly, the action space is formulated as: A = {0, 1} × [0, τmax ] × [0, min{lmax , qi (t)}].

(12)

act At any given
time ti , agent i selects a
hybrid action ai = ui , τi , li , which is recorded as tact i , ai . Transmitters act asynchronously; yet the long propagation delay couples their actions, and the resulting joint behavior determines the environment state and reward. Time-offset Observation Space O. The global environment state s(t) is the union of all agent observations, denoted as s(t) = {o1 (t), o2 (t), · · · , oN (t)}. However, due to the long propagation delay and half-duplex operation, each transmitter can only obtain partial and delayed information about the network status, leading to a partial observation. Hence, we construct the observation oi (t) of agent i at time t from three event histories in the local channel:

oi (t) = Hiact ∪ Hidata ∪ Hiack ,

(13)

where Hiact , Hidata , and Hiack are the local histories of agent i’s own actions, overheard data, and overheard ACK, respectively. Such historical events implicitly contain the spatio-temporal topology and connectivity information of the network, which can help intelligent transmitters infer the network status and other transmitters’ behavior. Each history keeps at most L×N latest events, where N is the number of transmitters and L is a tunable history length controlling the trade-off between observation richness and complexity. Specifically, the three types of events are recorded as follows: (1) Hiact : (tact i , ai , fi ), indicating that transmitter i takes action ai at time tact and receives feedback fi ; (2) Hidata : i

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

7

, j, l), indicating that transmitter i receives a data packet (tdata i sent by transmitter j ∈ N with packet size l at time tdata ; (3) i Hiack : (tack i , j), indicating that transmitter i receives an ACK corresponding to a packet from transmitter j at time tack i . Time-offset Observation Encoding. The time-offset mechanism is proposed to provide effective observations for agents at any given time. Since the time in the observation is represented as the absolute value of the network runtime, it leads to a non-uniform and unbounded observation space, which is not conducive to transmission strategy training. To address this issue, time offset weight is defined using linear decay. At any given observation time t, each timestamp ts in the observation is replaced by its corresponding time-offset weights w(t, ts ), computed as   |t − ts | s , (14) w(t, t ) = max 0, 1 − L · τmax where ts denotes a timestamp recorded in one of the observation histories, i.e.,tact in Hiact , tdata in Hidata , or tack in Hiack . The time offset weight w(t, ts ) ∈ [0, 1] preserves the timeliness of observations while yielding a consistent observation representation for decisions made at arbitrary asynchronous epochs. Time-offset Reward Function R. The reward function evaluates the executed action and provides guidance for future decisions. In our paper, an individual reward is given to each agent at each asynchronous triggered slot, instead of team reward. Since the objective of agents is to maximize network throughput and transmission success rate, the reward function is defined based on the executed actions and the feedback received. The reward function r of action ai = {ui , τi , li } at time tact is defined as: i   0, ui = 0,      li τmax ri = α lmax t − tact , ui = 1 and fi = 1, (15) i    τmax   , ui = 1 and fi = −1, − t − tact i

We instantiate the above To-Dec-POMDP with a MAPPObased centralized training and decentralized execution (CTDE) actor–critic framework. Each transmitter i executes a decentralized actor πθi (ai | oi ) to sample a hybrid action ai from its local observation oi (t) at each decision epoch, while a centralized critic is trained with access to the global state s(t) to stabilize learning under partial observability and coupled long-delay interactions. During asynchronous execution, the actor outputs a candidate action, which is applied only if approved by the load-aware fairness guard (Section IV-C). Compared with standard MAPPO, we incorporate two key adaptations to improve stability and efficiency under the asynchronous triggered slot paradigm: the time-offset observation encoding in (14) and the time-aware discounted reward formulation in (16)–(17). These adaptations improve the consistency of observation representation and the accuracy of credit assignment across asynchronous decision epochs. We refer to the resulting method as time-offset asynchronous MAPPO. The corresponding training and execution workflow is further adapted to this asynchronous setting, as covered in Section IV-D.

where t − tact is the time offset of the last action time relative i to the current time, and α is a reward function coefficient to adjust the emphasis of agents on throughput and success rate. The optimization goal of AT-MAC is to maximize the cumulative reward throughout the network lifetime. Hence, we define the time-aware discounted long-term reward R of agents with non-uniform interaction intervals as: X k Ri = γ ∆ti /τmax · ri (tki ), (16)

bi (t) = ⟨Biavail (t), Bideliv (t)⟩.

tk+1 ≤T i

where ∆tki = tk+1 − tki is the elapsed time between two i consecutive decision epochs of transmitter i, ri (tki ) is the reward corresponding to the action at time tki , T is the network lifetime. The discount factor γ ∈ (0, 1) is applied with a time-normalized exponent to properly handle non-uniform interaction intervals, using τmax as the normalization scale. Finally, we define the network-level objective by aggregating individual returns as X R= Ri . (17) i∈N

C. Load-Aware Fairness Guard Mechanism To pursue the long-term load-aware fairness objective JF (π) in (7) under decentralized execution, we design a load-aware fairness guard mechanism that infers network-wide fairness from local overhearing and suppresses transmissions that would violate the prescribed tolerance. The flowchart of the load-aware fairness guard mechanism is illustrated in Fig. 5. Each transmitter i infers the sentto-available ratios of other transmitters by leveraging the broadcast nature of the network. The load-aware fairness guard mechanism facilitates the convergence of these ratios among transmitters, thereby improving the network-wide load-aware fairness. The complete flow is as follows: 1) Load Unit Construction and Broadcast: At each decision time t, transmitter i forms a load unit (18)

Biavail (t) denotes the available data amount over the observation horizon h, and Bideliv (t) denotes the successfully delivered data amount over the same horizon, which are calculated according to (5), as: Z t avail Bi (t) = qi (t − h) + λi (τ ) dτ, t−h X (19) Bideliv (t) = I{fi (tki ) = 1} · li (tki ), tk i ∈[t−h, t]

where qi (t − h) is the queue length at the beginning of the observation horizon, λi (τ ) is the data arrival rate at time τ , and tki is the k-th decision epoch of transmitter i. The load unit bi (t) is disseminated through two phases: (i) Data Broadcast Phase: During data transmission, transmitter i piggybacks its latest bi (t) on the data packet. Since acoustic communication is broadcast, nearby transmitters may overhear this piggybacked load unit even when the data packet is not destined for them;

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

Load Unit Construction and Broadcast Phase 1: Data Broadcast Sink Transmitter i Load Unit of Transmitter i

Attached

Transmitter j

Phase 2: ACK Broadcast

Attached

Load Unit of Transmitter i

Load Unit Table Maintenance and Inference Load Unit Table Source: Transmitter N … Source: Transmitter 1 Arrival Load Time&Phase Unit …

estimated sent-to-available ratio then follows as ρ̂j (t) =

B̂jdeliv (t) B̂javail (t)

, ∀j ∈ N \ {i},

(23)

Finally, transmitter i derives the optimal fairness reference ratio ρ̂⋆ (t) based on its locally observed sent-to-available ratio ρi (t) and the estimated ratios ρ̂j (t) of other transmitters:  1 X ρ̂⋆ (t) = ρ̂j (t) + ρi (t) , (24) N

Inference Load Fairness Status Actual

Sink

8

Optimal

j∈N \{i}

Transmitter i

Transmitter j

Tolerance-Based Fairness Guarding Candidate Action

Executed Action

Fig. 5. The flowchart of the load-aware fairness guard mechanism. Each transmitter infers the network-wide load fairness status from local observations, and suppresses transmission decisions from time-offset asynchronous MAPPO that would violate the fairness tolerance.

(ii) ACK Broadcast Phase: When a data packet is successfully received, the sink relays the parsed load unit in the ACK. In the one-hop network, the sink is within the communication range of all transmitters. Therefore, the ACK phase provides network-wide dissemination, albeit with additional relay latency. 2) Load Unit Table Maintenance and Inference: Each transmitter i maintains a load unit table TiTU = {Bi,1 , Bi,2 , . . . , Bi,N },

(20)

where Bi,j stores the load units originated from any other transmitter j. Each stored record takes the form (tarr , b, ph), where tarr is the reception time at transmitter i, b is the load unit content, and ph ∈ {D, A} indicates whether the unit is obtained from the data (D) or ACK (A) broadcast phase. Due to propagation delay and half-duplex collisions, TiTU can be incomplete and delayed. To partially remove phasedependent offset, transmitter i calibrates each record timestamp to an approximate generation time: ( tarr − 2dmax /c, ph = A, t̃ = arr (21) t − dmax /c, ph = D, where dmax is the maximum one-hop distance to the sink. Given the calibrated history in Bi,j , transmitter i extrapolates the current load statistics of any other transmitter j at decision time t. Let [t̃jlast , Bjavail (tjlast ), Bjdeliv (tjlast )] denote the last calibrated record for transmitter j. We approximate the current Bjavail (t) and Bjdeliv (t) as follows: B̂javail (t) = Bjavail (tjlast ) + vjavail (t) · t − t̃j ,
B̂jdeliv (t) = Bjdeliv (tjlast ) + vjdeliv (t) · t − t̃jlast ,
last

(22)

where vjavail (t) and vjdeliv (t) are estimated change rates obtained via weighted piecewise linear extrapolation [40]. The

which corresponds to the ideal condition that maximizes network-wide load-aware fairness F (t) defined in (6). 3) Tolerance-Based Fairness Guarding: Let ūi (t) ∈ {0, 1} denote the candidate transmission decision suggested by the underlying time-offset asynchronous MAPPO. The fairness guard post-processes ūi (t) to produce the executed action: ( 0, ūi (t) = 1 and ρ̂i (t) > (1 + ϵ)ρ̂⋆ (t), ui (t) = (25) ūi (t), otherwise, where ϵ ≥ 0 is a hyperparameter controlling the tolerance for fairness deviation, with smaller ϵ imposing stricter fairness requirements but potentially limiting channel utilization. D. Workflow of AT-MAC AT-MAC follows a CTDE workflow. Specifically, a centralized simulation or digital twin controller is deployed at the sink node to collect the agent decisions and corresponding environment state, enabling centralized training of all agents’ strategies and safe policy updates under significant environment changes [41]. However, the agents must rely on partial observations during execution. The workflow of AT-MAC is detailed below. 1) Decentralized Action Selection: Let the decentralized actor of agent i be denoted as πθi (ai | oi ) with policy parameters θi . Given oi (t), the actor produces a candidate hybrid action āi (t) = (ūi (t), τ̄i (t), ¯li (t)). Since the action contains a discrete transmission indicator and continuous control variables, we apply exploration noise only to the continuous components: (u)

ūi (t) ∼ πθi (· | oi (t)),
(τ̄i (t), ¯li (t)) ∼ N(µθ oi (t) , σ 2 ). i

(26)

t

where µθi (·) denotes the actor output for the continuous variables. The exploration variance decays over training as σt2 = σ02 e−βt . The candidate action āi (t) is post-processed by the fairness guard in (25) to obtain the executed action. 2) Separate Memory Buffer: Each agent interacts with the environment, generating trajectory segments at asynchronous time points t: ζi = {(oi (t), ai (t), r(t), oi (t′ ))}Tt=0 .

(27)

Due to asynchronous decision-making, agents’ trajectories are misaligned and may have various lengths. Therefore, trajectories from multiple agents cannot be aggregated into a shared memory buffer. To preserve temporally ordered, agentconsistent rollouts, we maintain a separate memory buffer for

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

9

Algorithm 1 Workflow of AT-MAC each agent’s actors {πθi }N i=1 with random weights θi and centralized critic Vϕ with weights ϕ; 2: Initialize replay buffers {Di }N i=1 ← ∅, batch size B. 3: for episode = 1 to M do 4: ▷ Asynchronous Decentralized Execution Phase 5: for each agent i ∈ {1, . . . , N } in parallel do 6: Reset environment, get initial observation oi (t) and global state s(t). 7: while t ≤ Tmax do 8: Sample action āi (t) according to (26); 9: Update load unit table TiTU using received packets; 10: Correct āi (t) → ai (t) via fairness guard (25); 11: Execute ai (t), receive reward r(t) and oi (t′ ), s(t′ ); 12: Store transition (s(t), oi (t), ai (t), r(t), s(t′ )) in Di ; 13: oi (t) ← oi (t′ ), s(t) ← s(t′ ), t ← t′ . 14: end while 15: end for 16: ▷ Centralized Policy Optimization Phase 17: for each agent i ∈ {1, . . . , N } when |Di | ≥ B do 18: Sample mini-batch {(s, o, a, r, s′ )} ∼ Di ; 19: Update ϕ by minimizing TD error as (29); 20: Compute advantage  according to (31); 21: Update θi via gradient ascent as (32); 22: Reset buffer: Di ← ∅. 23: end for 24: end for 1: Initialize

(a) Danjiang Lake, Nanyang, China

(b) Jiaozhou Bay, Qingdao, China

(c) Songhua Lake, Jilin, China

Fig. 6. Field trials for collecting real-world data, including: (a) Danjiang Lake trial in October 2024, (b) Jiaozhou Bay trial in March 2025, and (c) Songhua Lake trial in October 2025.

where ρ(θi ) = πθi (ait |oit )/πθold (ait |oit ) is the importance sampling ratio. The clipping range ϵ constrains policy updates to prevent destructive large steps. The trained optimal policy parameters are encapsulated for decentralized execution. Each agent decides transmission based solely on local partial observations without additional information exchange. Therefore, AT-MAC’s collaborative strategy does not incur additional system communication overhead, requiring only the extra lightweight information necessary for the fairness guard mechanism. V. T RACE -BASED VALIDATION AND H ARDWARE F EASIBILITY A NALYSIS

each agent, which is essential for correctly computing timedependent return and advantage estimates (e.g., TD-based or GAE-style) under asynchronous decision epochs. Each agent’s trajectory is aggregated with global state s(t) into its memory buffer Di : Di ← Di ∪ {(s(t), oi (t), ai (t), r(t), s(t′ ))}.

(28)

3) Centralized Policy Optimization: An asynchronous centralized policy optimization strategy is adopted in AT-MAC. Specifically, centralized policy training is initiated as soon as each agent’s memory buffer accumulates a sufficient number of trajectories. A centralized critic Vϕ (s) is trained by minimizing the time-aware TD error: i h 2 (29) Lcritic (ϕ) = E(s,o,a,r,s′ )∼D (y − Vϕ (s)) , y = r + γ ∆t/τmax Vϕ (s′ ),

(30)

where ∆t is the time offset between the current and next decision epochs. The critic’s value estimates guide policy updates through the advantage function: Â = y − Vϕ (s).

To evaluate the effectiveness of AT-MAC in real-world, this section conducts trace-based convergence validation using actual field trial data. Furthermore, AT-MAC’s computational feasibility and energy consumption are evaluated on the embedded platform deployed in field trials.

(31)

Actors update their policies by maximizing the clipped surrogate objective:  " # min ρ(θi )Ât , Lactor (θi ) = Et (32)
 clip ρ(θi ), 1 − ϵ, 1 + ϵ Ât

A. Overview of Field Data Traces and Hardware Platform As illustrated in Fig. 6, we have conducted several multinode field trials across diverse environments, ranging from shallow lakes to deep-sea scenarios. These trials provide realworld data, including node connectivity under various topologies, link signal-to-noise ratios, and sound speed profiles. The same underwater node unit was utilized across these three trials, comprising: (i) a transducer for underwater electroacoustic conversion, (ii) an acoustic modem for baseband signal processing, and (iii) an embedded protocol-stack board, as shown in Fig. 7. Tailored to the transducer’s characteristics, the acoustic modem employs orthogonal frequency-division multiplexing (OFDM) combined with spread spectrum modulation to enable efficient underwater communication. To ensure reliable physical layer communication, we set the transmission power to the maximum value of 64 W, which can achieve a stable maximum communication range of up to 5.5 km under typical underwater acoustic channel conditions in our field trials. The capabilities of the underwater node unit have been validated in multiple field trials, with specific parameters listed in Table II.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

10

TABLE III. Training Parameters

(a) Underwater node unit

(b) Shore-based test

Parameters Discount Factor GAE Parameter Clip Range Entropy Coeff. Opt. Epochs Reward Coeff. Fairness Horizon

Values 0.95 0.95 0.2 0.01 10 2 100 × N s

Parameters Actor Learning Rate Critic Learning Rate Batch Size Update Horizon Episode Duration Observation History Length Fairness Guard Tolerance

Values 1 × 10−5 1 × 10−4 128 4096 10,000 s 5 0.3

Fig. 7. Hardware implementation of the underwater node unit and shore-based test scenario.  Parameter Effective Transmission Rate Maximum Packet Size Preamble Delay Operating Frequency Maximum Power Stable Maximum Communication Range

Value 1000 bps 200 Bytes 0.3 s 10 kHz - 20 kHz 64 W 5500 m

B. Convergence Validation on Real-Trial Trace 1) Scenario Settings: To validate AT-MAC’s real-world effectiveness, we reconstruct trace-driven simulation environments from three field trials across Danjiang Lake, Jiaozhou Bay, and Songhua Lake, each featuring a five-node network configuration. The nodes were deployed on moored vessels, with their topology and measured connectivity illustrated in Fig. 6. We employ Aqua-Sim-FG to replicate the trial’s network environment, including topology, link connectivity, and communication parameters [42]. Aqua-Sim-FG features a high-fidelity physical layer module that accurately models underwater acoustic communication dynamics, including distance-dependent attenuation, ambient noise, and spatiotemporal packet collisions. The measured sound speed profile is incorporated into the Bellhop model of Aqua-Sim-FG to derive the channel characteristics, ensuring that the simulated environment mirrors the real-world conditions. Data generation follows a Poisson process with rate λpkt , which denotes the average arrival rate of maximum-sized packets, while rare stochastic bursts (with probability 1 × 10−4 ) temporarily amplify the arrival rate by 200% for 1,000 s. 2) Training Parameters: The observation history length L is set to 5, and the reward function coefficient α in (15) is set to 2. The fairness horizon h in (5) is set to 100 × N seconds, and the fairness guard tolerance ϵ in (25) is set to 0.3. The chosen parameters are the result of preliminary grid search tuning, which demonstrated robust performance across different scenarios. Each agent’s hybrid actor network comprises two fully connected (FC) layers with 256 hidden units and Tanh activation, followed by an output layer composed of three parallel FC layers corresponding to the three elements of the hybrid action. The critic network has two FC layers with 256 hidden units and a single-unit output layer. The scenario is trained for 10,000 episodes, with the key training parameters summarized in Table III.

5HZDUG

TABLE II. Communication Parameters  

6RQJKXD/DNH 'DQMLDQJ/DNH <HOORZ6HD

 







7UDLQLQJHSLVRGHV



Fig. 8. Convergence curves of AT-MAC in the reconstructed real-world scenarios. The shaded regions indicate the 95% confidence intervals.

3) Convergence Results: We train AT-MAC 5 times using different random seeds, and report the mean episode reward with the corresponding 95% confidence interval (shaded region). Fig. 8 shows the reward convergence of AT-MAC in the reconstructed scenarios. The reward increases consistently during training and reaches a clear plateau within 8,000 episodes, while the confidence band progressively narrows and remains stable thereafter. This pattern indicates that AT-MAC can consistently learn a stable transmission policy under the field-reconstructed topology and real modem parameters. C. Computational and Energy Feasibility A critical concern for DRL-based protocols is whether the inference overhead is acceptable for resource-constrained underwater nodes. We evaluate this on the specific embedded board that was deployed across all aforementioned field trials. The embedded board is powered by an N100 processor, an energy-efficient platform with a thermal design power of 6 W. This low-power platform is responsible for executing the underwater protocol stack and real-time policy inference. 1) Model Complexity Analysis: The Actor network of ATMAC comprises two shared fully connected (FC) layers with 256 hidden units each (activated by Tanh), branching into three parallel FC output heads for hybrid action generation. Fig. 9 illustrates the parameter count of the Actor network across different network sizes and observation history lengths. As observed, the parameter scales linearly with the input dimension, which is determined by the number of transmitters (N ) and the history window length (L). Notably, even in the most complex scenario (N = 7, L = 7), the network contains merely 154.88 × 103 parameters, which requires only 0.31 M

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

11

3DUDPHWHUVîñ

  

L=3 L=4 L=5 L=6 L=7

 









1XPEHURIWUDQVPLWWHUVN



Fig. 9. Model complexity of AT-MAC’s actor network, measured in the number of parameters across different numbers of transmitter N and observation history lengths L.

(a) Syn-7

Fig. 10. Synthetic large-scale scenarios. •

floating-point operations (FLOPs) per inference and occupies approximately 605 KB of memory. 2) On-Board Benchmarking Results: We deployed the trained policy network to the embedded protocol-stack board and measured the inference latency and power consumption over 1,000 independent runs. • Inference Latency: The average time required to generate an action is 0.8 ms. Given that the typical acoustic transmission slot and propagation delay are in the order of seconds (e.g., > 1000 ms), the decision-making latency occupies less than 0.1% of the operational cycle, making it computationally negligible. • Energy Consumption: The average power consumption remains stable at approximately 4600 mW, with no statistical increase observed when DRL inference is activated compared to the baseline operation (running only the OS and protocol stack). This is attributed to the lightweight nature of the Actor network, which incurs minimal CPU load. These results confirm that AT-MAC is lightweight enough to be deployed on existing underwater hardware without requiring hardware upgrades or compromising battery life. VI. P ERFORMANCE E VALUATION This section conducts a comprehensive performance evaluation of AT-MAC using real-world, public dataset, and synthetic large-scale topologies. Furthermore, ablation studies are performed to validate the effectiveness of AT-MAC’s loadaware fairness guard mechanism and time-offset observation encoding. The experimental design focuses on four key aspects: • Performance and Generalization Profiling: We evaluate AT-MAC’s performance and generalization across varying packet arrival rates in a reconstructed real-world scenario (Section VI-B). • Scalability on Diverse Scenarios: We evaluate AT-MAC across scenarios, which exhibit different inter-node interference relationships and network scales (Section VI-C). • Generalization of the Fairness Guard: We validate the generalization of AT-MAC’s fairness guard mechanism under unseen dynamic traffic patterns (Section VI-D).

(b) Syn-8

Ablation Studies: We conduct ablation studies to validate the effectiveness of the fairness guard mechanism and time-offset observation encoding (Section VI-E).

A. Evaluation Setup 1) Experimental Settings: AT-MAC is evaluated under three categories of scenarios: • Reconstructed Real-World Scenarios: This category includes three scenarios from Danjiang Lake (5-node), Jiaozhou Bay (5-node), and Songhua Lake (5-node). The specific topologies and connectivity relationships are illustrated in Fig. 6. • Dataset-Derived Scenarios: We utilize the ASUNA dataset [43], which provides measured topologies from small-scale nearshore trials in Italy, Germany, and Israel. Four topologies are selected: Garda Lake-1 (5-node), Garda Lake-2 (5-node), Hadera-1 (5-node), and Werbellin Lake-3 (6-node). We preserve the original topology and connectivity, while scaling the maximum transmitter-tosink distance to 5000 m to match the real trial scale [44]. • Synthetic Large-Scale Scenarios: Since the maximum number of nodes in the aforementioned scenarios is limited to 6, we designed two synthetic large-scale topologies to evaluate scalability, consisting of Syn-7 (7-node) and Syn-8 (8-node) topologies, as illustrated in Fig. 10. The communication parameters are consistent with the real field trials, as listed in Table II. The training parameters for AT-MAC are consistent with those used in the convergence validation, as summarized in Table III. All experiments are conducted on a workstation equipped with an AMD 5995WX CPU, 128GB RAM, and an NVIDIA RTX 4090 GPU. 2) Baseline Methods and Evaluation Metrics: In our simulation, AT-MAC is compared to six baseline methods: • TDMA: A classical protocol that allocates a fixed-length time slot to each transmitter, where each slot is long enough to accommodate a complete data–ACK exchange to avoid collisions. • ALOHA: A fundamental random-access protocol where transmitters transmit uncoordinatedly. Unacknowledged packets trigger a random backoff and retransmission, up to a maximum retry limit before being discarded [45]. • UW-ALOHA-Q: Employs Q-learning to optimize transmission slot selection within a long frame, improving

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

TDMA

ALOHA

12

UW-AlOHA-Q

Async-DL-MAC

DR-DLMA

MOMA-MAC

18

1.0

15

ηE (bits/J)

0.8

SR

Thr (bps)

240

160

AT-MAC

12

0.6 9

80 0.4 0.01 0.02 0.03 0.04 0.05 0.07 0.1

Packet arrival rate (λ

pkt

0.2

0.3

0.01 0.02 0.03 0.04 0.05 0.07 0.1

Packet arrival rate (λ pkt )

)

(a) Throughput

0.2

0.3

0.01 0.02 0.03 0.04 0.05 0.07 0.1

(b) Success Rate

Packet arrival rate (λ pkt )

0.2

0.3

(c) Energy Efficiency

4500 0.88

̄ (s) δe2e

3000

F5%

0.80

1500

0.72

0 0.01 0.02 0.03 0.04 0.05 0.07 0.1

Packet arrival rate (λ pkt )

0.2

0.3

(d) End-to-End Delay

0.01 0.02 0.03 0.04 0.05 0.07 0.1

Packet arrival rate (λ pkt )

0.2

0.3

(e) Fairness pkt

Fig. 11. Performance across varying packet arrival rates λ

channel temporal reuse and mitigating collisions compared to standard ALOHA [23]. • Async-DL-MAC: Allows transmitters to vary the start timing of data transmissions within a slot. Using a DRL algorithm, it flexibly exploits idle temporal channels caused by the spatial-temporal uncertainty of UANs [27]. • DR-DLMA: Divides the channel into micro-slots equal to the packet transmission delay. It utilizes a single-agent DRL algorithm to opportunistically exploit slots left by other transmitters [25]. • MOMA-MAC: Adopts the short-slot design of DRDLMA but introduces MAPPO, enabling multiple transmitters to collaboratively learn and exploit idle channel resources to enhance overall network performance [34]. The evaluation metrics include: (a) throughput (Thr) in bps, the total successfully delivered data per unit time; (b) success rate (SR), the ratio of successfully delivered data to total attempted data; (c) end-to-end delay (δ̄e2e ) in seconds, the mean elapsed time per delivered byte from packet generation to successful ACK reception, capturing queuing, transmission, and retransmission overheads; (d) energy efficiency (ηE ) in bits/J, the ratio of delivered data to the total energy consumed by data and ACK transmissions; and (e) worst-case load-aware fairness F5% , defined as the 5th percentile of the real-time fairness F (t) in (6), which serves as a robust lower bound. B. Performance and Generalization Profiling Under Varying Traffic Loads This subsection profiles AT-MAC in a reconstructed fieldmeasured scenario (Danjiang Lake) under varying packet

in the reconstructed Danjiang Lake scenario. arrival rates λpkt . The policy is trained at λpkt = 0.1 and evaluated across a range of λpkt to assess its generalization to different traffic loads. As shown in Fig. 11??, the throughput of all protocols increases with the packet arrival rate λpkt under light traffic loads, owing to the available channel capacity. However, as λpkt increases, the throughput trends diverge significantly, with AT-MAC exhibiting a substantial advantage. Specifically, AT-MAC achieves its peak throughput of 299.5 bps at its training rate (λpkt = 0.1) and maintains superiority across other untrained loads, outperforming the most competitive baselines, UW-ALOHA-Q (222.6 bps) and Async-DL-MAC (216.5 bps), by approximately 34.6% and 38.3%, respectively. This gain is attributed to its asynchronous triggered slot mechanism, which breaks the rigid constraints of synchronized scheduling and mitigates the impact of long underwater propagation delays, thereby maximizing spatial reuse. Conversely, MOMA-MAC performs poorly in this scenario, yielding a substantially lower throughput of only 172.2 bps, as its idealized environmental assumptions fail to hold in realistic UANs, preventing its policy from converging to the theoretical optimum. As shown in Fig. 11??–??, the trends in Success Rate (SR) and energy efficiency are consistent. As λpkt increases, the SR of most baselines deteriorates due to intensified channel contention. In contrast, AT-MAC exhibits a lower SR under light loads (e.g., 83.2% at λpkt = 0.01). This stems from imperfect generalization: trained specifically at λpkt = 0.1, the AT-MAC policy yields suboptimal scheduling decisions in low-load states. However, as λpkt approaches the training regime, the SR of AT-MAC stabilizes near 100% (e.g., 98.3%

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

TDMA

ALOHA

13

UW-AlOHA-Q

Async-DL-MAC

DR-DLMA

MOMA-MAC

AT-MAC

1.0 16

ηE (bits/J)

0.8

SR

Thr (bps)

240

160

0.6

12

8

80

0.4 1

2

3

4

5

Topology index

6

7

8

1

2

(a) Throughput

3

4

5

Topology index

6

7

8

1

(b) Success Rate

2

3

4

5

Topology index

6

7

8

(c) Energy Efficiency

4800 0.88

̄ (s) δe2e

4200

F5%

0.80

3600

0.72

3000

1

2

3

4

5

Topology index

6

7

8

1

(d) End-to-End Delay

2

3

4

5

Topology index

6

7

8

(e) Fairness pkt

Fig. 12. Performance across diverse network topologies at λ = 0.1. Indices 1–8 correspond to: (1) Jiaozhou Bay, 5-node; (2) Songhua Lake, 5-node; (3) Garda Lake-1, 5-node; (4) Garda Lake-2, 5-node; (5) Hadera-1, 5-node; (6) Werbellin Lake-3, 6-node; (7) Syn-7, 7-node; and (8) Syn-8, 8-node. at λpkt = 0.1). The energy efficiency closely follows this SR trend, since a high SR indicates fewer packet collisions and retransmissions, which eliminates redundant energy expenditure and maximizes the delivered bits per unit of energy. As shown in Fig. 11??, the end-to-end delay of all protocols increases with λpkt , primarily due to exacerbated queuing delays. Across all evaluated load levels, AT-MAC consistently exhibits superior delay performance. This advantage is attributed to its superior throughput and high success rate, which alleviate queue congestion and minimize time-consuming retransmissions over long-delay underwater acoustic channels. As shown in Fig. 11??, the fairness of all protocols shows a non-linear trend with increasing λpkt . At λpkt = 0.01, fairness is high across all protocols (e.g., 0.92 for ATMAC) due to sufficient channel capacity. At medium loads (λpkt ∈ [0.02, 0.07]), AT-MAC experiences a slight fairness drop (decreasing to 0.86 at λpkt = 0.04). This occurs because its fairness guard mechanism effectively suppresses greedy nodes but cannot actively compensate disadvantaged ones, reflecting imperfect generalization under non-training loads. As λpkt increases further, the fairness of all protocols exhibits an upward trend, since severe congestion uniformly constrains the resource acquisition of all nodes. Despite this mid-load fluctuation, AT-MAC consistently outperforms all baselines. This confirms that the proposed fairness guard mechanism successfully mitigates disproportionate channel occupation to maintain network-wide fairness. Overall, the profiling results demonstrate that despite minor

generalization limitations under non-training regimes, ATMAC consistently achieves superior performance across varying traffic loads. C. Scalability Across Diverse Topologies This subsection evaluates the scalability of AT-MAC across a diverse set of network topologies, including reconstructed real-world deployments (Jiaozhou Bay and Songhua Lake), public dataset-derived topologies from ASUNA (Garda Lake 1/2, Hadera 1, and Werbellin Lake 3), and two synthetic largescale topologies (Syn-7 and Syn-8). The packet arrival rate is fixed at λpkt = 0.1. As shown in Fig. 12??, AT-MAC consistently achieves the highest throughput, peaking at 303.27 bps in topology 4 (Garda Lake-2). This validates the robust scalability of ATMAC, demonstrating its ability to sustain superior performance across varying network scales and complex interference conditions. However, as the network scale increases, the throughput of all intelligent protocols converges toward the TDMA baseline. Specifically, in the largest topology (Syn-8), the throughput of AT-MAC drops to 182.04 bps, approaching TDMA’s rigidly scheduled 160.00 bps. This occurs because a larger network scale exacerbates the complexity of internode interference, diminishing the opportunities for concurrent transmissions. Consequently, all protocols are forced to degrade to more conservative scheduling strategies, leading to performance approaching the level of TDMA.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

14

TABLE IV. Packet arrival rates (λpkt ) of each transmitter under different traffic load patterns. Node

Phase 1

1 2 3 4

0.04 0.04 0.04 0.04

Alternating Phase 2 Phase 3 0.06 0.06 0 0

(a) Alternating

0.06 0 0.06 0

Phase 4

Phase 1

0 0.06 0 0.06

0.04 0.04 0.04 0.04

Burst Phase 2 Phase 3 0.04 0.04 0.04 0.04

0.04 0.08 0.04 0.04

(b) Burst

Phase 4

Phase 1

0.04 0.08 0.04 0.04

0.04 0.04 0.04 0.04

Random Phase 2 Phase 3 0.04 0.06 0.04 0

0.04 0.04 0.04 0.08

Phase 4 0.06 0.04 0.06 0.04

(c) Random

Fig. 13. The generalization of AT-MAC under various traffic load patterns, comparing three metrics in each phase: Initial (backlog at phase start), Generated (new arrivals during the phase), and Sent (successfully delivered data). Optimal fairness is achieved when all transmitters transmit data in equal proportion to their available data (initial plus generated). Furthermore, most protocols perform better in topologies 3 and 4 (Garda Lake-1 and Garda Lake-2) compared to topologies 1, 2, and 5 (Jiaozhou Bay, Songhua Lake, and Hadera-1), despite all containing 5 nodes. This discrepancy exists because the Garda Lake topologies possess a much higher variance in transmitter-to-sink propagation delays, which creates more opportunities for spatial-temporal channel reuse. MOMA-MAC performs particularly poorly in Syn-8, achieving only 120.15 bps, which is because its idealized assumptions violate more severely in larger and more complex topologies, preventing it from learning a robust policy. As shown in Fig. 12??–??, the trends in Success Rate (SR) and energy efficiency further validate the scalability of ATMAC. AT-MAC maintains a robust and stable SR of approximately 98.0% across all evaluated topologies. Consequently, its energy efficiency remains consistently high at around 17.5 bits/J across all topologies, as a high SR indicates fewer collisions and retransmissions. As illustrated in Fig. 12??, the end-to-end delay of all protocols escalates with increasing network scale. Constrained by the low throughput and success rate caused by more complex interference relationships, all protocols face more severe queue congestion and retransmission overheads in larger networks, leading to significantly increased end-to-end delays. Nevertheless, AT-MAC consistently achieves the lowest delay, although its delay increases to 4161.40 s in the largest topology (Syn-8), closely approaching TDMA’s 4308.67 s. As shown in Fig. 12??, the fairness of all protocols generally improves as the network size increases. This trend is mainly because severe contention and increased interference in larger networks equally limit the transmission opportunities of all nodes. Nevertheless, AT-MAC consistently outperforms the baselines across all evaluated topologies, achieving a high fairness index F5% of 0.9138 in the largest topology

(Syn-8). This demonstrates that its fairness guard mechanism effectively maintains fair network-wide channel access, even in the complex interference environments of large-scale UANs. In summary, AT-MAC maintains a performance advantage across diverse network scales and complex interference conditions. Notably, the convergence of AT-MAC’s performance toward TDMA in larger topologies should not be interpreted as a performance degradation, but rather as an indication of its ability to approach the theoretical performance upper bound under severe contention and limited spatial reuse opportunities. D. Generalization Across Different Traffic Patterns In UANs, the spatial diversity of deployment and the dynamic nature of tasks lead to variable traffic patterns among transmitters. To evaluate the generalization of AT-MAC under various traffic patterns, we conduct tests in the reconstructed Danjiang Lake scenario, configured with three distinct traffic patterns as follows: • Alternating: Different subsets of transmitters alternate in packet generation, simulating periodic task shifts. • Burst: A transmitter experiences a sharp increase in packet generation rate, emulating a sudden traffic surge. • Random: All transmitters exhibit time-varying packet generation rates, representing an unpredictable condition. The simulation runs for 10,000 seconds, divided into four phases of 2,500 seconds each. The AT-MAC policy is trained at λpkt = 0.1, and then evaluated under the above traffic patterns without further training to assess its generalization to unseen traffic dynamics. The detailed traffic pattern settings are shown in Table IV. Fig. 13 illustrates the generalization of AT-MAC under different traffic patterns: i) In the alternating pattern, starting from phase 2, pairs of transmitters alternate in data generation.

 ZR)DLUQHVV*XDUG Z)DLUQHVV*XDUG

 

 7LPHV

15

  



ZR)DLUQHVV*XDUG Z)DLUQHVV*XDUG

 7LPHV

(a)



(b)

Fig. 14. Fairness over time in the Danjiang Lake scenario with and without the load-aware fairness guard mechanism. (a) The mean fairness across 100 runs with 95% confidence intervals; (b) Specific fairness curves.

=1





7ROHUDQFH

7KUESV



$YHUDJH7KURXJKSXWESV



/RDG$ZDUH)DLUQHVV

/RDG$ZDUH)DLUQHVV

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021





 

 



)DLUQHVVF5%

Fig. 15. The sensitivity of AT-MAC’s performance to the fairness guard tolerance ϵ. As shown in Fig. 13??, during phases 2-4, the two active transmitters secure over 75% of the full-load throughput by opportunistically utilizing idle channels via overheard network states, which is significantly higher than the expected 50% for two active transmitters; ii) In the burst pattern, transmitter 2 experiences a data generation surge during phases 2-4. As shown in Fig. 13??, AT-MAC provides transmitter 2 with more transmission opportunities, exceeding 35% of the total throughput; iii) In the random pattern, all transmitters exhibit varying λpkt during phases 2-4. Fig. 13?? shows that AT-MAC can dynamically adjust the transmission opportunities for each transmitter based on their traffic load variations, achieving a fair distribution among transmitters. In summary, these evaluations reveal a fundamental tradeoff between strict fairness and maximum throughput, as yielding channels to accommodate uneven traffic load disrupts the optimal scheduling required for peak channel utilization. Nevertheless, the results also demonstrate AT-MAC’s reasonable generalization in non-training traffic patterns. Although accompanied by some performance degradation, it successfully preserves network-wide fairness. E. Ablation Study 1) Effectiveness of the Load-Aware Fairness Guard Mechanism: To isolate the specific contribution of the load-aware fairness guard mechanism, we conduct an ablation study by evaluating the standard AT-MAC against a variant with this mechanism disabled. We run both variants in the Danjiang

7'0$ $/2+$

$70$& (YHU\3NWV

   







7LPHV





Fig. 16. Average throughput (smoothed by a 40 seconds sliding window) of each protocol under clock desynchronization.

Lake scenario under λpkt = 0.05, while the AT-MAC policy is trained at λpkt = 0.1. As shown in Fig. 14??, the AT-MAC with the fairness guard achieves a higher mean fairness of 0.94 compared to 0.78 without the guard. More importantly, it exhibits a much narrower 95% confidence interval. This demonstrates that the guard mechanism effectively maintains a high and stable fairness level across different runs. Furthermore, Fig. 14?? illustrates two real-time fairness curves from individual runs. Without the guard, the fairness index drops sharply to 0.64 at one point. In contrast, AT-MAC with the guard keeps the fairness index consistently high, despite occasional brief dips. This validates that the guard mechanism can infer the current traffic load state to suppress fairness-disrupting transmission decisions, thereby ensuring network-wide fairness. 2) Sensitivity to the Fairness Guard Tolerance: To evaluate the impact of the fairness guard tolerance ϵ, we train AT-MAC using various ϵ values in the Danjiang Lake scenario under λpkt = 0.1. Fig. 15 illustrates the sensitivity of AT-MAC’s performance to ϵ. At ϵ = 0, the strict zero tolerance blocks almost all transmission attempts. Consequently, the RL agent lacks valid training samples, which prevents the policy from converging and leads to poor throughput and fairness. As ϵ increases, network performance steadily improves. Specifically, ϵ = 0.3 is the optimal value, achieving the best balance between high throughput and network fairness. However, increasing ϵ beyond 0.3 brings little benefit: the throughput barely grows, while the fairness drops significantly. This is because a larger tolerance fails to suppress greedy nodes, rendering the fairness guard ineffective. 3) Robustness of Time-Offset Observation Encoding against Clock Desynchronization: In underwater acoustic networks, clock desynchronization can disrupt slot alignment and degrade the performance of synchronization-dependent MAC protocols [46]. This subsection evaluates the robustness of time-offset observation encoding against clock desynchronization in the Danjiang Lake scenario. We set up two typical scenarios of clock desynchronization: •

Clock Drift: The node clock drifts at a constant rate, simulating a gradual desynchronization over time. Transmitter 1’s clock is behind by 0.001 seconds every second.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

Clock Jump: A node undergoes a fault and restarts, causing its clock to jump. Transmitter 4’s clock jumps forward by 4 seconds at 3000s. Fig. 16 illustrates the average throughput under clock desynchronization. TDMA suffers significant performance degradation, dropping to 120 bps at 3000 s due to a sudden clock jump that causes Transmitter 4’s packets to collide with Transmitter 3’s ACKs, and plummeting to 40 bps at 6000 s as cumulative drift leads to collisions between Transmitters 1 and 2. Conversely, ALOHA remains largely unaffected (fluctuating between 20-150 bps), as its random access mechanism does not systematically increase collision probability due to unaligned time slots. AT-MAC maintains a high and stable throughput of 290 bps. By utilizing independent timelines, ATMAC’s DRL time-offset observation space accurately captures the timing changes of other nodes, making it resilient to clock drift. Although the sudden clock jump at 3000 s temporarily distorts its historical observation weights and causes a brief performance drop, AT-MAC recovers once the observation window is replenished with new, accurate data. •

VII. C ONCLUSION This paper advocates a paradigm shift in underwater DRLMAC design, transitioning from synchronized slot to asynchronous scheduling. This design philosophy allows the underwater MAC protocol to natively embrace, rather than counteract, the inherent asynchrony of the underwater acoustic channel. To realize this vision, we propose an asynchronous triggered MAC, denoted as AT-MAC. AT-MAC integrates a triggered slot paradigm, an asynchronous time-offset MAPPO algorithm, and a low-overhead load-aware fairness guard mechanism to achieve collaborative efficient and fair channel access using solely local information. Trace-based and onboard experiments validate the feasibility and computational practicality of AT-MAC. Extensive simulation results further demonstrate its superiority and adaptability across various traffic load conditions and scenarios. Ablation studies confirm the effectiveness of the fairness guard mechanism and timeoffset observation encoding. Future work will explore broader applications of asynchronous event-triggered mechanisms, with particular emphasis on cross-layer designs integrating routing and MAC protocols in multi-hop UANs, thereby enhancing overall performance in more complex network environments. ACKNOWLEDGMENTS This work was supported in part by the National Natural Science Foundation of China under Grant 62471201 and Grant 62501250; in part by the Postdoctoral Science Foundation of China under Grant 2025M771509; in part by the Postdoctoral Fellowship Program of CPSF under Grant Number GZC20250178; in part by the Fundamental Research Funds for the Central Universities under Grant Number 45124031D041. R EFERENCES [1] E. M. Sozer, M. Stojanovic, and J. G. Proakis, “Underwater acoustic networks,” IEEE journal of oceanic engineering, vol. 25, no. 1, pp. 72– 83, 2000.

16

[2] I. F. Akyildiz, D. Pompili, and T. Melodia, “Underwater acoustic sensor networks: research challenges,” Ad hoc networks, vol. 3, no. 3, pp. 257– 279, 2005. [3] L. Wang, Y. Lei, B. Li, and J.-H. Cui, “Towards achieving long-lifespan and self-sustained monitoring of coastal environments,” in 2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2014, pp. 3413–3418. [4] M. Jahanbakht, W. Xiang, L. Hanzo, and M. R. Azghadi, “Internet of underwater things and big marine data analytics—a comprehensive survey,” IEEE Communications Surveys & Tutorials, vol. 23, no. 2, pp. 904–956, 2021. [5] Y. Noh, U. Lee, S. Han, P. Wang, D. Torres, J. Kim, and M. Gerla, “Dots: A propagation delay-aware opportunistic mac protocol for mobile underwater networks,” IEEE Transactions on Mobile Computing, vol. 13, no. 4, pp. 766–782, 2014. [6] J. Guo, S. Song, J. Liu, L. Wan, Y. Zhao, and G. Han, “An efficient medium access control scheme based on mc-cdma for mobile underwater acoustic networks,” IEEE Network, vol. 36, no. 3, pp. 167–173, 2022. [7] Y. Gou, T. Zhang, J. Liu, T. Yang, S. Song, and J.-H. Cui, “Achieving fair-effective communications and robustness in underwater acoustic sensor networks: A semi-cooperative approach,” IEEE Transactions on Mobile Computing, 2023. [8] M. Stojanovic and J. Preisig, “Underwater acoustic communication channels: Propagation models and statistical characterization,” IEEE communications magazine, vol. 47, no. 1, pp. 84–89, 2009. [9] K. Chen, M. Ma, E. Cheng, F. Yuan, and W. Su, “A survey on mac protocols for underwater wireless sensor networks,” IEEE Communications Surveys & Tutorials, vol. 16, no. 3, pp. 1433–1447, 2014. [10] P. A. Van Walree, “Propagation and scattering effects in underwater acoustic communication channels,” IEEE Journal of Oceanic Engineering, vol. 38, no. 4, pp. 614–631, 2013. [11] R. Petroccia and D. Spaccini, “Comparing the sunset and desert frameworks for in field experiments in underwater acoustic networks,” IEEE, 2013. [12] S. N. Le, Y. Zhu, Z. Peng, J.-H. Cui, and Z. Jiang, “Pmac: a real-world case study of underwater mac,” in Proceedings of the 8th International Conference on Underwater Networks & Systems, 2013, pp. 1–8. [13] A. A. Syed and J. Heidemann, “Time synchronization for high latency acoustic networks,” in INFOCOM 2006. 25th IEEE International Conference on Computer Communications, Joint Conference of the IEEE Computer and Communications Societies, 23-29 April 2006, Barcelona, Catalunya, Spain, 2006. [14] A.-R. Cho and Y. Choi, “Pta-sync: Packet-train-aided time synchronization for underwater acoustic applications,” Applied Sciences, vol. 13, no. 2, 2023. [15] M. Molins and M. Stojanovic, “Slotted fama: a mac protocol for underwater acoustic networks,” in OCEANS 2006-Asia Pacific. IEEE, 2006, pp. 1–7. [16] Z. Zhou, Z. Peng, J.-H. Cui, and Z. Jiang, “Handling triple hidden terminal problems for multichannel mac in long-delay underwater sensor networks,” IEEE Transactions on Mobile Computing, vol. 11, no. 1, pp. 139–154, 2011. [17] B. Peleato and M. Stojanovic, “Distance aware collision avoidance protocol for ad-hoc underwater acoustic sensor networks,” IEEE Communications letters, vol. 11, no. 12, pp. 1025–1027, 2007. [18] K. Kredo II, P. Djukic, and P. Mohapatra, “Stump: Exploiting position diversity in the staggered tdma underwater mac protocol,” in IEEE INFOCOM 2009. IEEE, 2009, pp. 2961–2965. [19] N. Morozs, P. Mitchell, and Y. V. Zakharov, “Tda-mac: Tdma without clock synchronization in underwater acoustic networks,” IEEE Access, vol. 6, pp. 1091–1108, 2017. [20] X. Guo, M. R. Frater, and M. J. Ryan, “Design of a propagation-delaytolerant mac protocol for underwater acoustic sensor networks,” IEEE journal of oceanic engineering, vol. 34, no. 2, pp. 170–180, 2009. [21] X. Zhuo, F. Qu, H. Yang, Y. Wei, Y. Wu, and J. Li, “Delay and queue aware adaptive scheduling-based mac protocol for underwater acoustic sensor networks,” IEEE Access, vol. 7, pp. 56 263–56 275, 2019. [22] S. Jiang, “State-of-the-art medium access control (mac) protocols for underwater acoustic networks: A survey based on a mac reference model,” IEEE communications surveys & tutorials, vol. 20, no. 1, pp. 96–131, 2017. [23] S. H. Park, P. D. Mitchell, and D. Grace, “Reinforcement learning based mac protocol (uw-aloha-q) for underwater acoustic sensor networks,” IEEE access, vol. 7, pp. 165 531–165 542, 2019. [24] W. Xie, X. Shen, L. Sun, C. Wang, Y. Yan, and H. Wang, “Dynamic optimization of slot management mac protocol for large-scale iout based

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

on pomdp,” IEEE Internet of Things Journal, vol. 12, no. 15, pp. 29 784– 29 796, 2025. [25] X. Ye, Y. Yu, and L. Fu, “Deep reinforcement learning based mac protocol for underwater acoustic networks,” IEEE Transactions on Mobile Computing, vol. 21, no. 5, pp. 1625–1638, 2022. [26] H. Du, X. Wang, W. Sun, and J. Zhang, “An adaptive mac protocol for underwater acoustic networks based on deep reinforcement learning,” in 2024 6th International Conference on Communications, Information System and Computer Engineering (CISCE), 2024, pp. 1271–1275. [27] X. Geng and Y. R. Zheng, “Exploiting propagation delay in underwater acoustic communication networks via deep reinforcement learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 12, pp. 10 626–10 637, 2022. [28] T. Qiu, Z. Zhao, T. Zhang, C. Chen, and C. P. Chen, “Underwater internet of things in smart ocean: System architecture and open issues,” IEEE transactions on industrial informatics, vol. 16, no. 7, pp. 4297–4307, 2019. [29] C. M. Gussen, P. S. Diniz, M. L. Campos, W. A. Martins, F. M. Costa, and J. N. Gois, “A survey of underwater wireless communication technologies,” J. Commun. Inf. Sys, vol. 31, no. 1, pp. 242–255, 2016. [30] S. H. Park, P. D. Mitchell, and D. Grace, “Reinforcement learning based mac protocol (uw-aloha-qm) for mobile underwater acoustic sensor networks,” IEEE Access, vol. 9, pp. 5906–5919, 2020. [31] S. Tomovic, “Dr-aloha-q: A q-learning-based adaptive mac protocol for underwater acoustic sensor networks,” Sensors, vol. 23, p. 4474, 05 2023. [32] I. B. Alhassan and P. D. Mitchell, “Packet flow based reinforcement learning mac protocol for underwater acoustic sensor networks,” Sensors, vol. 21, no. 7, 2021. [33] J. Shi, K. Tian, and J. Zhang, “Delay-fluctuation-resistant underwater acoustic network access method based on deep reinforcement learning,” Sensors, vol. 25, no. 21, p. 6673, 2025. [34] J. Jiang, Y. Dong, G. Han, and G. Su, “Underwater acoustic mac protocol for multi-objective optimization based on multi-agent reinforcement learning,” Drones, vol. 9, no. 2, p. 123, 2025. [35] J. Huang, X. Ye, Y. Wang, and L. Fu, “Leveraging propagation delays: A delay-aware multi-agent reinforcement learning mac protocol for underwater acoustic networks,” IEEE Internet of Things Journal, pp. 1–1, 2025. [36] C.-C. Hsu, K.-F. Lai, C.-F. Chou, and K.-J. Lin, “St-mac: Spatialtemporal mac scheduling for underwater sensor networks,” in IEEE INFOCOM 2009. IEEE, 2009, pp. 1827–1835. [37] J. Mao, S. Chen, Y. Liu, J. Yu, and Y. Xu, “Lt-mac: A location-based tdma mac protocol for small-scale underwater sensor networks,” in 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER). IEEE, 2015, pp. 1275– 1280. [38] Z. Zhang, W. Shi, Q. Niu, Y. Guo, J. Wang, and H. Luo, “A load-based hybrid mac protocol for underwater wireless sensor networks,” IEEE Access, vol. 7, pp. 104 542–104 552, 2019. [39] Y. Su, X. Liu, G. Han, and X. Fu, “A traffic load-aware ofdma-based mac protocol for distributed underwater acoustic sensor networks,” IEEE Transactions on Vehicular Technology, vol. 70, no. 10, pp. 10 501– 10 513, 2021. [40] R. L. Burden and J. D. Faires, Numerical Analysis, 9th ed. Boston, MA: Brooks/Cole, Cengage Learning, 2010. [41] S. Song, B. Huangfu, J. Guo, J. Liu, J. Cui, and X. Shen, “A digital twinbased intelligent network architecture for underwater acoustic sensor networks,” IEEE Trans. Mob. Comput., vol. 24, no. 9, pp. 8196–8213, 2025. [42] J. Guo, S. Song, H. Chen, B. Huangfu, J. Liu, and J. Cui, “Aquasim fourth generation: Toward general and intelligent simulation for underwater acoustic networks,” IEEE Internet Things J., vol. 12, no. 15, pp. 30 203–30 214, 2025. [43] P. Casari, F. Campagnaro, E. Dubrovinskaya, R. Francescon, A. Dagan, S. Dahan, M. Zorzi, and R. Diamant, “Asuna: A topology data set for underwater network emulation,” IEEE Journal of Oceanic Engineering, vol. 46, no. 1, pp. 307–318, 2020. [44] L. Pu, Y. Luo, H. Mo, S. Le, Z. Peng, J.-H. Cui, and Z. Jiang, “Comparing underwater mac protocols in real sea experiments,” Computer Communications, vol. 56, pp. 47–59, 2015. [45] N. Chirdchoo, W.-S. Soh, and K. C. Chua, “Aloha-based mac protocols with collision avoidance for underwater acoustic networks,” in IEEE INFOCOM 2007-26th IEEE International Conference on Computer Communications. IEEE, 2007, pp. 2271–2275. [46] Y. Weng, T. Matsuda, Y. Sekimori, J. Pajarinen, J. Peters, and T. Maki, “Time synchronization scheme of underwater platforms using wireless

17

acoustic and optical communication,” in 2022 IEEE/OES Autonomous Underwater Vehicles Symposium (AUV), 2022, pp. 1–6.

Bingwen Huangfu received the B.E. degree in computer science and technology from Jilin University, Changchun, China, in 2022. He is currently working toward the PhD degree at the College of Computer science and technology at Jilin University, Changchun, China. His current research interests include network architecture, resource allocation, and machine learning for underwater acoustic networks.

Jiani Guo received the BS degree (2016) in computer science and technology from Beijing Jiaotong University, Beijing, China, received PhD degree (2024) in Jilin University, Changchun, China. She is currently a Postdoctoral Researcher with the Department of Computer science and technology, Jilin University. Her current research interests include MAC protocols design and performance analysis for underwater acoustic networks.

Shanshan Song (Member, IEEE) received the BS degree (2011) and MS degree (2014) in computer science and technology from Jilin University, China, received PhD degree (2018) in Management science and engineering from Jilin University, China. She was a Post-Doctoral Researcher with the Department of Computer science and technology, Jilin University, Changchun, China. She is currently an associate professor with the Department of Computer science and technology, Jilin University. Her major research focuses on underwater data collection, localization and navigation and machine learning. She serves as the WUWNet’ 2023 Publication chair.

Nan Sun received the B.E. degree in software engineering from Yanshan University, Qinhuangdao, China, in 2023. He is currently working toward the M.S. degree with the College of Software Engineering, Jilin University, Changchun, China. His current research involves medium access control protocols for underwater acoustic networks.

Jun Liu (Member, IEEE) received the BS degree (2002) in computer science from Wuhan University, China, the PhD degree (2013) in Computer Science and Engineering from University of Connecticut, USA. Currently, he is a professor of the School of Electronic and Information Engineering at Beihang University, Beijing, China, also a part-time professor of the Robotics Research Center, Peng Cheng Laboratory, Shenzhen, China. His major research focuses on underwater acoustic networking, time synchronization, localization, network deployment, and also interested in operating system, cross layer design. He is a member of the IEEE Computer Society.

Related documents

• An Asynchronous Triggered MAC Protocol for Underwater Acoustic Networks#610040
• An Asynchronous Triggered MAC Protocol for Underwater Acoustic Networks#608978
• From Localization to Discovery: Bayesian Ranking of Electromagnetic Counterparts to Gravitational-Wave Events#986165
• Equivariant Filter Design for Acoustic and Depth Aided Inertial Navigation Systems#1014330
• Wave Breaking and Structure-Preserving Fully Discrete Method for Stochastic Camassa--Holm Equation with Additive Noise#1116459