IEEE XXX, VOL. XX, NO. XX, MONTH 202X
1
EvoOMG: An Evolution-Oriented Multi-Agent Guidance Framework for Heterogeneous Legacy-and-MLO Wi-Fi Networks
arXiv:2607.07045v1 [cs.NI] 8 Jul 2026
Junjie Wu, Lingjian Zhou, Zerui Shao, Yi Zou, Tianrui Li Senior Member, IEEE, Yi Zhang Senior Member, IEEE, Ziyuan Yang Member, IEEE
Abstract—The gradual deployment of Wi-Fi 7/8 multi-link operation (MLO) will lead to long-term coexistence between legacy non-MLO stations (STAs) and MLO-capable STAs in WLANs. This mixed deployment makes throughput optimization challenging because legacy STAs follow single-link contention and transmission, whereas MLO-capable STAs can exploit multiple links with richer access opportunities. Existing learning-based methods usually treat such networks as homogeneous systems and directly map the current observation to a complete MAC action, which cannot faithfully represent both legacy singlelink and MLO multi-link behaviors. To address this issue, we propose EvoOMG, an evolution-oriented multi-agent guidance framework for heterogeneous legacy-and-MLO Wi-Fi networks. EvoOMG reformulates throughput optimization as a standardconstrained staged multi-agent decision problem. Each agent encodes recent channel, queue, contention, and transmission histories, first generates contention guidance, and then produces aggregation guidance conditioned on the preceding access stage and standard-specific feasibility constraints. This autoregressive design follows the Wi-Fi MAC order of “contention before transmission” while preserving distinct protocol behaviors of legacy and MLO-capable STAs. NS-3 evaluations show that EvoOMG improves scheduled goodput, convergence stability, and MLO link utilization over static enhanced distributed channel access (EDCA), one-step MADDPG, and independent-learning baselines, achieving substantial performance gains in representative mixed-standard scenarios. Index Terms—Wi-Fi 7/8, legacy-and-MLO coexistence, multilink operation, multi-agent guidance, autoregressive control, heterogeneous WLAN optimization.
I. I NTRODUCTION IRELESS local area networks (WLANs) are entering a long coexistence period in which non-multi-link operation stations (STAs) and emerging multi-link operation (MLO)-capable Wi-Fi 7/8 stations operate in the same deployment. This coexistence makes Wi-Fi throughput optimization
W
J. Wu, Z. Shao, and T, Li are with the School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756, China. (e-mail: [email protected], [email protected], [email protected]). L. Zhou and Y. Zou are with the School of Software, Nanchang Hangkong University, Nanchang 330063, China. (e-mail: [email protected], [email protected]). Y. Zhang is with the School of Cyber Science and Engineering, Sichuan University, Chengdu 610207, China. (e-mail: [email protected]). Z. Yang is with the Nanyang Technological University, Singapore 639798, Singapore. (e-mail: [email protected]). This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
substantially more challenging than in conventional homogeneous WLANs [1], [2]. Since different device types exhibit distinct transmission behaviors and channel-access opportunities, the optimization must account for both device heterogeneity and their asymmetric access to wireless resources [3], [4]. In homogeneous WLAN scenarios, STAs usually follow similar single-link access mechanisms and expose homogeneous medium access control (MAC) configuration spaces. Under this setting, existing methods can effectively optimize MAC-layer contention and scheduling policies, since the underlying optimization reduces to a relatively stationary resource allocation problem and is not affected by standard-induced action heterogeneity or multi-link feasibility constraints [5], [6]. However, the coexistence of legacy STAs (non-MLO STAs) and MLO-capable STAs invalidates the assumptions underlying prior studies. Legacy STAs usually operate on a single available link, whereas MLO-capable STAs can use multiple links and may follow different simultaneous transmit-and-receive (STR) or non-simultaneous transmit-andreceive (NSTR) constraints and traffic identifier (TID)-tolink mapping mapping rules [7]–[10]. As a result, different STAs may have different feasible actions and channel-access behaviors. Therefore, they cannot be modeled in the same way as homogeneous STAs in conventional WLANs. This heterogeneity makes previous works difficult to extend in this challenging and practical scenario. Specifically, a policy that outputs all MAC-layer actions at once assumes that all STAs can be controlled in the same way. This is not true in a mixed legacy-and-MLO network, the same action may have different meanings for different device types. Moreover, WiFi MAC control follows a natural order. Contention-window decisions first affect whether a STA can access the channel, while aggregation and link-scheduling decisions matter only after a transmission opportunity is obtained [11]–[13]. Therefore, heterogeneous Wi-Fi optimization should model both the differences among STA types and the ordered stages of MAC control, instead of treating all MAC actions as one flat decision. Based on this observation, we revisit Wi-Fi optimization in mixed-standard environments from the perspective of closing the heterogeneity-induced performance gap via behavioral decomposition . The fundamental challenge arises from the coexistence of heterogeneous STA capabilities: MLO-capable STAs [14], [15] can exploit multi-link transmission, STR/NSTR
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
(a) Flat one-shot control under mixed standards.
2
(b) Standard-aware autoregressive staged control.
Fig. 1: Motivation of EvoOMG. Conventional one-shot control treats legacy and MLO devices with a flat action space, whereas EvoOMG follows a standard-aware staged process that generates contention guidance before aggregation decisions. switching, and TID-to-link mapping, whereas legacy STAs are constrained to single-link contention-based access. This discrepancy induces a structural gap between the optimal performance envelope enabled by MLO and the achievable performance of learning-based MAC policies. We argue that this gap is not merely a function approximation issue, but fundamentally stems from a behavioral mismatch: existing methods attempt to directly learn a flat joint policy over contention, link scheduling, and aggregation decisions, without explicitly respecting either the standard-induced feasibility constraints or the inherent temporal ordering of the Wi-Fi MAC protocol. As a result, the high-dimensional heterogeneous action space is treated as a single-step mapping, which obscures the sequential dependency between contention, access, and transmission [16]. To bridge this heterogeneity gap, we rethink the problem as a structured sequential decision process [17], [18], where the original high-dimensional joint action is decomposed into a series of conditional sub-decisions aligned with the MAC execution pipeline. Specifically, we reformulate the optimization problem from a flat policy learning task into a heterogeneityaware staged control problem, in which contention control, link-level scheduling, and aggregation configuration are generated sequentially and conditioned on both device standard and prior decisions. This reformulation transforms the original coupled optimization into a structured decision hierarchy that explicitly aligns with protocol causality. This perspective leads to a central question: How can we reformulate the high-dimensional, standardconstrained joint action optimization problem into a structured sequential decision process to bridge the performance gap, by explicitly decomposing MAC-layer control into conditional, causality-aligned sub-decisions? To answer this question, we propose EvoOMG, an Evolution-Oriented Multi-agent Guidance framework for heterogeneous legacy-and-MLO Wi-Fi networks. As shown in Fig. 1, EvoOMG differs from flat one-shot optimizers by encoding recent protocol histories, including channel, queue, contention, and transmission states, and then generating MAC guidance in a staged manner: contention guidance is produced first, while aggregation and link-related guidance are
generated conditioned on the preceding access stage and standard-specific feasibility constraints. This design allows legacy STAs and MLO-capable STAs to be optimized within a unified MADRL framework while preserving their distinct protocol behaviors. EvoOMG follows the centralized-training and decentralized-execution paradigm [19], and can optionally incorporate federated aggregation for distributed deployment under heterogeneous Wi-Fi domains [20]–[22]. We implement EvoOMG in an NS-3-based heterogeneous Wi-Fi environment that exposes protocol-level observations such as SNR, PER, idle-time ratio, previous CW settings, aggregation decisions, and per-device throughput, enabling evaluation under realistic mixed legacy/MLO deployments with STR/NSTR constraints [7], [14]. The main contributions are summarized as follows: • We identify standard-induced action heterogeneity and MAC-stage temporality as two key challenges in mixed legacy-and-MLO WLANs, and formulate the problem as a standard-constrained staged multi-agent decision process. • We propose EvoOMG, a standard-aware autoregressive MADRL framework that encodes recent protocol histories and generates MAC actions following the contention-toaggregation order. This design enables unified yet deviceaware control for legacy single-link STAs and MLO multilink STAs. • We conduct NS-3-based evaluations under legacy/MLO deployments and compare EvoOMG with static EDCA, one-step MADDPG, and independent-learning baselines. The results show that EvoOMG improves scheduled goodput, convergence stability, and MLO link utilization while maintaining standard-compliant action generation. II. S YSTEM M ODEL AND P ROBLEM F ORMULATION We consider an infrastructure-based heterogeneous WLAN implemented on the NS-3 Wi-Fi stack, as illustrated in Fig. 2. One access point (AP) serves a set of STAs N = Nℓ ∪ Nm , where Nℓ denotes legacy STAs, i.e., non-MLO STAs in the considered mixed deployment, and Nm denotes MLO-capable STAs. Legacy STAs mainly operate with single-link access, whereas MLO-capable STAs can exploit multiple candidate
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
3
adaptation and retransmissions, and the idle-time ratio ιi,k (t) captures MAC-layer contention and residual channel opportunity. For MLO-capable STAs, these statistics are collected separately over all candidate links k ∈ Kj , enabling the controller to distinguish congested high-rate links from more reliable low-rate links. In contrast, legacy STAs do not have link-selection flexibility, and their performance is determined by the single-link channel condition, contention process, and TID/AC-specific queue behavior. This difference motivates the later STA–AC–link goodput model, where legacy goodput is computed over one link while MLO goodput is aggregated over active feasible links. Fig. 2: System model of the heterogeneous legacy-and-MLO Wi-Fi network. Legacy STAs use single-link access, whereas MLO-capable STAs schedule TID/AC traffic over feasible links under MLO constraints. links, such as 2.4 GHz, 5 GHz, and 6 GHz links [23], [24]. Each STA i is associated with a standard indicator τi , where τi = 0 represents a legacy single-link STA and τi = 1 represents an MLO-capable STA. For a legacy STA i ∈ Nℓ , the feasible link set is a singleton, denoted by Ki = {0}. For an MLO-capable STA j ∈ Nm , the feasible link set is Kj = {1, . . . , Kj }, where Kj ≥ 2. The traffic is further differentiated by TIDs, which are mapped to access categories (ACs). Let A = {VO, VI, BE, BK} denote the set of ACs, corresponding to voice, video, best-effort, and background traffic [7]. Different ACs may have different contention parameters, queue priorities, and throughput weights. Therefore, the considered heterogeneous Wi-Fi system is modeled at the STA–AC–link level. A. Channel, Mobility, and Link Model The considered heterogeneous WLAN operates in an indoor environment. The AP is fixed at position p0 , while STAs move within a bounded service area according to the NS3 RandomWalk2dMobilityModel. Let pi (t) denote the position of STA i at decision epoch t, and let di (t) = ∥pi (t) − p0 ∥ be the AP–STA distance. The large-scale path loss is modeled as [25]: di (t) P Li (t) = P L(d0 ) + 10η log10 + Xσ , (1) d0 where d0 is the reference distance, η is the path-loss exponent, and Xσ denotes shadowing. For a legacy STA i ∈ Nℓ , the channel condition is defined on its single feasible link k = 0. For an MLO-capable STA j ∈ Nm , the channel condition is link-specific because candidate links may operate on different frequency bands and experience different propagation, interference, and contention conditions [26], [27]. For each feasible STA–link pair (i, k), the NS-3 PHY/MAC stack provides the SNR, packet error ratio (PER), and channel idle-time ratio, denoted by γi,k (t), ei,k (t), and ιi,k (t), respectively. The SNR γi,k (t) characterizes PHY-layer channel quality, the PER ei,k (t) reflects transmission reliability after rate
B. TID-Aware Traffic Scheduling The heterogeneous WLAN carries traffic with different TIDs. Following the Wi-Fi QoS mechanism, TIDs are mapped into four ACs, denoted by A = {VO, VI, BE, BK}, corresponding to voice, video, best-effort, and background traffic, respectively. Let Ua denote the set of TIDs belonging to AC a. Recent studies on real-time industrial IoT and wireless timesensitive networking also show that traffic differentiation and predictable latency are becoming increasingly important for Wi-Fi-based low-latency services [28]. The considered system is modeled at the STA–AC–link level. For a legacy STA i ∈ Nℓ , all AC queues are served over the single feasible link k = 0. For an MLO-capable STA j ∈ Nm , AC traffic can be scheduled over a candidate link set Kj . Let ρi,a,k (t) ∈ {0, 1} denote whether AC a traffic of STA i is scheduled on link k at epoch t. The legacy single-link scheduling constraint is ρi,a,0 (t) = 1, i ∈ Nℓ , a ∈ A. For an MLO-capable STA j ∈ Nm , AC traffic can be assigned to one or multiple feasible links according to the MLO capability, TID-to-link mapping, and link availability: ρj,a,k (t) ∈ {0, 1},
j ∈ Nm , a ∈ A, k ∈ Kj .
(2)
If a certain TID/AC is not allowed to use link k by the TIDto-link mapping or standard configuration, then ρj,a,k (t) = 0. C. MAC Feasibility under Legacy and MLO The access behavior of each AC is governed by EDCAlike contention and aggregation procedures. For STA i, AC a, and link k, let ci,a,k (t) denote the contention-windowrelated configuration, and let li,a,k (t) denote the aggregation frame length, e.g., the A-MPDU aggregation level. These two variables jointly affect the probability of obtaining a transmission opportunity and the amount of useful payload delivered once the channel is accessed. The feasible MAC configurations are standard-, AC-, and link-dependent: ci,a,k (t) ∈ Wi,a,k ,
li,a,k (t) ∈ Li,a,k .
(3)
where Wi,a,k and Li,a,k denote the feasible contentionwindow and A-MPDU aggregation-length sets of STA i for AC a on link k, respectively. For legacy STAs, the feasible sets are defined only on the single link k = 0. For MLO-capable STAs, the feasible sets are link-specific because different links may have different channel quality, contention intensity, and MLO availability.
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
4
To model MLO link activation, let zj,k (t) ∈ {0, 1} indicate whether link k of MLO-capable STA j is active at epoch t. The AC-level scheduling indicator must satisfy ρj,a,k (t) ≤ zj,k (t),
j ∈ Nm , a ∈ A, k ∈ Kj .
(4)
Under STR, multiple links can be active concurrently if they are feasible, while practical performance still depends on cross-link interference, traffic steering, and link-access constraints [29]. Under NSTR, conflicting links cannot be active at the same time. Let EjNSTR denote the set of conflicting link pairs of MLO-capable STA j. The NSTR constraint is: j ∈ Nm , (k, k ′ ) ∈ EjNSTR .
zj,k (t) + zj,k′ (t) ≤ 1,
D. Scheduled Goodput Model Since the system is implemented on the NS-3 Wi-Fi stack, packet transmission is not modeled by an ideal Shannonrate expression. Instead, the delivered payload is generated by the protocol-level process, including EDCA contention, backoff, PHY rate control, frame aggregation, retransmission, acknowledgement or block acknowledgement, and MLO link scheduling. Therefore, we use scheduled goodput as the throughput metric. succ Let Bi,a,k (t) denote the successfully delivered applicationlayer payload bits of STA i with AC a over link k during interval ∆t. This quantity excludes MAC/PHY headers, control signaling, failed retransmissions, and acknowledgement overhead, while such signaling and retransmission costs are still reflected in the consumed channel time in NS-3. The STAAC link goodput is defined as: Gi,a,k (t) =
i∈Nℓ a∈A
Gmlo (t) =
(9)
wa Gj,a (t),
where wa denotes the AC-specific service weight. In general, wVO and wVI can be set larger than wBE and wBK to reflect the higher priority of delay-sensitive traffic. The total scheduled goodput of the heterogeneous Wi-Fi system is: Gsys (t) = Gleg (t) + Gmlo (t).
(10)
This goodput formulation captures TID/AC-level service differentiation, legacy single-link scheduling, MLO multi-link traffic distribution, and the impact of CW and A-MPDU configurations on protocol-level payload delivery. E. Problem Formulation The objective is to optimize the long-term scheduled goodput of the heterogeneous Wi-Fi system under standard-specific, AC-specific, and MLO-mode-specific feasibility constraints. Let π denote the control policy that determines AC-level MAC configurations and link-scheduling decisions over time. Based on the scheduled goodput model in Eq. (10), the instantaneous utility is defined as: U (t) = Gsys (t),
(11)
where Gsys (t) is the total scheduled goodput of the heterogeneous legacy-and-MLO Wi-Fi system. This utility characterizes the performance under mixed legacy-and-MLO constraints and implicitly reflects the gap between feasible learned policies and the MLO-enabled performance envelope. The throughput-oriented optimization problem is formulated as: " T # X P : max Eπ U (t) (12a) π
s.t. .
(mlo)
X X j∈Nm a∈A
(5)
This feasibility model captures the essential access difference between legacy and MLO-capable STAs. Legacy STAs perform AC-specific contention and aggregation on one link, whereas MLO-capable STAs must jointly consider AC-to-link scheduling, per-link contention behavior, aggregation length, and STR or NSTR-compliant link activation.
succ Bi,a,k (t)
condition, and the NS-3 process. The class-wise goodput of legacy and MLO-capable STAs is computed as: X X (leg) Gleg (t) = wa Gi,a (t),
t=1
ρi,a,0 (t) = 1,
∀i ∈ Nℓ , a ∈ A,
(12b)
(6)
ρj,a,k (t) ∈ {0, 1},
For a legacy STA i ∈ Nℓ , all AC traffic is served on the single link. Thus, the AC-level goodput is:
∀j ∈ Nm , a ∈ A, k ∈ Kj , (12c)
ρj,a,k (t) ≤ zj,k (t),
∀j ∈ Nm , a ∈ A, k ∈ Kj , (12d)
(leg)
∆t
Gi,a (t) = Gi,a,0 (t).
(7)
For an MLO-capable STA j ∈ Nm , the AC-level goodput is aggregated over the scheduled active links: X (mlo) Gj,a (t) = ρj,a,k (t)Gj,a,k (t). (8)
zj,k (t) + zj,k′ (t) ≤ 1, ci,a,k (t) ∈ Wi,a,k ,
∀i ∈ N , a ∈ A, k ∈ Ki , (12f)
li,a,k (t) ∈ Li,a,k ,
∀i ∈ N , a ∈ A, k ∈ Ki , (12g)
k∈Kj
The goodput of each AC is affected by both contention and aggregation, which is consistent with recent studies showing that traffic allocation, channel selection, and MAC parameter tuning jointly determine MLO performance [26]. Hence, Gi,a,k (t) is implicitly determined by the protocol-level interaction among qi,a (t), ci,a,k (t), li,a,k (t), ρi,a,k (t), the link
∀j ∈ Nm , (k, k ′ ) ∈ EjNSTR , (12e)
π ∈ Πstd ,
(12h)
where T denotes the finite optimization horizon, i.e., the number of decision epochs considered in one control period. Eπ [·] denotes the expectation over trajectories induced by policy π and the NS-3 protocol dynamics. Constraint (12b) enforces the single-link access behavior of legacy STAs, where
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
5
all AC traffic of a legacy STA is served on its only feasible link. Constraint (12c) defines the binary AC-to-link scheduling decision for MLO-capable STAs. Constraint (12d) couples AC-level scheduling with link activation, ensuring that AC traffic can be assigned to link k only when this link is active. Constraint (12e) imposes the NSTR feasibility requirement, where two conflicting links in EjNSTR cannot be activated simultaneously. Constraint (12f) restricts the CW-related action to the feasible set Wi,a,k determined by the device standard, AC type, and link condition. Constraint (12g) restricts the AMPDU aggregation-length action to the feasible set Li,a,k . Constraint (12h) further requires the policy to belong to the standard-compliant policy space Πstd , which excludes invalid MAC configurations and infeasible legacy/MLO actions. III. E VO OMG D ESIGN This section presents EvoOMG, an evolution-oriented multiagent guidance framework for heterogeneous legacy-andMLO Wi-Fi networks. Based on the STA–AC–link scheduled goodput formulation in Section II, EvoOMG learns distributed MAC guidance policies that jointly consider TID/AC service differentiation, legacy single-link access, MLO multi-link scheduling, and the protocol-level coupling between contention-window configuration and aggregationframe length. As shown in Fig. 3, EvoOMG integrates a history-aware state encoder, a Transformer-based autoregressive actor, centralized critics for CTDE training, and an optional federated aggregation module for coordinating heterogeneous Wi-Fi domains.
τi ∈ {0, 1} indicates legacy or MLO capability, and µi ∈ {SL, STR, NSTR} denotes the access mode. For legacy STAs, µi = SL and Ki = {0}. The local observation of STA i is oi (t) = {oi,a,k (t), ∀a ∈ A, k ∈ Ki } ∪ {di }. To capture temporal MAC dynamics, EvoOMG maintains a sliding history: Oi (t) = [oi (t − S + 1), . . . , oi (t)] ,
(15)
where S is the history length. The history-aware decision feature is obtained by: hi (t) = Enc (Oi (t), di ) ,
(16)
where Enc(·) denotes the Transformer-based history encoder. This history-aware encoding is aligned with recent AI-native Wi-Fi studies that emphasize environment-adaptive protocol control [30], [31]. 2) Action. The local action is defined at the STA–AC–link level: ai (t) = {ρi,a,k (t), zi,k (t), ci,a,k (t), li,a,k (t)}a∈A, k∈Ki . (17) Here ρi,a,k (t) is the AC-to-link scheduling indicator, zi,k (t) is the link-activation indicator, ci,a,k (t) is the CW-related action, and li,a,k (t) is the A-MPDU aggregation-length action. For a legacy STA i ∈ Nℓ , the action degenerates to the single-link case: (ℓ)
ai (t) = {ci,a,0 (t), li,a,0 (t)}a∈A ,
(18)
with ρi,a,0 (t) = 1 and zi,0 (t) = 1. For an MLO-capable STA j ∈ Nm , the action preserves the multi-link structure: (m)
(t) = {ρj,a,k (t), zj,k (t), cj,a,k (t), lj,a,k (t)}a∈A, k∈Kj . (19) The generated action is finally projected onto the standardcompliant feasible set. In the current NS-3 implementation, the learnable action vector consists of AC-specific CW configurations and the A-MPDU aggregation length. The AC-tolink scheduling variable ρi,a,k (t) and link-activation variable zi,k (t) are determined by the standard-compliant feasibility projection, TID-to-link mapping, and STR/NSTR constraints rather than being independently optimized as unconstrained neural-network outputs. 3) Objective and Reward. The optimization utility is the scheduled goodput of the heterogeneous Wi-Fi system: aj
A. Decentralized Partially Observable Markov Decision Process EvoOMG formulates heterogeneous Wi-Fi MAC guidance as a decentralized partially observable Markov decision process (Dec-POMDP): GD = (N , S, {Oi }i∈N , {Bi }i∈N , P, R, δ) ,
(13)
where N is the STA set, S is the global network state, Oi and Bi are the local observation and action spaces of STA i, P is the protocol-induced transition kernel, R is the training reward, and δ ∈ (0, 1) is the discount factor. 1) State. At decision epoch t, each STA observes its local STA–AC–link protocol state. For STA i, AC a, and feasible link k ∈ Ki , the local observation is defined as: oi,a,k (t) = [χi,k (t), qi,a (t), ψi,a,k (t − 1), Gi,a,k (t − 1)] , (14) where the link-state vector is χi,k (t) = [γi,k (t), ei,k (t), ιi,k (t)], and the previous MAC-action vector is ψi,a,k (t − 1) = [ρi,a,k (t − 1), zi,k (t − 1), ci,a,k (t − 1), li,a,k (t − 1)]. Here χi,k (t) contains the SNR, PER, and idle-time ratio of link k; qi,a (t) is the AC-level queue backlog; ψi,a,k (t − 1) records the previous scheduling, link activation, CW, and A-MPDU length; and Gi,a,k (t − 1) is the previous scheduled goodput. To distinguish different Wi-Fi standards, each STA is associated with a device descriptor di = [τi , µi ], where
U (t) = Gsys (t).
(20)
This utility is consistent with the problem formulation in Eq. (11). EvoOMG does not introduce additional explicit penalty terms for collision, delay, or packet dropping in the objective. These factors affect the delivered scheduled goodput through the NS-3 MAC/PHY process and are therefore reflected implicitly in Gsys (t). For stable neural-network training, the reward used by the autoregressive MADRL agent is a normalized version of the utility: e (t) = Norm(Gsys (t)) , R(t) = U
(21)
where Norm(·) denotes a normalization operator, such as min– max normalization or running-statistics normalization. This normalization only improves numerical stability and does not
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
6
Fig. 3: Overall architecture of EvoOMG. Each agent uses a transformer-based autoregressive actor to generate CW actions before A-MPDU length decisions. Centralized critics support CTDE-based multi-agent training, while optional federated aggregation enables distributed training across heterogeneous Wi-Fi domains. change the underlying optimization objective. The normalization is introduced solely for numerical stability in learning and does not alter the underlying objective of maximizing the heterogeneity-constrained system goodput. B. Autoregressive Staged Action Generator Different from prior learning-based MLO controllers that mainly focus on traffic steering or channel selection, EvoOMG further imposes the protocol order from contention guidance to aggregation guidance [29]. It factorizes the local policy into a contention stage and a transmission stage. For STA i, the factorized policy is written as: πi (ai (t) | Oi (t), di ) = πicw (ci (t) | Oi (t), di ) × πitr (ui (t) | ci (t), Oi (t), di ) ,
(22)
where ci (t) = {ci,a,k (t)}a,k is the contention-stage action and ui (t) = (ρi (t), zi (t), li (t)) is the transmission-stage action, with ρi (t) = {ρi,a,k (t)}a,k , zi (t) = {zi,k (t)}k , and li (t) = {li,a,k (t)}a,k . The first stage generates AC-link contention guidance: ĉi (t) = fθc (hi (t), di ) ,
(23)
where θc demotes contention head parameters. The contention guidance affects backoff behavior and channel access probability. The second stage generates aggregation guidance conditioned on the contention output: l̂i (t) = fθl (hi (t), di , ĉi (t)) ,
(24)
where θl demotes aggregation head parameters. The associated scheduling and link-activation variables are then obtained through the standard-compliant feasibility projection according to device capability, TID-to-link mapping, and STR/NSTR constraints. This staged factorization explicitly aligns with the heterogeneity-aware causality structure of Wi-Fi MAC operations, where contention decisions determine access feasibility and subsequent aggregation decisions depend on both access outcome and device-specific constraints. The contention-stage decision determines access opportunity and collision risk, while the transmission-stage decision determines how much payload is aggregated and how feasible links are used after the access condition is formed. To ensure protocol feasibility, let âi (t) = (ĉi (t), l̂i (t)) denote the raw learnable actor output. The final MAC action is obtained by ai (t) = PΩi (âi (t)), where Ωi denotes the feasible action set determined by the device standard, AC type, TID-to-link mapping, and STR/NSTR mode. For legacy STAs, invalid MLO link actions are masked out and only single-link CW and aggregation actions are retained. For MLO-capable STAs, AC-to-link scheduling and link activation are constrained by MLO feasibility and NSTR conflicting-link constraints. C. Standard-Aware Multi-Agent Policy Heads To address the heterogeneous action spaces of legacy and MLO-capable STAs, EvoOMG uses standard-aware policy heads. The sequence encoder and early actor layers are shared
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
7
across agents to capture common patterns, while the output heads are adapted according to the device descriptor di . For a legacy STA, the policy head only outputs single-link AC-specific CW and aggregation guidance. Its feasible action (leg) space is denoted by Ωi , which enforces ρi,a,0 (t) = 1 and zi,0 (t) = 1. For an MLO-capable STA, the feasible action (mlo) space is denoted by Ωi , which further depends on STR or NSTR operation and determines valid AC-to-link scheduling and link activation after projection. The final policy output is represented as: P (leg) (âi (t)) , i ∈ Nℓ , Ωi (25) ai (t) = P (mlo) (âi (t)) , i ∈ Nm , Ω i
where âi (t) is the raw actor output, PΩ (·) denotes the (leg) projection operator onto feasible action set Ω, Ωi and (mlo) Ωi denote the feasible action sets of legacy and MLOcapable STAs, respectively, and the superscripts (leg) and (mlo) represent the legacy single-link case and the MLOcapable multi-link case. This design allows EvoOMG to share useful protocol knowledge across heterogeneous STAs while avoiding invalid actions caused by mixing legacy single-link and MLO multi-link behaviors in one flat output space. D. Centralized Critic and Federated Training Procedure EvoOMG follows the centralized-training and decentralized-execution (CTDE) paradigm within each local Wi-Fi domain. During local training, the centralized critic observes the joint histories and joint actions of all STA agents in the local heterogeneous WLAN to estimate the global action value. During execution, each STA agent generates its MAC action using only its local observation history and device descriptor. To support distributed deployment across heterogeneous Wi-Fi domains, EvoOMG can further adopt an optional federated-assisted training procedure. Let v index a local training client, e.g., an AP-side Wi-Fi domain or an NS-3 training environment, and let N v denote the STA set in client v. The joint history and joint action in client v are defined as: O v (t) = {Oiv (t)}i∈N v ,
av (t) = {avi (t)}i∈N v ,
(26)
where Oiv (t) and avi (t) are the local history and action of STA i in client v, respectively. The local centralized critic is written as Qϕv (O v (t), av (t)), where ϕv denotes the critic parameters of client v. The local replay buffer of client v stores sequence transitions: Dv = {(O v (t), av (t), Rv (t), O v (t + 1))} .
(27)
Different from one-step replay, Dv stores history tensors with the form [batch, seq_len, state_dim], so that local temporal protocol continuity is preserved. For each sampled transition, the temporal-difference target is computed as: y v (t) = Rv (t) + γQϕ− O v (t + 1), av,− (t + 1) , (28) v
where γ is the discount factor, Qϕ− is the local target critic, v and av,− (t + 1) is generated by the local target actors. The local critic loss is: h i 2 LvQ (ϕv ) = EDv (Qϕv (O v (t), av (t)) − y v (t)) . (29) The actor parameters of STA i in client v are updated by deterministic policy gradient: ∇θiv Jiv = EDv ∇θiv πθiv (Oiv (t), θiv )∇avi Qϕv (O v (t), av (t)) , (30) where θiv denotes the actor parameters of STA i in client v, and θiv is its device descriptor. To support distributed deployment across heterogeneous WiFi domains, EvoOMG can optionally adopt a lightweight fed(r) erated aggregation mechanism. Let wv denote the trainable model parameters of client v after local training in communication round r, including the sequence encoder, staged actor, standard-aware policy heads, and critic parameters. The server aggregates the uploaded model parameters as: X X w(r+1) = αv wv(r+1) , αv = 1, (31) v∈M
v∈M
where M denotes the set of participating local Wi-Fi clients and αv is the aggregation weight. Each client then downloads w(r+1) to initialize the next local training round. Since different clients may contain different legacy/MLO ratios, traffic mixtures, and STR/NSTR modes, the standard-aware policy heads and feasibility masks are retained during local training to reduce cross-standard drift. In this paper, federated aggregation is used as an optional deployment mechanism rather than the main algorithmic contribution. E. Theoretical Analysis This section analyzes the performance gap induced by heterogeneous MAC constraints in mixed legacy-and-MLO Wi-Fi networks. Let the local decision state of STA i be denoted by xi (t) = (Oi (t), di ) , where Oi (t) is the local protocol history and di is the device descriptor. For STA i, define the contention-stage action as ci (t) = {ci,a,k (t)}a,k and the transmission-stage action as ui (t) = (ρi (t), zi (t), li (t)), where ρi (t) = {ρi,a,k (t)}a,k , zi (t) = {zi,k (t)}k , and li (t) = {li,a,k (t)}a,k . The scheduled goodput contribution of STA i can be abstracted as a stage-coupled function: Gi (t) = Ai (ci (t); xi (t)) Bi (ui (t); ci (t), xi (t)) ,
(32)
where Ai (·) denotes the access-efficiency term induced by AClevel contention and channel acquisition, and Bi (·) denotes the payload-efficiency term induced by AC-to-link scheduling, link activation, A-MPDU length, and STR/NSTR feasibility constraints. Consistent with the scheduled goodput definition in Eq. (10), the discounted goodput objective is defined as: "∞ # X t J(π) = Eπ δ Gsys (t) , δ ∈ (0, 1), (33) t=0
where δ is the discount factor. This objective implicitly measures the gap between the achievable performance of
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
8
the learned policy and the optimal STR-enabled performance envelope under standard heterogeneity.
the burden of learning asymmetric MAC dependencies implicitly from one flat action vector.
Theorem 1 (STR/NSTR Feasible-Set Envelope). Assume that all scheduled goodput contributions are nonnegative and that the NSTR feasible link-activation set is a subset of the STR ⋆ ⋆ feasible link-activation set. Let JNSTR and JSTR denote the optimal discounted scheduled-goodput objectives under NSTRconstrained and STR-enabled MLO operation, respectively. Then ⋆ ⋆ JNSTR ≤ JSTR . (34)
Assumption 1 (Lipschitz Continuity). The optimal actionvalue function is Lipschitz continuous with respect to the contention action, transmission action, and decision state. Namely, there exist constants Lc , Lu , and Lx such that, for any feasible contention-stage actions c and c′ , transmissionstage actions u and u′ , and decision states x and x′ ,
Proof. The NSTR mode restricts simultaneous activation of conflicting links, while STR allows a superset of feasible linkactivation patterns. Since each scheduled goodput contribution is nonnegative, enlarging the feasible link-activation set cannot decrease the optimal achievable discounted goodput. The result follows from feasible-set inclusion.
|Q⋆ (x, c, u) − Q⋆ (x, c′ , u)| ≤ Lc ∥c − c′ ∥,
(41)
|Q⋆ (x, c, u) − Q⋆ (x, c, u′ )| ≤ Lu ∥u − u′ ∥,
(42)
|Q⋆ (x, c, u) − Q⋆ (x′ , c, u)| ≤ Lx ∥x − x′ ∥.
(43)
and
Theorem 2 (Staged Optimal Factorization). Let Q⋆ (x, c, u) denote the optimal action-value function, where x is the local decision state, c is the contention-stage action, and u is the transmission-stage action. Then there exists an optimal staged policy of the following form:
Assumption 2 (Stage-wise Approximation Errors). Let x denote the ideal local decision state, and let x̂ denote the encoded decision-state representation generated by the history encoder. Given x̂, EvoOMG produces the contention-stage action ĉ(x̂) and the transmission-stage action û(x̂, ĉ). Their expected approximation errors relative to the optimal staged actions satisfy
π ⋆ (c, u | x) = πc⋆ (c | x)πu⋆ (u | c, x).
E[∥ĉ(x̂) − c⋆ (x)∥] ≤ εc ,
(44)
E[∥û(x̂, ĉ) − u⋆ (x, c⋆ (x))∥] ≤ εu ,
(45)
(35)
Proof. For any fixed decision state x, the optimal joint action solves (c⋆ (x), u⋆ (x)) ∈ arg max Q⋆ (x, c, u). (36)
and the decision-state representation mismatch satisfies
c,u
E[∥x̂ − x∥] ≤ εx .
(46)
The joint maximization in Eq. (36) can be equivalently written as the following nested maximization: h i max Q⋆ (x, c, u) = max max Q⋆ (x, c, u) . (37)
Theorem 3 (Performance Gap Bound). Under Assumptions 1– 2, let δ ∈ (0, 1) denote the discount factor and define
Therefore, one optimal contention-stage action can be selected as h i c⋆ (x) ∈ arg max max Q⋆ (x, c, u) . (38)
Then the performance gap between the optimal policy and EvoOMG is bounded as follows:
c,u
c
c
εQ = Lc εc + Lu εu + Lx εx .
u
u
For a given contention-stage action c, the corresponding optimal transmission-stage action is ⋆
⋆
u (x, c) ∈ arg max Q (x, c, u). u
(39)
Thus, the optimal joint decision can be represented by first selecting c⋆ (x) and then selecting u⋆ (x, c) conditioned on the contention-stage action and the decision state. Equivalently, the optimal policy can be written in staged form by defining πc⋆ (c | x) over the optimal contentionstage actions and πu⋆ (u | c, x) over the corresponding optimal transmission-stage actions. This gives π ⋆ (c, u | x) = πc⋆ (c | x)πu⋆ (u | c, x),
(40)
which proves Eq. (35). The above theorem shows that the staged policy adopted by EvoOMG is not merely a heuristic decomposition. Under the exact action-value function, contention-first and transmissionconditioned generation is lossless relative to joint action optimization. In practical learning, this factorization also reduces
J(π ⋆ ) − J(πEvoOMG ) ≤
2δ εQ . (1 − δ)2
(47)
(48)
Equivalently, J(π ⋆ ) − J(πEvoOMG ) ≤
2δ (Lc εc + Lu εu + Lx εx ) . (1 − δ)2 (49)
Proof. For a fixed decision state x, let c⋆ (x) and u⋆ (x, c⋆ (x)) denote the optimal staged actions. EvoOMG observes the encoded decision-state representation x̂ and produces ĉ(x̂) and û(x̂, ĉ). We define the following quantity ∆Q to characterize the value-function discrepancy induced by representation and policy approximation: ∆Q = |Q⋆ (x, c⋆ (x), u⋆ (x, c⋆ (x))) − Q⋆ (x̂, ĉ(x̂), û(x̂, ĉ))| . (50) Then, by adding and subtracting intermediate terms and using the Lipschitz continuity of Q⋆ , we have ∆Q ≤ Lc ∥ĉ(x̂) − c⋆ (x)∥ + Lu ∥û(x̂, ĉ) − u⋆ (x, c⋆ (x))∥ + Lx ∥x̂ − x∥ .
(51)
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
3.0 2.5 2.0 1.5 1.0 0.5 0.0
0
10
20
30
Proposed MADDPG IDDPG Conservative Greedy
40
Simulation Time (s)
50
(a) Total system reward.
60
Proposed MADDPG
Critic Loss (Log Scale)
Total System Reward
3.5
9
TABLE I: Key Simulation Parameters
IDDPG
Parameter
10 2
Value
Wi-Fi system configuration Simulator AP / MLO / Legacy 0
10
20
30
40
Simulation Time (s)
50
60
(b) Critic loss in log scale.
Fig. 4: Training convergence and critic stability. Taking expectation on both sides of Eq. (51) and applying Assumption 2 gives E [∆Q ] ≤ Lc εc + Lu εu + Lx εx = εQ .
(52)
Thus, εQ upper-bounds the expected action-value degradation caused by three sources: contention-stage approximation, transmission-stage approximation, and decision-state representation mismatch. By the standard discounted performance-difference bound for approximate policy improvement, an expected action-value degradation bounded by εQ leads to J(π ⋆ ) − J(πEvoOMG ) ≤
2δ εQ . (1 − δ)2
(53)
Substituting the definition of εQ completes the proof. The bound in Eq. (48) clarifies the role of each EvoOMG component. The sequence encoder reduces εx by preserving protocol history; the staged generator reduces εc and εu by explicitly modeling the contention-to-transmission dependency; and the standard-aware policy heads reduce approximation mismatch caused by heterogeneous legacy and MLO action spaces. Therefore, the theoretical analysis supports the main design principle of EvoOMG: heterogeneous Wi-Fi MAC control should be parameterized as a standard-aware staged decision process rather than a flat one-shot action mapping.
MLO bands MLO bandwidths Legacy band / bandwidth Propagation / mobility Traffic type Offered load Packet size CW range Aggregation range Interaction interval Warm-up / measurement time
NS-3 + ns3-ai 802.11be (Wi-Fi 7) 802.11be (Wi-Fi 7) 802.11ax (Wi-Fi 6) 2.4 / 5 / 6 GHz 40 / 80 / 320 MHz 5 GHz / 80 MHz Log-distance RandomWalk2d Downlink UDP 900 / 800 Mbit/s 1440 bytes [3, 1023] [1, 4096] 500 ms 10 s / 60 s
/ /
/
Learning configuration History length Transformer dimension / heads Actor / critic learning rates Replay buffer / batch size Discount factor Soft update coefficient
5 16 / 2 3 × 10−4 / 10−3 105 / 64 0.99 0.005
around 1.65–1.70, showing limited adaptability to dynamic Wi-Fi MAC conditions. Fig. 4(b) reports the critic loss in logarithmic scale. EvoOMG keeps a bounded critic loss, although its value is higher than those of MADDPG and IDDPG. This behavior is expected because EvoOMG uses a centralized critic to evaluate heterogeneous legacy/MLO interactions and autoregressive MAC actions. The lower critic losses of MADDPG and IDDPG do not lead to higher rewards, indicating that criticloss minimization alone is insufficient; the critic must guide the actor toward protocol-aligned contention and aggregation decisions.
IV. E XPERIMENTAL R ESULTS This section evaluates EvoOMG in an NS-3-based heterogeneous Wi-Fi environment with ns3-ai interaction. Following recent protocol-level MLO studies [23], [24], we consider one IEEE 802.11be AP serving both MLO STAs and IEEE 802.11ax legacy STAs under downlink user datagram protocol (UDP) traffic. The key system and learning settings are summarized in Table I [7], [16], [25]. The evaluation covers training stability, legacy/MLO heterogeneity, TID/ACaware traffic differentiation, and robustness under MLO PHY constraints, with MADDPG, IDDPG, conservative, and greedy policies as baselines. A. Training Convergence and Critic Stability Fig. 4 compares the training behavior of EvoOMG with representative baselines. As shown in Fig. 4(a), EvoOMG rapidly improves the system reward in the first few seconds and then stabilizes around 2.9–3.1. MADDPG and IDDPG reach moderate rewards in the early stage but gradually decline in the later stage, mostly staying around 1.8–2.2 after 40 s. Conservative and Greedy policies remain nearly unchanged
B. Performance under Legacy and MLO Heterogeneity Fig. 5 evaluates the impact of legacy/MLO device heterogeneity under three deployment ratios: legacy-dominant (2MLO : 8Legacy), balanced (5MLO : 5Legacy), and MLOdominant (8MLO : 2Legacy). Across all settings, EvoOMG achieves the highest or near-highest throughput for both MLO and legacy STAs, showing robust performance under mixedstandard coexistence. In the legacy-dominant case, EvoOMG improves the MLO throughput from 163 Mbps under MADDPG to 252 Mbps, while keeping the legacy throughput slightly higher than MADDPG. In the balanced case, EvoOMG reaches 568 Mbps and 737 Mbps for MLO and legacy STAs, respectively, outperforming MADDPG and IDDPG on both device types. In the MLO-dominant case, EvoOMG further increases MLO throughput to 1111 Mbps and significantly improves legacy throughput from 146 Mbps under MADDPG to 277 Mbps. These results indicate that EvoOMG can exploit MLO capability while preventing legacy STAs from being starved, owing to its autoregressive and standard-aware action generation.
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
871 854
1000
MLO STA (Wi-Fi 7) Legacy STA (Wi-Fi 6)
754
200
146 132 115 114
252 163 138 106 106
400
277
538 542
600
716 726
800
568 737 529 591 517 574 390 441 394 444
Average Throughput per STA (Mbps)
1200
Conserv. Greedy
1117 1055 988
Proposed MADDPG IDDPG
1400
10
0Scenario 1: Legacy Dom Scenarssio 2: Balanced Scenario 3: MLO Dom (5MLO : 5Legacy) (8MLO : 2Legacy) (2MLO : 8Legacy)
Fig. 5: Throughput per STA under different legacy/MLO deployment ratios. Three settings are evaluated: legacy-dominant (2MLO : 8Legacy), balanced (5MLO : 5Legacy), and MLOdominant (8MLO : 2Legacy). AC_VO
AC_VI
1500
AC_BK
D. Packet-Level Tail Latency 1148.3
1111.9
1000
840.5
832.5
500 0
ed
MADD
Greedy
(a) Stacked AC throughput. Proposed IDDPG Greedy MADDPG Conserv.
720
1000 800
IDDPGConservative
PG
Propos
309 70
200
163 158 163 161
250 252
267 277
400
383 371 358
600 421 412
System Throughput (Mbps)
AC_BE
1482.8
194 184 152 151
System Throughput (Mbps)
2000
0
AC_VO
AC_VI
AC_BE
AC_BK
(b) Per-AC throughput comparison.
Fig. 6: TID/AC-aware traffic differentiation results. IDDPG Conserv.
Greedy STA 1
STA 2 STA 3
STA 4
Link 1 (5GHz)
Fig. 8 reports the packet-level latency cumulative distribution functions (CDFs) for different AC traffic classes. For AC_VO in Fig. 8(a), EvoOMG significantly shifts the CDF to the left. Its median latency is around 75–80 ms, while MADDPG is around 150–160 ms. At the tail, EvoOMG approaches 100–110 ms, whereas MADDPG is close to 190 ms, showing a clear latency reduction for voice traffic. For AC_VI in Fig. 8(b), MADDPG has a slightly lower delay distribution than EvoOMG. The median latency of EvoOMG is around 180 ms, while MADDPG is around 145– 150 ms. This indicates that EvoOMG does not uniformly minimize delay for all ACs; instead, it trades part of the VI delay for higher system throughput and better global scheduling. For AC_BE in Fig. 8(c), EvoOMG has a much larger latency than MADDPG. The EvoOMG CDF rises mainly around 430– 460 ms, while MADDPG is concentrated around 200–260 ms. This is consistent with the per-AC throughput result, where BE traffic receives fewer transmission opportunities under the goodput-oriented utility. For AC_BK in Fig. 8(d), EvoOMG shifts the CDF to the left compared with MADDPG. Most BK packets under EvoOMG are completed around 100–130 ms, while MADDPG is closer to 220–260 ms. Overall, EvoOMG substantially improves VO and BK latency, while VI and BE reflect the trade-off between goodput maximization and AC-level delay fairness.
27 23 39 38
E. Performance under STR/NSTR PHY Constraints 29 1323 5613 14 5612 13 558 9 559 9
Link 0 (2.4GHz)
14 46 60 97 2535 59 56 2333 57 56 26 24 40 36
110137 4553 7689 4452 7892
54 52
113 141 56 53
100119
240
Proposed MADDPG
21
350 300 250 200 150 100 50 0
highest total system throughput, reaching 1482.7 Mbps, compared with 1148.3 Mbps for MADDPG, 1111.9 Mbps for IDDPG, 832.5 Mbps for Conservative, and 840.5 Mbps for Greedy. This corresponds to gains of 29.1%, 33.3%, 78.1%, and 76.4%, respectively. Fig. 6(b) further shows the per-AC throughput. EvoOMG achieves the best performance on AC_VO, AC_VI, and AC_BK, with 720 Mbps, 383 Mbps, and 309 Mbps, respectively. For AC_BE, EvoOMG obtains 70 Mbps, lower than MADDPG and IDDPG, indicating that the learned policy prioritizes VO, VI, and BK traffic under the goodput-oriented utility. This reflects service differentiation rather than uniform allocation across all ACs. Fig. 7 shows the STA-level traffic distribution over MLO links. EvoOMG mainly exploits Link 0 (2.4 GHz), while still assigning part of the traffic to Link 1 (5 GHz). Link 2 (6 GHz) carries much less traffic, mostly below 20 Mbps. This indicates that EvoOMG performs linkdependent scheduling instead of blindly activating all available MLO links.
Link 2 (6GHz)
Fig. 7: STA-level throughput breakdown over MLO physical links. C. TID-Aware Traffic Differentiation and Link Scheduling Fig. 6 evaluates TID/AC-aware scheduling and MLO link allocation. As shown in Fig. 6(a), EvoOMG achieves the
Fig. 9 evaluates EvoOMG under STR-only, NSTR-only, and mixed STR/NSTR settings. As shown in Fig. 9(a), EvoOMG and MADDPG have almost identical average delay across the three modes. The delays of EvoOMG are 14.84 ms, 48.97 ms, and 29.53 ms under STR-only, NSTR-only, and mixed settings, respectively, which are very close to MADDPG’s 14.74 ms, 49.05 ms, and 29.67 ms. This indicates that EvoOMG improves throughput without increasing average delay.
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
Proposed AC_VO
MADDPG AC_VO
100
Proposed AC_VI
MADDPG AC_VI
100
Proposed AC_BE
MADDPG AC_BE
80
60
60
60
20
20 0
40
4 × 101 6 × 101 102 Packet Latency (ms)
0
2 × 102
(a) AC_VO traffic.
CDF (%)
80
60
CDF (%)
80
40
40
20
0
2 × 102 3 × 102 Packet Latency (ms)
(b) AC_VI traffic.
MADDPG AC_BK
40
20
102
Proposed AC_BK
100
80
CDF (%)
CDF (%)
100
11
0
2 × 102 3 × 102 4 × 102 Packet Latency (ms)
100
(c) AC_BE traffic.
101 102 Packet Latency (ms)
(d) AC_BK traffic.
20
29.53 29.67
14.84 14.74
10 0
MADDPG
60
26
40 30
Proposed
MADDPG
24
24.24
23.60
23.27 23.45
23.77
24.23
22 20
STR-only
NSTR-only
(a) Average delay.
Mixed
58.48
Proposed
MADDPG
47.88
49.24
50 39.59
40
32.18
32.45
NSTR-only
Mixed
30 20 10
STR-only
NSTR-only
Mixed
0
STR-only
(b) Collision rate.
System Throughput (Mbps)
48.97 49.05
Link Utilization (%)
Proposed
50
Collision Rate (%)
System Average Delay (ms)
Fig. 8: Packet-level tail-latency CDFs for different AC traffic classes. MADDPG 1600 1471.50 Proposed 1420.65 1414.36 1400 1141.22 1200 1096.85 1094.84 1000 800 600 400 200 0 STR-only NSTR-only Mixed
(c) Link utilization.
(d) System throughput.
Fig. 9: Performance under different MLO PHY constraints. EvoOMG is compared with MADDPG under STR-only, NSTRonly, and mixed STR/NSTR settings.
F. Ablation Study Fig. 10 evaluates the contribution of each component using the following variants: • MLP-MADDPG: a one-shot MLP-based MADDPG baseline without Transformer encoding or autoregressive action generation. • w/o AR Gen: EvoOMG without autoregressive generation, where CW and A-MPDU length are generated as a flat joint action. • w/o History: EvoOMG without historical state encoding, using only the current observation. • w/o Mask: EvoOMG without standard-aware feasibility masking for legacy/MLO action constraints.
Jain Fairness
744.7 738.1
1106.0
556.8 555.1
1301.7
659.4 659.4
744.9 738.3
682.0 673.8
741.1 734.1
575.1 573.1
1141.2
1500 1250 1000 750 500 250 0 ed en ull PG ory itic ask ADD /o AR G /o Hist w/o M ev-Emb Cent-Cr posed F M P D w w o r L o o / / P M w w
1.0 0.9
Jain Fairness Index
Legacy Throughput 1471.5
1471.9
1344.4
MLO Throughput
1463.9
System Throughput
Throughput (Mbps)
Fig. 9(b) shows that the collision rates are also comparable. EvoOMG obtains 24.24%, 23.27%, and 23.77% under the three settings, while MADDPG obtains 23.60%, 23.45%, and 24.23%. Thus, the performance gain of EvoOMG is not mainly caused by collision reduction. The main advantage appears in Fig. 9(c). EvoOMG achieves link utilization of 58.48%, 47.88%, and 49.24%, clearly higher than MADDPG’s 39.59%, 32.18%, and 32.45%. This shows that the proposed policy can better exploit feasible MLO links under both STR and NSTR constraints. Fig. 9(d) further confirms this advantage. EvoOMG reaches 1471.50 Mbps, 1414.36 Mbps, and 1420.65 Mbps under STR-only, NSTR-only, and mixed settings, respectively, while MADDPG achieves 1141.22 Mbps, 1094.84 Mbps, and 1096.85 Mbps. The corresponding throughput gains are about 28.9%, 29.2%, and 29.5%. These results demonstrate that EvoOMG remains effective when full simultaneous transmission is available and when NSTR constraints restrict feasible link activation. The results show that EvoOMG consistently approaches the STR-enabled performance upper envelope even under NSTR constraints.
0.8 0.7 0.6 0.5 0.4
Fig. 10: Ablation study of EvoOMG core mechanisms. w/o Dev-Embed: EvoOMG without the device-standard embedding that distinguishes legacy and MLO STAs. • w/o Cent-Critic: EvoOMG without the centralized critic, relying on decentralized value estimation. • Proposed Full: the complete EvoOMG model with Transformer-based history encoding, autoregressive action generation, device embedding, feasibility masking, and centralized critic. •
Proposed Full achieves the highest system throughput of 1471.5 Mbps, with 744.7 Mbps MLO throughput and 738.1 Mbps legacy throughput. Compared with MLP-MADDPG, whose system throughput is 1141.2 Mbps, Proposed Full improves system throughput by about 28.9%. The w/o AR Gen variant reaches 1463.9 Mbps, close to Proposed Full, but its Jain fairness is lower, indicating that autoregressive CW-to-aggregation generation mainly improves coordinated allocation rather than only increasing raw throughput. The w/o History variant drops to 1344.4 Mbps, showing that recent channel, queue, contention, and aggregation states are useful for MAC decision making. The w/o Mask variant obtains 1471.9 Mbps, but its fairness is also lower than Proposed Full, suggesting that standard-aware masking helps maintain
IEEE XXX, VOL. XX, NO. XX, MONTH 202X
a better legacy/MLO balance. The w/o Dev-Embed variant decreases to 1301.7 Mbps, confirming the importance of explicitly encoding device type for legacy and MLO action grammars. Finally, w/o Cent-Critic performs the worst with 1106.0 Mbps, demonstrating that centralized training is critical for learning coupled multi-agent contention behavior. V. C ONCLUSION This paper argues that Wi-Fi optimization should be revisited from two first principles: standard heterogeneity and protocol sequentiality. Mixed deployments of legacy nonMLO STAs and MLO-capable STAs invalidate one-size-fits-all control, while the contention-to-transmission pipeline makes one-step static policies structurally mismatched to the MAC process. To address these issues, we propose EvoOMG, an evolution-oriented multi-agent guidance framework whose internal policy is sequence-aware and stage-aligned with WiFi access. The draft provides the system model, algorithm design, theoretical interpretation, and evaluation plan for a full NS-3 study. Our core thesis is that protocol-aligned staged guidance is a promising direction for adaptive intelligence in heterogeneous legacy-and-MLO Wi-Fi networks. R EFERENCES [1] C. Deng, X. Fang, X. Han, X. Wang, L. Yan, R. He, Y. Long, and Y. Guo, “Ieee 802.11be wi-fi 7: New challenges and opportunities,” IEEE Communications Surveys & Tutorials, vol. 22, no. 4, pp. 2136– 2166, 2020. [2] R. J. Jayabal, D. T. C. Wong, L. K. Goh, X. Zhang, C. M. Pang, and S. Sun, “Survey, design and evaluation of tgt-hc: A time-aware shaper mac for wireless tsn,” IEEE Transactions on Mobile Computing, vol. 24, no. 6, pp. 5433–5445, 2025. [3] S. Jung, S. Choi, Y. Yoon, H.-k. Son, and H. Kim, “Modeling and analysis of coexistence between mlo nstr-based wi-fi 7 and legacy wifi,” arXiv preprint arXiv:2509.01201, 2025. [4] S. Arthi, N. B. Mehta, and C. Singh, “Hybrid access mac protocol in wi-fi: Analysis and optimal resource allocation policy design,” IEEE Transactions on Mobile Computing, vol. 25, no. 4, pp. 5742–5757, 2026. [5] W. Wydma’nski and S. Szott, “Contention window optimization in ieee 802.11ax networks with deep reinforcement learning,” in Proc. IEEE Wireless Communications and Networking Conference (WCNC), 2021, pp. 1–6. [6] Q. Li, B. Lv, Y. Hong, and R. Wang, “Reinwifi: A reinforcementlearning-based framework for the application-layer qos optimization of wifi networks,” arXiv preprint arXiv:2405.03526, 2024. [7] J. Wu, X. Fang, and G. Min, “Deep reinforcement learning based multilink frame aggregation length optimization in next generation wi-fi networks,” IEEE Transactions on Wireless Communications, vol. 23, no. 10, pp. 14 482–14 497, 2024. [8] L. Galati-Giordano, G. Geraci, M. Carrascosa, and B. Bellalta, “What will wi-fi 8 be? a primer on ieee 802.11bn ultra high reliability,” IEEE Communications Magazine, vol. 62, no. 8, pp. 126–132, 2024. [9] R. Shafin, I. Val, Y. Qi, P. Nayak, V. V. Ratnam, B. Sadiq, S. Schelstraete, M. Martinez, and B. L. Ng, “Multi-device experience with peer-to-peer connectivity in ieee 802.11bn (wi-fi 8),” IEEE Journal on Selected Areas in Communications, vol. 43, no. 11, pp. 3695–3707, 2025. [10] D. Wei, L. Cao, L. Zhang, X. Gao, and H. Yin, “Optimized nonprimary channel access design in ieee 802.11bn,” arXiv preprint arXiv:2405.00227, 2024. [11] A. L’opez-Ravent’os and B. Bellalta, “Multi-link operation in ieee 802.11be wlans,” IEEE Wireless Communications, vol. 29, no. 4, pp. 94–100, Aug. 2022. [12] M. Carrascosa, G. Geraci, E. Knightly, and B. Bellalta, “An experimental study of latency for ieee 802.11be multi-link operation,” in ICC 2022 - IEEE International Conference on Communications, 2022, pp. 2507– 2512.
12
[13] B. Bellalta, M. Carrascosa, L. Galati-Giordano, and G. Geraci, “Delay analysis of ieee 802.11be multi-link operation under finite load,” IEEE Wireless Communications Letters, vol. 12, no. 4, pp. 595–599, 2023. [14] L. Zhang, H. Yin, S. Roy, L. Cao, X. Gao, and V. Sathya, “Ieee 802.11be network throughput optimization with multi-link operation and ap controller,” IEEE Internet of Things Journal, vol. 11, no. 13, pp. 23 850–23 861, 2024. [15] S. Avallone and P. Imputato, “Understanding the new enhanced multilink single radio feature of ieee 802.11be wlans,” IEEE Journal on Selected Areas in Communications, vol. 43, no. 11, pp. 3683–3694, 2025. [16] J. Wu, X. Fang, and X. Wang, “Service-differentiated joint distributed communication and computing resource allocation for wi-fi networks based on federated learning and madrl,” IEEE Internet of Things Journal, vol. 12, no. 17, pp. 36 302–36 317, 2025. [17] X. Shen, Z. Song, Y. Zhou, B. Chen, J. Liu, R. Zhang, R. A. Rossi, H. Tan, T. Yu, X. Chen, Y. Zhou, T. Sun, P. Zhao, Y. Wang, and J. Gu, “Numerical pruning for efficient autoregressive models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 19, 2025, pp. 20 418–20 426. [18] J. Han, J. Liu, Y. Jiang, B. Yan, Y. Zhang, Z. Yuan, B. Peng, and X. Liu, “Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2025, pp. 15 733–15 744. [19] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu, “The surprising effectiveness of ppo in cooperative multi-agent games,” in Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 24 611–24 624. [20] Z. Yang, Y. Chen, Z. Wang, H. Shan, Y. Chen, and Y. Zhang, “Patientlevel anatomy meets scanning-level physics: Personalized federated lowdose ct denoising empowered by large language model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 5154–5163. [21] Z. Yang, W. Xia, Z. Lu, Y. Chen, X. Li, and Y. Zhang, “Hypernetworkbased physics-driven personalized federated learning for ct imaging,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 2, pp. 3136–3150, 2025. [22] K. Wei, J. Li, M. Ding, and H. V. Poor, “Low-latency federated learning over wireless channels with differential privacy,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 1, pp. 290–307, 2022. [23] Y. Zou, M. Shen, H. Yin, and Y. Gao, “Delay analysis of multi-link devices coexisting with single-link devices in wi-fi 7,” in Proc. IEEE INFOCOM, 2025, pp. 1–10. [24] Y. Gao, M. Shen, Y. Zou, H. Yin, and S. Roy, “Latency optimal trafficto-link allocation for mlo/slo coexistence in wi-fi 7,” IEEE Journal on Selected Areas in Communications, vol. 43, no. 11, pp. 3633–3649, 2025. [25] J. Wu, X. Fang, J. Wang, B. Yin, and T. Li, “Topology-aware dynamic offloading for mec-enabled mapc wi-fi networks: A multi-agent hierarchical drl framework,” IEEE Transactions on Vehicular Technology, pp. 1–16, 2026. [26] S. Lian, J. Tong, J. Zhang, and L. Fu, “Intelligent channel allocation for ieee 802.11be multi-link operation: When mab meets llm,” IEEE Journal on Selected Areas in Communications, 2025, early access. [27] J. Jung, J. Baik, Y. Kim, H.-S. Park, and J.-M. Chung, “Otop: Optimized transmission power controlled obss pd-based spatial reuse for high throughput in ieee 802.11be wlans,” IEEE Internet of Things Journal, vol. 10, no. 19, pp. 17 110–17 123, 2023. [28] I. Behnke and H. Austad, “Real-time performance of industrial iot communication technologies: A review,” IEEE Internet of Things Journal, vol. 11, no. 5, pp. 7399–7410, 2024. [29] P. E. Iturria-Rivera, M. Chenier, B. Herscovici, B. Kantarci, and M. ErolKantarci, “Rl meets multi-link operation in ieee 802.11be: Multiheaded recurrent soft-actor critic-based traffic allocation,” in Proc. IEEE International Conference on Communications (ICC), 2023, pp. 1–6. [30] F. Wilhelmi, S. Szott, K. Kosek-Szott, and B. Bellalta, “Machine learning and wi-fi: Unveiling the path toward ai/ml-native ieee 802.11 networks,” IEEE Communications Magazine, vol. 63, no. 7, pp. 114–120, Jul. 2025. [31] B. Bellalta, K. Kosek-Szott, S. Szott, and F. Wilhelmi, “Towards an ai/ml-defined radio for wi-fi: Overview, challenges, and roadmap,” arXiv preprint arXiv:2405.12675, 2024.