Multi-Agent Transformer for Queue-Level XR Traffic Scheduling in TSN Networks

Multi-Agent Transformer for Queue-Level XR Traffic Scheduling in TSN Networks Marcos Carvalho∗‡ , Fatih Temiz‡ , Shavbo Salehi‡ Melike Erol-Kantarci‡ , Fellow, IEEE, Daniel F. Macedo∗ ∗ Universidade Federal de Minas Gerais, Brazil ‡ School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

arXiv:2608.05340v1 [cs.NI] 5 Aug 2026

E-mails:{marcoscarvalho, damacedo}@dcc.ufmg.br , {ftemi033, ssale038, melike.erolkantarci}@uottawa.ca

Abstract—Time-Sensitive Networking (TSN) and Mobile Edge Computing (MEC) hold strong potential for enabling ultrareliable low-latency communication for time-sensitive applications, such as eXtended Reality (XR). However, the widespread adoption of XR introduces significant challenges due to co-located services in MEC environments, leading to contention for shared network resources. Moreover, XR traffic types have distinct characteristics and criticality in terms of timing requirements, further increasing the complexity and dynamics of such environments. Although reinforcement learning has shown promise for TSN scheduling optimization in dynamic network scenarios, existing approaches rely on centralized or high-level multi-agent designs and are typically tailored to periodic and predictable industrial traffic, limiting their applicability to XR workloads. As a result, these approaches suffer from (i) limited ability to capture interqueue dependencies due to coarse-grained control, and (ii) poor adaptability to highly dynamic and heterogeneous XR traffic. To address these gaps, we propose a multi-agent reinforcement learning approach for queue-level XR traffic scheduling. We adopt the multi-agent transformer (MAT) to model inter-queue dependencies via attention over agents’ observations and actions, enabling implicit coordination across heterogeneous co-located XR applications. Our simulation results show that the proposed method outperforms baselines, achieving up to 71.42% latency reduction and up to 83.2% reduction in failure rate, while consistently achieving high reliability across all queues. Index Terms—Extended Reality, Mobile Edge Computing, Multi-Agent Reinforcement Learning, Time-Sensitive Networking, Ultra-Reliable Low-Latency Communication

I. I NTRODUCTION In recent years, eXtended Reality (XR) has rapidly expanded across various domains such as gaming, education, and industry [1]. This expansion poses significant challenges to network infrastructure, as XR applications rely on immersive and continuous sensing with stringent ultrareliable low-latency communication (URLLC) requirements. Meanwhile, mobile edge computing (MEC) has been a promising approach to support URLLC by reducing latency and improving reliability through proximity to end users [2]. Its integration with time-sensitive networking (TSN) has been explored to enhance timely service delivery [3]. TSN standards such as IEEE 802.1Qbv [4] further support this by enabling time-aware control of multiple queues in network devices. In practice, MEC environments co-locate multiple XR applications, where each is mapped to distinct TSN queues

to enable traffic isolation and fine-grained control. However, this resource-sharing strategy introduces competition among multiple applications for network resources. In addition, the diversity of XR applications creates a highly challenging environment. For instance, semantic augmented reality applications (SeAR) impose more stringent deadline constraints, as semantically critical content must be delivered within strict latency bounds, compared to traditional augmented reality (AR) applications, which typically operate under soft real-time (SRT) constraints [5]. Traditional TSN schedulers based on optimization methods, such as satisfiability modulo theories and integer linear programming, struggle to adapt schedules in a timely manner in such dynamic settings [6]. Alternatively, reinforcement learning (RL) has shown promise in improving TSN scheduling, as it enables adaptive decision-making through interaction with the environment. For example, [7] applies proximal policy optimization (PPO) to optimize gate timing across multiple queues, reducing endto-end latency in dynamic industrial networks. In addition, [8] proposes a DDPG-based approach to adapt time slot durations across the queues, aiming to mitigate transmission failures caused by external factors such as time-synchronization errors. Moreover, [9] proposes a hierarchical RL approach for jointly managing computational resources and TSN scheduling in edge environments, where a high-level policy allocates resources and a low-level policy controls transmission intervals across the queues. Recent studies explore multi-agent reinforcement learning (MARL) by proposing different levels of agent abstraction to address the complexities of TSN networks. For instance, [10] decouples the TSN scheduling problem into specialized routing and timing agents, enabling real-time traffic scheduling at the device level. Similarly, [11] implements a multi-agent architecture within a centralized control plane, where routing and scheduling agents collaborate to minimize latency in dynamic environments. Moreover, [12] models each TSN switch as an autonomous agent responsible for managing local traffic. However, existing designs remain centralized at the device level, limiting their ability to capture complex inter-queue dependencies under shared network resources and heterogeneous XR applications. To address these limitations, we propose a queue-level MARL framework in which each TSN queue is modeled as

an individual agent. The agents learn cooperative scheduling policies by dynamically allocating time-slot durations to minimize deadline misses across heterogeneous XR applications. We leverage a multi-agent transformer (MAT) [13] architecture to model inter-queue dependencies through attention mechanisms, enabling effective coordination among agents. To evaluate the effectiveness of the proposed approach, we compare it against single-agent, MARL, and non-learningbased baselines. The results show that MAT-based scheduling significantly outperforms the baselines by learning a more effective cooperative policy, leading to improved latency reduction up to 71.42% and up to 83.2% reduction in failure rate, while consistently maintaining high reliability.

(i)

where ts (pj ) denotes the service start time of packet pj in the (i) queue qi , and IP I(pj ) represents the arrival time of packet pj in queue qi . Based on the waiting time of all packets in the queue qi , the average waiting time is modeled as W̄ = PXi (i) 1 j=1 W (pj ), where Xi is the total number of packets. Xi Although W̄ reflects the overall latency across all packets, it does not capture worst-case scenarios. The age of the oldest packet (AOP), denoted by g, captures this worst-case latency by measuring the older packet in the queues, as formulated by the equation below: (i)

g = max W (pj ).

(3)

pj ∈qi

In this work, we consider both IFI and IPI as random variables following a Johnson’s SU distribution [15].

II. S YSTEM M ODEL AND P ROBLEM F ORMULATION A. System Architecture Overview In this paper, we consider an MEC-based XR streaming architecture (Fig. 1) supporting traditional AR and SeAR applications. SeAR applications differ from traditional AR by focusing on the transmission of semantic information rather than raw data [14]. Furthermore, we consider different deadline requirements between traditional AR and SeAR applications. We observe that SeAR applications require a more stringent latency bound compared to traditional AR. This observation stems from the fact that semantic frames are typically small but carry an extremely high value of information [5], [14]. In addition, we adopt the MEC XR server equipped with a TSNcapable network interface card (NIC) [3]. We assume the NIC with a set of queues, denoted by Q = {q0 , q1 , . . . , qn }, where each queue serves a specific XR application to ensure strict isolation. Additionally, we consider a downlink (DL) multiuser communication environment in which the number of users for each XR application varies randomly over time. B. XR Traffic and Latency Model XR applications have two temporal variables: inter-frame interval (IFI) and inter-packet interval (IPI). The IFI denotes the time between consecutive video frames and exhibits a nonperiodic nature due to external factors [15]. The IPI denotes the time between consecutive packets within a video frame and is modeled as: (i)

(f )

IP I(pj ) = tstart +

P −1 X

∆j + τ,

P = 1, 2, . . . , NIP

(1)

C. TSN Schedule Model We discretize time globally into fixed-duration cycles ci ∈ C, each with duration Cd = 1ms. Based on this, a schedule si ∈ S is constructed at each cycle in two steps. First, we define intra-cycle time-slot allocation for each queue, where each cycle is partitioned into a sequence of T time slots, with T ≤ |Q|. Formally, we define this step as:

T = t1 (q0 ), t2 (q1 ), . . . , tT (qQ ) ,

T X

tj = Cd ,

(4)

j=1

where tj denotes the duration of the j-th time slot (in µs) assigned to queue qi . The second step defines the service order of queues. We assume that the time-slot duration encodes queue urgency, so queues are scheduled in descending order of tj . We also assume that multiple queues share the same duration, where ties are resolved by prioritizing higher-indexed queues (i.e., q7 > q6 > · · · > q0 ), following driver-level prioritization [16]. The resulting service order is a permutation π(·) obtained by sorting queues in decreasing order of tk and breaking ties by index: π = arg, sortk∈1,...,Q (−tk , k). Thus, the schedule si for cycle ci is defined as:
si = [tπ(1) , π(1)], [tπ(2) , π(2)], . . . , [tπ(Q) , π(Q)] , (5) where tπ(k) is the assigned time-slot duration and π(k) denotes the index of the queue scheduled in the k-th position.

j=1

D. Problem Formulation

(f ) where tstart

denotes the starting time of video frame f , ∆j is the interval between consecutive packets within the same frame, τ captures additional stochastic variations (i.e., IP fragmentation), and NIP represents the total number of packets in a video frame. These packets are enqueued in the TSN queues according to their IPI and transmitted following a firstin, first-out (FIFO) policy during the transmission opportunity. Hence, latency is mainly determined by the queueing delay, formulated as: (i)

(i)

(i)

W (pj ) = ts (pj ) − IP I(pj ),

(2)

In this work, we assume a hard deadline scenario in which all packets of an XR video frame must be transmitted before the next frame arrives [17]. We further assume that SeAR imposes stricter deadlines than traditional applications. Thus, the main objective is to minimize deadline violations in colocated mixed-criticality XR applications by reducing the AOP of each queue relative to its deadline, while a secondary objective is to maximize bandwidth utilization by avoiding allocating more time than is actually needed. The scheduling problem is formulated as follows:

XR Applications

Multi-Agent Observation Encoding

MEC XR Server

TSN-Capable NIC XR Users

TSN Network

Application

Queue

q2

q7

q1

M L P

Attention

(ii) M Masked Masked L Attention Attention P Auto-Regressive Action Decoding

...

a0

2

Ô7 ... Ô1 Ô0

(i)

a6

...

M L P

3

am+1

q0 4

... ...

Packet

Agent

O7 ... O1 O0

Observation

1

Actions Deployment

a7

a2

a1

a0

Multi-Agent Transformer (MAT)

Figure 1: Overall System Model

s∗ = arg min s∈S

C X

" T Q XX

c=1

k=0 i=0

λ1

gi,tk ,c Dqi #

+

X

λ2 C(tk,qi ) − Ftx (tk,qi )

(6a)

tk ∈c

s.t.

X

xi (tk ) ≤ 1,

∀i, ∀c,

(6b)

Ftx (tk ) ≤ C(tk ), ∀tk , X tk = Cd , ∀c,

(6c)

tk ∈c

(6d)

oldest remaining packet (in µs), and W̄ is the average waiting time of packets (in µs). • Action: Each queue agent i decides its time-slot allocation by outputting a continuous value ai ∈ [0, 1] representing its desired fraction of the cycle time. The actual timeslot duration assigned to each queue is then computed as ai · Cd . ti = P|Q| j j=0 a

•

Reward: The reward function is designed as a cooperative team reward, encouraging all agents to minimize deadline violations across all queues and maximize the bandwidth utilization. The team reward Rt is formulated as: Q

tk ∈c

Di < IF Ii , ∀i,

(6e)

Dm < Dk , ∀m ∈ QSeAR ; ∀k ∈ QAR ,

(6f)

where gi,tk ,c denotes the AOP in queue qi after completing the transmission of its allocated time slot of duration tk within the cycle c. Di is the deadline of the XR application in qi ; For maximizing bandwidth utilization, C(tk,i ) represents the maximum number of packets that can be transmitted in slot L ·tk tk for queue qi , defined as C(tk ) = speed Psize , where Lspeed is the link speed, tk is the slot duration, and Psize is the packet size. Ftx (tk ) denotes the actual number of transmitted frames. Eq. (6b) enforces that each queue is allocated at most once per cycle. Eq. (6c) ensures that the amount of transmitted packets during time slot tk does not exceed the available capacity. Eq. (6d) guarantees that the total duration of all time slots within a cycle equals the predefined cycle duration Cd . Eq. (6e) ensures that the deadline Di of each application does not exceed its IFI. Finally, eq. (6f) enforces stricter deadline requirements for SeAR applications by ensuring that their deadlines are always smaller than those of AR applications.

Rt = − max Φi (t) − β i∈Q

1 X BW i (t) Q i=1

(7)

where the term Φi (t) represents the queueing latency penalty, which is computed as follows: ( ρi (t), if ρi (t) ≤ 1 Φi (t) = (8) 2 1 + λ(ρi (t) − 1) , if ρi (t) > 1 where ρi (t) is the AOP-deadline ratio in queue i, defined i (t) . Then, the team receives a linear penalty as ρi (t) = gD i based on the maximum ratio across all queues when no deadline is violated (ρi (t) ≤ 1). Once any deadline is violated (ρi (t) > 1), the penalty increases quadratically, strongly discouraging deadline violations. The term λ is a positive scaling factor that ensures a smooth transition between the non-violation and violation cases. The team is further penalized by the average bandwidth utilization term BW i (t), which is computed as C(tk,qi )−Ftx (tk,qi ). β controls the relative weight of the bandwidth penalty. III. MAT-BASED TSN S CHEDULING

E. MDP Formulation

A. MAT Framework

We formulated the TSN scheduling problem as a multiagent partially observable Markov decision process. Below, we describe the three essential components of an agent. i • Observation: The local observation o of each queue agent i after time slot duration tk is defined as oi = ⟨Ftx , B, g, W̄ ⟩, where Ftx is the number of transmitted packets, B is the residual backlog, g is the age of the

To efficiently address the optimization challenge, we employ MAT [13] framework as shown in Fig. 1, which comprises the following blocks: 1) Multi-Agent Observation Encoding: this block receives queue observations (o0 , o1 , . . . , oQ ) in an arbitrary order and then extracts information across them, where each block includes multi-head self-attention and a feedforward neural network. The output of the encoder is a

series of latent representations (ô0 , ô1 , . . . , ôQ ) containing both the individual information of each queue i and complex, highlevel interactions between them. 2) Auto-Regressive Action Decoding: this block employs an auto-regressive approach to generate a sequence of actions. It consists of a sequence of decoding blocks, where each block uses two masked self-attention mechanisms: (i) the first masked self-attention ensures that, when generating the action, attention is computed only with the previous actions. (ii) the second masked attention mechanism computes attention between the current action and the encoder outputs. The output of the final decoder block is a sequence of joint actions for all queues, which is fed to the final MLP to produce the probability distribution of each queue qi ’s action, representing its policy. B. Training and Inference Procedure The MAT algorithm (Algo. 1) adopts an actor–critic framework in which the encoder and decoder share a latent representation space. During training, the encoder estimates value functions Vϕ (ôi ) (line 22), encouraging the network to extract informative features from the TSN environment that are critical for scheduling decisions. Coordinated scheduling policies are learned by jointly processing observations and action sequences, where masked self-attention captures both the global network state and causal dependencies among agents (line 25). The model parameters are then updated in a single optimization step by minimizing the combined loss LEncoder (ϕ)+LDecoder (θ). For data collection, local observations oi for each agent are generated after each time slot duration tk (lines 5–7). The decoder receives these representations and generates joint actions auto-regressively, where each queue selects a time slot duration conditioned on previous decisions (lines 12–14). The environment then executes the actions, determines the service order, transitions to the next state, and returns a team reward Rt (lines 15–18). These transitions are stored in a replay buffer B for periodic minibatch updates (line 19). Due to space constraints, we omit the loss functions for the LEncoder (ϕ) and LDecoder (θ), as well as the definitions of the value function and joint advantage function, and refer the reader to the original MAT paper [13]. IV. P ERFORMANCE E VALUATION

Algorithm 1 MAT-Based TSN Scheduling 1: Initialize: Encoder {ϕ0 }, Decoder {θ0 }, Replay buffer B. 2: for each episode e ∈ {1, 2, . . . , N } do 3: Reset environment parameters. 4: for each step s ∈ {1, 2, . . . , 150} do 5: for tk ∈ T do

Collect oi = ⟨Ftx , B, g, W̄ ⟩ of queue i; end for Form observation vector os = (o0 , . . . , oQ ), // Inference Phase Encode observations into ô0 , . . . , ôQ . 11: Input (ô0 , . . . , ôQ ) to the decoder. 12: for m = 0, 1, . . . , Q do 13: Input a0 , . . . , am and infer am+1 . 14: end for 15: Execute joint action (a0 , . . . , aQ ) 16: Determine the service order by eq. 5. 17: Execute the schedule in the TSN network 18: Obtain the global reward Rt (eq. 7). 19: Store the tuple (o, a, Rt ) into B. 20: // The Training Phase 21: Sample a random minibatch of B tuples from B. 22: Compute Vϕ (ô0 ), . . . , Vϕ (ôQ ) from the encoder. 23: Calculate LEncoder (ϕ) 24: Compute the joint advantage function Ât based on Vϕ (ô0 ), . . . , Vϕ (ôQ ) with GAE. 25: Input ô0 , . . . , ôQ and a0 , . . . , aQ , generate πθ0 , . . . , πθQ at once with the decoder. 26: Calculate LEncoder (ϕ) and LDecoder (θ) 27: Update the encoder and decoder by minimizing LEncoder (ϕ) + LDecoder (θ) 28: end for 29: end for 6: 7: 8: 9: 10:

Table I: MEC and MAT parameters/values MEC parameter Network speed Max packet size # of users per queue

value MAT parameter value 10 Gbps entropy_coef 0.01 1428 bytes gae_lambda 0.95 1–15 episode_length 150 q0,q3: 3840×1920@90Hz/5.55 ms learning_rate 1e-4 AR setting/deadline q1,q4,q7: 3840×1920@72Hz/6.94 ms ppo_epoch 5 q2: 1920×720@60Hz/1 ms n_heads 1 q5: 1280×480@60Hz/1 ms n_block 2 SeAR setting/deadline q6: 2560×960@60Hz/1 ms n_embd 128

A. Experimental Settings and Baselines We consider a network scenario composed of eight queues supporting five distinct XR applications. Specifically, queues q0 , q1 , q3 , q4 , and q7 handle high-workload traditional AR traffic. In contrast, the remaining queues carry SeAR traffic with different video resolutions, as summarized in Table I. In addition, for traditional AR applications, we defined the deadline as 50% of their IFI. This choice reflects a latency budget distribution strategy, where the end-to-end latency is divided between the MEC TSN scheduling domain and the remaining system components (e.g., 5G network). For SeAR applications, we adopt a stricter deadline of 1 ms, requiring that packets be delivered within a single scheduling cycle, imposing a hard real-time constraint.

The MAT parameters are set according to commonly used configurations in the literature [18]. For a fair comparison, all baselines share the same hyperparameters, including learning rate, PPO epochs, and entropy coefficient. Each episode consists of 150 steps, with the number of users per queue randomly varying between 1 and 15. We set β = 0.2 and λ = 4 in the reward function. All results in the following subsection are reported as the mean and standard error of the mean over three seeds, with the methods’ performance evaluated over 4800 steps. Baselines: We compare MAT against single-agent, MARL, and non-learning-based heuristics, as described below: • Single-agent RL: PPO and advantage actor-critic

In contrast, MAT coordinates all queues without such tradeoffs by allocating time appropriately and responding faster to critical SeAR queues. This highlights the advantage of our queue-level design over centralized high-level single-agent approaches, as well as the ability of attention mechanisms to capture inter-queue dependencies and adapt resource allocation to varying urgency, outperforming HAPPO. MAT

p95 of AOP (ms)

(A2C) [19], representing centralized control. MARL: heterogeneous-agent proximal policy optimization (HAPPO) [20], where each queue is an agent. • Backlog-aware heuristic (BAH): prioritizes queues with larger backlogs by allocating longer time slots, computed as ti = P Bi Bj · Cd , where Bi is the backlog of queue j∈Q i and Cd is the scheduling cycle duration. • Deadline-aware heuristic (DAH): prioritizes queues closer to their deadlines based on the AOP-deadline ratio, allocating time slots as ti = P ρi ρj · Cd , where ρi j∈Q denotes the AOP-deadline ratio of queue i. •

B. Experimental Results

Average Cumulative Reward

Learning performance: As shown in Fig. 2, MAT achieves higher reward values, faster convergence, and lower variance across different seeds, indicating greater robustness. 50 1550 3050 4550 6050 7550 9050 10550 12050 13550 15050 16550 18050 19550 0

45 40 35 30 25 20 15 10 5 0

HAPPO

PPO

A2C Backlog-aware heuristic AR Queues

Deadline-aware heuristic SeAR Queues

10 8 2 1 q0

q1

q3

q4

q7

0

q2

q5

q6

Figure 3: 95th percentile of queueing latency 0 750 1500 2250 3000 3750 4500 5250 8000 2000

9000

MAT HAPPO PPO A2C 8000 10000

10000

4000 6000 Episodes

Figure 2: Average Cumulative Reward

To further assess performance, we evaluate overall service reliability, defined as the percentage of packets meeting their deadlines and expressed in “nines,” as reported in Table II. Table II: Service reliability (%) across the queues. Arrows indicate the percentage reduction in failure rate compared to the best baseline method. MAT

In contrast, HAPPO exhibits higher variance due to its randomly ordered sequential updates, which hinder dependency capture. A2C achieves a significantly higher final reward, which can be explained by the fact that A2C avoids the conservative updates induced by PPO’s clipped objective. However, A2C is limited by its centralized decision-making, which does not explicitly model the interactions among queues. Thus, the superior performance of MAT can be attributed to its sequential decision-making directly into the policy via an auto-regressive formulation, enabling explicit conditioning on previously selected actions and leading to more consistent and globally coordinated behavior. Network-centric performance: Fig. 3 shows the 95th percentile of the AOP across the queues. As observed, at the 95th percentile, MAT ensures that packets in a given AR queue are delivered within 10 ms, whereas other RL-based methods range from approximately 15 ms to 35 ms, representing a latency reduction of up to 71.42%. Moreover, while nonlearning baselines match MAT performance on AR queues, they perform poorly overall. As the backlogs on AR queues grow faster, the backlog policy prioritizes them, delaying SeAR packets by up to approximately 10 cycles (10 ms). Conversely, although the AOP-to-deadline ratio increases faster in SeAR queues, the deadline heuristic still incurs delays (up to 0.5 ms in q2 and > 1 ms in q6 ), showing that SeAR traffic cannot be effectively prioritized when other queues dominate, forcing both policies to sacrifice SeAR performance.

q0 q1 q2 q3 q4 q5 q6 q7

99.7284(0.01) ↑8.8% 99.6697(0.009) ↓3.8% 99.9692(0.02) ↑83.2% 99.7238(0.006) ↑8.3% 99.6699(0.006) ↑4.3% 99.9527(0.009) ↑61.4% 99.8564(0.02) ↑39.8% 99.7026(0.007) ↑3.2%

HAPPO

PPO

A2C

BAH

DAH

99.3531(0.03) 99.3110(0.02) 99.6603(0.02) 99.3258(0.03) 99.2570(0.02) 99.8838(0.03) 99.4185(0.05) 99.2064(0.07)

99.4832(0.009) 99.4540(0.01) 99.7302(0.02) 99.4900(0.01) 99.4593(0.01) 99.6782(0.01) 99.6206(0.02) 99.4591(0.01)

99.4450(0.01) 99.4072(0.01) 99.5885(0.03) 99.4407(0.02) 99.4244(0.01) 99.6446(0.01) 99.5222(0.01) 99.4142(0.01)

99.6291 99.5472 97.2819 99.6829 99.5527 97.0370 98.9155 99.6343

99.7021 99.6723 99.7719 99.6986 99.6551 99.8773 99.7559 99.6929

As shown, MAT consistently outperforms all baselines, achieving reliability improvements of up to 0.2–0.4 percentage points. This improvement translates into a reduction in failure rates ranging at least from 3% to 8%. Larger performance gaps are observed in the SeAR queues, where the difference between MAT and the baselines can reach a shift from twonines to three-nines reliability, representing a reduction in failure rate of up to 83.2%. This improvement can be attributed to the superior cooperative policy learned by MAT, driven by the reward design, which enforces coordination among queues and encourages the agent to allocate resources according to their relative urgency. Due to high contention, 100% reliability is not expected. We thus evaluate MAT and DAH under increased network speeds to assess performance under higher capacity, as shown in Fig. 4. As expected, reliability increases for both methods as network speed increases, since more packets can be transmitted per cycle. However, the results indicate that the deadline-aware heuristic requires higher network capacity to achieve reliability levels that are already attained by MAT under lower bandwidth conditions, suggesting better efficiency and scalability of the proposed approach. Additionally, MAT

Service Reliability

demonstrates stronger generalization across different network speeds, maintaining consistently high reliability. 100.00 99.98 99.96 99.94 25 Gbps 75 Gbps 100 Gbps 99.92 99.90 99.88 MAT 99.86 Deadline-aware heuristic q0 q1 q2 q3 q4 q5 q6 q7 q0 q1 q2 q3 q4 q5 q6 q7 q0 q1 q2 q3 q4 q5 q6 q7

Figure 4: Service reliability (%) under different network speed Finally, we evaluate the MAT inference time on an RTX 5090 GPU (107.6 TFLOPS, 31.8 GB VRAM) and an RTX PRO 6000 S GPU (93.6 TFLOPS, 95.6 GB VRAM). On the RTX 5090, MAT achieves a latency of 1.26 ± 0.037 ms, while on the RTX PRO 6000 S it achieves 0.9777 ± 0.0039 ms. Indeed, MAT needs high-performance hardware to keep inference latency below the cycle time, mainly due to sequential autoregressive decoding across queues and transformer overhead. V. C ONCLUSION This paper addresses scheduling challenges arising from the co-location of heterogeneous XR applications in MECTSN integration. We propose a queue-level MARL framework that models inter-queue dependencies using attention and auto-regressive decoding, enabling effective cooperative TSN scheduling. Simulation results show that MAT outperforms baselines, reducing queueing latency and achieving satisfactory reliability. Unlike greedy policies that sacrifice strict SeAR performance, MAT effectively balances heterogeneous deadlines, highlighting the potential of transformerbased MARL for URLLC in XR applications. Future work may investigate advanced knowledge and policy distillation techniques to further reduce inference latency. ACKNOWLEDGMENT This work was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nı́vel Superior - Brasil (CAPES) - Finance Code 001, CNPq (funding agency from the Brazilian federal government), FAPEMIG (Minas Gerais State Funding Agency), and São Paulo Research Foundation (FAPESP) with Brazilian Internet Steering Committee (CGI.br), grants 2018/23097-3 and 2020/05182-3, NSERC Canada Research Chairs Program. The authors also gratefully acknowledge the support of the University of Ottawa Visiting Researchers Program/Office. R EFERENCES [1] F. Temiz, S. Salehi, and M. Erol-Kantarci, “Edge learning via federated split decision transformers for metaverse resource allocation,” in IEEE International Conference on Communications (ICC). IEEE, 2026. [2] S. Salehi, P. E. Iturria-Rivera, M. Elsayed, M. Bavand, R. Gaigalas, Y. Ozcan, and M. Erol-Kantarci, “Self-play ensemble q-learning enabled resource allocation for network slicing,” in GLOBECOM 2024-2024 IEEE Global Communications Conference. IEEE, 2024, pp. 4528– 4533.

[3] M. Carvalho and D. F. Macedo, “Performance analysis of the integration of dynamic cloud computing environments and tsn networks,” in 2025 IEEE 11th International Conference on Network Softwarization (NetSoft). IEEE, 2025, pp. 406–414. [4] “IEEE Standard for local and metropolitan area networks—bridges and bridged networks—amendment 25: Enhancements for scheduled traffic,IEEE Standard 802.1qbv-2015, 2016, pp. 1–57,” 2016. [5] L. Luo, Y. Zhou, J. Shen, H. Luo, W. Wu, and H. Yu, “Enhancing mobile immersive streaming experience via deadline-aware scheduling and learning-enhanced congestion control,” IEEE Internet of Things Journal, 2025. [6] T. Stüber, L. Osswald, S. Lindner, and M. Menth, “A survey of scheduling algorithms for the time-aware shaper in time-sensitive networking (tsn),” Ieee Access, vol. 11, pp. 61 192–61 233, 2023. [7] A. Roberty, Q. Besnard, S. B. H. Said, F. Ridouard, H. Bauer, and A. Geniet, “Configuring the ieee 802.1 q time-aware shaper with deep reinforcement learning,” in NOMS 2024-2024 IEEE Network Operations and Management Symposium. IEEE, 2024, pp. 1–7. [8] B. Zhou and L. Cheng, “Mitigation of scheduling violations in timesensitive networking using deep deterministic policy gradient,” in Proceedings of the 4th FlexNets Workshop on Flexible Networks Artificial Intelligence Supported Network Flexibility and Agility, 2021, pp. 32–37. [9] A. A. Saruhan, M. Abuibaid, A. H. Ghorab, M. St-Hilaire, and I. Moldován, “Cooperative resource allocation and traffic scheduling for iiot controllers in edge clouds: A hierarchical reinforcement learning approach,” in 2025 International Conference on Smart Applications, Communications and Networking (SmartNets). IEEE, 2025, pp. 1–6. [10] M. Guo, S. He, C. Gu, X. Guo, J. Chen, T. Gao, and T. Wang, “Towards distributed flow scheduling in ieee 802.1 qbv time-sensitive networks,” ACM Transactions on Sensor Networks, vol. 20, no. 5, pp. 1–30, 2024. [11] S. Garcia-Cantón, C. Ruiz de Mendoza, C. Cervelló-Pastor, and S. Sallent, “Multi-agent reinforcement learning-based routing and scheduling models in time-sensitive networking for internet of vehicles communications between transportation field cabinets,” Applied Sciences, vol. 15, no. 3, p. 1122, 2025. [12] S. Tan, C. Dong, and W. Wen, “Sharp: A study on safe heterogeneous agent reinforcement learning paradigm for 5g-tsn traffic scheduling,” in 2025 3rd International Conference On Mobile Internet, Cloud Computing and Information Security (MICCIS). IEEE, 2025, pp. 1–9. [13] M. Wen, J. Kuba, R. Lin, W. Zhang, Y. Wen, J. Wang, and Y. Yang, “Multi-agent reinforcement learning is a sequence modeling problem,” Advances in Neural Information Processing Systems, vol. 35, pp. 16 509–16 521, 2022. [14] E. Uysal, O. Kaya, A. Ephremides, J. Gross, M. Codreanu, P. Popovski, M. Assaad, G. Liva, A. Munari, B. Soret et al., “Semantic communications in networked systems: A data significance perspective,” IEEE Network, vol. 36, no. 4, pp. 233–240, 2022. [15] D. G. Morı́n, D. Medda, A. Iossifides, P. Chatzimisios, A. G. Armada, A. Villegas, and P. Peréz, “An extended reality offloading ip traffic dataset and models,” IEEE Transactions on Mobile Computing, vol. 23, no. 6, pp. 6820–6834, 2023. [16] M. Bosk, F. Rezabek, K. Holzinger, A. G. Marino, A. A. Kane, F. Fons, J. Ott, and G. Carle, “Methodology and infrastructure for tsn-based reproducible network experiments,” IEEE Access, vol. 10, pp. 109 203– 109 239, 2022. [17] P. Paymard, S. Paris, A. Amiri, T. E. Kolding, F. S. Moya, and K. I. Pedersen, “Pdu-set scheduling algorithm for xr traffic in multi-service 5g-advanced networks,” in ICC 2024-IEEE International Conference on Communications. IEEE, 2024, pp. 758–763. [18] Z. Zhou, Z. Zhang, J. J. Zeng, and H.-C. Chao, “Multi-agent transformer approach for collaborative task offloading and resource optimization in noma-based vehicular edge computing,” Ad Hoc Networks, p. 104222, 2026. [19] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in International conference on machine learning. PmLR, 2016, pp. 1928–1937. [20] J. G. Kuba, R. Chen, M. Wen, Y. Wen, F. Sun, J. Wang, and Y. Yang, “Trust region policy optimisation in multi-agent reinforcement learning,” arXiv preprint arXiv:2109.11251, 2021.

Related documents

• Cayley Graph Optimization for Scalable Multi-Agent Communication Topologies#10306
• A theoretical model for task routing in mixture-of-expert transformers#271858
• FUSION: Forecast-Embedded Agent Scheduling with Service Incentive Optimization over Distributed Air-Ground Edge Networks#1063822
• Agentic IoT: Architectures, Applications, and Challenges Toward the Internet of Agents#343412
• Snow leopard beetle optimization and deep learning: a hybrid approach for routing and intrusion detection in WSN#149509