QoE-Aware Service Function Chain Orchestration in Space-Air-Ground Integrated Networks for In-Flight Internet

1

QoE-Aware Service Function Chain Orchestration in Space–Air–Ground Integrated Networks for In-Flight Internet

arXiv:2610.17851v1 [cs.NI] 13 Oct 2026

Feng Liu, Bing Du, Member, IEEE, Ziyang Guo, Xiaoqi Zhang, Member, IEEE, Haijun Zhang, Fellow, IEEE

Abstract—In remote regions lacking terrestrial infrastructure, aviation users increasingly demand Internet services comparable to ground cellular networks. This paper proposes a space–air–ground integrated network (SAGIN) architecture that combines ground networks, civil aviation networks, and low Earth orbit (LEO) satellites. Based on real North Atlantic flight data, a heterogeneous aviation Internet is constructed to support diverse aircraft service requirements. A composite Quality of Experience (QoE) metric, characterized by transmission delay, jitter, and content similarity, guides efficient resource utilization. The dynamic connectivity of SAGIN is modeled using a reconfigurable time-expanded graph (RTEG). With Network Function Virtualization (NFV), data delivery is formulated as a Service Function Chain (SFC) deployment across ground stations, relay aircraft, and satellites, resulting in a longterm QoE maximization problem. A tailored Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm incorporating LCSS-based path filtering and softmax-based action selection is developed. Simulation results demonstrate that the proposed approach enables seamless connectivity for heterogeneous aircraft and consistently outperforms game-theoretic and MATD3 methods in terms of QoE. Index Terms—Space-air-ground integrated network (SAGIN), heterogeneous dynamic networks, Quality of Experience (QoE), Multi-Agent Deep Deterministic Policy Gradient (MADDPG), aeronautical internet formation, Service Function Chain (SFC)

I. INTRODUCTION WITH the rapid evolution of communication technologies, mobile users increasingly demand ubiquitous and seamless connectivity, and air travelers now expect in-flight Internet services comparable to terrestrial cellular networks. Current civil aviation communications mainly rely on ground-based Internet Gateway Stations (IGSs) to establish air-to-ground (A2G) links. However, over oceans and remote regions where A2G coverage is limited or unavailable, air-to-space (A2S) communication becomes the primary means of connectivity. Despite its global coverage, A2S communication is constrained by limited satellite bandwidth and high latency. In addition, the high mobility of aircraft—up to 250 m/s—leads to large-scale, highly dynamic network topologies. Under these constraints, providing reliable, low-latency connectivity This study was supported by the National Natural Science Foundation of China under Grant 62471031. (Corresponding author: Bing Du) Feng Liu, Bing Du, Ziyang Guo, Xiaoqi Zhang, and Haijun Zhang are with the School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China. (e-mail: m202310616@xs.ustb.edu.cn; dubing@ustb.edu.cn; M202420965@xs.ustb.edu.cn; zhangxiaoqi@ustb.edu.cn; haijunzhang@ieee.org).

comparable to terrestrial networks during transoceanic flights remains a major challenge. To overcome these limitations, Aerial Ad Hoc Networks (AANETs) have emerged as a promising solution [1]. AANETs enable aircraft-to-aircraft (A2A) communication through long-range, highly directional wireless links, allowing aircraft within communication range to form dynamic, flexible, and cost-effective mesh networks. By exploiting aircraft mobility and spatial distribution, AANETs effectively improve access efficiency across the airspace. Accordingly, extensive efforts have been devoted to enhancing AANET communication performance [2]–[5]. When combined with conventional air-to-ground (A2G) links, A2A communications allow aircraft to relay data to ground gateways via multi-hop paths, thereby extending the effective coverage of ground base stations. Nevertheless, the limited communication range of relay aircraft may leave some aircraft isolated in sparsely populated airspace, posing challenges to achieving global connectivity solely through AANETs. Satellite networks are therefore incorporated into the AANET framework to mitigate this limitation. Through the synergistic integration of abundant ground network resources, the flexibility and mobility of airborne networks, and the ubiquitous coverage provided by satellite systems, the Space–Air–Ground Integrated Network (SAGIN) architecture is formed. This architecture enables seamless wide-area connectivity for aircraft operating in highly dynamic and heterogeneous environments [6]–[10]. Building on this paradigm, recent studies have investigated the use of unmanned aerial vehicles (UAVs) as an additional layer for collaborative resource optimization [9], [11], [12]. However, despite their agility, UAVs suffer from inherent constraints such as limited endurance, payload capacity, and onboard energy, which restrict their applicability in civil aviation scenarios. Consequently, increasing attention has been directed toward exploiting the high density and mobility of commercial aircraft themselves to enable mutual assistance. For instance, Hoffmann et al. proposed a genetic algorithm–based framework for joint optimization of gateway allocation, routing, and scheduling in integrated aircraft–ground–satellite networks, aimed to minimize average packet delay [13]. Similarly, Guo et al. designed a three-layer architecture that combined satellite and ground-assisted aircraft caching to reduce transmission latency for onboard passengers [14]. Most existing studies evaluate aviation Internet performance using traditional Quality of Service (QoS) metrics, such as latency, throughput, and bit error rate. However, QoS-centric

2

approaches exhibit inherent limitations. They primarily emphasize technical indicators while overlooking user perception and content characteristics, and often rely on rigid resource allocation strategies that may result in low utilization efficiency and poor adaptability to highly dynamic network conditions [15]. With the diversification of in-flight services, aviation networks are required not only to support bandwidthintensive applications such as video streaming but also to accommodate delay-sensitive services like video conferencing, alongside more delay-tolerant applications including web browsing, email, social media, and data synchronization [16]. Given the heterogeneous service requirements and varying user tolerance levels, optimization based solely on QoS is insufficient to capture overall system performance. Therefore, this paper advocates the adoption of a more flexible and usercentric metric—Quality of Experience (QoE)—to evaluate and optimize aviation Internet services within the three-layer SAGIN architecture [17]–[19]. Additionally, the high mobility and heterogeneity of SAGIN introduce significant real-time variability and management complexity. To tackle these challenges, Network Function Virtualization (NFV) is adopted to decouple network functions from underlying hardware, thereby improving resource utilization and management efficiency [20]. Within the NFV framework, data delivery from source nodes to destination aircraft can be abstracted as the deployment of a Service Function Chain (SFC), in which each node along a transmission path implements a Virtual Network Function (VNF). Accordingly, routing in SAGIN can be formulated as a multi-task SFC scheduling problem, which is non-convex and NP-hard, motivating the use of reinforcement learning–based solutions. Existing work, such as [21], combines game theory with actor–critic learning to optimize bandwidth allocation for individual data streams. In contrast, this study focuses on the concurrent servicing of multiple aircraft and aims to derive cooperative SFC decisions that maximize the overall network Quality of Experience (QoE). To this end, a Multi-Agent Reinforcement Learning (MARL) framework is introduced, enabling distributed agents to learn coordinated strategies. Specifically, we employ the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, which adopts an actor–critic architecture and exhibits strong convergence and robustness in multi-agent environments [22]. An extension of MADDPG is developed to handle discrete and time-varying action spaces arising from dynamic path availability, by incorporating the Longest Common Subsequence (LCSS)-based path filtering together with a softmax mechanism. Each agent outputs a probability distribution over feasible paths, enabling deterministic execution while preserving differentiability during training. To accurately characterize the highly dynamic topology of SAGIN, a Reconfigurable Time-Extended Graph (RTEG) model is further developed. The aerial layer is constructed from real North Atlantic flight data involving 571 aircraft during peak periods, the satellite layer consists of 66 Iridium LEO satellites, and the ground layer includes 20 coastal gateway stations. These layers are integrated into a unified SAGIN framework, within which aircraft may connect directly

to ground stations, via relay aircraft, or through satellites when terrestrial links are unavailable. On this basis, the proposed space–air–ground integrated Multi-Agent Deep Deterministic Policy Gradient (SAG-MADDPG) algorithm adaptively optimizes SFCs deployment for service-demanding aircraft to enhance overall QoE. Finally, the impacts of service types, buffer capacities at ground stations and relay aircraft, and the number of destination aircraft are systematically evaluated, with comparisons against scenarios excluding satellite assistance. For clarity, the main contributions of this work are as follows: • We propose a three-tier SAGIN architecture in which the aerial network serves as a functional service layer rather than a supplementary component. By exploiting the dynamic density of airborne networks, the architecture enables direct and opportunistic relay transmissions between aircraft, effectively extending ground coverage. An RTEG model captures the highly dynamic topology, while Network Function Virtualization reformulates resource management as a coordinated SFC deployment problem. • QoE is introduced as a performance metric to support diverse service types in the aviation network, allowing flexible and efficient allocation of network resources to better meet passenger needs. The definition of QoE incorporates both objective indicators—such as latency and jitter—and a subjective factor, content similarity. • The proposed SAG-MADDPG algorithm, incorporating LCSS-based path filtering and a softmax mechanism, is employed to derive optimal SFCs for aircraft with data demands. Simulation results based on real North Atlantic flight data show that, when the SAGIN scales to approximately 650 nodes, the proposed approach outperforms the Multi-Agent Delay-Deterministic Policy Gradient (MATD3) and game-theoretic methods, achieving faster convergence and superior QoE. The remainder of this paper is organized as follows. Section II presents the network architecture, the reconfigurable timeextended graph model, and the four SFC transmission modes. Section III introduces the communication model, the QoE formulation, and the optimization objective. Section IV outlines the algorithmic framework and describes the reconstruction of the algorithm based on the problem model. Section V evaluates the performance of the proposed method, and Section VI concludes the paper with a summary of the main findings. The main variables and their corresponding definitions used in this paper are summarized in Table I. II. SYSTEM MODEL In this section, we first present the architecture of the SAGIN. Subsequently, a reconfigurable time-expanded graphbased model of the SAGIN is constructed using real transNorth Atlantic flight trajectory data. Finally, we introduce the four SFC transmission modes supported within the SAGIN framework. A. Network Infrastructure The architecture of the SAGIN is illustrated in Fig. 1. The satellite layer comprises LEO satellites from the Iridium con-

3

TABLE I S UMMARY OF K EY N OTIONS Symbol I X V MD MR VD VR G(t) RG(tn ) η τ J ξ χ χ̃ Pi Pdim i ϱk−hop i ϱS i ϱact i

Definition Set of N Internet gateway stations (IGS), i.e., {I1 , · · · , IN } Set of K satellites, i.e., {X1 , · · · , XK } Set of M air vehicles, i.e., {V1 , · · · , VM } Number of destination aircraft Number of relay aircraft D }, Set of MD destination aircraft, i.e., {V1D , · · · , VM D VD ∈ V R R Set of MR relay aircraft, i.e., {V1 , · · · , VM }, R VR ∈ V The SGAIN dynamic graph The reconstructed snapshot at tn The rate of the change of the edges between the adjacent subgraphs Delay of the entire transmission process Jitter of transmission Similarity of content before and after transmission The pre-transmission content eigenmatrix The post-transmission content eigenmatrix The feasible path selection space for aircraft in VD The number of optional paths for ViD Total number of k-hop path options for ViD , with k ∈ {1, 2, 3} The number of optional A2S paths for ViD The serial number of path selected by ViD

stellation, denoted by the set X = {X1 , · · · , Xk , · · · , XK }, where K is the total number of satellites. The aerial layer consists of civil aviation aircraft cruising at high altitudes, represented by the set V = {V1 , · · · , Vm , · · · , VM }, with M denoting the number of aircraft. The ground layer includes Internet gateway base stations deployed across terrestrial locations, defined as I = {I1 , · · · , In , · · · , IN }, where N is the total number of base stations. Within the SAGIN framework, aircraft can establish three types of communication links: • Air-to-Ground (A2G) enables communications between aircraft and ground base stations. While ground stations offer abundant communication resources, their coverage radius is limited to approximately 200 nmi due to hardware constraints [23]. • Air-to-Air (A2A) supports direct communications between aircraft, with a typical transmission range of 250 nmi. Aircraft in the network can not only exchange information but also act as relays for others. This capability extends the effective coverage of A2G links and enables flexible, dynamic networking based on the mobility of aircraft. In addition, A2A communication generally incurs lower costs compared to satellite-based links. • Air-to-Space (A2S) offers near-global coverage and ensures consistent connectivity for all aircraft. LEO satellites in the Iridium constellation orbit at an altitude of approximately 780 km, and the latency is in the range of 20-250 ms [24]. By fully leveraging the complementary strengths of these three communication modes, the proposed SAGIN architecture aims to deliver reliable, flexible, and cost-effective connectivity

for aircraft, thereby enhancing in-flight service quality for passengers while ensuring robust communication across the airspace. B. Reconfigurable Time-Expanded Graph

Fig. 1.

SAGIN Reconfigurable Time-Expanded Graph

Given the highly dynamic nature of SAGIN, we adopt a reconfigurable time-expanded graph (RTEG) to model its timevarying topology. RTEG represents the evolving network as a compact sequence of graph snapshots by identifying critical topology-change events [25]. In SAGIN, such events mainly arise from link establishment or disruption, typically caused by nodes entering or leaving communication ranges or by link quality crossing operational thresholds. Aircraft positions are predicted in real time using actual flight trajectory data and combined with known locations of ground gateway stations and LEO satellites. This enables continuous inference of the dynamic network topology, denoted by G(t) = {V(t), E(t)}, where V(t) includes satellites, aircraft, and ground stations, and E(t) denotes the set of available wireless links. A set of critical time instants t1 , t2 , . . . , tN is defined, each corresponding to one or more topology-change events. The network is assumed quasi-static between adjacent critical instants. Accordingly, a sequence of reconstructed snapshots {RGt1 , RGt2 , . . . , RGtN } is generated, where each snapshot RGtn represents a static topology over the interval {tn , tn+1 }. These snapshots are then temporally connected to form the complete RTEG model. RTEG construction includes: • Node replication, where each physical node v is replicated as (v, tn ) at every critical time point; • Spatial edges, which reflect valid connectivity within each snapshot and are weighted by propagation distance; • Temporal edges connect the replica nodes (v, tn ) and (v, tn+1 ) of the same physical node, capturing the continuity of node states across adjacent time intervals and the cost of waiting or local persistence. To efficiently identify critical time points, we introduce a topology change sensitivity metric η, defined as the ratio of cumulative link additions and deletions since the last critical time tn to the number of edges in RGtn : η(t) =

NE (RG(t) − RG(tn )) , NE (RG(tn ))

(1)

A threshold ηth (e.g., 1%) is applied: when η(t) ≥ ηth , a new critical time point is created and a new snapshot is

4

reconstructed. An overly small threshold leads to redundant snapshots, while an excessively large one may miss significant topology changes. Proper tuning of ηth therefore balances modeling accuracy and computational complexity.

C. SFC Transmission Modes The SAGIN architecture provides a unified hardware and connectivity platform for Network Function Virtualization (NFV). In this framework, ground Internet gateway stations I and satellites X act as data sources, while aircraft with data demands serve as destination nodes VD ⊂ V. Aircraft without data requests operate as relay nodes VR ⊂ V, forwarding data toward destinations. Virtual Network Functions (VNFs) are deployed on relay aircraft, ground gateways, and satellites, forming Service Function Chains (SFCs), where each SFC is an ordered set of VNFs that together provide an end-to-end network service.

III. NETWORK MODEL AND PROBLEM FORMULATION In this section, we first introduce the communication network model based on the SAGIN architecture. Then, the proposed Quality of Experience (QoE) model is presented. Finally, a QoE-based utility function is formulated to capture the objective of SAGIN SFC optimization at the network level. A. Communication Model In the SAGIN network, both A2A and A2G communications are assumed to operate in the VHF band and share spectral resources, whereas A2S communications are conducted in the L-band [6]. The A2A, A2S and A2G communications are essentially at high altitude, so both can be approximated as Line-of-sight (LOS) propagation links [26]. The network consists of N IGSs, K satellites, and M aircraft, including MD destination aircraft and MR relay aircraft, where MD + MR < M . This paper focuses primarily on downlink transmissions from ground stations or satellites to destination aircraft. The following sections use the A2G downlink as a representative example for illustration. The distance of A2G link connecting the vehicle ViD and the base station In is denoted as dA2G i,n . The channel gain of an A2G GA2G LOS link can be calculated by the free-space i path loss model, and the path loss exponent is taken as 2. V 2 gTI gR c , A2G (4πf di,n ) 2

GA2G = i

Fig. 2.

Overview of the Four SFC Transmission Modes in SAGIN

Figure 2 illustrates the feasible SFC transmission modes within the SAGIN network. As shown, aircraft V1D is located within the coverage area of ground station I3 , enabling airto-ground direct transmission (A2G-DT). In this case, SFC1, I3 → V1D , is completed within a single subgraph. Aircraft V4D can communicate with I2 via relay aircraft V3R and V2R , supporting air-to-air connection transmission (A2A-CT). Accordingly, SFC2, I2 → V2R → V3R → V4D , is also completed within one subgraph. For aircraft V2D , when neither A2G nor A2A links are available, data can be transmitted from ground station I4 to relay aircraft V2R for temporary caching until the SFC V2R → V2D becomes available. This transmission mode corresponds to air-to-air opportunistic transmission (A2A-OT). Consequently, SFC3 I4 → V1R → V2D spans adjacent subgraphs and is constrained by buffer capacity and delay tolerance. In scenarios where both ground stations and relay aircraft are unavailable, as illustrated by V3D , data are delivered via satellite links, enabling air-to-space direct transmission (A2SDT). In this case, SFC4 is given by X1 → V3D .

(2)

where gTI denotes the transmitting antenna gain of the IGS, V gR represent the receiving antenna gain of the air vehicle, c is the speed of light, f is the carrier frequency. The pIt n represents the transmit power of IGS In . According to [27], the signal-to-interference-plus-noise ratio (SINR) is calculated as follows: pIt n Gi , k 2 k∈C int pt Gk + σ

SINRi = P

(3)

i

where σ 2 is the noise variance. Let Ciint be the set of transmitters within the interference range of aircraft ViD , defined by the maximum distance threshold dint max . The total interference is then the sum of the transmission powers from all nodes in this set. Then the maximum achievable transmission rate ri can be derived by: ri = W log2 (1 + SINRi ).

(4)

In (4), W represents the total bandwidth between the transmitter and receiver. During the flight, the destination aircraft ViD establishes a SFC with the ground base station via A2A or A2G, with the help of VNFs, to obtain the required data resources. Let SFCi = In , V1R , · · · , VϑR , ViD define the path In → V1R → · · · → VϑR → ViD , ϑ is the number of relay VNFs in this path. The delay at node ViD equals the total delay across all hops along the path:

5

τi =

P

v∈SFC′i

Lv−,v + δT, Bvach

(5)

where Lv− ,v represents the length of the data transmitted between two nodes. v − denotes the predecessor of node v in SFCi , i.e., the node responsible for forwarding the required data to aircraft v. SFC′i refers  to the subset of nodes in SFCi excluding the IGS, SFC′i = V1R , · · · , VϑR , ViD . Due to the limited coverage and data resources of the base stations and the relay nodes, the aircraft may not have a feasible path at all times, so δ is the number of time slots for which ViD has been waiting for transmission. T represents the length of a time slot. Depending on τi , the jitter experienced by the receiver can be defined as follows: Ji = |τi (t) − τi (t − 1)| .

B. QoE Model The relationship between latency and QoE exhibits a negative correlation characterized by a distinct threshold effect. When latency is low, QoE remains high and stabilizes; however, once latency exceeds a certain threshold, QoE experiences a significant decline. Similarly, the relationship between jitter and QoE is also negatively correlated, demonstrating an approximately exponential decay. Additionally, content similarity is commonly employed to assess the quality of user experience. The definition of similarity varies based on the form of the transmitted content: for text transmission, it refers to semantic similarity [28], while for images and videos, it is typically measured using the Structural Similarity Index Measure (SSIM) [29]. The fundamental principle behind these metrics involves extracting features from the pre-transmission and post-reception content, and then calculating the similarity between these features. Therefore, in this paper, content similarity ξi is defined as follows: χi (χ̃i )T , ||χi || · ||χ̃i ||

(7)

where χi denotes the pre-transmission content eigenmatrix, χ̃i represents the post-transmission content eigenmatrix, (•)T indicates the transpose of a matrix, and ∥•∥ denotes the l2 norm of a matrix. ξi ∈ [0, 1], the similarity is maximized when ξi = 1, indicating no loss or degradation of resources during transmission, whereas ξi = 0 corresponds to the opposite scenario. It can be inferred that ξi exhibits a nonlinear positive correlation with QoE: when ξi exceeds a certain threshold ξ th , QoE increases sharply, while below this threshold, QoE declines gradually and steadily, stabilizing at qlow , and qlow ∈ (0, 1) [29], [30]. The QoE of the aircraft ViD can be formulated by

(8)

In (8), τ th denotes the latency threshold, θ1 and θ2 respectively represent the positive attenuation coefficients associated with delay and jitter, and θ3 denotes the growth rate of similarity. The parameters ρ1 , ρ2 , and ρ3 represent the weighting coefficients corresponding to the contributions of latency τ , jitter J, and content similarity ξ to the receiver’s QoE, respectively. These coefficients satisfy the constraint ρ1 + ρ2 + ρ3 = 1. According to (8), the gain of the whole network can be expressed as U=

(6)

It refers to the variation in the delay of the user in two adjacent transmissions.

ξi =

ρ1 ρ2 + (1 + eθ1 (τi −τ th ) ) eθ2 Ji   (1 − qlow )
+ ρ3 qlow + θ (ξth −ξ ) . i e3

QoEi = 5 ·

MD X

QoEi .

(9)

i=1

C. Problem Formulation The objective of this study is to enable each aircraft with data requests (i.e., the destination nodes in the RTEG G(t)) to select an SFC that meets predefined constraints, such that the overall QoE across the SAGIN network is maximized. The objective function can be written as max U = max

MD X

QoEi .

(10)

P

max ca Ln,k ≤ BSn k∈Cn max

(10a)

τi ≤ τ

(10b)

0 ≤ QoEi ≤ 5

(10c)

i=1

s.t.

In (10a), Cnca denotes the set of aircraft that need to store data at node n. Equation (10a) enforces the buffer-capacity constraint for relay aircraft and IGSs in SAGIN, requiring the cached data of each VNF to remain within its maximum buffer limit BSmax . Equation (10b) ensures that the transmission delay of the selected chain for each destination node remains within its maximum tolerable delay τ max . Equation (10c) defines the permissible range of the QoE for each destination node. IV. ALGORITHM This section begins with an overview of the basic MADDPG algorithm, followed by a Markov Decision Process (MDP) based formulation of the problem, and concludes with the introduction of the proposed SAG-MADDPG algorithm. A. MADDPG Algorithm The Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm extends DDPG from single-agent continuous control to multi-agent environments. In multi-agent settings, non-stationarity caused by simultaneous policy updates degrades learning performance. MADDPG addresses this issue by adopting a centralized training and decentralized

6

execution (CTDE) paradigm [31]. During training, each critic accesses global information to evaluate joint actions, whereas during execution, each actor selects actions based solely on local observations, ensuring scalability and practicality. In this section, “M ” denotes the number of agents, “m” indexes a specific agent, “θ” represents network parameters, and “τ ” is the soft-update rate; these notations differ from those used previously. 1) Actor-Critic Architecture: Each agent m maintains an actor network πm (om (t), |, θπm ) and a critic network Qm (s(t), a(t), |, θQm ), along with corresponding target net′ works πm and Q′m . The actor maps the local observation om (t) to a deterministic action am (t), while the critic evaluates the Q-value using the global state s(t) = o1 (t), . . . , oM (t) and the joint action a(t) = a1 (t), . . . , aM (t). Target networks are updated via soft updates to stabilize training, and experience replay is employed to improve sample efficiency. While critics exploit global information during training, execution remains fully decentralized. 2) Training Process of Actor Network: Training samples are stored in a centralized replay buffer as tuples ⟨s(t), a(t), r(t), s(t + 1)⟩, where r(t) = r1 (t), . . . , rM (t). Each agent optimizes its policy by maximizing the expected Q-value estimated by its critic: ∇θπm Jm = Es [∇θπm πm (om (t)) · ∇âm Qm (s(t), â(t))] . (11) For a mini-batch of Nmb samples, gradients are computed per sample and averaged: N

∇ θ πm J m =

mb 1 X k ∇θπm Jm . Nmb

(12)

k=1

Finally, the actor parameters are updated by gradient ascent: θπm ← θπm + β · ∇θπm Jm .

(13)

3) Training Process of Critic Network: For each sampled transition, target actions are generated by target actor networks, and the target Q-value is computed as: k

′

k ym (t) = rm (t) + γQ′ (skm (t + 1), â′ m (t + 1)|θQm ).

(14)

The temporal-difference (TD) error is the difference between the current critic’s estimate and the target value: k k δm = Q(skm (t), âkm (t)|θQm ) − ym (t).

(15)

The critic is trained by minimizing the mean squared TD error: θ

Qm

Qm

←θ P 1

− α · ∇θQm Lm .

(16)

k 2 ) . where L ∗ m = 2N ∗mb k (δm To stabilize learning, all target networks are updated through a soft update mechanism, where their parameters are periodically synchronized with the main networks using a small update rate τ ,

θtarget ← τ θ + (1 − τ )θtarget .

(17)

B. Problem Reformulation Within the proposed framework, every destination aircraft ViD ∈ VD generating a data request is represented as an autonomous agent that determines its SFC by leveraging observable network states and past learning experience. Each destination aircraft is assumed to issue a request once every three time slots. Given the predictable aircraft positions, the location of each aircraft at each time slot can be estimated. Accordingly, the feasible chain space Pi for each ViD is constructed from subgraphs {RGt1 , RGt2 , RGt3 }, representing the network topology over three consecutive time slots. Each aircraft selects an SFC from P, and the joint decisions of all aircraft in VD determine the final aeronautical internet topology, upon which QoE is evaluated. To solve the optimization problem using MADDPG, the system is formulated as an MDP, defined by its state space, action space, and reward function, which together describe agent-environment interactions during training. 1) State Space: In the system, there are MD destination aircraft. At the beginning of time step t, each destination aircraft ViD , where i ∈ (1, MD ), acquires its path information from the network, which forms its local observation oi (t). Each aircraft can only obtain information relevant to itself. Therefore, the global state s(t) at time step t is constructed by aggregating the local observations oi (t) of all destination aircraft. The definitions of the local observation oi (t) and the global state s(t) are given as follows. 1−hop 2−hop 3−hop S oi (t) = {τi (t−1), Pdim , ϱi , ϱi , ϱi } (18) i (t), ϱi

oi (t) denotes the local observation of the i-th destination aircraft at time step t, where 1) τi (t − 1): The delay of the agent i at the previous data transmission, . 2) Pdim i (t): The number of optional paths for the airplane i at time step t. 3) ϱ1−hop : The number of one-hop paths in the optional i SFCs, i.e., A2G-DT links. 4) ϱ2−hop : The number of two-hop paths in the optional i SFCs containing A2A-CT and A2A-OT links. 5) ϱ3−hop : The number of three-hop paths in the optional i SFCs containing the A2A-CT and A2A-OT links. 6) ϱSi : Number of satellite communication SFCs in the optional paths, including A2S-DT links. Each agent collects its own observation, and the state of the current time step t is available: s(t) = {o1 (t), · · · , oi (t), · · · , oMD (t)}

(19)

and thus, the state space can be expressed as S = {s(t), t ∈ (0, T tot )}. 2) Action Space: Based on oi (t), the corresponding action ai (t) can be obtained: ai (t) = {ϱact i }

(20)

where ϱact indicates the serial number of the SFC in Pi (t) i selected by destination aircraft i. The set of all destination

7

Fig. 3.

SAG-MADDPG Algorithm

aircraft actions at the current time step t is represented as follows: a(t) = {a1 (t), · · · , ai (t), · · · , aMD (t)}

(21)

The action space should be expressed as A = {a(t), t ∈ (0, T tot )}. The output action ai (t) range is determined by the SFC space size Pdim i (t) for each destination aircraft. 3) Reward Function: Once the chains for all destination aircraft are determined, the resulting network topology initiates data transmission, and the QoE for each aircraft is subsequently measured. The primary objective of this study is to maximize the overall QoE of the aviation interconnection network. Since the SFC selection of each aircraft influences not only its own QoE but also that of others, cooperative decisionmaking among aircraft is essential to optimize the global reward. To enable coordinated learning, all aircraft share a common reward signal, encouraging convergence toward a globally optimal policy. The reward function for aircraft i is defined as follows: ri (t) =

MD 1 X QoEm MD m=1

(22)

The reward for each aircraft is the average of the QoE of all destination nodes. And the reward for the current time step t is then r(t) = {r1 (t), · · · , ri (t), · · · , rMD (t)}. 4) Discrete Action Space Adaptation: In this study, the action space is discrete and varies across aircraft and time slots, making discrete MARL methods with fixed action dimensions (e.g., MADQN) unsuitable [32]. To overcome this limitation, we adopt an LCSS-based SFC filtering mechanism and integrate it with a softmax-based action selection strategy, thereby enabling the continuous-action MARL framework MADDPG to operate effectively in the proposed discrete and variable action space. Specifically, the actor network outputs a real-valued vector, which is transformed into a probability distribution over

feasible SFCs using the softmax function. This probabilistic representation enables smooth gradient propagation during training while maintaining compatibility with discrete decision-making. To accommodate the variable action space, we define a maximum action dimension Pdim max = F . The Longest Common Subsequence (LCSS) algorithm is employed to measure structural similarity among candidate chains and filter out highly redundant ones, thereby ensuring sufficient diversity and maintaining high-quality representations within the action space. From the original candidate set P̃i (t), up to F representative chains are retained to form the effective action space Pi (t), whose dimension is given by Pdim = i dim min(P̃dim , F ). Accordingly, the actor network produces P i i logits, which are normalized by the softmax function to obtain a valid probability distribution. If the number of available SFCs is smaller than F , only the first P̃dim outputs are retained i and normalized. During execution, a deterministic policy is adopted: each aircraft selects the SFC with the highest softmax probability as its final action. In contrast, during training, the softmax formulation preserves differentiability with respect to the actor parameters, enabling efficient policy-gradient updates within the MADDPG framework. In this way, the proposed design bridges continuous policy learning and discrete SFC selection in a principled and scalable manner. C. SAG-MADDPG Algorithm Building upon the previously defined state space, action space, and reward function, we propose a multi-agent algorithm named SAG-MADDPG, based on the MADDPG framework, as outlined in Algorithm 1. The overall workflow of Algorithm 1 is illustrated in Fig. 3. As illustrated in Fig. 3 and detailed in Algorithm 1, at the beginning of each slot t, each destination aircraft obtains its initial path information oi (t) from the environment and inputs it into its actor network to generate a probability distribution vector containing all feasible paths, given by ăi (t) = πi (oi (t) | θπi ) + ζ, where ζ denotes the exploration

8

Algorithm 1 SAG-MADDPG Algorithm 1: for episode = 1 to NEP do 2: Each destination aircraft is initialized with an initial observation oi (t). 3: for t = 1 to Nstep do 4: for agent i = 1 to MD do 5: Obtains probability distribution vector ăi = πi (oki | θπi ) + ζ. 6: Takes action ai . 7: end for 8: Execute actions a = {a1 , . . . , aMD }, receives reward r = {r1 , . . . , rMD } by (22), and the next state s(t + 1) = {o1 (t + 1), . . . , oMD (t + 1)}. 9: Store experience ⟨s, a, r, s(t + 1)⟩ in D 10: Randomly select Nmb samples from the experience pool to create a mini-batch. 11: for agent i = 1 to MD do 12: Update actor and critic networks via (13) and (16), respectively. 13: Soft update target networks via (17). 14: end for 15: s(t) ← s(t + 1)) 16: end for 17: end for

latitude, altitude, heading, and speed, assuming a cruising altitude of 10,000 meters. The airborne network topology was built using a distributed cone-based control algorithm [34], ensuring each 120° cone around an aircraft contains at least one connectable neighbor. During the chosen time, 571 aircraft over the North Atlantic formed the airborne layer, classified as relay or destination nodes. The simulation used 100-200ms time slots, with each service cycle spanning three slots (300-600ms), requiring all data transmissions to complete within this period. Algorithm 1 was used to generate SFCs for destination aircraft over 4 service cycles (12 time slots). The A2A and A2G spectrum sharing model in [27] is adopted, allowing the A2A link to utilize idle A2G bandwidth. Due to the heterogeneous SAGIN architecture, communication parameters differ across A2A, A2G, and A2S links. Table II summarizes these parameters, with additional settings in Table III. TABLE II D ISTINCT SIMULATION PARAMETERS FOR A2G, A2A, A2S Parameter

A2G

A2A

A2S

Carrier frequency f (Hz) Transmitting antenna gain gdB Transmit power pt (dBW) Noise variance σ 2 (dBm/Hz) Maximum coverage radius Rmax (nmi)

137M 3.98 17.8 -164.88 200

137M 20.97 12.4 -164.88 250

1.62G 21.99 11.8 -174.99 1350

noise, and then selects the final path based on this. Once all destination aircraft have chosen their paths, the joint action a(t) is executed, resulting in the formation of a new aviation interconnection network topology. Upon the formation of the updated topology, the QoE—corresponding to the instantaneous reward r(t)—is calculated for each aircraft, after which the environment transitions to a new global state s(t + 1). The experience tuple of this interaction is stored in the replay buffer D. A mini-batch is then randomly sampled from D to train the actor and critic networks, following the update rules defined in (13) and (16). Finally, the target networks are updated using a soft update mechanism in (17), and the SAGIN system proceeds to the next time step. The computational complexity and convergence properties of SAG-MADDPG are analyzed in detail in the Appendix. V. SIMULATION Although the SAGIN architecture features a highly dynamic three-layer network, especially in the aerial and satellite layers, the predictable trajectories of LEO satellites and the fixed flight schedules of commercial aircraft under normal conditions enable the accurate estimation of aircraft positions in real-time. In this section, we evaluate the SAGMADDPG algorithm using real-world flight data, including global commercial flight trajectories, 20 terrestrial Internet gateway stations along the North Atlantic coast, and 66 Iridium LEO satellites. Satellite trajectories were simulated with STK software. A 24-hour period from September 17 to 23, 2017, was analyzed, selecting the 17:22 UTC slot on September 18 due to peak aircraft density [33]. Flight paths were modeled as great circle arcs between airports, with aircraft positions calculated from longitude,

TABLE III T HE REMAINING SIMULATION PARAMETERS Parameter

Value

Sector type Air space Number of IGSs on shore Number of total aircraft Number of LEO satellites Flight altitude(km) Pathloss exponent Flight velocity (km/h) The number of destination aircraft Buffer of the IGS (Mbit) Buffer of the relay aircraft (Mbit)

En-Route trans-Atlantic 20 571 66 10 2 900 5-35 300-800 100-500

In (8), the coefficients ρ1 , ρ2 , and ρ3 denote the weights assigned to delay, jitter, and content similarity, respectively. By adjusting these parameters, the characteristics of different types of data streams in practical aviation networks can be reflected. To reflect varying service characteristics, this study considers video, image, and text as representative data types. Similarity evaluations are conducted using publicly accessible pretrained models, with detailed parameter settings and evaluation approaches summarized in Table IV. TABLE IV DATA TYPES AND CHARACTERISTICS Data type

Weighting coefficients

Similarity model

Video Image Text

ρ3 > ρ1 , ρ2 (similarity sensitive) ρ2 > ρ1 , ρ3 (jitter sensitive) ρ1 > ρ2 , ρ3 (delay sensitive)

SSIM [29] ResNet [35] BERT [28]

9

A. Convergence of SAG-MADDPG

multiple aircraft, thereby reducing the efficiency of collaborative learning [37]. Fig. 5 illustrates the optimal network architecture obtained by the SAG-MADDPG algorithm across two consecutive cycles for a scenario with 20 destination aircraft. It can be observed that the optimal topology varies from cycle to cycle, reflecting the dynamic nature of aircraft and satellite positions over time. B. Impact of Service Type

Fig. 4.

Convergence of SAG-MADDPG

We first analyze the impact of different service types on the overall QoE of the SAGIN network. Specifically, we compare scenarios involving the exclusive transmission of video, images, and text across the network, as well as a scenario with mixed transmission of all three data types. Fig. 6(a) shows that video services, despite incurring higher latency and jitter due to large data sizes, still achieve high QoE thanks to strong content similarity. In contrast, text services, characterized by small packet sizes, experience lower latency and jitter, resulting in high QoE. Image services exhibit the lowest QoE overall, likely due to frequent link switching in the highly dynamic network, which introduces considerable jitter. Mixed data transmission maintains moderate values across all metrics, balancing the diverse requirements of different service types. This suggests that user-perceived quality depends on a holistic assessment of multiple performance metrics rather than a single factor. Fig. 6(b) further shows that the A2A-CT ratio is highest for text data and lowest for image data, indicating that A2A-CT is favorable for low-latency transmission but less stable due to the need for multi-VNF coordination. C. Comparison with AGIN

Fig. 5.

Network architecture of SAGIN

In this section, we evaluate the convergence performance of the proposed algorithm and compare it with the MATD3 algorithm. We also present the optimal network architecture identified in the SAGIN system. As shown in Fig. 4, the reward curve during the training process of the SAG-MADDPG algorithm indicates that convergence is approximately achieved after 1500 episodes. The learning rates are set to α = 10−4 for the actor network and β = 10−3 for the critic network. During the initial 200 episodes, the reward distribution is highly scattered due to a deliberately high exploration rate, which encourages agents to explore a broader range of actions. Although the reward curve exhibits fluctuations even in the convergence phase, these are primarily attributed to stochastic factors in the transmission process, such as interference, noise, and Doppler shifts. As a result, even with identical SFCs, performance metrics may vary across episodes, leading to fluctuations in reward values. Nevertheless, despite these fluctuations, the proposed algorithm exhibits superior learning efficiency and stability. In particular, it achieves faster convergence and a higher QoE compared with MATD3 and game-theoretic [36]. The delayed policy update mechanism employed by MATD3 leads to asynchronous learning among

We first evaluate the connectivity of the Air–Ground–Internet (AGIN) network considering only A2G and A2A links, as illustrated in Fig. 7(g). Simulations are conducted using flight data from 17:22 to 17:31 UTC, with a maximum of five hops per chain. In each round, 50 destination aircraft are randomly selected, and the minimum hop count to reach an IGS is recorded over 20 repetitions. As shown in Fig. 7(g), approximately 88% of aircraft can reach an IGS within three hops, while about 2% fail to establish valid connections due to isolation and the lack of nearby relay nodes. Trajectory analysis indicates that such isolation may persist over time, leading to prolonged connectivity disruptions. We then compare the performance of SAG-MADDPG under AGIN and SAGIN scenarios. In AGIN, three transmission modes are supported—A2G-DT, A2A-CT, and A2AOT—with each chain limited to five hops. Network planning is performed over 12 consecutive time slots. Due to inherent topological constraints, some aircraft remain unreachable within five hops; therefore, only aircraft capable of reaching a ground station within this limit are included. To further reduce complexity, the search for 4-hop and 5-hop chains is restricted to aircraft unreachable within three hops. Fig. 7 compares QoE, delay, jitter, and similarity under AGIN and SAGIN for different destination-node counts. As

10

(a)

(b)

Fig. 6. QoE metrics comparison of SAG-MADDPG under varying service types. MD = 20. The buffer size of IGS is 500 Mb, and the buffer size of the relay aircraft is 300 Mb. (a) QoE, delay, jitter and similarity versus network service type. (b) Ratio of A2S-DT, A2G-DT, A2A-CT, A2A-OT transmission modes.

(a)

(b)

(e)

(c)

(f)

(d)

(g)

Fig. 7. Comparison between AGIN and SAGIN network. The network service type is hybrid data transmission. The buffer size of IGS is 500 Mb, and the buffer size of the relay aircraft is 300 Mb. (a) QoE for SAGIN and AGIN versus the number of destination nodes. (b) Delay for SAGIN and AGIN versus the number of destination nodes. (c) Jitter for SAGIN and AGIN versus the number of destination nodes. (d) Similarity for SAGIN and AGIN versus the number of destination nodes. (e) Ratio of 1-hop, 2-hop, 3-hop, 4-hop, 5-hop chains in AGIN. (f) Ratio of A2G-DT, A2A-CT, and A2A-OT transmission modes in AGIN. (g) Coverage capabilities of A2A and A2G links.

shown in Fig. 7(a), the average QoE of both architectures is comparable, with differences below 0.05 except when MD = 10. As illustrated in Fig. 7(b–d), AGIN achieves lower delay due to the absence of high-latency satellite links, whereas SAGIN exhibits lower jitter, benefiting from the greater stability of A2S transmissions. In contrast, AGIN suffers higher jitter caused by unstable multi-hop A2A relaying, where frequent relay changes amplify delay variations. The QoE advantage of SAGIN at MD = 10 is further explained in Fig. 7(e): approximately 25% of the chains involve four or five hops, which in SAGIN are predominantly

A2S-based, providing more stable albeit slower transmissions. Fig. 7(f) additionally highlights the effective utilization of A2A-CT links to support high-quality connectivity demands.

D. Impact of the Number of Destination Aircraft We examine how various performance metrics in the SAGMADDPG algorithm vary with the number of receiver aircraft. To ensure generality, MD receiver aircraft were randomly selected within 60°W–2°E longitude and 25°S–64°N latitude over the North Atlantic.

11

(a)

(b)

(c)

Fig. 8. QoE metrics of SAG-MADDPG under different values of MD . The network service type is pure video data transmission. The buffer size of IGS is 500 Mb, and the buffer size of the relay aircraft is 300 Mb. (a) QoE, delay, jitter, and similarity versus the number of destination aircraft. (b) Ratio of 1-hop, 2-hop, 3-hop, and A2S chain. (c) Ratio of A2S-DT, A2G-DT, A2A-CT, A2A-OT transmission modes.

(a)

(b)

(c)

(d)

Fig. 9. QoE metrics of SAG-MADDPG under varying IGS buffer sizes. MD = 25. The buffer size of the relay aircraft is 300 Mb. The network service type is pure video data transmission. (a) QoE, delay, jitter, and similarity versus the size of the IGS buffer. (b) Ratio of 1-hop, 2-hop, 3-hop, and A2S chain. (c) Ratio of A2S-DT, A2G-DT, A2A-CT, A2A-OT transmission modes. (d) Iterative numbers of achieving the Nash equilibrium.

As shown in Fig. 8(a), increasing the number of receiver aircraft leads to lower similarity, reducing QoE in the similaritysensitive SAGIN. This decline is due to degraded channel conditions and longer transmission distances, as more destinations reduce the number of available relay aircraft MR . Shared relays and longer hops increase interference and latency. Fig. 8(b) shows that 2-hop and 3-hop SFCs dominate, highlighting the role of mutual relaying in extending communication range and maintaining quality. Fig. 8(c) further confirms the central role of A2A links, which constitute a significant portion of total transmissions. E. Impact of IGS Buffer Size We evaluate how varying the base station buffer size BSmax I affects SAG-MADDPG performance and convergence in a similarity-sensitive network with high data demand. As shown in Fig. 9(a), average QoE first increases and then decreases with buffer size. A small buffer limits service capacity at popular base stations, forcing aircraft to reroute frequently and lowering QoE. In contrast, an excessively large buffer increases simultaneous connections, leading to higher interference and degraded performance. Fig. 9(b) and 9(c) show that with a 300Mb buffer, fewer A2G-DT links and more 2-hop chains occur, indicating that buffer constraints push aircraft toward indirect transmissions. Fig. 9(d) shows convergence trends: most configurations converge in 1000 rounds. Notably, BSmax = 500Mb achieves faster, more stable I convergence, while 300Mb requires more exploration.

F. Impact of Relay Buffer Size This section examines how relay aircraft buffer size (BSmax R ) affects overall performance and the convergence of SAG-MADDPG. To highlight its influence, more destination nodes are selected from aircraft farther offshore with fewer nearby nodes. As shown in Fig. 10(a), QoE varies only slightly with buffer size, as destination aircraft (MD = 35) account for less than 10% of the total, and most are in dense regions with multiple chains. However, Fig. 10(b) and (c) show that with BSmax = R 100Mb, the use of A2S links increases, indicating insufficient relay buffer capacity limits A2A-OT transmissions to ground stations. Fig. 10(d) further reveals that smaller buffers lead to longer convergence times, while oversized buffers offer no significant performance benefit. G. Results Summary By summarizing and analyzing the experimental results, the following conclusions can be drawn: 1) When the total number of nodes in the SAGIN network is approximately 650, and the number of destination nodes ranges from 5 to 35, all variants of the proposed Algorithm SAG-MADDPG can converge to a Nash equilibrium within 1500 iterations. SAG-MADDPG with the same network settings achieves better QoE than MATD3 and the game.

12

(a)

(b)

(c)

(d)

Fig. 10. QoE metrics of SAG-MADDPG under varying relay buffer sizes. MD = 35. The network service type is pure video data transmission. The buffer size of IGS is 500 Mb, and the buffer size of the relay aircraft is 300 Mb. (a) QoE, delay, jitter, and similarity versus the relay buffer size. (b) Ratio of 1-hop, 2-hop, 3-hop, and A2S chain. (c) Ratio of A2S-DT, A2G-DT, A2A-CT, A2A-OT transmission modes. (d) Iterative numbers of achieving the Nash equilibrium.

2) Integrating ground, satellite, and airborne networks is essential for reliable communication services in civil aviation. A2A communications take advantage of the mobility of airborne networks to provide flexible connectivity among aircraft, while A2S communications ensure wide-area access and coverage. 3) Variations in service types lead to different user sensitivities toward key performance indicators, resulting in diverse QoE outcomes. To ensure a satisfactory user experience, it is essential to allocate network resources flexibly and dynamically in accordance with the specific requirements of each service. 4) The buffer capacity of VNFs has a significant impact on network formation and performance, and both insufficient and excessive buffering can be detrimental. For 5–35 destination nodes, a base station buffer of 500 Mb is optimal. Similarly, while too small a too-small relay buffer limits opportunistic transmission (OT), increasing it beyond 300 Mb yields no further performance gains under the current settings. VI. CONCLUSION This paper proposes a three-layer heterogeneous SAGIN architecture to enable seamless in-flight Internet services in remote areas without terrestrial infrastructure. By integrating multiple transmission links and Service Function Chains under the NFV framework, the proposed system effectively supports heterogeneous service demands with limited communication resources. A tailored SAG-MADDPG algorithm is developed to solve the resulting multi-task SFC scheduling problem with the objective of maximizing overall Quality of Experience. Simulation results based on real flight data show that the proposed algorithm converges reliably within a finite number of iterations and consistently outperforms benchmark schemes in terms of QoE. These results demonstrate the effectiveness and robustness of the proposed framework for large-scale and highly dynamic aviation networking scenarios. With the continued growth of the aviation tourism industry and a surge in airborne users, more computational tasks will emerge at aircraft terminals. Future research may explore task offloading and edge processing strategies at the terminal level to further enhance service quality and network efficiency.

A PPENDIX C OMPUTATIONAL C OMPLEXITY AND C ONVERGENCE A NALYSIS OF SAG-MADDPG In this appendix, we analyze the computational complexity and convergence properties of the proposed SAG-MADDPG algorithm. A. Computational Complexity Consider a SAGIN with MD destination aircraft, each modeled as an agent equipped with an actor–critic pair under the MADDPG framework. Let Pdim max denote the maximum number of feasible SFCs retained after LCSS-based filtering, and let Nmb be the mini-batch size. In each training iteration, the dominant computational overhead arises from forward and backward propagation in the actor and critic networks. The overall per-iteration complexity,
therefore, scales as O MD · Nmb · Pdim . max The LCSS-based SFC filtering incurs an additional cost of O(|Pi |2 · L) per agent, where |Pi | is the original candidate set size and L is the average SFC length. Since this procedure is triggered only upon significant topology changes in the reconfigurable time-expanded graph, its amortized cost is negligible compared with the online training complexity. Overall, SAG-MADDPG exhibits polynomial-time complexity and scales linearly with MD , making it suitable for large-scale SAGIN deployments. B. Convergence Properties The SAG-MADDPG algorithm follows the deterministic policy gradient framework under a centralized training and decentralized execution (CTDE) paradigm. During training, each agent employs a centralized critic with access to global state and joint actions, effectively mitigating the non-stationarity induced by concurrent multi-agent learning. Under standard assumptions in deep reinforcement learning—such as bounded rewards, sufficiently small learning rates, and expressive function approximators—the deterministic policy gradient method converges to a locally optimal policy. In the cooperative multi-agent setting considered here, the centralized critic guides learning toward a local Nash equilibrium of the SAGIN topology optimization game.

13

Moreover, experience replay and soft target-network updates reduce gradient variance and smooth parameter evolution, significantly enhancing training stability. Simulation results in Section V confirm this analysis: training rewards converge within a finite number of episodes, and SAG-MADDPG achieves faster convergence and higher QoE than the MATD3 baseline. R EFERENCES [1] E. Sakhaee, A. Jamalipour, and N. Kato, “Aeronautical ad hoc networks,” in IEEE Wireless Communications and Networking Conference, 2006. WCNC 2006., vol. 1, 2006, pp. 246–251. [2] Q. Luo and J. Wang, “Multiple qos parameters-based routing for civil aeronautical ad hoc networks,” IEEE Internet of Things Journal, vol. 4, no. 3, pp. 804–814, 2017. [3] J. Zhang, D. Liu, S. Chen, S. X. Ng, R. G. Maunder, and L. Hanzo, “Multiple-objective packet routing optimization for aeronautical ad-hoc networks,” IEEE Transactions on Vehicular Technology, vol. 72, no. 1, pp. 1002–1016, 2023. [4] D. Liu, J. Cui, J. Zhang, C. Yang, and L. Hanzo, “Deep reinforcement learning aided packet-routing for aeronautical ad-hoc networks formed by passenger planes,” IEEE Transactions on Vehicular Technology, vol. 70, no. 5, pp. 5166–5171, 2021. [5] B. Du, X. Di, D. Liu, and H. Ning, “Coalitional dynamic graph game for aeronautical ad hoc network formation,” IEEE Internet of Things Journal, vol. 9, no. 8, pp. 5773–5784, 2022. [6] J. Liu, Y. Shi, Z. M. Fadlullah, and N. Kato, “Space-air-ground integrated network: A survey,” IEEE Communications Surveys & Tutorials, vol. 20, no. 4, pp. 2714–2741, 2018. [7] J. Ye, S. Dang, B. Shihada, and M.-S. Alouini, “Space-air-ground integrated networks: Outage performance analysis,” IEEE Transactions on Wireless Communications, vol. 19, no. 12, pp. 7897–7912, 2020. [8] H. Zhang, X. Yao, K. Xu, Z. Wu, W. Li, Y. Lu, and J. Yang, “Trustworthiness evaluation toward 6g support of space-air-ground integrated network,” IEEE Wireless Communications, vol. 32, no. 2, pp. 34–40, 2025. [9] H. Zhang, W. Song, X. Liu, M. Sheng, W. Li, K. Long, and O. A. Dobre, “Intelligent channel prediction and power adaptation in leo constellation for 6g,” IEEE Network, vol. 37, no. 2, pp. 110–117, 2023. [10] H. Zhang, J. Xu, X. Liu, K. Long, and V. C. M. Leung, “Joint optimization of caching placement and power allocation in virtualized satellite-terrestrial network,” Trans. Wireless. Comm., vol. 22, no. 11, p. 7932–7943, Nov. 2023. [Online]. Available: https://doi.org/10.1109/TWC.2023.3257178 [11] C. Huang, G. Chen, P. Xiao, Y. Xiao, Z. Han, and J. A. Chambers, “Joint offloading and resource allocation for hybrid cloud and edge computing in sagins: A decision assisted hybrid action space deep reinforcement learning approach,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 5, pp. 1029–1043, 2024. [12] P. Qin, M. Fu, Y. Fu, R. Ding, and X. Zhao, “Collaborative edge computing and program caching with routing plan in c-noma-enabled spaceair-ground network,” IEEE Transactions on Wireless Communications, vol. 23, no. 12, pp. 18 302–18 315, 2024. [13] F. Hoffmann, D. Medina, and A. Wolisz, “Joint routing and scheduling in mobile aeronautical ad hoc networks,” IEEE Transactions on Vehicular Technology, vol. 62, no. 6, pp. 2700–2712, 2013. [14] Z. Guo, F. Tang, X. Chen, L. Luo, and M. Zhao, “Deep-reinforcementlearning-based content caching in satellite–terrestrial assisted airborne communications,” IEEE Internet of Things Journal, vol. 11, no. 12, pp. 22 779–22 789, 2024. [15] A.-E. M. Taha, “Quality of experience in 6g networks: Outlook and challenges,” Journal of Sensor and Actuator Networks, vol. 10, no. 1, 2021. [Online]. Available: https://www.mdpi.com/2224-2708/10/1/11 [16] L. Toka, E. A. Papp, T. Cinkler, I. Gódor, and L. Hévizi, “Dimensioning space-air-ground integrated networks for in-flight 6g slice orchestration,” Vehicular Communications, vol. 51, p. 100866, 2025. [Online]. Available: https://www.sciencedirect.com/ science/article/pii/S2214209624001414 [17] M. Fiedler, T. Hossfeld, and P. Tran-Gia, “A generic quantitative relationship between quality of experience and quality of service,” IEEE Network, vol. 24, no. 2, pp. 36–41, 2010. [18] J. Zhang and N. Ansari, “On assuring end-to-end qoe in next generation networks: challenges and a possible solution,” IEEE Communications Magazine, vol. 49, no. 7, pp. 185–191, 2011.

[19] H. Gacanin and M. Wagner, “Artificial intelligence paradigm for customer experience management in next-generation networks: Challenges and perspectives,” IEEE Network, vol. 33, no. 2, pp. 188–194, 2019. [20] J. He, N. Cheng, Z. Yin, C. Zhou, H. Zhou, W. Quan, and X.-H. Lin, “Service-oriented network resource orchestration in space-air-ground integrated network,” IEEE Transactions on Vehicular Technology, vol. 73, no. 1, pp. 1162–1174, 2024. [21] H. He, K. Sun, S. Chen, X. Jiang, R. Zhu, and J. Yang, “Deep reinforcement learning based routing for non-cooperative multi-flow games in dynamic aanets,” IEEE Transactions on Vehicular Technology, vol. 73, no. 12, pp. 19 495–19 510, 2024. [22] H. Sun, Y. Zhou, H. Zhang, L. Ale, H. Dai, and N. Zhang, “Joint optimization of caching, computing, and trajectory planning in aerial mobile edge computing networks: An maddpg approach,” IEEE Internet of Things Journal, vol. 11, no. 24, pp. 40 996–41 007, 2024. [23] V. Megas, S. Hoppe, M. Ozger, D. Schupke, and C. Cavdar, “A combined topology formation and rate allocation algorithm for aeronautical ad hoc networks,” IEEE Transactions on Mobile Computing, vol. 23, no. 1, pp. 12–28, 2024. [24] M. Vondra, M. Ozger, D. Schupke, and C. Cavdar, “Integration of satellite and aerial communications for heterogeneous flying vehicles,” IEEE Network, vol. 32, no. 5, pp. 62–69, 2018. [25] Z. Jia, Y. Cao, L. He, Q. Wu, Q. Zhu, D. Niyato, and Z. Han, “Service function chain dynamic scheduling in space-air-ground integrated networks,” IEEE Transactions on Vehicular Technology, vol. 74, no. 7, pp. 11 235–11 248, 2025. [26] Q. Feng, J. McGeehan, E. Tameh, and A. Nix, “Path loss models for air-to-ground radio channels in urban environments,” in 2006 IEEE 63rd Vehicular Technology Conference, vol. 6, 2006, pp. 2901–2905. [27] B. Du, R. Xue, L. Zhao, and V. C. Leung, “Coalitional graph game for air-to-air and air-to-ground cognitive spectrum sharing,” IEEE Transactions on Aerospace and Electronic Systems, vol. 56, no. 4, pp. 2959– 2977, 2020. [28] L. Yan, Z. Qin, C. Li, R. Zhang, Y. Li, and X. Tao, “Qoe-based semantic-aware resource allocation for multi-task networks,” Trans. Wireless. Comm., vol. 23, no. 9, p. 11958–11971, Sep. 2024. [Online]. Available: https://doi.org/10.1109/TWC.2024.3386807 [29] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. [30] X. Li, J. Zhang, and X. Liu, “Mode selection and resource allocation for d2d communications with tradeoff of throughput and user satisfaction,” IEEE Transactions on Communications, vol. 71, no. 9, pp. 5607–5620, 2023. [31] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multiagent actor-critic for mixed cooperative-competitive environments,” CoRR, vol. abs/1706.02275, 2017. [Online]. Available: http://arxiv.org/ abs/1706.02275 [32] T. T. Nguyen, N. D. Nguyen, and S. Nahavandi, “Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications,” IEEE Transactions on Cybernetics, vol. 50, no. 9, pp. 3826–3839, 2020. [33] OAG Aviation Worldwide Ltd., “Flight data sets,” Available: https:// www.oag.com/flight-data-sets, 2025, accessed: Jun. 9, 2025. [34] L. Li, J. Halpern, P. Bahl, Y.-M. Wang, and R. Wattenhofer, “A conebased distributed topology-control algorithm for wireless multi-hop networks,” IEEE/ACM Transactions on Networking, vol. 13, no. 1, pp. 147–159, 2005. [35] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778. [36] J. Ackermann, V. Gabler, T. Osa, and M. Sugiyama, “Reducing overestimation bias in multi-agent domains using double centralized critics,” CoRR, vol. abs/1910.01465, 2019. [Online]. Available: http://arxiv.org/abs/1910.01465 [37] H. Nekoei, A. Badrinaaraayanan, A. Sinha, M. Amini, J. Rajendran, A. Mahajan, and S. Chandar, “Dealing with non-stationarity in decentralized cooperative multi-agent deep reinforcement learning via multi-timescale learning,” 2023. [Online]. Available: https: //arxiv.org/abs/2302.02792

Related documents

• QoE-Aware Service Function Chain Orchestration in Space-Air-Ground Integrated Networks for In-Flight Internet#1343055
• A Bi-Objective Routing Framework for Hybrid Terrestrial-Satellite Quantum Networks#1014153
• A Bi-Objective Routing Framework for Hybrid Terrestrial-Satellite Quantum Networks#978386
• Spectrum Management and Broadcasting: Current Issues#1275498
• Regulatory Pricing Rules To Neutralize Network Dominance,#942300