arXiv CSopen access

Learning-Based Spectrum Cartography in Low Earth Orbit Satellite Networks: An Overview

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

1

Learning-Based Spectrum Cartography in Low Earth Orbit Satellite Networks: An Overview

arXiv:2605.10359v1 [cs.NI] 11 May 2026

Liping Tao, Xindi Tong, and Chee Wei Tan, Senior Member, IEEE,

Abstract—Low earth orbit (LEO) satellite networks are emerging as a key infrastructure for global connectivity and spacebased sensing. Many tasks in such systems can be formulated as measurement-set-to-spatial-inference problems, where spatial variables are inferred from sparse and heterogeneous wireless observations. Spectrum cartography provides a unifying framework for this paradigm, encompassing representative tasks such as satellite-assisted localization and radio map reconstruction, as well as map-informed resource allocation. Yet the highly dynamic orbital geometry, complex propagation conditions, and reliability-varying nature of LEO measurements pose fundamental challenges for traditional model-driven and interpolation-based methods. This article surveys the literature from 1964 to 2026 on learning-based spectrum cartography as applied to LEO satellite networks, with a particular focus on attention mechanisms as a principled operator for adaptive and reliability-aware measurement fusion across localization, radio map reconstruction, and resource allocation tasks. We review modeling foundations and key challenges of representative tasks, and analyze how attention-based learning enables flexible fusion of heterogeneous measurements for both inference and map-informed decision-making. Representative formulations and simulation studies are provided to illustrate the framework and demonstrate its effectiveness, offering a unified perspective for measurement-driven inference and decision-making in LEO satellite networks. Index Terms—Spectrum cartography, Low earth orbit satellite, Machine learning, Attention, Radio map, Satellite localization.

I. INTRODUCTION Recent deployment of fifth-generation (5G) networks have significantly improved communication capacity, yet its reliance on terrestrial infrastructure limits coverage in rural, remote, and maritime regions. This has motivated sixth-generation (6G) systems targeting seamless global connectivity and integrated sensing and communication [1]. The rise of AInative 6G platforms, such as NVIDIA’s Sionna [2], further reflects a shift toward data-driven and simulation-integrated network design [3]. In this context, low earth orbit (LEO) satellite constellations have emerged as a key enabler of 6G and beyond [4]. Compared with geostationary systems, LEO satellites operate at lower altitudes, offering reduced latency, improved link budgets, and flexible beam coverage [5]. With the rapid deployment of mega-constellations such as Starlink and OneWeb [6], LEO networks are evolving into highly dynamic systems with rapidly varying orbital geometry, enabling applications in global broadband, internet of things (IoT) connectivity, positioning, and remote sensing [7], [8]. Liping Tao ([email protected]), Xindi Tong ([email protected]), and Chee Wei Tan ([email protected]) are with the College of Computing and Data Science, Nanyang Technological University, Singapore, 639798. Liping Tao and Xindi Tong contributed equally to this work.

Among emerging paradigms for LEO satellite networks, spectrum cartography (SC) provides a unifying framework for constructing spatial radio intelligence from sparse measurements, guided by the principles of coherence, coverage, and structure [9]–[11]. Within this framework, radio map reconstruction, LEO satellite localization, and map-informed resource allocation can be viewed as complementary tasks forming a unified inference-and-decision pipeline. Specifically, radio map reconstruction infers the spatial–spectral–temporal distribution of radio signals from sparse observations, enabling applications such as spectrum sharing, interference management, and network planning [12], [13]. LEO satellite localization estimates user position from heterogeneous measurements such as range and signal strength, with performance governed by noise and rapidly varying satellite geometry [14]– [16]. Beyond inference, map-informed representations support downstream decision-making: resource allocation strategies, such as water-filling, beam switching, and beam hopping, can leverage spatial and temporal radio intelligence to enable topology-aware power and spectrum management in dynamic LEO networks [17], [18]. These three tasks share a common measurement-driven structure in which spatial radio intelligence, once constructed, bridges sensing and resource management across the network. Many spectrum cartography tasks share a common measurement-driven structure that spans both spatial inference and downstream decision-making. Two representative examples are radio map reconstruction and LEO satellite localization, which, although often studied separately, exhibit similar characteristics. Localization estimates a low-dimensional state from satellite observations whose utility depends on noise, geometry, and temporal context [19]–[21], while radio map reconstruction infers a spatial field from sparse measurements shaped by anisotropic correlation, sensing geometry, and reliability [22]–[24]. In both cases, inputs form heterogeneous measurement sets with unequal informativeness and uncertainty, rather than regular grid-structured data. This shared structure motivates a unified view of these tasks as measurement-set-tospatial-inference problems, requiring adaptive, context-aware mechanisms for information selection and fusion. However, the irregular and reliability-varying nature of measurements, driven by orbital geometry and propagation conditions [25], [26], poses challenges for both model-driven methods and learning architectures with predefined structures, such as convolutional neural networks (CNNs) [27] and graph neural networks (GNNs) [28]. This motivates learning-based approaches that operate directly on unstructured measurement sets, enabling adaptive inference beyond grid- or graph-constrained representations.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

SC in LEO Satellite Networks LEO sattellite

Sensor

2

Machine Learning-Based Method Measurement

Attention

Query

Ground surface

...

...

...

...

Adaptive weighting

Representative Applications

Key

Value

Gated Mechanism

Weighted aggregation

Reliabilityaware

Fig. 1. Unified view of learning-based spectrum cartography (SC) in low earth orbit (LEO) satellite networks. Left: measurement-driven sensing with LEO satellites. Middle: attention-based learning enables adaptive and reliability-aware aggregation of heterogeneous measurements. Right: representative applications, including radio map reconstruction, LEO satellite localization, and map-informed resource allocation.

To address these challenges, attention mechanisms have emerged as an effective framework for adaptive information aggregation. Originally developed for neural machine translation [29], [30] and later generalized by Transformers [31], attention can be viewed as a data-dependent weighted aggregation operator, closely related to the Nadaraya–Watson estimator [32], [33] but extended with learned, context-dependent similarity functions. This operator view highlights key properties that make attention well suited for LEO spectrum cartography. First, it naturally handles permutation-invariant, set-structured inputs, enabling flexible processing of irregular measurements [34], [35]. Second, it enables reliabilityaware fusion by emphasizing informative observations and suppressing noisy or geometrically weak ones under geometrydependent measurement quality. Third, it captures global interactions through self-attention, modeling long-range dependencies induced by inter-satellite coordination, multi-beam interference, and dynamic network topology [31], [36]. Recent work further shows that attention-based models can directly perform spatial interpolation from irregular measurements in radio map estimation, outperforming grid-based approaches while offering lower complexity, full spatial resolution, and inherent equivariance. For example, Viet et al. [37] proposed a gridless spatial transformer estimator with state-of-the-art performance and active sensing capability, while Tao et al. [38] developed an accelerating attention kernel regression framework with learned preconditioning that improves computational efficiency while maintaining high reconstruction accuracy. Together, these properties position attention as a principled and flexible operator for measurement-driven spectrum cartography in LEO satellite networks. Motivation. These observations position attention-based models as a promising paradigm for measurement-driven spectrum cartography in LEO satellite networks, spanning spatial inference tasks (e.g., radio map reconstruction and localization) and map-informed resource allocation (cf. Fig. 1). However, their integration into LEO systems remains underexplored, with key challenges including reliability-aware modeling under complex propagation, scalability to large and dynamic constellations, and principled integration with physics-based inference models.

Scope and Contribution. This article surveys the literature from 1964 to 2026 on learning-based spectrum cartography for LEO satellite networks, with a focus on attention-based methods for adaptive and reliability-aware measurement fusion across localization, radio map reconstruction, and resource allocation. The main contributions are summarized as follows: • We establish spectrum cartography as a unifying framework for measurement-driven learning and inference in LEO satellite networks, bridging spatial inference and map-informed decision-making under heterogeneous and reliability-varying observations. • We systematically review representative applications, including radio map reconstruction, LEO satellite localization, and map-informed resource allocation such as water-filling power allocation problem, and analyze their shared structure as heterogeneous measurement-set-tospatial-inference problems with unequal informativeness and geometric uncertainty. • We provide a comprehensive analysis of attention mechanisms for spectrum cartography, characterizing them as principled operators for adaptive information selection, reliability-aware weighting, and context-dependent fusion, illustrated with representative examples. • We investigate the integration of attention-based models into LEO satellite systems through representative methods and simulations, and identify future directions toward scalable, physics-aware, and uncertainty-informed designs. The source code is publicly available.1 Organization. As shown in Fig. 2, Section II introduces spectrum cartography, including its definition, core components, principles, and both classical and learning-based methods. Section III presents representative applications in LEO satellite networks, including radio map reconstruction, localization, and map-informed resource allocation, and analyzes their modeling perspectives and shared structural challenges. Section IV reviews the fundamentals of attention mechanisms and establishes their suitability for spectrum cartography tasks. Section V develops attention-based models, methods, and simulation studies for LEO satellite networks. Finally, Section VI concludes the paper and outlines directions for future research. 1 Source code is available at https://github.com/convexsoft/LearnSCLEO.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

3

Section I

Section II

Section III

Section IV

Section V

Section VI

Introduction

Spectrum Cartography

Representative Applications

Attention-Based Learning

Models And Analysis

Conclusion

A. Core Components B. Key Principles C. Classical Method D. Deep Learning-Based Method

A. Radio Map Reconstruction B. LEO Satellite Localization C. Map-Informed Resource Allocation

A. Nadaraya--Watson Estimator B. Attention in Deep Learning C. Transformer Attention D. Gated Attention E. Software and Framework F. Capabilities and Limitations G. Attention-Based LEO Satellite Localization H. Case Study

A. LEO Satellite Localization via Attention B. Radio Map Reconstruction via Learnable Attention

Fig. 2. Organization of the paper, progressing from spectrum cartography fundamentals and representative applications to attention-based models and analysis.

II. SPECTRUM CARTOGRAPHY Spectrum cartography studies the reconstruction of spatially continuous radio fields such as received signal strength (RSS), pathloss, interference power, or power spectral density (PSD) from sparse and geo-tagged measurements, aiming to characterize radio-frequency (RF) power utilization over a geographical region. The resulting radio maps capture spectrum usage, interference propagation, and channel characteristics across space, frequency, and time [39], [40]. Over time, spectrum cartography has evolved from spatial interpolation methods such as Kriging [41] and kernel-based approaches to regularized inverse formulations exploiting sparsity and low-rankness, and more recently to deep generative modeling [42], [43]. This evolution positions spectrum cartography as a measurementdriven field inference problem under physical constraints, extending beyond static interpolation toward adaptive, contextdependent inference where measurement importance depends on geometric relevance. Modern frameworks further encompass the full radio map lifecycle, supporting downstream tasks such as localization and interference management [44], [45]. In the following section, we introduce the core components, principles, and representative methods. A. Core Components To operationalize spectrum cartography, we decompose it into five interrelated components: radio field representation, measurements and side information, map inference, map updating and surveying, and map exploitation (cf. Fig. 3). Early work focused on constructing and maintaining radio maps from spatially distributed observations [44]. Subsequent research has extended this pipeline to multi-domain modeling across space, frequency, and time [39], hybrid model- and data-driven inference under shadowing [40], side-informationaware prediction from environmental context such as building maps and aerial imagery [43], [46], and active data acquisition via autonomous sensing agents [47]. A concise summary is provided in Table I. 1) Radio-Field Representation: Radio-field representation is a core component of spectrum cartography, defining the target radio field to be inferred. Spectrum cartography models the radio environment as a continuous function over space, capturing quantities such as received power, pathloss, channel

gain, or PSD [44]. This representation can be extended to multi-domain settings. For example, PSD cartography models the field as a joint space–frequency function, reflecting both spatial and spectral characteristics [39]. More generally, this component determines the form of the inference target, e.g., scalar, multi-frequency, or spatiotemporal fields. 2) Measurements and Side Information: Measurements and side information constitute a core component of spectrum cartography, defining the inputs available for radio field inference. Classical approaches rely on sparse geo-referenced measurements [39], [48], [49], but the input space has steadily expanded. Location-free methods replace explicit coordinates with signal-derived features under unreliable positioning [50], while learning-based approaches incorporate auxiliary information such as environmental geometry, transmitter descriptors, and aerial imagery [46], [51]. More recent formulations further cast spectrum cartography as a fusion problem, integrating uncertain priors with sparse but reliable observations [43]. As a result, spectrum cartography is governed not only by measurement locations, but also by the richness of available contextual information. 3) Map Inference: Map inference is a central component of spectrum cartography, transforming incomplete and heterogeneous inputs into a continuous radio field. It encompasses tasks such as map construction from measurements or priors, reconstruction from sparse samples, completion of partially observed maps or tensors, and prediction at unseen locations or conditions. These tasks are addressed through diverse paradigms, including kernel-based regression [38], [52], semiparametric models [48], adaptive basis expansions [49], tensor completion [40], and learning-based predictors [43], [46]. Despite their methodological differences, all aim to infer an unobserved radio field from partial evidence. 4) Map Updating and Surveying: Map updating and surveying extend spectrum cartography beyond static estimation. Spectrum cartography inherently involves maintaining radio maps as new data becomes available [44], and further encompasses active surveying, where sensing locations are adaptively selected based on current estimates and uncertainty [47]. Consequently, spectrum cartography includes not only passive reconstruction from fixed datasets, but also online updating, uncertainty-aware refinement, and adaptive data acquisition.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

4

Five Core Components of Spectrum Cartography Located samples

P(x) Power

Tx1

Online update

Tx1

L(x) Pathloss

Sparse input

Tx2

Map-aided planning

Tx1

Tx2

Tx1

Localization

Inferred field Tx2

Tx2

S(x, f) PSD

(1) Field Representation (2) Measurements and Side Information

Sharing zone

Next sensing location

Signal features + context

(3) Map Inference

(4) Map Updating and Surveying

(5) Map Exploitation

Fig. 3. Core components of spectrum cartography. The framework comprises five key components: radio-field representation, measurements and side information, map inference, map updating and surveying, and map exploitation.

5) Map Exploitation: Map exploitation focuses on leveraging inferred radio fields for downstream decision-making. Radio maps support applications such as spectrum sharing, interference management, network planning, and localization [44]. Recent approaches further integrate map inference with optimization, treating learned radio maps as differentiable representations for system-level design and control [43]. Thus, spectrum cartography serves not only as a reconstruction tool, but also as a bridge from sparse observations to actionable environmental knowledge. B. Key Principles Inspired by the notion of information cartography [9], spectrum cartography can be understood through three key principles: coherence, coverage, and structure (cf. Fig. 4). These principles characterize when reliable reconstruction from sparse measurements is possible and provide guidance for the design of learning-based inference methods. 1) Coherence: Coherence describes the spatial correlation of the radio field, where nearby locations tend to exhibit similar propagation characteristics due to shared large-scale effects such as pathloss and shadowing. This property enables generalization from sparse measurements and forms the foundation of classical interpolation methods. In spectrum cartography, coherence is particularly important for radio map reconstruction, where spatial smoothness allows reliable interpolation between nearby measurements. Example 1: Consider two measurement points at (0, 0) and (1, 0) meters with RSS values −60 dBm and −62 dBm. Due to spatial coherence, the RSS at (0.5, 0) is expected to lie between them, e.g., around −61 dBm. If a prediction at (0.5, 0) deviates significantly (e.g., −75 dBm), it violates the coherence assumption. 2) Coverage: Coverage refers to how well the measurement locations span the region of interest. Accurate reconstruction requires sufficient spatial coverage; otherwise, predictions rely on extrapolation and become unreliable. This principle is critical in both radio map construction and LEO satellite localization, where uneven or sparse measurements can lead to large uncertainty in unobserved regions. Example 2: Suppose measurements are only collected along the line y = 0 in a 10 × 10 m2 area, with values ranging from −60 to −70 dBm. While interpolation along this line is

Coherence

Coverage

Structure

spatial correlation

-60 dBm (0, 0)

Nearby locations

-61 dBm ? (0.5, 0)

-62 dBm (1, 0)

Similar propagation

void

Sparse coverage

extrapolation

-65 dBm

Extrapolation error

Proximity

-90 dBm

Similarity

Fig. 4. Key principles of spectrum cartography. Coherence captures spatial correlation, coverage reflects measurement sparsity, and structure emphasizes propagation-driven similarity.

accurate, predicting the RSS at (5, 8) relies on extrapolation and may incur large error (e.g., predicted −65 dBm vs. true −80 dBm), due to insufficient coverage. 3) Structure: Structure captures the geometry- and environment-dependent patterns that govern the radio field, including obstacles, propagation paths, and measurement geometry. Unlike coherence, which assumes smooth variation, structure explains systematic deviations caused by physical constraints. This principle is especially important in complex environments, such as urban radio mapping with blockage and LEO satellite localization where measurement informativeness depends on satellite geometry. Example 3: Consider two nearby locations (2, 2) and (2.5, 2) meters separated by a building. The measured RSS values may be −65 dBm and −90 dBm, respectively, due to blockage, despite their proximity. In LEO localization, two satellites may both have measurement noise of 1 m, but due to different geometric configurations, one may reduce position error by 5 m while the other only contributes 1 m. These differences cannot be explained by coherence alone but are governed by underlying structure. C. Classical Method Spectrum cartography aims to reconstruct spatial, spectral, or joint spatio-spectral radio maps from geographically distributed yet typically sparse measurements. A generic observation model can be written as: Y = PΩ (X) + N,

(1)

where X is the complete radio map, Ω denotes the set of sampled locations, PΩ (·) is the sampling operator, and N

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

5

TABLE I C ORE COMPONENTS OF SPECTRUM CARTOGRAPHY. Component Radio-field representation Measurements & side information

Map inference

Map updating & surveying

Map exploitation

Key Roles and Elements • Target: continuous radio field, e.g., power, PSD, pathloss, or channel gain • Domain: space, or joint space–frequency–time • Role: defines the inference objective and field structure • Inputs: spatial samples and signal-derived features • Context: geometry, aerial data, and environmental priors • Property: sparse, heterogeneous, and often incomplete • Tasks: construction, reconstruction, completion, and prediction • Methods: kernel-based, model-based, and learning-based approaches • Goal: infer the unobserved radio field from partial evidence • Updating: online refinement as new measurements arrive • Awareness: uncertainty-aware estimation and revision • Surveying: adaptive sensing and measurement acquisition • Applications: spectrum sharing, interference management, and localization • Planning: network design and coverage optimization • Integration: optimization and control using learned radio maps

denotes measurement noise. Classical methods address this illposed inverse problem by imposing handcrafted priors such as smoothness, sparsity, or low rank [39], [40]. This line of research traces back to early interference cartography and radio map construction, where spatial maps of interference or received power were built from geo-localized measurements for cognitive radio applications [53]. Kriging. A representative classical approach is kriging, which models the radio map as a random field and yields a linear minimum mean-square error estimator [44], [53], [54]: p̂(s0 ) = µ(s0 ) + k(s0 )⊤ K + σn2 I

−1

(y − µ),

(2)

where µ(·) is the mean function, K is the covariance matrix over sampled locations, and k(s0 ) contains cross-covariances with the target location s0 . Kriging is attractive for its statistical interpretability and uncertainty quantification, but its performance depends strongly on the assumed covariance model [40]. Kernel Methods. Another important family includes kernel and basis-expansion methods such as thin-plate splines, reproducing kernel Hilbert space regression [38], and radial basis function interpolation [44], [49], where the map is expressed as: M X p̂(s) = αm κ(s, µm ). (3) m=1

Here, κ(·, ·) is a kernel, µm are kernel centers, and αm are coefficients estimated from the measurements. These methods adapt naturally to irregular sampling, although their performance still depends on the chosen kernel family. Structured Recovery. A third class formulates spectrum cartography as structured recovery, including compressive sensing, dictionary learning, matrix completion, and tensor completion [39], [40]. For spatio-spectral maps, one may write: R X X≈ Sr ◦ cr , (4) r=1

Representative Work [39], [44]

[43], [46], [50], [51]

[39], [40], [48], [49]

[44], [47]

[43], [44]

where Sr is the spatial loss field of emitter r and cr is its spectral signature. This formulation motivates matrix/tensor recovery methods that jointly exploit spatial and spectral correlation. Overall, classical methods remain valuable for their interpretability, statistical grounding, and effectiveness in datalimited regimes. However, their reliance on handcrafted priors can be restrictive in complex propagation environments, motivating the shift to learning-based spectrum cartography. D. Deep Learning-Based Method Deep learning-based spectrum cartography replaces handcrafted priors with representations learned directly from data. The central idea is that radio maps lie on a low-dimensional manifold that can be learned from training samples and used to infer missing values from sparse measurements [44]. This is particularly effective in complex propagation environments, where assumptions such as smoothness or low rank may be overly restrictive [40]. A representative starting point is the deep completion paradigm, where a network reconstructs a complete radio map from a masked input. Given a complete training map Qn and a sampling mask Mn , training can be formulated as: N

θ̂ = arg min θ

1 X 2 ∥fθ (Mn ⊙ Qn ) − Qn ∥F , N n=1

(5)

where fθ (·) is the completion network. In encoder–decoder form, fθ (Q) = gθd (pθe (Q)), the latent representation provides a compact description of the radio-map manifold and supports interpolation from sparse observations [40], [44]. To reduce the difficulty of learning aggregated radio maps, subsequent work introduces emitter-wise decomposition as in (4). By learning individual components rather than the aggregated tensor directly, this approach improves generalization while preserving physical structure [39]. Among practical architectures, CNNs have become the dominant backbone for radio map prediction. They treat

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

6

Fig. 5. Illustration of the radio map reconstruction. Left: data acquisition with one transmitter and multiple sensors in a built environment. Right: radio map reconstruction from sparse measurements, showing the estimated spatial distribution of signal strength under blockage and shadowing effects.

environmental information, transmitter locations, and sparse measurements as image-like inputs and learn: X̂ = fθ (Ienv , ITx , Imeas ) .

(6)

A prominent example is RadioUNet, which uses a UNet-style encoder–decoder to infer path-loss maps from city maps and transmitter positions [46]. Subsequent studies further explore benchmark datasets, aerial-image-based prediction, and robustness to imperfect side information [43], [51]. Meanwhile, lightweight data-driven schemes such as kernel-densitybased learning have also been investigated, highlighting the tradeoff between model expressiveness and computational complexity [55]. In summary, learning-based spectrum cartography has evolved from neural completion to structured decomposition and CNN-based scene-aware prediction, and is now moving toward more expressive models that capture global spatiotemporal dependencies and heterogeneous observations [56]. In this context, attention- and Transformer-based architectures extend the representational scope of CNNs and have emerged as a key direction for next-generation spectrum cartography [39], [40], [44]. III. REPRESENTATIVE APPLICATIONS A. Radio Map Reconstruction Radio maps [10], [11], [57], [58] describe the spatial–spectral–temporal distribution of radio signals by integrating measurements such as RSS, RSRP, and PSD across location, frequency, and time, offering fine-grained and continuously updated views of propagation beyond static coverage maps (cf. Fig. 5). More than visualization tools, they serve as inference frameworks that fuse sparse and heterogeneous observations to estimate unobserved conditions and predict interference, blockage, and coverage dynamics [59]. Recent advances further position radio maps as multi-domain, evolving intelligence systems that incorporate cross-layer and environmental information, casting their construction as a robust inference problem under sparse and noisy data rather than simple interpolation [60].

1) Problem Formulation: Radio map reconstruction aims to infer an unknown spatiotemporal radio field from sparse and noisy observations. The radio map is modeled as a continuous function F (x, f, t) : Rd × F × T → R, where x denotes location, f frequency, and t time. Assume Ns s measurement-capable devices located at {xi }N i=1 , each providing yi (f, t) = F (xi , f, t) + ϵi (f, t), where ϵi (f, t) captures measurement noise. Observations are sparse, heterogeneous, and potentially unreliable. Particularly, radio map reconstruction can be formulated as a structured inverse problem, written as y = Hz + ϵ, where z denotes latent radio variables and H encodes sampling geometry and propagation effects. Recovering the radio field is generally ill-posed due to sparsity and noise. From a functional perspective, reconstruction is cast as: F̂ = arg min

F ∈H

Ns X

ωi L(F (xi , f, t), yi (f, t)) + R(F ),

(7)

i=1

where ωi captures measurement reliability and spatial relevance. Abstractly, this defines a mapping from sparse obsers vations to a continuous field, M : {(xi , yi )}N i=1 → F̂ (x), and Ns in practice, F̂ (xr ) = M (xr ; {(xi , yi )}i=1 ). 2) Fundamental Challenges: Radio map reconstruction differs fundamentally from conventional spatial interpolation, as radio measurements are governed by propagation physics, environmental geometry, and network deployment rather than smooth spatial variation. The resulting radio field exhibits highly anisotropic and context-dependent spatial correlations, where proximity does not necessarily imply relevance [22]. Thus, simple distance-based kernels or naive averaging schemes often lead to biased or oversmoothed estimates [23]. Effective reconstruction must therefore account for geometry-aware propagation characteristics and adapt to complex sensing conditions [12]. In addition, reconstruction relies on sparse, uneven, and heterogeneous observations collected from diverse sensing sources with varying noise levels and reliability [61]. Such conditions render the inverse problem inherently ill-posed, especially under clustered or limited sampling, necessitating the integration of structural priors and spatial redundancy [59], [62]. Moreover, radio measurements are often corrupted or unreliable, requiring mechanisms to assess and downweight low-quality data [24]. Since reconstruction uncertainty also varies across space, it must be jointly modeled with the radio field itself [13], motivating adaptive estimation strategies that selectively emphasize informative observations and support uncertainty-aware sensing [47]. 3) Reconstruction Methods: Existing radio map reconstruction methods estimate the underlying radio field from distributed measurements, and can be broadly categorized based on their use of propagation modeling, statistical inference, and sensing strategies (summarized in Table II). Direct Interpolation. Direct interpolation methods estimate the radio state at unobserved locations by aggregating nearby observations without explicitly modeling propagation: F̂ (x) =

Ns X i=1

wi (x) yi ,

(8)

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

7

TABLE II C OMPARISON OF R ADIO M AP R ECONSTRUCTION PARADIGMS

17.5

−44

15.0

Spatial weighting (IDW, Kriging)

Comp.

Robust.

#

#

Data Eff. #

Physics #

G #

Model-Based

Propagation laws (Ray-tracing)

Sparse/Bayes

Statistical inference (CS, GPR)

G #

Active Sensing

Closed-loop adaptive sampling

G #

12.5

7.5

Metal shelf (NLoS wall) Link (width ∝ ωi)

# # G

ω2=0.42 d r =5.8 2

xr (query)

ωd4r =0.06 =3.1 (↓) 4

x4 Shelf (NLoS) r (NLoS) ω1d=0.23 1 =4.1 x1

5.0

2.5

# G

−46

x2

d r =7.1

ω33=0.29 Base station (Tx) Reliable sensor (LoS) Corrupted sensor (NLoS) Query point xr

10.0

̂ r) (dBm) F(x

Direct Interp.

Key Mechanism

x2-coordinate (m)

Paradigm

x3

+0.9 dB -45.6 dBm -46.6 dBm

−48

-5.2 dB −50

BS (Tx) 3.5 GHz

−52

-51.8 dBm

0.0

0.0

2.5

5.0

7.5

10.0

12.5

15.0

17.5

Uniform IDW

20.0

x1-coordinate (m)

(a) Indoor factory geometry

Reliabilityaware

Ground truth

(b) Estimated field at xr

Notes: High, G # Moderate, # Low. Comp.: Computational Complexity; Data Eff.: Data Efficiency.

where weights wi (x) are determined by spatial proximity or statistical correlation [24]. Representative approaches include inverse distance weighting (IDW) [63], nearest neighbor [64], spline interpolation [65], natural neighbor interpolation [66], and Kriging [41]. These methods are computationally efficient and easy to deploy, but are sensitive to non-uniform sampling, noise, and heuristic weighting rules. Model-Based Methods. Model-based methods incorporate propagation physics by estimating channel or transmitter parameters and predicting the radio field via a parametric model: F̂ (x) = g(x; θ̂),

(9)

where θ̂ denotes inferred propagation parameters [24]. While physically interpretable and accurate under reliable environmental knowledge, these methods are sensitive to model mismatch and rely on strong priors. Hybrid approaches combine interpolation with model-based refinement, improving accuracy at the cost of increased complexity. Sparse and Bayesian Inference. Recent approaches formulate reconstruction as a structured inverse problem under sparse and probabilistic settings [62], [67]. Let z denote latent radio variables. A typical Bayesian formulation estimates: p(z | y) ∝ p(y | z) p(z),

(10)

where sparsity-promoting priors and probabilistic noise models enhance robustness under limited and noisy observations. Compared with deterministic interpolation, these methods provide principled uncertainty quantification and improved performance in high-dimensional or data-scarce scenarios. Active Sensing. Beyond passive estimation, recent work considers sensing-aware reconstruction with mobile platforms such as UAVs [12], [47]. In this setting, measurement locations are sequentially selected based on current estimates and uncertainty, leading to a joint estimation–control problem. By prioritizing informative or uncertain regions, active sensing significantly improves reconstruction efficiency. To illustrate the impact of measurement reliability on reconstruction accuracy, we consider a Sionna RT-simulated indoor factory scenario, where a corrupted NLoS observation introduces significant bias under uniform aggregation. Example 4 (Reliability-Aware Radio Map Reconstruction): Consider a Sionna RT-simulated indoor factory (InF) scenario

Sensor

yi (dBm)

diTx (m)

dir (m)

wiunif

wirel

x1

-36.9

10.44

4.12

0.280

0.225

x2

-50.1

18.87

5.83

0.140

0.424

x3

-42.2

13.42

7.07

0.095

0.288

x4 (NLoS)

-62.7

15.27

3.13

0.485

0.063

(c) Sensor measurements and weights Fig. 6. Reliability-aware radio map reconstruction in a Sionna RT-simulated indoor factory scenario (InF-DH, fc = 3.5 GHz, 3GPP TR 38.901). (a) Sensors x1 –x3 are LoS (blue); x4 is NLOS due to metal shelving (red), with link width proportional to ωi . (b) Reliability-aware aggregation (−45.6 dBm, +0.9 dB error) substantially outperforms uniform IDW (−51.8 dBm, −5.2 dB error) against the Sionna ground truth (−46.6 dBm). (c) Although x4 is the nearest sensor (dr4 = 3.13 m), its NLOS-induced bias suppresses its weight from w4unif = 0.485 to w4rel = 0.063.

[2], with a base station deployed at (2, 2) m on a 20 × 16 m factory floor, transmitting at fc = 3.5 GHz (5G NR FR1). Path loss follows the 3GPP TR 38.901 InF-Dense-High (InF-DH) model: FLoS (x) = Pt − [31.84 + 21.5 log10 d + 19.0 log10 (fGHz )], (11) FNLoS (x) = Pt − [33.63 + 21.9 log10 d + 20.0 log10 (fGHz )], (12) with transmit power Pt = 23 dBm. Four sensors are placed at x1 = (12, 5), x2 = (18, 12), x3 = (8, 14), and x4 = (16, 8.1) m. Sensors x1 –x3 operate under LoS conditions, while x4 is obstructed by a metal shelving panel modeled as a metallic obstacle with ITU-R P.2040-3 electromagnetic properties in the Sionna scene. The Sionna ray-traced received powers are [−38.6, −47.1, −40.5, −52.3] dBm; after adding Gaussian measurement noise (σ = 1.5 dB) and non-line-ofsight (NLOS) lognormal shadowing (σNLoS = 7.2 dB per 3GPP TR 38.901 InF-DH), the measurements are: y = [−36.9, −50.1, −42.2, −62.7] dBm.

(13)

To estimate the field at xr = (13, 9) m, uniform IDW weights (wiunif ∝ d−2 i ) assign the largest weight (0.485) to x4 due to its proximity (dr4 = 3.13 m), yielding F̂unif (xr ) = −51.8 dBm—a −5.2 dB error against the Sionna ground

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

truth of −46.6 dBm. Reliability-aware weights ωi ∝ wiunif · e−β|yi −med(y)| suppress x4 to w4rel = 0.063, improving the estimate to F̂rel (xr ) = −45.6 dBm (error: +0.9 dB), illustrating the necessity of reliability-aware aggregation in radio map reconstruction (cf. Fig. 6). 4) Recent Advances and Trends: Recent advances in radio map reconstruction reflect a clear transition from interpolationdriven techniques toward data-driven and structure-aware frameworks. Early work established reconstruction as a field inference problem under sparse, noisy, and heterogeneous observations [59], and broadly categorized approaches into model-based, interpolation-based, and learning-based paradigms. Subsequent studies further emphasized the evolution from static signal databases to multi-domain radio map frameworks that jointly capture spatial, spectral, and temporal dynamics across complex environments [61]. This progression highlights the growing role of radio maps as inference-oriented systems for proactive wireless resource management. Recent methodological advances increasingly leverage deep generative and self-supervised learning to address severe data sparsity and complex propagation conditions. Diffusion-based approaches enable high-resolution reconstruction through progressive denoising under limited measurements [68], while self-supervised learning strategies improve generalization by exploiting structural information from partially observed environments [69]. In addition, generative models such as variational autoencoders provide uncertainty-aware reconstruction by producing multiple plausible realizations of the radio field [70], extending reconstruction from deterministic estimation to probabilistic modeling. Alongside learning-based developments, increasing attention has been devoted to incorporating structural priors and improving data realism. Gaussian process formulations with environmental features enable joint modeling of radio propagation and geographic context [71], while graph-based representations capture spatial dependencies over irregular regions [22]. The availability of high-fidelity datasets, such as UrbanMIMOMap and OpenPathNet [72], [73], further supports realistic evaluation and model development. Overall, these advances indicate a shift toward learning-based, structure-aware, and uncertainty-aware reconstruction frameworks, while also revealing the need for adaptive mechanisms that can effectively fuse heterogeneous measurements under varying reliability. B. LEO Satellite Localization The rapid deployment of large-scale LEO satellite constellations is reshaping global connectivity and positioning capabilities [74], [75], as illustrated in Fig. 7, which visualizes a representative Sionna RT-based localization scenario, including satellite–observer geometry and propagation characteristics. Compared with traditional global navigation satellite systems (GNSS) [76], [77], LEO satellites operate at lower altitudes and higher velocities, creating new opportunities for positioning, navigation, and timing (PNT) services in nonterrestrial networks [78]. However, LEO systems differ fundamentally from conventional GNSS. Their fast motion induces strong Doppler

8

LEO Satellite Localization LEO Satellite 2

LEO Orbit LEO Satellite 1

Earth Target Location LEO Satellite 3

Fig. 7. Illustration of a LEO satellite localization scenario, where multiple LEO satellites are used to determine the location of a ground target.

effects [79], while limited synchronization leads to asynchronous measurements. The satellite–user geometry also evolves rapidly, resulting in time-varying observability and geometric dilution of precision. Despite these characteristics, many existing methods adopt estimation frameworks originally developed for terrestrial or GNSS systems, often assuming homogeneous noise or equal reliability across observations [80], [81]. In practice, measurement quality varies significantly across satellites and time due to signal conditions, geometry, and dynamics, causing uneven contributions to localization accuracy [19]–[21]. Motivated by these challenges, this section provides a unified background on LEO-based localization with a focus on measurement heterogeneity. We first formulate the problem within a general estimation framework, then review classical models and methods, highlighting the need for reliabilityaware weighting in dynamic LEO environments. 1) Problem Formulation: We consider a localization scenario where a user with unknown state is observed by multiple LEO satellites moving along known trajectories. Due to dynamic geometry and varying signal conditions, the resulting measurements exhibit heterogeneous reliability. State and Geometry. Let x ∈ Rnx denote the unknown user state. For position-centric localization we set x = p ∈ R3 , where p represents the user position in the earth-centered earth-fixed (ECEF) coordinate system [25]. The formulation can be extended to augmented states including velocity and clock parameters. Consider N LEO satellites indexed by S = {1, 2, . . . , N }. The position and velocity of satellite i at time t are denoted by si (t) and ṡi (t) and are assumed known from ephemeris information. Generic Measurement Model. Let m ∈ M index an individual observation, and let Gm denote the associated satellite geometry, which may involve a single satellite or a satellite pair. A general measurement model is ym = hm (x, Gm ) + nm ,

m ∈ M,

(14)

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

9

geometry-only model for all i is: yir (t) = ρi (t) + nri (t),

Fig. 8. Sionna RT scene of the link geometry between the observer and STARLINK-11166 [DTC] at UTC 2026-04-13 05:03:42.75. The satellite subpoint is at (1.222477◦ , 104.572422◦ ) with an altitude of 360.890 km, while the observer sees the satellite at 75.257◦ elevation, 95.476◦ azimuth, and a slant range of 372.471 km.

where hi (·) denotes a generally nonlinear measurement function governed by the satellite–user geometry and the underlying signal propagation mechanism, while ni represents the corresponding measurement noise. Both {hi } and {ni } may vary across satellites and over time, thereby reflecting the heterogeneity of the observation conditions. Unified Estimation Objective. Based on (14), localization can be formulated in weighted quadratic form: X 2 ∥ym − hm (x, Gm )∥Wm . (15) x̂ = arg min x

m∈M

where Wi ⪰ 0 is a weighting matrix for measurement i, reflecting its reliability without specific statistical assumptions. Problem Characteristics. This formulation highlights several characteristics of LEO localization. Rapid satellite motion produces strongly time-varying geometry and observability, while measurements from different satellites and sensing modalities exhibit heterogeneous reliability. Measurement quality further depends on signal conditions, geometry, and temporal alignment [25]. Consequently, the contribution of each observation varies over time, motivating adaptive mechanisms that modulate measurement influence during estimation. To make this generic formulation concrete, we next instantiate the measurement functions hi (·) using classical observables commonly used in LEO satellite localization. 2) Measurement Models: This subsection summarizes classical measurement models used in LEO satellite localization. Although the observables originate from satellite navigation and wireless positioning, LEO systems exhibit rapidly varying geometry and heterogeneous link conditions, making measurement reliability strongly input dependent [17], [18]. The models are presented in a geometry-centric form to emphasize the coupling between the unknown user state and the satellite–user line-of-sight (LOS). Preliminaries and Notation. Let the user position be p ∈ R3 in ECEF coordinates, and let the position and velocity of satellite i at time t be si (t) and ṡi (t). Define ri (t) ≜ p−si (t), ρi (t) ≜ ∥ri (t)∥, and the LOS unit vector ui (t) ≜ ∥rrii (t) (t)∥ . Range / TOA-Type Measurements. Range-type observations are fundamental in satellite navigation [78], [82]. A simplified

(16)

where nri (t) represents receiver noise and propagation effects. More complete GNSS models include clock offsets and atmospheric errors, while (16) isolates the geometric term that governs Jacobian structure and dilution-of-precision behavior [14], [78], [83]. Doppler / FDOA-Type Measurements. Because LEO satellites move at high orbital velocities, Doppler observables are particularly informative for LEO-based PNT [15], [17], [84]. A standard Doppler (range-rate) model is yid (t) = ρ̇i (t)+ndi (t) with ρ̇i (t) = u⊤ i (t)(ṗ(t)− ṡi (t)). Thus, Doppler is determined by the projection of relative velocity onto the LOS [25]. If ṗ is omitted, it can be modeled through an augmented state or short-time motion model. An frequency difference of arrival (FDOA) measurement between satellites i and j is: fd yij (t) = ρ̇i (t) − ρ̇j (t) + nfd ij (t).

(17)

RSS / Power-Based Measurements. In opportunistic LEO localization, RSS or related power metrics may also be available [18]. A commonly used path-loss model is:
(18) yirss (t) = P0 − 10α log10 ρi (t) + ηi (t), where P0 is a reference power, α is the path-loss exponent, and ηi (t) captures shadowing and power fluctuations [85]. Compared with time of arrival (TOA) and Doppler, RSS provides weaker geometric sensitivity and is more environment dependent, with errors influenced by blockage, antenna patterns, beam scheduling, and multipath [17]. Hybrid and Multi-Source Measurements. LEO localization often combines time-, frequency-, and power-based measurements [14], [18]. Let M denote the available observations, td fd e.g., yir , yid , yij , yij , and yirss . A unified representation follows the generic model in (14), where both the observation function and the error term depend on the modality, geometry, receiver processing, and propagation conditions. Summary and Reliability Heterogeneity. Table III summarizes the main characteristics of these modalities. In LEO localization, two properties are particularly important. First, the information content of each measurement is geometry dependent through LOS diversity and temporal evolution. Second, the measurement reliability is intrinsically heterogeneous due to time-varying signal-to-noise ratio (SNR), blockage, antenna effects, and modality-specific error statistics [18]. Consequently, uniform treatment of all measurements is generally suboptimal. Given these heterogeneous observation models, the next step is to examine how classical estimators convert such measurements into state estimates and how weighting enters the estimation process. 3) Estimation Models: Based on the measurement models above, this subsection reviews classical estimation frameworks for satellite localization. These methods infer the unknown user state from noisy observations under statistical assumptions [90] and provide the basis for subsequent discussions on uncertainty, geometry, and measurement weighting. Generic Estimation Formulation. Let x denote the unknown user state, which may include position, velocity, and clock

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

10

TABLE III C LASSICAL MEASUREMENT MODALITIES IN LEO LOCALIZATION : GEOMETRY DEPENDENCE AND RELIABILITY HETEROGENEITY. Modality

Measurement Model

Geometry Sensitivity

Dominant Error Drivers

Range / TOA

ρi = ∥p − si ∥

LOS diversity, satellite–user geometry

Bandwidth, SNR, multipath, residual biases [25], [82], [85]

G #

Doppler / FDOA

ρ̇i = u⊤ i (ṗ − ṡi )

Velocity alignment, temporal geometry variation

Oscillator stability, tracking errors [15], [25], [85]

G #–

TDOA

∆ρij = ρi − ρj

Satellite pair separation, relative geometry

Clock offsets, synchronization, pairing, residual biases [85]–[87]

RSS

Pr,i = P0 − 10α log10 (ρi /ρ0 ) + ηi

Weak, indirect through distance

Shadowing, blockage, beam patterns [18], [88], [89]

Notes: G # Medium,

High,

Reliability Hetero.

Very High. Reliability heterogeneity refers to the variance of measurement quality across space, time, and frequency.

parameters. Let h(·) denote a possibly nonlinear measurement function, and let the observations be corrupted by noise and modeling error. Localization can then be formulated by estimating x through a suitable loss or negative log-likelihood determined by the assumed statistical model, encompassing least-squares, maximum-likelihood, Bayesian, and robust estimators [90]. Least-Squares and Weighted Least-Squares. Least-squares (LS) estimates the state by minimizing the squared residual, while weighted least-squares (WLS) introduces the matrix: x̂WLS = arg min(y − h(x))⊤ W(y − h(x)). x

(19)

Assume that the measurement noise is zero-mean Gaussian. In this case, weighted least-squares (WLS) is equivalent to maximum likelihood estimation when W = R−1 , where R is the noise covariance matrix [90]. LS and WLS are standard in satellite navigation and have been applied to LEO localization [14], [82]. Maximum Likelihood and Bayesian Estimation. Maximum likelihood (ML) estimation solves: x̂ML = arg max p(y|x), x

(20)

while Bayesian estimation incorporates prior information through p(x) and infers the posterior p(x|y) [91]. Sequential Bayesian methods such as the Kalman and extended Kalman filters are widely used when temporal dynamics are considered. Their performance, however, depends on accurate likelihood and prior models, which can be difficult to maintain in rapidly varying LEO environments [17]. Robust Estimation. Robust estimators mitigate the impact of outliers through loss-function design or residual reweighting, with representative examples including the Huber and Tukey formulations [92]. Despite their effectiveness in handling anomalous perturbations, these methods are principally intended to suppress outlier effects, rather than to capture the continuous and geometry-dependent variations in measurement informativeness. Information-Theoretic Limits and Fisher Information. Estimator performance is fundamentally limited by the information contained in measurements. The Fisher Information Matrix (FIM) and the Cramér–Rao Lower Bound (CRLB) characterize these limits [91], and Cov(x̂) ⪰ J−1 (x). For

additive Gaussian noise with state-independent covariance, the local Fisher information matrix can be written as J(x) = H⊤ (x)R−1 H(x),

(21)

where H(x) is the Jacobian and R is the noise covariance. Hence, while covariance-based weighting accounts for measurement noise, the actual information contribution of each observation also depends on the state-dependent geometry through H(x). Geometric Observability and Conditioning. Observation geometry determines the rank and conditioning of the Jacobian and thus directly affects localization accuracy [18]. In LEO systems these effects are pronounced because satellite visibility, LOS directions, and relative velocities vary rapidly. Different modalities exhibit distinct geometric sensitivities: Doppler measurements mainly capture radial velocity components [14], [82], TOA / time difference of arrival (TDOA) are subject to dilution of precision [25], while RSS is often dominated by propagation effects. Unfavorable configurations like satellites clustered in similar angular sectors, may render the FIM illconditioned and produce anisotropic uncertainty [17], [93]. Summary and Limitations. Classical estimators, including LS, WLS, ML, Bayesian, and robust methods, form the foundation of localization, but their performance depends on uncertainty modeling. Most rely on predefined noise models and use variance as the main reliability measure. Yet Fisher information shows that measurement utility depends on both noise and geometry through the Jacobian. In LEO systems with dynamic geometry and heterogeneous observations, static or noise-only weighting can misjudge measurement value. Robust methods handle outliers, but not continuous geometrydriven variability. This limitation motivates a closer examination of how uncertainty is shaped not only by noise statistics but also by satellite–user geometry, which is discussed next. 4) Uncertainty and Geometry: The previous subsection reviewed classical estimation frameworks. Here we focus on uncertainty in LEO localization, which is geometry-driven and input dependent because of rapid satellite motion, short visibility, and heterogeneous links, rather than well described by static noise-only models [94]. Noise Versus Geometry. For y = h(x) + n, uncertainty is often linked to noise variance. In localization, however,

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

11

Thus u1 = (0, −1), u2 = (−1, 0), u3 = (1, 0) and:   0 −1 H = −1 0  . 1 0

(23)

s1 σ=5 m

10

0

visibility window

Zoom

5 y (km)

error depends jointly on noise and geometric sensitivity: noise determines measurement accuracy, while geometry determines which state directions are constrained. Thus, even measurements with identical noise may contribute differently depending on LOS directions, relative velocities, and satellite–user configuration [25], [26]. This is particularly relevant in LEO systems. TOA/TDOA constrain range directions and are subject to dilution of precision, Doppler captures radial velocity and contributes through temporal/spatial diversity [95]–[97], while RSS is more environment dependent and often dominated by propagation effects [18]. Fisher-Information Perspective. A unified view is provided by the FIM, which yields CRLB [90], [91]. The locally linearized Gaussian model is the same as (21). While R scales information, directionality and observability are determined by H(x). Since H(x) depends on both the geometry and the state, the FIM is time-varying. Consequently, the informativeness of the measurements is also time-varying. Static noiseonly weights therefore cannot fully represent the time-varying information contribution of individual measurements [94]. Anisotropy, Conditioning, and Observability. The structure of J(x) determines error anisotropy: information is often strong in some directions but weak in others, leading to elongated uncertainty ellipsoids. These effects are amplified in LEO due to rapidly changing LOS directions and visibility [16]. Ill-conditioning arises when columns of H(x) become nearly dependent (e.g., satellite clustering), causing large estimation variance and relating to classical DOP effects [25], [26]. Empirical studies further show strong performance variation with geometry, satellite selection, and measurement type [96], [97]. Implications for Weighting and Fusion. Measurement utility depends jointly on noise and geometry through H⊤ R−1 H and varies with time and state [18]. Noise-aware weighting is therefore not equivalent to information-aware weighting. Robust estimators suppress outliers but do not capture continuous geometry-driven variability [92], [98]. Reliable LEO localization thus requires geometry-aware and input-dependent fusion mechanisms. To illustrate the effect of geometry, we consider a simple numerical example. The example shows that, even when all measurements have identical noise variance, their contribution to localization accuracy can differ due to geometry. Example 5 (Geometry-Driven Heterogeneity Under Homogeneous Noise): Consider 2D localization with unknown position p = [x, y]⊤ and satellites s1 = (0, 10) km, s2 = (10, 0) km, s3 = (−10, 0) km. Let the true position be p⋆ = (0, 0) and range measurements yi = ρi (p) + ni , ρi (p) = ∥p − si ∥, ni ∼ N (0, σ 2 ), with identical noise σ = 5 m. Linearizing at p⋆ , the Jacobian row equals the line-of-sight unit vector: (p⋆ − si )⊤ . (22) u⊤ i = ∥p⋆ − si ∥

LOS diversity ≈ 180 ∘

s3 σ=5 m

User

4 km −5

s2 σ=5 m

1σ CRLB 2σ CRLB User Satellite

−10 −10

−5

0 x (km)

5

10

Fig. 9. Geometry-driven uncertainty anisotropy under homogeneous range noise. Although three satellites provide identical measurement variance, satellite–user geometry leads to direction-dependent Fisher information and an elongated CRLB ellipse.

The Fisher information matrix is: J=

 1 ⊤ 1 2 H H = σ2 σ2 0

 0 , 1

(24)

yielding std(x) ≥ √σ2 and std(y) ≥ σ. Removing s1 gives:     1 2 0 −1 0 H23 = , J23 = 2 . (25) 1 0 σ 0 0 The illustration of this example is shown in Fig. 9. The FIM becomes rank deficient, so the y direction is unobservable. Even with identical noise, measurement informativeness depends strongly on geometry. 5) Reliability-Aware Weighting: The preceding analysis establishes that measurement utility in LEO localization is jointly determined by signal reliability and geometrydependent observability. Therefore, LEO satellite localization should be viewed as a heterogeneous measurement fusion problem. Measurements collected from LEO constellations, especially signals of opportunity, can differ substantially in signal quality, geometric contribution, and statistical reliability across satellites and time [16], [99]. Weighting is therefore not merely an algorithmic detail, but a core modeling component that determines how heterogeneous information is fused into a consistent estimate. Generic Estimation Perspective. Consider the nonlinear measurement model: zi = hi (x) + vi ,

(26)

where x is the unknown state and zi is the i-th observation. A weighted least-squares estimator is: x̂ = arg min x

M X

2

∥zi − hi (x)∥Wi .

(27)

i=1

If the measurement covariance Ri is accurately known, the classical choice is Wi = R−1 i . In LEO localization, however, measurement statistics are rarely stationary or fully known because of time-varying propagation conditions, receiver dynamics, and ephemeris or clock uncertainty [100], [101]. Static

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

12

(a) SNR-only selection

(b) Geometry-aware fusion

Zoom

12

12 visibility window

10

10

6 4

s2 σ =s1 m 1 σ=1 m

2 0 −2

Zoom

visibility window

8 1σ CRLB 2σ CRLB User Satellite

y (km)

y (km)

8

s3 σ=5 m

LOS diversity ≈ 6 ∘ User 3 km 0.0

2.5

5.0 7.5 x (km)

10.0

4 2 0 −2

12.5

1σ CRLB 2σ CRLB User Satellite

6

LOS diversity ≈ 90 ∘

s2 σ =s1 m 1 σ=1 m

User 3 km 0.0

2.5

5.0 7.5 x (km)

10.0

12.5

Fig. 10. SNR-only selection versus geometry-aware fusion in Example 6. Although satellites s1 and s2 have higher SNR, their nearly collinear LOS directions lead to poor observability and an elongated CRLB ellipse. Including the lower-SNR but geometrically diverse satellite s3 significantly reduces directional uncertainty, showing that measurement informativeness depends jointly on noise and geometry.

or pre-defined weighting is therefore often mismatched to the actual data-generating process. Signal- and Geometry-Dependent Heterogeneity. From a signal perspective, LEO measurements exhibit rapid reliability fluctuations due to elevation-dependent attenuation, beam scheduling, Doppler dynamics, and environmental effects. Empirical studies using Starlink, OneWeb, Orbcomm, and Iridium signals show that positioning accuracy is highly sensitive to instantaneous signal conditions and satellite availability [95], [102]. Uniform weighting therefore implicitly assumes homogeneous and stationary noise, which is inconsistent with practice. From a geometric perspective, comparable noise levels do not necessarily imply comparable information contribution. As discussed in the previous subsection, measurement utility depends jointly on noise statistics and geometry-dependent sensitivity, so geometrically redundant observations may provide limited additional information [103], [104]. To motivate the need for geometry-aware weighting, consider the following example, where SNR-based selection performs poorly because high-SNR measurements are geometrically redundant, while a lower-SNR measurement improves observability. Example 6 (Geometry-Limited Informativeness of High-SNR Measurements): Consider 2D range localization with true user position p⋆ = (0, 0) and satellites s1 = (10, 0) km, s2 = (9, 1) km, s3 = (0, 10) km. Each satellite provides yi = ∥p − si ∥ + ni , ni ∼ N (0, σi2 ), with σ1 = σ2 = 1 m and σ3 = 5 m. At p⋆ , the LOS vectors are u1 = (−1, 0), u2 = (−9,−1) √ ≈ (−0.9939, −0.1104), u3 = (0, −1). The Fisher 82 information matrix is: J=

N X 1

σ2 i=1 i

ui u⊤ i .

(i) SNR-only selection: using {s1 , s2 }    1.9878 0.1098 1 −1 J12 ≈ , J12 ≈ 0.1098 0.0122 −9

(28)

(ii) Geometry-aware fusion: including s3    1.9878 0.1098 0.5691 −1 J123 ≈ , J123 ≈ 0.1098 0.0522 −1.1968

 −1.1968 , 21.6755 (30)

yielding std(x) ≳ 0.75 m and std(y) ≳ 4.66 m. The illustration of (i) and (ii) is shown in Fig. 10. Despite lower SNR, satellite s3 improves observability by providing LOS diversity, illustrating that measurement utility depends jointly on noise and geometry. Statistical Mismatch and Robustness. LEO measurements may also exhibit non-Gaussian errors due to multipath and NLOS propagation, synchronization faults, and model mismatch. This motivates robust formulations based on iteratively reweighted least squares and M -estimation [92], [105]. Robust and constrained WLS methods have been widely applied in TOA/TDOA/FDOA localization to reduce sensitivity to outliers and initialization errors [106]–[108]. However, such methods usually adapt weights based on residual magnitude alone and do not explicitly account for geometry or crossmeasurement interactions. Implications for Weighting Design. These observations indicate that effective weighting in LEO localization should depend jointly on signal reliability, geometry-dependent information contribution, and consistency with the current state and other measurements. Heuristic rules based on a single factor, such as SNR thresholds or elevation masks, are therefore generally insufficient for heterogeneous multi-source fusion [109]. More broadly, LEO localization calls for inputdependent weighting strategies that jointly model reliability, geometry, and consistency, providing the foundation for the more expressive fusion mechanisms developed in the following sections. C. Map-Informed Resource Allocation



−9 , 163

giving std(x) ≳ 1.00 m and std(y) ≳ 12.77 m.

(29)

Map-informed resource allocation bridges classical communication-theoretic optimization and spectrum cartography. Classical approaches, exemplified by waterfilling, perform continuous power allocation over channels

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

13

SAT 2 SAT 1

SAT 3

Active beam

Inactive beam

Radio Map

Fig. 12. Illustration of adversarial water-filling. (a) Transmit-side water-filling for p⋆ , with dashed line ν denoting the transmit water level. The orange bars σ +n⋆ indicate the effective base levels i β i , and the blue segments denote the i ⋆ transmit powers pi . (b) Interference-side water allocation for n⋆ , with dashed 1 line −µ denoting the interference water level. The total bar height is σ +n ⋆, and the yellow segment represents β1ν .

i

i

i

0.2

1.6

Inferred

2.4

1.9

0.5

from sparse & heterogeneous observations

Map-Informed Resource Allocation

Water-Filling （Continuous Allocation）

Beam Switching （Spatial Selection）

Beam Hopping （Spatiotemporal Scheduling）

Fig. 11. Illustration of map-informed resource allocation in LEO satellite networks. Beam qualities are inferred from sparse observations to form a radio map, which guides resource allocation via water-filling (continuous allocation), beam switching (spatial selection), and beam hopping (spatiotemporal scheduling).

with known quality. In emerging LEO satellite networks, however, resource allocation extends beyond this paradigm: decision variables are often discrete, and the allocation domain spans not only frequency but also space and time. In particular, beam switching and beam hopping enable spatial selection and spatiotemporal scheduling, respectively, generalizing water-filling to geometry-aware and dynamic settings. From a cartography perspective, the key challenge is not only how to allocate resources given known conditions, but also how to infer favorable dimensions from sparse observations and adapt decisions in real time. Fig. 11 illustrates this framework and its representative mechanisms. 1) Water-Filling-Based Resource Allocation: Consider a system with m parallel Gaussian channels. Let p = [p1 , . . . , pm ]T ∈ Rm denote the transmit power allocation, where pi ≥ 0 and the total power budget satisfies 1T p = P . With channel gain βi > 0 and noise power σi > 0, the optimal solution takes the water-filling form p⋆i = max{ν − σβii , 0} for i = 1, . . . , m, where ν is the water level chosen to satisfy the total power constraint [110]–[112]. This expression captures the key principle of allocating more power to favorable channels while possibly deactivating unfavorable ones. In spectrum cartography, however, the favorable dimensions are not directly observable and must be inferred from spatially distributed, heterogeneous measurements. Radio maps provide this information by offering a measurement-driven representation of channel quality, interference, and spectrum availability [53]. In this view, water-filling determines how resources are

allocated once opportunities are known, whereas radio maps determine where and when such opportunities arise, thereby tightly coupling inference and optimization. This interpretation extends resource allocation beyond conventional frequencydomain settings, where the channel index i may represent not only spectral subchannels but also spatial locations, beams, or time slots. Consequently, water-filling can be viewed more broadly as allocating resources over measurement-informed dimensions across space, time, and frequency, which is particularly relevant in LEO satellite networks with highly dynamic geometry and visibility conditions. In practice, radio maps are inherently imperfect due to sparse observations, outdated measurements, and rapidly varying interference, which is especially critical in LEO satellite networks. Such uncertainty propagates directly to mapinformed resource allocation, motivating robust formulations that explicitly account for worst-case conditions. A natural extension is to formulate resource allocation as a minimax game, where the transmitter allocates power to maximize capacity while an adversary allocates interference power to degrade performance [113]. Example 7 (Adversarial and Minimax Water-Filling for LEO Networks.): Following [114], let p = [p1 , . . . , pm ]T and n = [n1 , . . . , nm ]T denote the transmit-power and interferencepower allocations over m Gaussian channels. The adversarial water-filling problem is:   m X βi pi max min log 1 + (31) p n σi + n i i=1 s.t.

1T p = P, 1T n = N, p ≥ 0, n ≥ 0.

(32)

For fixed n, the optimal transmit power retains the classical water-filling form with σi + ni acting as the effective noise floor. For fixed p, the adversary allocates interference according to its coupled optimal strategy, where µ is the interferencepower water level determined by 1T n = N [114]. For active channels, the two water levels satisfy: µ=

1 1 − . βi ν σi + n⋆i

(33)

The illustration of adversarial water-filling is shown in Fig. 12. Therefore, adversarial water-filling is a coupled two-player generalization of classical water-filling: ν governs transmitresource allocation, while µ governs worst-case interference

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

14

allocation. From the viewpoint of spectrum cartography, this is particularly relevant because a learned radio map should be treated as an uncertain state estimate rather than exact ground truth. In LEO satellite networks, where beam footprints, visibility windows, and interference relationships vary rapidly, adversarial water-filling provides a principled robust extension of radio-map-aware resource allocation [114]. 2) Beam Switching for Spatial Resource Selection: Beam switching [115], [116] addresses the problem of selecting the most favorable transmission direction or link from a set of candidate beams. Let B = {1, . . . , B} denote the set of available beams, and let rb denote the achievable rate associated with beam b, defined as rb = log2 (1 + SINRb ), where SINRb captures the signal-to-interference-plus-noise ratio under the channel gain, interference, and visibility conditions of beam b. The beam-switching problem can be formulated as b⋆ = arg maxb∈B rb , which characterizes beam switching as a discrete selection problem over spatial dimensions. In practice, SINRb is only partially observable and may vary rapidly in LEO satellite networks due to orbital dynamics and time-varying interference. From the perspective of spectrum cartography, the quantities {rb } can be inferred from radio maps that aggregate sparse and heterogeneous observations. Accordingly, beam switching can be interpreted as selecting the most favorable spatial dimension identified by the radio map, extending classical resource allocation beyond continuous power control to geometry-aware link selection in highly dynamic satellite environments. 3) Beam Hopping for Spatiotemporal Scheduling: Beam hopping [117], [118] extends resource allocation to the temporal domain by dynamically activating beams across different spatial regions. Let sb (t) ∈ {0, 1} denote whether beam b ∈ B is active at time slot t, and let rb (t) denote the achievable rate of beam b at time t, reflecting time-varying channel conditions and spatially heterogeneous traffic demands. A typical beamhopping problem can be formulated as:

IV. ATTENTION-BASED LEARNING FOR SPECTRUM CARTOGRAPHY The attention mechanism, originally introduced for neural machine translation [29], [30], addresses the limitations of fixed-length representations by enabling models to dynamically focus on the most relevant inputs when producing each output. Instead of uniformly aggregating all observations, attention assigns data-dependent weights based on their relevance, allowing effective modeling of complex dependencies. From a statistical perspective, this mechanism is closely related to classical non-parametric regression, such as the Nadaraya–Watson estimator [32], [33], where predictions are formed via similarity-weighted averaging. Neural attention generalizes this idea by learning the similarity function, enabling adaptive and context-aware information fusion. These properties make attention particularly well suited for spectrum cartography, where radio map reconstruction requires aggregating sparse, heterogeneous, and reliabilityvarying measurements. In this section, we first revisit the NW estimator as a classical foundation for similarity-based aggregation, followed by neural attention models and the Transformer multi-head attention mechanism. We then discuss practical implementation aspects and analyze the strengths and limitations of attention-based approaches. Finally, we illustrate their application to LEO satellite localization and radio map reconstruction, highlighting their ability to perform geometryaware and data-adaptive fusion. A. Nadaraya–Watson Estimator From a statistical perspective, the attention mechanism can be interpreted as a non-parametric regression operator. Given observations {(xi , yi )}ni=1 , the NW estimator [32], [33] predicts the response at a query point x through a similarityweighted aggregation of the observed targets [119]–[121]: ŷ(x) =

n X i=1

max

{sb (t)}

s.t.

X

T X X

sb (t) rb (t)

(34)

t=1 b∈B

sb (t) ≤ Bmax ,

sb (t) ∈ {0, 1},

∀ b, t,

(35)

b∈B

which characterizes beam hopping as a spatiotemporal scheduling problem under resource constraints, where at most Bmax beams can be simultaneously activated at any time slot. Unlike beam switching, which selects a single best direction at a given instant, beam hopping distributes resources across multiple regions over time, enabling flexible adaptation to heterogeneous traffic demands and time-varying channel conditions. From a spectrum cartography perspective, rb (t) can be interpreted as a map-informed quantity inferred from sparse observations, so that beam hopping amounts to scheduling resource allocation over spatiotemporal dimensions revealed by radio maps.

α(x, xi ) yi ,

k(x, xi ) , (36) α(x, xi ) = Pn j=1 k(x, xj )

where the normalized weights {α(x, xi )} form a probability distribution over the samples, assigning greater influence to observations that are more similar to the query. Here, k(·, ·) denotes a kernel function that quantifies similarity. A commonly adopted choice is the Gaussian kernel [122], [123], yielding: k(x, xi ) = exp (−

∥x − xi ∥2 ), 2σ 2

(37)

where σ > 0 is the bandwidth parameter controlling the locality of the weighting. Example 8 (NW Estimator for Reliability-Aware LEO Satellite Localization): Consider a simplified one-dimensional LEO satellite localization scenario, where a ground receiver aims to estimate its position x ∈ R using signed pseudorangelike measurements from three visible LEO satellites. Assume that the true receiver position is x⋆ = 0, and the satellite positions and corresponding noisy measurements are (s1 , ρ1 ) = (−10, −10.2), (s2 , ρ2 ) = (0, 0.1), and (s3 , ρ3 ) =

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

15

Estimated location x(x)

NW Regression View of Attention-Based LEO Satellite Localization (1D) 0.4 0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2 15

s1

2 = 0.708

s3

h(l+1) = f (l) (h(l) ),

NW curve x(x) Samples (si, xi) Query x0 = 0 NW output x(x0) = 0.17 True position x * = 0

10

Deep learning models learn hierarchical representations by composing multiple layers of parameterized nonlinear transformations [124], [125]. Formally, a deep neural network can be expressed as a sequence of layer-wise operators:

s2

1 = 0.096

5 0 5 Query / prior location x

10

15

(8, 8.6). The measurements follow a simplified signed model ρi ≈ si − x⋆ + εi , where εi denotes a fixed noise realization. Assume that satellite 2 provides a higher-quality measurement due to a favorable elevation angle and signal-tonoise ratio. Each measurement induces an individual candidate position estimate x̂i = si − ρi , yielding x̂1 = 0.2, x̂2 = −0.1, and x̂3 = −0.6. Note that while εi models the measurement noise, it does not explicitly appear in x̂i = si −ρi , as the noise realization is already absorbed into the observed ρi . To fuse these candidates, we employ a NW estimator with a coarse prior x0 = 0: 3 X

α(x0 , si ) x̂i ,

i=1

k(x0 , si ) α(x0 , si ) = P3 . j=1 k(x0 , sj ) (38)

Choosing a Gaussian similarity kernel with σ = 5: k(x0 , si ) = exp (−

(x0 − si )2 ), 2σ 2

(39)

we obtain: k(x0 , s1 ) ≈ 0.135, k(x0 , s2 ) = 1, k(x0 , s3 ) ≈ 0.278, (40) leading to normalized weights: α(x0 , s1 ) ≈ 0.096, α(x0 , s2 ) ≈ 0.708, α(x0 , s3 ) ≈ 0.197. (41) Consequently, x̂(x0 ) ≈ 0.096 × 0.2 + 0.708 × (−0.1) + 0.197 × (−0.6) ≈ −0.17.

l = 0, 1, . . . , L − 1,

(43)

3 = 0.197

Fig. 13. Nadaraya–Watson (NW) regression view of LEO satellite localization in a one-dimensional setting. Satellites at positions si generate candidate estimates x̂i = si − ρi , which are fused using Gaussian-kernel similarity weights relative to a query x0 . The estimate x̂(x0 ) ≈ −0.17 is dominated by the nearby satellite, illustrating the attention analogy.

x̂(x0 ) =

B. Attention in Deep Learning

(42)

Fig. 13 illustrates NW-based localization as similarityweighted fusion, where nearby measurements dominate the estimate. This behavior directly mirrors attention mechanisms, in which relevance is determined by learned or predefined similarity.

where h(0) = x denotes the input and each layer f (l) (·) typically consists of a structured linear transformation followed by a nonlinear activation. Through end-to-end optimization, these layers progressively transform raw inputs into task-aligned representations. Beyond function approximation, deep learning also relies on aggregation operators to regulate how information from multiple features or intermediate representations is combined [126]. Abstractly, such operations can be written as z = A({hi }i∈S ), where A(·) aggregates a collection of features. The design of this aggregation operator plays a central role in controlling information flow within a network. 1) Pooling as Fixed Aggregation: Early deep architectures relied on deterministic pooling mechanisms to perform feature aggregation. For example, convolutional networks commonly employ max or average pooling over a local neighborhood N ,i.e., h = P({zi }i∈N ), where P(·) denotes a fixed rule such as mean or max pooling [127]. Pooling improves robustness by promoting invariance to small perturbations and spatial variations, but assigns static importance to features and cannot adapt to input-dependent relevance. 2) Attention as Adaptive Aggregation: A fundamental shift occurred with the introduction of attention mechanisms, which replaced fixed aggregation with learnable, data-dependent routing. In neural machine translation, Bahdanau et al. [29] introduced attention to alleviate the fixed-length encoder bottleneck. The context vector ci is computed as a weighted combination of encoder states: ci =

Tx X

exp(eij ) αij hj , αij = PTx , eij = a(si−1 , hj ), k=1 exp(eik ) j=1 (44)

where the normalized weights αij determine the relative importance of each feature. In this view, attention can be interpreted as an adaptive aggregation operator that dynamically routes information based on learned relevance scores. 3) Attention for Set-Structured Learning: The aggregation perspective becomes particularly important when inputs are unordered sets. Zaheer et al. [34] showed that any permutationinvariant function over a set P X = {x1 , . . . , xM } admits the decomposition f (X) = ρ ( x∈X ϕ(x)), which formalizes pooling as a fundamental operator for set modeling. Attention naturally extends this formulation by replacing uniform summation with adaptive weighting: X X f (X) = ρ ( α(x; X)ϕ(x)), α(x; X) = 1. (45) x∈X

x∈X

This idea has been widely adopted in permutation-invariant learning tasks such as multiple instance learning. Ilse et

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

16

From Pooling to Attention: An Operator-Level View learned aggregation

adaptive weighting Fixed Pooling z1

z2

z3 z4

z5

Uniform aggregation max / mean pooling ℎ = 𝑃({𝑧ᵢ})

Kernel-weighted Pooling

Learnable Attention

z1

z1

z2

z3 z4

z5

Similarity-based weighting distance / kernel functions ℎ = Σ 𝑤ᵢ 𝑧ᵢ

z2

z3 z4

z5

Data-dependent routing learned relevance weights ℎ = Σ 𝛼ᵢ 𝑧ᵢ

Fig. 14. From pooling to attention: an operator-level view. Aggregation progresses from fixed pooling, to similarity-based weighted pooling, to learnable attention with adaptive weights, reflecting a shift from static aggregation to context-aware information routing.

al. [128] proposed attention-based pooling for PK PKbag-level representation learning z = k=1 ak hk , where k=1 ak = 1 with a gated scoring function:  ak ∝ exp w⊤ (tanh(V hk ) ⊙ σ(U hk )) . (46) Building on this idea, the set transformer [35] employs selfattention and pooling-by-multihead-attention to perform expressive permutation-invariant aggregation over sets. As shown in Fig. 14, these developments position attention as a general-purpose deep learning operator for adaptive information aggregation. Rather than relying on fixed pooling rules, attention enables networks to dynamically determine the relevance of different inputs. This operator-centric view provides the conceptual foundation for the multi-head softmax attention mechanism used in modern Transformer architectures, which we introduce next. C. Transformer Attention: Multi-Head Softmax Attention The Transformer architecture introduced by Vaswani et al. [31] demonstrated that attention can serve as the primary computational mechanism for sequence modeling, eliminating the need for recurrent or convolutional structures. Instead of processing tokens sequentially, Transformer models allow each element in a sequence to directly interact with all others through attention, enabling efficient modeling of long-range dependencies and global context. From the operator perspective introduced in the previous section, Transformer attention can be viewed as a learnable mechanism for adaptive information aggregation. In this section, we review the standard formulation of multi-head softmax attention, including the query–key–value projections, scaled dot-product attention, and the multi-head aggregation mechanism used in Transformer architectures. 1) Query–Key–Value Linear Projections: Given an input sequence X ∈ Rn×dmodel , where n denotes the sequence length and dmodel the model dimension, the first step in Transformer attention is to construct three representation spaces corresponding to queries, keys, and values. This is achieved through learned linear projections: Q = XWQ ,

K = XWK ,

V = XWV ,

(47)

where WQ , WK ∈ Rdmodel ×dk , WV ∈ Rdmodel ×dv are trainable projection matrices and Q, K ∈ Rn×dk , V ∈ Rn×dv denote the resulting query, key, and value representations.

These projections assign each token three roles in the attention operation: queries specify what information is sought, keys determine how elements are matched, and values represent the content to be aggregated. Attention weights are then computed from query–key similarities, and the corresponding values are combined according to these weights. 2) Scaled Dot-Product Attention: Given the query, key, and value representations introduced above, as shown in Fig. 15, the core computational unit of Transformer attention is the scaled dot-product attention (SDPA). This mechanism maps queries and associated key–value pairs to contextdependent representations by computing similarity scores between queries and keys and using them to aggregate the corresponding values. Formally:   QK⊤ V, (48) Attention(Q, K, V) = softmax √ dk where Q, K ∈ Rn×dk , V ∈ Rn×dv for a single attention head, √ and n denotes the sequence length. The scaling factor dk mitigates the growth of dot-product magnitudes, stabilizing gradients during training. The softmax function is applied row-wise to transform the query–key similarity scores into normalized attention weights, ensuring that the weights associated with each query form a probability distribution over all tokens in the sequence. The resulting attention output therefore remains in Rn×dv while incorporating contextual information from the entire sequence. SDPA can be interpreted as a three-stage procedure: Compatibility assessment: scaled dot products QK⊤ measure the alignment between queries and keys. • Relevance normalization: softmax converts these scores into attention weights. • Contextual aggregation: the value vectors are combined according to these weights.

•

Although the final output is a linear combination of value vectors, the weights themselves are produced by a nonlinear transformation involving query–key interactions and softmax normalization. This combination of nonlinear weighting and linear aggregation underlies the expressive power of softmax attention. 3) Multi-Head Attention: In the multi-head formulation, the scaled dot-product attention mechanism is applied in parallel across h attention heads, each with its own set of projection matrices. The outputs of these heads are concatenated and linearly transformed to produce the final representation: O = Concat(head1 , . . . , headh )WO ,

(49)

where headi = Attention(Qi , Ki , Vi ) ∈ Rn×dv and WO ∈ Rhdv ×dmodel . This design enhances expressiveness by enabling different heads to capture diverse interaction patterns and dependencies. To further illustrate these mechanisms, we consider LEO satellite localization in Example 9, where attention dynamically prioritizes measurement quality by assigning higher weights to high-SNR observations and down-weighting those degraded by interference or poor link conditions.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

17

Multi-Head Attention

Input sequence

Linear projection to Q₁, K₁, V₁

Head 1 softmax(Q₁K₁ᵀ / √dₖ)V₁

Linear projection to Q₂, K₂, V₂

softmax(Q₂K₂ᵀ / √dₖ)V₂

MultiHead(Q,K,V) = Concat(head₁,…,head_h)Wᴼ

Head 2 Concatenate

X = [x₁, …, xₙ]

[head₁; … ; headh ]

Wᴼ Output

Head 3

Linear projection to Q₃, K₃, V₃

softmax(Q₃K₃ᵀ / √dₖ)V₃

Linear projection to Q h , K h , Vh

softmax(Q h K h ᵀ / √dₖ)Vh

Concatenation and output projection

Head h

Parallel attention heads Shared input tokens

Per-head linear projections

Scaled dot-product attention in each head

Fig. 15. Multi-head attention in transformers projects the input sequence into multiple query, key, and value sets, each defining an attention head. Each head applies scaled dot-product attention to compute query–key similarity and aggregate values. The head outputs are then concatenated and linearly projected to form the final representation.

Example 9 (Attention-Based Fusion for LEO Satellite Localization): Consider a one-dimensional LEO satellite localization scenario with three satellites. The objective is to estimate the target position x by fusing measurements from multiple satellites with varying reliability. Each satellite i at position si provides a pseudorange measurement ρi that relates to the target position via:

The final position estimate is obtained as the attentionweighted combination of the value vectors:

ρi ≈ si − x + noise.

This example illustrates selective fusion in attention-based localization. Satellites 1 and 3, with strong signal quality and zero residuals, receive most of the attention (about 50% and 46%), while Satellite 2 is downweighted due to its low SNR (SNR = 0.4) and large residual (|r| = 2). Consequently, the attention-based estimate aligns with the consistent measurements (v = 70), whereas uniform averaging is biased by the outlier, yielding x̄ = 69.3. Fig. 16 provides a visual interpretation of this behavior: attention concentrates weight on reliable observations in the weight domain (Fig. 16a), which directly translates into a more robust position estimate in the spatial domain (Fig. 16b). This highlights its role as an adaptive aggregation mechanism that prioritizes compatibility over uniformity for robust localization under unreliable measurements.

(50)

This yields a position candidate vi = si − ρi for each satellite. Given the satellite positions and measurements: s = [100, 140, 180],

ρ = [30, 72, 110],

(51)

the candidate positions form the value matrix: V = [v1 , v2 , v3 ]⊤ = [70, 68, 70]⊤ .

(52)

To weight each measurement by its reliability, we construct key vectors using two quality indicators: signal quality, represented by normalized SNR values [0.9, 0.4, 0.8] that reflect measurement precision, and consistency, quantified by residuals |ri | = |(si − x0 ) − ρi | relative to a prior estimate x0 = 70, yielding |r| = [0, 2, 0]. The key matrix encodes these features as [SNR, −|r|]:   0.9 0 K = 0.4 −2 . (53) 0.8 0 The query q = [1, 1] expresses a preference for high signal quality and low residuals. Computing dot-product similarity: scores = qK ⊤ = [0.9, −1.6, 0.8], and applying softmax normalization αi = yields attention weights:

(54)

exp(scorei ) P3 j=1 exp(scorej )

α = softmax(scores) ≈ [0.503, 0.041, 0.455].

(55)

x̂ = α⊤ V =

3 X

αi vi

i=1

≈ 0.503 · 70 + 0.041 · 68 + 0.455 · 70 ≈ 69.9.

(56)

D. Gated Attention Despite its empirical success, standard softmax attention exhibits structural limitations that motivate gating mechanisms: • Low-rank constraint: The value (WV ) and output (WO ) projections are linear and sequential, collapsing into a low-rank mapping that limits representational capacity, especially in multi-head or grouped-query settings. • Attention sink: Softmax normalization can induce redundant mass allocation and instability. As shown in [129], this leads to the attention sink phenomenon, where

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

18

Uniform Pooling vs. Attention-Based Weighting Uniform pooling Attention weights

0.6 0.503

Weight

0.5 0.4 0.3

0.333

where X ∈ Rn×dmodel denotes the hidden states used to compute the gating scores, Wθ ∈ Rdmodel ×dv represents the learnable gate parameters, and σ(·) is an activation function, typically the sigmoid function. The operator ⊙ denotes the element-wise product. The gating scores are computed from the hidden states X, which in the case of SDPA output gating correspond to the query-dependent representations, and the gating term σ(XWθ ) ∈ Rn×dv acts as a dynamic filter that modulates the attention output, selectively preserving informative features while suppressing irrelevant components. The gated attention output is projected through the output layer WO ∈ Rhdv ×dmodel :

0.455 0.333

0.333

0.2 0.1

0.041

0.0

Sat 1

SNR=0.9, |r|=0

Sat 2

Sat 3

SNR=0.4, |r|=2

SNR=0.8, |r|=0

Satellite Index

O = MultiHeadgated (Q, K, V )WO .

(a) Uniform pooling vs attention weights

It introduces query-dependent nonlinearity and adaptive sparsity, mitigating attention sinks and improving training stability, especially under large learning rates. Example 10 (Gated Attention for Reliability-Aware LEO Satellite Localization): We revisit the one-dimensional LEO satellite localization example 9 and illustrate how a gating mechanism further refines the attention output by suppressing unreliable contributions. Using the same attention weights:

Candidate Positions and Fusion Estimates v2 = 68

Satellite candidates Uniform: x = 69.33 Attention: x = 69.92 66

67

v1 = 70

SNR=0.4 2 = 0.041

v3 = 70

SNR=0.9 1 = 0.503

SNR=0.8 3 = 0.455

α ≈ [ 0.503, 0.041, 0.456 ] 68

69

Position

70

71

72

(b) Candidate positions and fusion estimates Fig. 16. Visualization of attention-based fusion in a one-dimensional LEO satellite localization example 9. (a): comparison between uniform pooling and learned attention weights, highlighting the suppression of unreliable measurements. (b): candidate position estimates and fused results, illustrating how attention-based aggregation emphasizes consistent, high-quality observations.

excessive weight is assigned to the first token, causing activation spikes. Gating mechanisms, widely used in recurrent and statespace models [130], [131], provide a natural remedy but remain underexplored in standard softmax attention and are often entangled with other components such as sparse attention or mixture-of-experts routing. Recent work [132] introduces a lightweight query-dependent gate implemented as a headspecific sigmoid on the SDPA output, significantly improving attention with negligible overhead. This gating introduces two effects: nonlinearity, breaking the linear composition of WV and WO , and query-dependent sparsity, suppressing irrelevant outputs beyond softmax. It improves stability, mitigates attention sinks, and enhances long-context extrapolation; in particular, it reduces first-token attention from 46.7% to 4.8%, eliminates loss spikes, and improves performance by over 10 points. 1) Formulation: To enhance the expressiveness and stability of the attention output, a gating mechanism can be introduced [132]. Let Y denote the feature representation to be modulated, for example the √ output of scaled dot-product attention Y = softmax(QK ⊤ / dk )V , Y ∈ Rn×dv . As shown in Fig. 17, the gated representation Y ′ is defined as: Y ′ = g(Y, X, Wθ , σ) = Y ⊙ σ(XWθ ),

(58)

(57)

(59)

⊤

and value vectors V = [ 70, 68, 70 ] , the standard attention estimate is: x̂atten ≈ 69.9.

(60)

We now introduce a gated attention mechanism applied to the attention output. The gate is computed from the hidden state features X, which encode reliability cues such as signal quality and residual consistency. For simplicity, assume a scalar gate per satellite, computed as: gi = σ(x⊤ i w),

w = [ 2, 1 ]⊤ ,

and feature vectors identical to the key features:   0.9 0 X = 0.4 −2 . 0.8 0 This yields gating scores:     σ(1.8) 0.858 g ≈ σ(−1.2) ≈ 0.231 . σ(1.6) 0.832

(61)

(62)

(63)

The gated attention output is then computed as: x̂gated =

3 X

αi gi vi

(64)

i=1

≈ 0.503 · 0.858 · 70 + 0.041 · 0.231 · 68 + 0.456 · 0.832 · 70 ≈ 69.6.

(65)

Obviously, compared with standard attention, the gated mechanism further suppresses the contribution of the second satellite, whose measurement exhibits both low signal quality and large residual inconsistency. This example illustrates how gated attention introduces an additional, input-dependent filtering stage beyond softmax weighting, enabling finer control over information flow and improving robustness against unreliable observations.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

19

Gated Attention Mechanism Query Q = XWQ

Input tokens X = [x₁,...,xₙ]

Key K = XWk

Scaled Dot-Product Attention

Sigmoid Gate

A = softmax(QKᵀ / √dₖ)V

g = σ(Wg q) ෩= g ⊙ A A

Output ෩ o 𝑌 = AW

Value V = XWV

Fig. 17. Simplified gated attention mechanism. The input tokens are projected into queries, keys, and values, followed by scaled dot-product attention. A query-dependent sigmoid gate modulates the attention output, allowing adaptive control of information flow across attention heads.

2) Research Advance: Recent advances increasingly view gating as a principled mechanism to overcome key limitations of standard self-attention, including limited expressivity, training instability, inefficient long-context modeling, and hardware inefficiency. Rather than a fixed aggregation operator, gated attention introduces input-dependent control signals that modulate information flow throughout the attention pipeline, improving representation capacity, stability, and efficiency. A first line of work focuses on selective computation, where gates dynamically activate a subset of tokens, projections, or scales. SwitchHead [133] employs mixture-of-experts routing to sparsify projection layers, while token-level gating [134] uses learnable masks to enable sparse attention. Extensions such as native sparse attention [135] further incorporate multiscale gating to balance global context and local precision. A second line of work leverages gating for feature modulation, where one representation controls another. The gated attention unit [136] reformulates attention as a modulation process, enabling efficient softmax-free architectures, while gated linear attention [137] introduces data-dependent forget gates for selective memory retention, improving long-context extrapolation. Related ideas have been adopted in diffusion transformers for efficient high-resolution modeling [138]. A third line of work integrates gating directly into attention scoring and dynamics. For example, forgetting-based attention [131] introduces learnable decay to model recency effects, while head-wise gating [139] enables no-op updates to stabilize training and support low-bit quantization. More recently, lightweight query-dependent gating [132] has been shown to significantly improve training stability by mitigating attention sinks with minimal overhead. These developments reflect a shift from static, globally normalized attention toward dynamic, input-conditioned modulation, where gating serves as a unifying mechanism for enhancing efficiency, stability, and expressivity. E. Software and Framework 1) PyTorch: PyTorch is a widely used deep learning framework designed around an imperative define-by-run execution model, which enables flexible construction of attention mechanisms with data-dependent control flow and adaptive structures [140]. Its automatic differentiation engine supports reversemode differentiation for dynamic programs, recording only

local computation dependencies at runtime without relying on static graphs or explicit tapes, thereby facilitating efficient gradient computation for complex attention-based models [141], [142]. More recently, PyTorch has incorporated compilation and graph-level optimization techniques that combine the flexibility of eager execution with improved performance and scalability, allowing attention-heavy models to be efficiently deployed across heterogeneous hardware platforms [143]. 2) TensorFlow: TensorFlow is a large-scale machine learning system built around a dataflow-graph abstraction that represents both computation and mutable state, and maps graph nodes across distributed machines and heterogeneous devices such as CPUs, GPUs or accelerators to support both training and inference [144]. It was open-sourced by Google in 2015 (Apache 2.0), implemented with a high-performance C++ core and exposed via convenient Python/C++ APIs; this “flowchart” programming model separates model design from execution, enabling deployment from clusters down to mobile devices [145]. TensorFlow further provides automatic differentiation to generate backpropagation and related gradient computations, and includes tooling such as TensorBoard for graph visualization and training monitoring. 3) JAX: Developed by Google Research, JAX [146], [147] is a high-performance numerical computing framework that extends the NumPy [148] API with automatic differentiation and hardware acceleration. Built on the XLA compiler, it enables near-native execution on GPUs and TPUs while supporting composable program transformations such as jit (just-in-time compilation), grad (automatic differentiation), and vmap/pmap (vectorization and parallelization). These features make JAX well suited for large-scale, high-dimensional workloads in attention models and Transformers. Combined with libraries such as Flax and Haiku, it supports modular and scalable model development for complex learning tasks, including attention-based systems in communication and sensing applications [149]. F. Capabilities and Limitations of Attention Mechanisms 1) Capabilities: Attention mechanisms provide a datadependent weighting strategy that enables adaptive information selection during prediction [121], [150]. Rather than compressing inputs into fixed-length representations, attention assigns context-aware weights to different elements, allowing

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

selective aggregation of relevant information while suppressing irrelevant signals [151]. This behavior can be interpreted as a learned alignment process, closely related to similarity-based regression, and has proven effective across sequence modeling, structured prediction, and multimodal learning tasks [152], [153]. At the same time, attention naturally captures longrange and global dependencies by enabling direct interactions between all input elements, avoiding the locality constraints of convolutional models and the sequential bottlenecks of recurrent architectures [150], [154]. Through the Query–Key– Value paradigm, it constructs globally informed representations that underpin the success of Transformer-based models across diverse domains [36], [155]. Beyond its representational power, attention offers strong architectural flexibility and computational advantages. It can be seamlessly integrated as either a modular component or a primary backbone, supporting various forms such as selfattention, encoder–decoder attention, co-attention, and multihead attention for heterogeneous data fusion [156]–[160]. Unlike CNNs or graph neural networks that rely on fixed local structures [28], attention learns input-dependent connectivity patterns, enabling flexible interactions across arbitrary elements. In addition, attention mechanisms are highly parallelizable, allowing simultaneous computation over all input positions and improving efficiency compared to sequential models [150]. The weighting process also acts as an implicit noise filter, enhancing robustness under noisy or heterogeneous observations while providing interpretable signals through the learned attention weights [36], [153], [161]. 2) Limitations: A primary limitation of standard attention, particularly self-attention, is its quadratic time and memory complexity with respect to sequence length [162]. The need to compute all pairwise interactions leads to large matrix operations, making attention computationally expensive for long sequences and high-dimensional inputs [163], [164]. To mitigate this, various efficient variants such as sparse, lowrank, and localized attention have been proposed, though often at the cost of reduced modeling flexibility. Moreover, attention lacks strong inductive biases, as its dot-product interactions do not explicitly encode locality, order, or hierarchical structure [165]. This can hinder generalization, especially in datalimited settings, and motivates the use of additional architectural constraints or hybrid designs [150]. In practice, localized or window-based attention is often used to reduce complexity, but this may fragment global context and weaken long-range dependencies [121], [155]. Beyond computational and structural issues, attention mechanisms also face challenges in reliability modeling and practical deployment. While attention adaptively reweights inputs, it does not explicitly account for measurement reliability or uncertainty, and may capture spurious correlations under noisy or limited data [150]. In addition, despite their theoretical parallelism, attention models can incur significant computational overhead due to large matrix multiplications and limited hardware efficiency, especially on resource-constrained devices [36], [154]. Standard softmax attention further produces dense weight distributions, which may dilute informative signals and propagate noise when many inputs are irrelevant [121].

20

Combined with potential gradient instability caused by large dot-product magnitudes, these issues often require additional normalization, scaling, or architectural modifications to ensure stable and efficient training [151], [153]. G. Attention-Based LEO Satellite Localization Recent advances in attention-based LEO satellite localization reflect a shift from architecture-centric designs toward problem-driven formulations that explicitly account for measurement reliability, spatio-temporal dynamics, and heterogeneous data fusion. In this context, attention is better understood not merely as a feature extraction tool, but as a data-adaptive weighting mechanism that enables context-aware estimation under challenging sensing conditions. This subsection adopts a unified modeling perspective and reviews representative approaches accordingly (cf. Table IV). 1) Unified Modeling Framework: From this perspective, attention-based localization can be formulated as a datadependent weighted estimation process: x̂ =

M X i=1

αi (z)hi ,

exp(g(z, zi )) . αi (z) = PM j=1 exp(g(z, zj ))

(66)

Here, hi denotes the measurement or feature from the i-th source, and αi (z) is a context-dependent weight determined by a learned compatibility function g(·, ·). This formulation highlights attention as a soft, data-driven selection mechanism that emphasizes informative observations while suppressing unreliable ones, which is particularly important in degraded or heterogeneous LEO sensing environments. This model can be further interpreted as a generalized NW estimator, where the kernel is implicitly learned through attention: M X K(z, zi ) x̂(z) = xi . (67) PM j=1 K(z, zj ) i=1 Here, softmax attention corresponds to a normalized, dataadaptive kernel that links classical nonparametric regression with learning-based localization, providing a unified interpretation of attention as similarity-weighted information fusion. 2) Representative Advances: Recent advances in attentionbased LEO satellite localization highlight its effectiveness in addressing measurement reliability, spatio-temporal dynamics, and heterogeneous data fusion. A key challenge lies in the highly variable reliability of measurements under NLOS and multipath conditions. Attention provides a principled solution by learning data-dependent weighting patterns from heterogeneous signal features. For instance, the environmental transformer in [166] employs self-attention to jointly model pseudo-range, Doppler, satellite geometry, and environmental cues for NLOS identification, improving robustness over CNN- and LSTM-based approaches. Similar ideas have been explored in GNSS monitoring and interference detection, where attention weights offer interpretable indicators of measurement reliability across receivers [169]. In addition, LEO systems exhibit strong non-stationarity due to orbital dynamics and environmental perturbations, making

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

21

TABLE IV R EPRESENTATIVE ATTENTION -BASED M ETHODS FOR LEO S ATELLITE L OCALIZATION . Work

Target

Problem / Motivation

Attention Mechanism

Key Challenge

[166]

NLOS/LOS state

Performance degradation under NLOS conditions

Transformer-based multimodal feature fusion

Noise robustness and sensor synchronization

[167]

Orbit trajectory

Non-stationary dynamics and orbital perturbations

Global–local dual-scale attention

Model interpretability and uncertainty

[168]

User position

Cross-environment generalization gap

Adaptive attention-based model weighting

Extreme scenarios and scalability

[169]

Interference

Complex spatio-temporal interference patterns

Multi-head global–local fusion

Multi-source detection and latency

[170]

User position

Multipath and signal blockage errors

Attention-driven GNSS bias correction

GNSS outages and LEO-only precision

[171]

Clock bias

Long-term non-stationary clock drift

Attention-augmented LSTM modeling

Non-stationarity and drift accumulation

[172]

Orbit anomaly

Mismatches between models and real-world data

Transformer-based residual modeling

Onboard efficiency and compute cost

[173]

Orbit prediction

Perturbation-induced prediction errors

Attention-based error refinement

Complexity–accuracy trade-off

fixed-order temporal models inadequate. Attention-based architectures, particularly Transformers, effectively capture both long-range dependencies and local variations. For example, GloLoSAT [167] integrates global and local attention to refine orbit prediction, while [173] combines physical dynamics with Transformer-based error correction to model complex perturbations. Attention has also been applied to anomaly detection and residual modeling, demonstrating robustness under sparse and irregular time-series observations [172]. Beyond reliability and dynamics, attention further enables flexible fusion of heterogeneous measurements across modalities, time scales, and reliability levels. DeepSatLoc [170] applies attention to refine GNSS observables before integrating them with Doppler, angle-of-arrival, and inertial measurements, reducing multipath-induced errors without explicit environmental modeling. Similar principles appear in terrestrial systems, where attention dynamically selects models under varying propagation conditions [168]. Moreover, attention has been explored in related applications such as satellite clock bias prediction and LEO networking, demonstrating its versatility in modeling large-scale satellite systems [171], [174]. 3) Opportunities and Challenges: Attention mechanisms enable adaptive reliability weighting, allowing localization systems to respond to time-varying measurement noise without requiring manually designed noise models. They also support multi-scale temporal modeling that captures both long-term orbital trends and localized perturbations. Furthermore, attention provides a unified abstraction for heterogeneous measurement fusion, facilitating the integration of diverse observations such as pseudorange, Doppler, and inertial data through learned weighting strategies. Despite these advantages, significant challenges remain. Existing models often lack explicit mechanisms for enforcing physical constraints or incorporating uncertainty structures within the learning process. The interpretability and generalization of learned attention weights across different operational regimes remain limited, which is problematic for safety-critical

applications. Moreover, the theoretical relationship between attention-based fusion and classical state-space estimation frameworks is not yet fully understood, highlighting the need for more principled modeling approaches.

H. Case Study: Attention-Based GPS Correction in LEOAssisted Localization We consider DeepSatLoc [170] as a representative framework to illustrate attention in practical localization. The system improves urban positioning by combining global positioning system (GPS) with LEO communication signals, integrating (i) an attention module for adaptive GPS error correction and (ii) model-based fusion with LEO and inertial measurement unit (IMU) measurements via an extended kalman filter (EKF). Here, attention serves as a data-adaptive weighting mechanism over multi-satellite and temporal features within a hybrid estimation framework. 1) Problem Formulation: The goal is to estimate the user trajectory in urban environments where global navigation satellite system (GNSS) measurements suffer from NLOS and multipath effects. Conventional pseudorange positioning becomes unreliable due to large biases or insufficient highquality satellites, while IMU-based dead reckoning accumulates drift. LEO SatCom signals provide complementary geometric diversity and Doppler information, improving robustness despite lacking precise ranging. Let ru ∈ R3 denote the user position and rgps,k the k-th satellite position. The pseudorange measurement is: pk = |ru − rgps,k | + c∆t + Ik + Tk + ϵNLoS,k + ϵk ,

(68)

where ϵNLoS,k represents NLOS-induced bias, which typically dominates in urban scenarios. While conventional methods estimate (ru , ∆t) via residual minimization, the residuals also indicate measurement reliability.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

Positioning Error Distribution

Satellite Index

Empirical CDF

0.8 0.6 0.4 0.2 0.0

0

5

10

15

20

25

Localization Error (m)

30

35

Trajectory Comparison: GPS vs. Attention-Corrected

Learned Satellite Reliability via Spatial Attention

0

NLoS Event

0.5

400

2

0.4

500

4

0.3

6

0.2

8

0.1 0

20

40

60

80

Time Index (test set)

(a) CDF

100

120

140

y (m)

Baseline LS (GPS-only) Attention-corrected

Attention Weight

1.0

22

Ground Truth GPS Baseline Attention-Corrected

600 700 800 1600

1400

(b) Heatmap

1200

1000

800

x (m)

600

400

200

(c) Trajectory

Fig. 18. Illustrative attention-based correction for satellite localization under bursty NLOS conditions. (a) Cumulative distribution function (CDF) of positioning error comparing baseline Gauss–Newton and attention-corrected estimates, showing reduced large-error tails. (b) Spatial attention weights over time with ground-truth NLOS events (red), indicating learned reliability down-weighting. (c) 2D trajectory comparison, where attention improves accuracy under obstruction-dominated segments.

2) Attention Instantiation: In DeepSatLoc, attention is used as a dedicated correction module (LocAttNet) for mitigating GPS bias, acting as a data-driven analogue of classical satellite weighting with richer contextual and temporal cues. At time t, each visible satellite k provides a feature vector xt,k ∈ Rdf such as SNR, residuals, geometry, and prior estimates, forming: ⊤ ⊤ K×df Xt = [x⊤ . t,1 , . . . , xt,K ] ∈ R

(69)

A sliding window of length T yields Gt ∈ RT ×K×df , capturing temporal context. Spatial Attention. For each epoch Xτ , self-attention across satellites learns inter-satellite reliability: Zτ = softmax (

Xτ WQ (Xτ WK )⊤ √ )Xτ WV , dk

(70)

producing reliability-weighted satellite representations. Here, WQ , WK , WV ∈ Rdf ×dk . Temporal Attention. The per-epoch outputs {Zt−T +1 , . . . , Zt } are first aggregated into global tokens via a permutation-invariant operator (e.g., mean pooling): τ = t − T + 1, . . . , t.

zτ = Pool(Zτ ),

(71)

Stacking these tokens yields: ⊤ ⊤ ⊤ T ×dz Zt = [zt−T . +1 , . . . , zt ] ∈ R

(72)

Multi-head temporal attention is then applied to capture motion-consistent patterns. For each head h: (h)

Qt

(h)

= Zt WQ ,

(h)

Kt

(h)

(h)

= Zt WK ,

Vt

(h)

= Zt WV , (73)

and the attention weights are computed as: (h)

(h)

At

= softmax (

(h) ⊤

Qt Kt √ dk

).

The head outputs are aggregated and fused: (1) (H)
Z̃t = Concat Z̃t , . . . , Z̃t WO ,

(74)

(75)

and the representation at the current epoch is taken as the last token z̃t = Z̃t [T, :]. This mechanism captures long-range temporal dependencies and emphasizes motion-consistent patterns while suppressing transient anomalies.

Output and Learning. The fused representation predicts a GPS correction: ∆rut = fθ (Gt ) ≈ Wout z̃t + b,

t r̂ut = rgps + ∆rut ,

(76)

where Wout and b denote a lightweight regression head. The network parameters θ are learned by minimizing a supervised regression loss: X L(θ) = ∥r̂ut − rugt (t)∥22 , (77) t

where rugt (t) denotes the ground-truth position. Integration with Model-Based Localization. DeepSatLoc integrates LocAttNet as an EKF front-end. The EKF fuses IMU propagation with corrected GPS observations r̂ut , mitigating NLOS bias and drift. LEO signals add constraints via Doppler and AoA, improving robustness under degraded GPS. 3) Illustrative Experiment: We consider a simplified 2D simulation to illustrate the role of attention in urban satellite localization. A vehicle follows a smooth trajectory over 500 timesteps with K=10 satellites uniformly distributed in azimuth. Bursty NLOS errors (20–50 timesteps, 50 ± 18 m) are injected with an overall contamination rate of ∼12%. Baseline. A regularized Gauss–Newton estimator is used without reliability weighting. The attention-based corrector takes four per-satellite features (SNR, elevation proxy, residual magnitude, signed residual) over a window of length T =10, forming Gt ∈ RT ×K×4 . Trained on 340 samples, it predicts corrections ∆ru to obtain r̂u = rgps + ∆ru . Results. We evaluate the attention-based corrector through error distributions, learned attention weights, and trajectorylevel comparisons. Fig. 18a shows the CDF of positioning errors on a 150-sample test trajectory. The baseline GPS exhibits significant dispersion, with a mean error of 13.2 m and a long tail beyond 30 m due to NLOS bias. In contrast, the attention-corrected results reduce the mean error to 9.8 m (26% improvement) and noticeably tighten the distribution: the 90th percentile decreases from 22.5 m to 16.8 m, and the maximum error from 35.2 m to 24.6 m. This indicates that attention primarily improves robustness by suppressing large NLOS-induced errors rather than simply shifting the average. Fig. 18b shows the learned attention weights αt,k alongside ground-truth NLOS events (red markers). A clear inverse correlation is observed: NLOS-affected satellites are consistently downweighted (α < 0.05), while unobstructed ones

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

retain higher weights (α ≈ 0.15–0.25), despite no explicit reliability supervision. Satellites undergoing NLOS periods exhibit persistently low weights, whereas LOS satellites maintain stable contributions, indicating that attention learns data-driven reliability weighting from features such as SNR, geometry, and residuals. Fig. 18c compares 2D trajectories. The baseline GPS estimate (red) deviates significantly from ground truth (black) during NLOS-dominated segments, especially near turns, while the attention-corrected trajectory (blue) more closely follows the true path with reduced scatter and bias. Shorter error vectors, particularly in high-error cases, confirm improved accuracy and robustness. 4) Insights and Limitations: DeepSatLoc provides a practical instantiation of attention for LEO-assisted localization. By exploiting per-satellite quality and geometric cues, attention learns a data-driven robustness rule that suppresses NLOS measurements without explicit propagation models. Restricting attention to GPS bias correction while delegating dynamics and fusion to an EKF yields a modular and interpretable design. While LEO Doppler and AoA add geometric constraints, the main gains come from attention-corrected GPS updates that re-anchor the state. Empirically, LocAttNet reduces heavytailed GPS errors, and EKF fusion further improves trajectory accuracy under challenging conditions. Despite strong performance, several limitations remain. First, attention weights are implicit and lack a probabilistic interpretation of reliability. Second, integrating corrected measurements into an EKF requires principled covariance modeling, yet current methods rely on heuristics. Third, the approach assumes reliable LEO Doppler and AoA, which may degrade in dense urban settings, motivating reliability-aware gating. Finally, training is often scenario-specific, and robust generalization across environments, hardware, and spectrum conditions remains open. V. ATTENTION-BASED SPECTRUM CARTOGRAPHY: MODELS AND ANALYSIS Building on the attention mechanisms introduced earlier, this section explores their integration into spectrum cartography for LEO satellite systems. We focus on two representative tasks: satellite-assisted localization and radio map reconstruction. For each, we present attention-based formulations that combine learnable aggregation with physical measurement models, illustrating how attention enables adaptive information fusion and improved robustness under sparse and heterogeneous observations.

23

1) NW Attention Formulation: For each index i, the attention weights are defined by a positive kernel with row K normalization: K̂ij = Pn′ ijK ′ . Here, Kij encodes temporal ij j =1 proximity and measurement similarity, while the bandwidth parameter h controls the effective attention radius along the observation sequence. 2) State Estimation: Let zj denote a localization hypothesis derived from xj . The NW attention estimator Pn measurement Pn is ŷ = n1 i=1 j=1 K̂ij zj . This estimator performs convex local aggregation, suppressing inconsistent observations while preserving the smooth variation induced by orbital motion. The bandwidth h is learned jointly with the localization module, enabling adaptive control of attention span across different satellite passes. Overall, the model can be viewed as a lightweight and physically interpretable kernel-based attention mechanism tailored to the sequential structure of LEO trajectories. 3) Numerical Example: We illustrate NW-attention localization using a synthetic two-dimensional trajectory generated over observation times {ti }ni=1 :   cos(2πti ) + 0.08 cos(6πti ) ⋆ yi = . (78) 0.8 sin(2πti ) + 0.05 sin(4πti ) Noisy localization hypotheses are formed as zi = yi⋆ + ηi , where ηi is Gaussian noise with standard deviation σ = 0.1, and 7.5% of observations are further corrupted as outliers. We compare three estimators: raw hypotheses, a moving-average Pn smoother, and the NW-attention estimatorŷi = j=1  K̂ij zj , (t −t )2

with Gaussian kernel weights Kij = exp − i2σ2j , where σ controls the attention span. Performance is evaluated using root mean square error (RMSE) and a trajectory-smoothness metric based on discrete second-order differences. For a representative run with n = 160, the RMSE values are RMSEraw = 0.265, RMSEMA = 0.064, RMSENW = 0.068. Both smoothing methods substantially outperform the raw hypotheses. Although NW attention yields a similar RMSE to moving averaging, it produces a smoother trajectory and is less sensitive to isolated outliers. As shown in Fig. 19, the raw hypotheses are highly scattered around the ground-truth orbit, while the movingaverage estimator only partially suppresses local corruption. In contrast, NW attention more closely follows the true trajectory and yields a more stable time-series estimate. The learned weight matrix exhibits a banded structure along the diagonal, indicating that each measurement primarily attends to a local temporal neighborhood. Overall, NW attention provides an interpretable and lightweight alternative to fixed-window smoothing for sequential satellite localization.

A. LEO Satellite Localization via Attention We consider LEO satellite localization from a time-ordered sequence of ground observations, D = {xi }ni=1 , where each xi contains Doppler, angular, or timing measurements collected during a satellite pass. The objective is to estimate the satellite state y. Rather than using fixed-window filtering or transformer-style dot-product attention, we adopt a NW attention mechanism for locality-aware aggregation over sequential measurements.

B. Radio Map Reconstruction via Learnable Attention 1) Problem: Consider a simple illustrative example of radio map reconstruction from sparse power measurements. Let Ω ⊂ R2 denote a two-dimensional geographical region, and let g : Ω → R represent the unknown radio power field. A sparse set of measurements is available: D = {(xi , yi )}N i=1 ,

xi ∈ Ω,

yi = g(xi ) + εi ,

(79)

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

0.0

0.5

0.5

0.0

140

0.5

1.0 Ground truth trajectory Noisy hypotheses Moving average NW attention Outliers 1.5 1.0 0.5 0.0 x

0.5

1.0

100

1.5

20

(a)

0.2

0.4 0.6 0.8 Normalized time

(b)

1.0

0.04

60 40

0.0

0.06

80

1.0

1.5

0.08

120 Query index i

1.0

2.0

Ground truth x(t) Noisy hypotheses Moving average NW attention Outliers

1.5

0.5

1.5

NW attention weight matrix

x-coordinate estimation over time

LEO localization demo: 2D trajectory recovery

x

y

1.0

24

00

0.02 50 100 Measurement index j

150

0.00

(c)

Fig. 19. Illustration of LEO satellite localization with NW-attention aggregation. (a) 2D trajectory reconstruction from noisy hypotheses. (b) x-coordinate estimation over time. (c) Attention weights K̂ij , showing temporal locality.

where xi denotes the measurement location and yi is the observed received power corrupted by noise. The objective is to estimate the field value at any query location q ∈ Ω: ĝ(q) ≈ g(q),

(80)

and thereby reconstruct the REM over a dense spatial grid. 2) Learnable Attention Method: Instead of relying on fixed spatial interpolation, we treat REM reconstruction as a queryconditioned aggregation over the measurement set. For a query point q, each observation (xi , yi ) is assigned a relevance score:
ei (q) = aθ [ q, xi , yi ] , (81) where aθ (·) is a learnable scoring function. The corresponding attention weights are obtained through softmax normalization: exp(ei (q)) αi (q) = PN . j=1 exp(ej (q))

(82)

The predicted field value is then obtained by aggregating the measurements using these query-dependent weights: ĝ(q) = fθ q,

N X

αi (q) yi ,

(83)

i=1

where fθ (·) denotes a learnable prediction function. Given training queries {qm }M m=1 and their ground-truth field values {g(qm )}M , the model parameters θ can be learned m=1 by minimizing the mean squared error (MSE): M

L(θ) =

2 1 X ĝ(qm ) − g(qm ) . M m=1

(84)

3) Simulation: To provide an illustrative comparison, we generate a synthetic radio field over a two-dimensional square region Ω = [0, 1]2 with several smooth source-like components and mild spatial variations. A sparse set of noisy measurements is randomly sampled from this field and used as the available observations. For the attention-based model, additional query locations are sampled from the same grid

as supervised training points, and the model parameters are learned by minimizing the MSE objective in (84). Baselines. We compare attention with two classical nonparametric interpolation methods: • IDW: estimates the field value at a query location by a weighted average of nearby measurements, where the weights are inversely proportional to the spatial distance. Formally: PN wi (q) yi 1 ĝIDW (q) = Pi=1 , , wi (q) = N ∥q − xi ∥p i=1 wi (q) (85) •

where p > 0 is a distance-decay parameter. NW: replaces inverse-distance weights with a normalized kernel function, producing a smooth non-parametric regression estimate: PN k(∥q − xi ∥) yi ĝNW (q) = Pi=1 , (86) N i=1 k(∥q − xi ∥)

where k(·) is chosen as a Gaussian kernel. Metrics. We report the RMSE, mean absolute error (MAE), maximum absolute error (MaxAE), and coefficient of determination (R2 ) to compare the reconstruction quality of different methods. Results. Fig. 20 presents the absolute reconstruction error maps produced by IDW, NW, and the attention-based method. Brighter regions indicate larger reconstruction errors. Both IDW and NW exhibit pronounced localized errors, particularly around areas with strong signal variations and high-intensity peaks, reflecting their limited ability to capture complex spatial structures under sparse sampling. In contrast, the attentionbased model produces substantially smaller and more spatially uniform errors across the domain, indicating improved reconstruction fidelity and better preservation of the underlying field structure. The quantitative comparison in Fig. 21 confirms these observations. The attention model significantly reduces RMSE,

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

IDW Error

1.0

25

NW Error

1.0

Attention Error

1.0

1.75

0.6

0.6

0.6

1.50 1.25

y

1.00

0.4

0.4

0.4

0.2

0.2

0.2

0.75

Absolute error

0.8

y

0.8

y

0.8

0.50 0.25 0.0 0.0

0.2

0.4

0.6

0.8

0.0 0.0

1.0

0.2

x

0.4

0.6

x

0.8

1.0

0.0 0.0

0.00 0.2

0.4

0.6

0.8

1.0

x

Fig. 20. Absolute reconstruction error maps for IDW, NW, and the attention-based method. Brighter regions indicate larger reconstruction errors. The attentionbased approach produces significantly smaller and more uniformly distributed errors across the spatial domain.

2.00

IDW NW Attention

1.75

Metric value

1.50 1.25 1.00

loss decreases rapidly in the early epochs and gradually stabilizes after several hundred iterations. The smoothed curve highlights a steady convergence trend with limited oscillation. This behavior indicates that the learnable attention parameters can be reliably optimized from sparse supervision, enabling accurate reconstruction of the underlying radio field.

0.75 0.50

VI. CONCLUSION

0.25 0.00

RMSE

MAE

MaxAE

R2

Fig. 21. Quantitative comparison of radio map reconstruction performance for IDW, NW, and the attention-based method. The attention model achieves substantially lower root mean square error (RMSE), mean absolute error (MAE), and maximum absolute error (MaxAE), while attaining an R2 value close to one, indicating a significantly better fit to the ground-truth radio field. 1.0

Final loss: 1.0533e-02

MSE Loss

0.8

0.6 Training loss Smoothed loss

0.4

0.2

0.0 0

100

200

300

400

500

Epoch

Fig. 22. Training loss curve of the attention-based radio map reconstruction model. The mean squared error (MSE) loss decreases rapidly during early training and gradually stabilizes as the model converges, indicating stable optimization of the learnable attention parameters.

MAE, and maximum absolute error compared with both IDW and NW, while achieving an R2 value close to one, indicating a much better fit to the ground-truth field. This improvement suggests that the learnable attention weights can adaptively emphasize more informative measurements instead of relying solely on distance-based interpolation. Finally, Fig. 22 shows the training curve of the attention model, where the MSE

This paper presented a systematic overview of learningbased spectrum cartography for low earth orbit (LEO) satellite networks, with a focus on attention-based modeling for measurement-driven inference. We framed spectrum cartography as a unifying paradigm for constructing spatial radio intelligence from sparse and heterogeneous observations, encompassing localization, radio map reconstruction, and mapinformed resource allocation. A key insight is that these tasks share a common structure as measurement-set-to-spatialinference problems, where inputs are irregular, reliabilityvarying, and geometrically dependent, conditions under which classical model-driven and interpolation-based approaches are fundamentally limited. Attention mechanisms were introduced as a principled operator for adaptive information aggregation, enabling reliability-aware weighting and flexible fusion of heterogeneous measurements. Representative models and simulation studies demonstrated their effectiveness across LEO satellite localization and radio map reconstruction tasks. Several promising directions remain open for future research. First, integrating attention with dynamic constellation geometry and time-varying propagation models would improve adaptability to rapidly evolving LEO environments. Second, incorporating physics-aware priors into attention-based learning could enhance interpretability and sample efficiency under sparse observation regimes. Third, developing scalable and cooperative sensing frameworks leveraging inter-satellite links and distributed inference, represents a critical step toward next-generation spectrum cartography in large-scale space networks. As LEO constellations continue to expand, learningbased spectrum cartography is poised to play an increasingly central role in enabling intelligent, adaptive, and globally connected satellite systems.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

R EFERENCES [1] X. Luo, H.-H. Chen, and Q. Guo, “LEO/VLEO satellite communications in 6G and beyond networks–technologies, applications, and challenges,” IEEE Network, vol. 38, no. 5, pp. 273–285, 2024. [2] G. Chander and S. Singh. Powering AI-native 6G research with the NVIDIA Sionna research kit. NVIDIA. [Online]. Available: https://developer.nvidia.com/blog/ powering-ai-native-6g-research-with-the-nvidia-sionna-research-kit/ [3] S. Zhang, M. Feizarefi, and A. Mirzaei, “AI-native integrated sensing and communications for self-organizing wireless networks: Architectures, learning paradigms, and system-level design,” arXiv preprint arXiv:2601.02398, 2025. [4] D.-R. Emenonye, H. S. Dhillon, and R. Michael Buehrer, “Fundamentals of LEO-based localization,” IEEE Transactions on Information Theory, vol. 71, no. 7, pp. 5277–5311, 2025. [5] H. Al-Hraishawi, H. Chougrani, S. Kisseleff, E. Lagunas, and S. Chatzinotas, “A survey on nongeostationary satellite systems: The communication perspective,” IEEE Communications Surveys & Tutorials, vol. 25, no. 1, pp. 101–132, 2022. [6] J. C. McDowell, “The low earth orbit satellite population and impacts of the SpaceX Starlink constellation,” The Astrophysical Journal Letters, vol. 892, no. 2, p. L36, 2020. [7] M. Y. Abdelsadek, A. U. Chaudhry, T. Darwish, E. Erdogan, G. Karabulut-Kurt, P. G. Madoery, O. B. Yahia, and H. Yanikomeroglu, “Future space networks: Toward the next giant leap for humankind,” IEEE Transactions on Communications, vol. 71, no. 2, pp. 949–1007, 2023. [8] M. Hui, S. Zhai, D. Wang, T. Hui, W. Wang, P. Du, and F. Gong, “A review of LEO satellite communication payloads for integrated communication, navigation, and remote sensing: Opportunities, challenges, future directions,” IEEE Internet of Things Journal, 2025. [9] D. Shahaf, C. Guestrin, E. Horvitz, and J. Leskovec, “Information cartography,” Communications of the ACM, vol. 58, no. 11, pp. 62– 73, 2015. [10] K. Song, M. Zingraff, and H. Zeng, “Spectrum shortage for radio sensing? leveraging ambient 5G signals for human activity detection,” in IEEE INFOCOM 2026-IEEE Conference on Computer Communications. IEEE, 2026, pp. 1–10. [11] Z. Xu, J. Li, and W. Gong, “Practical downlink communication for backscatter tags with commodity radios,” in IEEE INFOCOM 2026IEEE Conference on Computer Communications. IEEE, 2026, pp. 1–10. [12] W. Liu and J. Chen, “UAV-aided radio map construction exploiting environment semantics,” IEEE Transactions on Wireless Communications, vol. 22, no. 9, pp. 6341–6355, 2023. [13] R. Shrestha, T. N. Ha, P. Q. Viet, and D. Romero, “Radio map estimation in the real-world: Empirical validation and analysis,” in 2023 IEEE Conference on Antenna Measurements and Applications (CAMA). IEEE, 2023, pp. 169–174. [14] D. Wang, H. Qin, and Z. Huang, “Doppler positioning of LEO satellites based on orbit error compensation and weighting,” IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1–11, 2023. [15] C. Shi, Y. Zhang, and Z. Li, “Revisiting Doppler positioning performance with LEO satellites,” GPS Solutions, vol. 27, no. 3, p. 126, 2023. [16] G. Fan, X. Chen, Z. Chen, R. Zhang, P. Wu, Q. Wei, W. Xu, J. Dai, and L. Cao, “Toward massive satellite signals of opportunity positioning: Challenges, methods, and experiments,” Space: Science & Technology, vol. 4, p. 0191, 2024. [17] F. S. Prol, R. M. Ferre, Z. Saleem, P. Välisuo, C. Pinell, E. S. Lohan, M. Elsanhoury, M. Elmusrati, S. Islam, K. Çelikbilek, K. Selvan, J. Yliaho, K. Rutledge, A. Ojala, L. Ferranti, J. Praks, M. Z. H. Bhuiyan, S. Kaasalainen, and H. Kuusniemi, “Position, navigation, and timing (PNT) through low earth orbit (LEO) satellites: A survey on current status, challenges, and opportunities,” IEEE Access, vol. 10, pp. 83 971–84 002, 2022. [18] W. Stock, R. T. Schwarz, C. A. Hofmann, and A. Knopp, “Survey on opportunistic PNT with signals from LEO communication satellites,” IEEE Communications Surveys & Tutorials, vol. 27, no. 1, pp. 77–107, 2025. [19] A. H. Sayed, A. Tarighat, and N. Khajehnouri, “Network-based wireless location: Challenges faced in developing techniques for accurate wireless location information,” IEEE Signal Processing Magazine, vol. 22, no. 4, pp. 24–40, 2005.

26

[20] J. A. del Peral-Rosado, R. Raulefs, J. A. López-Salcedo, and G. SecoGranados, “Survey of cellular mobile radio localization methods: From 1G to 5G,” IEEE Communications Surveys & Tutorials, vol. 20, no. 2, pp. 1124–1148, 2017. [21] Y. Zhu, M. Chen, S. Wang, Y. Hu, Y. Liu, C. Yin, and T. Q. Quek, “Passive inter-satellite localization accuracy optimization in low earth orbit satellite networks,” IEEE Transactions on Wireless Communications, 2025. [22] A. Shibli and T. Zanouda, “Data-driven radio environment map estimation using graph neural networks,” in 2024 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 2024, pp. 650–655. [23] C. Phillips, M. Ton, D. Sicker, and D. Grunwald, “Practical radio environment mapping with geostatistics,” in 2012 IEEE International Symposium on Dynamic Spectrum Access Networks. IEEE, 2012, pp. 422–433. [24] M. Pesko, T. Javornik, A. Kosir, M. Stular, and M. Mohorcic, “Radio environment maps: The survey of construction methods,” KSII Transactions on Internet and Information Systems, vol. 8, no. 11, pp. 3789– 3809, 2014. [25] E. D. Kaplan and C. Hegarty, Understanding GPS/GNSS: Principles and applications. Artech House, 2017. [26] R. B. Langley et al., “Dilution of precision,” GPS World, vol. 10, no. 5, pp. 52–59, 1999. [27] K. O’shea and R. Nash, “An introduction to convolutional neural networks,” arXiv preprint arXiv:1511.08458, 2015. [28] F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2009. [29] D. Bahdanau, “Neural machine translation by jointly learning to align and translate,” International Conference on Learning Representations, 2015. [30] K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv preprint arXiv:1406.1078, 2014. [31] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems, vol. 30, 2017. [32] E. A. Nadaraya, “On estimating regression,” Theory of Probability & Its Applications, vol. 9, no. 1, pp. 141–142, 1964. [33] G. S. Watson, “Smooth regression analysis,” Sankhyā: The Indian Journal of Statistics, Series A, pp. 359–372, 1964. [34] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola, “Deep sets,” Advances in Neural Information Processing Systems, vol. 30, 2017. [35] J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh, “Set transformer: A framework for attention-based permutation-invariant neural networks,” in International Conference on Machine Learning. PMLR, 2019, pp. 3744–3753. [36] M.-H. Guo, T.-X. Xu, J.-J. Liu, Z.-N. Liu, P.-T. Jiang, T.-J. Mu, S.H. Zhang, R. R. Martin, M.-M. Cheng, and S.-M. Hu, “Attention mechanisms in computer vision: A survey,” Computational Visual Media, vol. 8, no. 3, pp. 331–368, 2022. [37] P. Q. Viet and D. Romero, “Spatial transformers for radio map estimation,” in ICC 2025-IEEE International Conference on Communications. IEEE, 2025, pp. 6155–6160. [38] L. Tao and C. W. Tan, “Accelerating regularized attention kernel regression for spectrum cartography,” arXiv preprint arXiv:2604.25138, 2026. [39] X. Chen, J. Wang, and Q. Huang, “Dynamic spectrum cartography: Reconstructing spatial-spectral-temporal radio frequency map via tensor completion,” IEEE Transactions on Signal Processing, vol. 73, pp. 1184–1199, 2025. [40] S. Shrestha, X. Fu, and M. Hong, “Deep spectrum cartography: Completing radio map tensors using learned neural models,” IEEE Transactions on Signal Processing, vol. 70, pp. 1170–1184, 2022. [41] M. A. Oliver and R. Webster, “Kriging: A method of interpolation for geographical information systems,” International Journal of Geographical Information System, vol. 4, no. 3, pp. 313–332, 1990. [42] S. Timilsina, S. Shrestha, L. Cheng, and X. Fu, “Domain-factored untrained deep prior for spectrum cartography,” IEEE Signal Processing Letters, vol. 32, pp. 3440–3444, 2025. [43] F. Jaensch, Ç. Yapar, G. Caire, and B. Demir, “Radio map prediction from noisy environment information and sparse observations,” arXiv preprint arXiv:2602.11950, 2026.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

[44] D. Romero and S.-J. Kim, “Radio map estimation: A data-driven approach to spectrum cartography,” IEEE Signal Processing Magazine, vol. 39, no. 6, pp. 53–72, 2022. [45] Y. S. Reddy, A. Kumar, O. J. Pandey, and L. R. Cenkeramaddi, “Spectrum cartography techniques, challenges, opportunities, and applications: A survey,” Pervasive and Mobile Computing, vol. 79, p. 101511, 2022. [46] R. Levie, Ç. Yapar, G. Kutyniok, and G. Caire, “RadioUNet: Fast radio map estimation with convolutional neural networks,” IEEE Transactions on Wireless Communications, vol. 20, no. 6, pp. 4001–4015, 2021. [47] R. Shrestha, D. Romero, and S. P. Chepuri, “Spectrum surveying: Active radio map estimation with autonomous UAVs,” IEEE Transactions on Wireless Communications, vol. 22, no. 1, pp. 627–641, 2022. [48] D. Romero, S.-J. Kim, and G. B. Giannakis, “Stochastic semiparametric regression for spectrum cartography,” in 2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP). IEEE, 2015, pp. 513–516. [49] H. Ids, M. Hamid, T. Jordbru, L. R. Cenkeramaddi, B. Beferull-Lozano et al., “Spectrum cartography using adaptive radial basis functions: Experimental validation,” in 2017 IEEE 18th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC). IEEE, 2017, pp. 1–4. [50] Y. Teganya, D. Romero, L. M. L. Ramos, and B. Beferull-Lozano, “Location-free spectrum cartography,” IEEE Transactions on Signal Processing, vol. 67, no. 15, pp. 4013–4026, 2019. [51] F. Jaensch, G. Caire, and B. Demir, “Radio map prediction from aerial images and application to coverage optimization,” IEEE Transactions on Wireless Communications, vol. 25, pp. 308–320, 2026. [52] B. Schölkopf and A. J. Smola, Learning with kernels: Support vector machines, regularization, optimization, and beyond. MIT Press, 2002. [53] A. B. H. Alaya-Feki, S. B. Jemaa, B. Sayrac, P. Houze, and E. Moulines, “Informed spectrum usage in cognitive radio networks: Interference cartography,” in 2008 IEEE 19th International Symposium on Personal, Indoor and Mobile Radio Communications, 2008, pp. 1–5. [54] A. Ben Hadj Alaya-Feki, B. Sayrac, S. Ben Jemaa, and E. Moulines, “Interference cartography for hierarchical dynamic spectrum access,” in 2008 3rd IEEE Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2008, pp. 1–5. [55] Z. Xu, B. Huang, and B. Jia, “An efficient radio map learning scheme based on kernel density function,” IEEE Transactions on Vehicular Technology, vol. 70, no. 12, pp. 13 315–13 324, 2021. [56] Ç. Yapar, S. Bakirtzis, A. Lutu, I. Wassell, J. Zhang, and G. Caire, “The sampling-assisted pathloss radio map prediction competition,” in 2025 IEEE 35th International Workshop on Machine Learning for Signal Processing (MLSP). IEEE, 2025, pp. 1–6. [57] Z. Li, H. Wang, Z. Shen, S. Zhang, H. Jiang, and Z. Liu, “A secure wireless transmission scheme: Reconstructing spatial radio environment map and redirecting electromagnetic signal propagation path,” IEEE Open Journal of the Communications Society, 2025. [58] A. Chen, S. Mao, Z. Li, H. Zhang, D. Niyato, and Z. Han, “A Gaussian splatting approach to continuous radio map construction,” in IEEE INFOCOM 2025-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS). IEEE, 2025, pp. 1–6. [59] S. Bi, J. Lyu, Z. Ding, and R. Zhang, “Engineering radio maps for wireless resource management,” IEEE Wireless Communications, vol. 26, no. 2, pp. 133–141, 2019. [60] N. A. Khalek, D. H. Tashman, and W. Hamouda, “Advances in machine learning-driven cognitive radio for wireless networks: A survey,” IEEE Communications Surveys & Tutorials, vol. 26, no. 2, pp. 1201–1237, 2023. [61] O. E. Dare, K. Okokpujie, and E. Adetiba, “Radio environment map construction: A mini-review,” in 2023 2nd International Conference on Multidisciplinary Engineering and Applied Science (ICMEAS), vol. 1. IEEE, 2023, pp. 1–7. [62] J. Wang, Q. Zhu, Z. Lin, J. Chen, G. Ding, Q. Wu, G. Gu, and Q. Gao, “Sparse Bayesian learning-based hierarchical construction for 3D radio environment maps incorporating channel shadowing,” IEEE Transactions on Wireless Communications, vol. 23, no. 10, pp. 14 560– 14 574, 2024. [63] D. Shepard, “A two-dimensional interpolation function for irregularlyspaced data,” in Proceedings of the 1968 23rd ACM National Conference, 1968, pp. 517–524. [64] E. Fix, Discriminatory analysis: Nonparametric discrimination, consistency properties. USAF School of Aviation Medicine, 1985, vol. 1. [65] C. De Boor and C. De Boor, A practical guide to splines. Springer New York, 1978, vol. 27.

27

[66] R. Sibson, “A brief description of natural neighbour interpolation,” Interpreting Multivariate Data, pp. 21–36, 1981. [67] H. Liu, L. Ziao, Z. Zhang, and K. Chen, “Neural network parameterized Bayesian nonstationary radio map estimation with uncertain location,” in IEEE INFOCOM 2026-IEEE Conference on Computer Communications. IEEE, 2026, pp. 1–10. [68] Z. Ye, F. Wu, C. Zhang, Y. Shao, W. Fan, B. Tang, and Y. Liu, “A diffusion framework for accurate fine-grained radio map reconstruction,” in IEEE Global Communications Conference (GLOBECOM). IEEE, 2025, pp. 1–6. [69] M. Jianping, Z. Zhang, J. Chen, K. Chen, and S. Cui, “Self-supervised learning informed radio environment map estimation with few samples,” in IEEE Global Communications Conference (GLOBECOM). IEEE, 2025, pp. 1–6. [70] P. Yang, N. Zhang, Z. Zhang, J. Chen, K. Chen, and S. Cui, “RadioVAE: Generating probabilistic radio map via variational autoencoder with UNet,” in IEEE Global Communications Conference (GLOBECOM). IEEE, 2025, pp. 1–6. [71] H. Liu, K. Chen, Z. Zhang, L. Wang, and S. Cui, “Land feature aware radio environment map construction using radio oriented heterogeneous multitask Gaussian process,” in IEEE Global Communications Conference (GLOBECOM). IEEE, 2025, pp. 1–6. [72] H. Jia, X. Wang, N. Cheng, R. Sun, and C. Li, “UrbanMIMOMap: A ray-traced MIMO CSI dataset with precoding-aware maps and benchmarks,” in IEEE Global Communications Conference (GLOBECOM). IEEE, 2025, pp. 1–6. [73] L. Liu, X. Chen, and W. Zhang, “OpenPathNet: An open-source RF multipath data generator for AI-driven wireless systems,” arXiv preprint arXiv:2512.17286, 2025. [74] J. G. Walker, “Satellite constellations,” Journal of the British Interplanetary Society, vol. 37, p. 559, Dec. 1984. [75] M. Handley, “Delay is not an option: Low latency routing in space,” in Proceedings of the 17th ACM Workshop on Hot Topics in Networks, 2018, pp. 85–91. [76] B. Hofmann-Wellenhof, H. Lichtenegger, and J. Collins, Global positioning system: Theory and practice. Springer Science & Business Media, 2012. [77] S. Bancroft, “An algebraic solution of the GPS equations,” IEEE Transactions on Aerospace and Electronic Systems, no. 1, pp. 56–59, 2007. [78] P. K. Enge, “The global positioning system: Signals, measurements, and performance,” International Journal of Wireless Information Networks, vol. 1, no. 2, pp. 83–105, 1994. [79] W. Guier and G. Weiffenbach, “A satellite Doppler navigation system,” Proceedings of the IRE, vol. 48, no. 4, pp. 507–516, 2007. [80] D. J. Torrieri, “Statistical theory of passive location systems,” IEEE Transactions on Aerospace and Electronic Systems, no. 2, pp. 183– 198, 2007. [81] S. Gezici, Z. Tian, G. B. Giannakis, H. Kobayashi, A. F. Molisch, H. V. Poor, and Z. Sahinoglu, “Localization via ultra-wideband radios: A look at positioning aspects for future sensor networks,” IEEE Signal Processing Magazine, vol. 22, no. 4, pp. 70–84, 2005. [82] I. Del Portillo, B. G. Cameron, and E. F. Crawley, “A technical comparison of three low earth orbit satellite constellation systems to provide global broadband,” Acta Astronautica, vol. 159, pp. 123–135, 2019. [83] W. Li, Q. Yang, X. Du, M. Li, Q. Zhao, L. Yang, Y. Qin, C. Chang, Y. Wang, and G. Qin, “LEO augmented precise point positioning using real observations from two CENTISPACE™ experimental satellites,” GPS Solutions, vol. 28, no. 1, p. 44, 2024. [84] M. Neinavaie, J. Khalife, and Z. M. Kassas, “Acquisition, Doppler tracking, and positioning with Starlink LEO satellites: First results,” IEEE Transactions on Aerospace and Electronic Systems, vol. 58, no. 3, pp. 2606–2610, 2021. [85] M. S. Braasch and A. Van Dierendonck, “GPS receiver architectures and measurements,” Proceedings of the IEEE, vol. 87, no. 1, pp. 48–64, 2002. [86] K. Ho and Y. Chan, “Solution and performance analysis of geolocation by TDOA,” IEEE Transactions on Aerospace and Electronic Systems, vol. 29, no. 4, pp. 1311–1322, 1993. [87] N. Jardak and Q. Jault, “The potential of LEO satellite-based opportunistic navigation for high dynamic applications,” Sensors, vol. 22, no. 7, p. 2541, 2022. [88] A. Goldsmith, Wireless communications. Cambridge University Press, 2005.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

[89] A. Al-Hourani, S. Kandeepan, and S. Lardner, “Optimal LAP altitude for maximum coverage,” IEEE Wireless Communications Letters, vol. 3, no. 6, pp. 569–572, 2014. [90] H. L. Van Trees, Detection, estimation, and modulation theory, part I: Detection, estimation, and linear modulation theory. John Wiley & Sons, 2004. [91] Y. Bar-Shalom, X. R. Li, and T. Kirubarajan, Estimation with applications to tracking and navigation: Theory algorithms and software. John Wiley & Sons, 2001. [92] P. J. Huber, “Robust estimation of a location parameter,” in Breakthroughs in statistics: Methodology and distribution. Springer, 1992, pp. 492–518. [93] M. L. Psiaki, “Block acquisition of weak GPS signals in a software receiver,” in Proceedings of the 14th international technical meeting of the satellite division of The Institute of Navigation (ION GPS 2001), 2001, pp. 2838–2850. [94] P. J. Teunissen, O. Montenbruck et al., Springer handbook of global navigation satellite systems. Springer, 2017, vol. 10. [95] H. Benzerrouk, Q. Nguyen, F. Xiaoxing, H. Rasaee, R. J. Landry et al., “LEO satellites based Doppler positioning using distributed nonlinear estimation,” IFAC-PapersOnLine, vol. 52, no. 12, pp. 496–501, 2019. [96] A. Baron, P. Gurfil, and H. Rotstein, “Implementation and accuracy of Doppler navigation with LEO satellites,” NAVIGATION: Journal of the Institute of Navigation, vol. 71, no. 2, 2024. [97] A. Allahvirdi-Zadeh, A. El-Mowafy, and K. Wang, “Doppler positioning using multi-constellation LEO satellite broadband signals as signals of opportunity,” NAVIGATION: Journal of the Institute of Navigation, vol. 72, no. 2, 2025. [98] P. J. Huber, “Robust statistics,” in International encyclopedia of statistical science. Springer, 2011, pp. 1248–1251. [99] Z. M. Kassas, S. Kozhaya, H. Kanj, J. Saroufim, S. W. Hayek, M. Neinavaie, N. Khairallah, and J. Khalife, “Navigation with multiconstellation LEO satellite signals of opportunity: Starlink, OneWeb, Orbcomm, and Iridium,” in 2023 IEEE/ION Position, Location and Navigation Symposium (PLANS). IEEE, 2023, pp. 338–343. [100] J. Ma, P. Zheng, X. Liu, Y. Zhang, A. A. Nasir, and T. Y. Al-Naffouri, “Positioning using LEO satellite communication signals under orbital errors,” arXiv preprint arXiv:2511.06060, 2025. [101] C. Chang, Q. Zhao, M. Li, and W. Li, “Augmentation message design for LEO-enhanced precise positioning: In-orbit performance assessment,” Measurement, vol. 243, p. 116314, 2025. [102] S. Kozhaya, J. Saroufim, and Z. Z. M. Kassas, “Unveiling Starlink for PNT,” NAVIGATION: Journal of the Institute of Navigation, vol. 72, no. 1, 2025. [103] W. L. Brogan, “Improvements and extensions of the geometrical dilution of precision concept for selecting navigation measurements,” in PLANS 1980-Position Location and Navigation Symposium, 1980, pp. 27–32. [104] R. Yarlagadda, I. Ali, N. Al-Dhahir, and J. Hershey, “GPS GDOP metric,” IEE Proceedings-Radar, Sonar and Navigation, vol. 147, no. 5, pp. 259–264, 2000. [105] P. W. Holland and R. E. Welsch, “Robust regression using iteratively reweighted least-squares,” Communications in Statistics-Theory and Methods, vol. 6, no. 9, pp. 813–827, 1977. [106] Y. T. Chan and K. C. Ho, “A simple and efficient estimator for hyperbolic location,” IEEE Transactions on Signal Processing, vol. 42, no. 8, pp. 1905–1915, 1994. [107] B. Jin, X. Xu, and T. Zhang, “Robust time-difference-of-arrival (TDOA) localization using weighted least squares with cone tangent plane constraint,” Sensors, vol. 18, no. 3, p. 778, 2018. [108] L. Yang and K. Ho, “An approximately efficient TDOA localization algorithm in closed-form for locating multiple disjoint sources with erroneous sensor positions,” IEEE Transactions on Signal Processing, vol. 57, no. 12, pp. 4598–4615, 2009. [109] Y. Tao, Y. Guo, S. Wang, C. Yu, and Z. Zhu, “Satellite selection strategy and method for signals of opportunity navigation and positioning with LEO communication satellites,” Sensors, vol. 25, no. 1, p. 267, 2025. [110] W. Yu, W. Rhee, S. Boyd, and J. Cioffi, “Iterative water-filling for Gaussian vector multiple-access channels,” IEEE Transactions on Information Theory, vol. 50, no. 1, pp. 145–152, 2004. [111] D. P. Palomar and J. R. Fonollosa, “Practical algorithms for a family of waterfilling solutions,” IEEE Transactions on Signal Processing, vol. 53, no. 2, pp. 686–695, 2005. [112] D. P. Palomar and M. Chiang, “A tutorial on decomposition methods for network utility maximization,” IEEE Journal on Selected Areas in Communications, vol. 24, no. 8, pp. 1439–1451, 2006.

28

[113] A. Ghosh and S. Boyd, “Minimax and convex-concave games,” 2003, EE392o Lecture course notes, Stanford University, Stanford, CA. [114] X. Tong, C. W. Tan, and H. V. Poor, “Adversarial water-filling: Minimax resource allocation optimization with proximal decomposition in open RAN,” in IEEE Global Communications Conference (GLOBECOM), 2025, pp. 3933–3938. [115] V. Va, T. Shimizu, G. Bansal, and R. W. Heath, “Beam design for beam switching based millimeter wave vehicle-to-infrastructure communications,” in 2016 IEEE International Conference on Communications (ICC). IEEE, 2016, pp. 1–6. [116] V. Va, X. Zhang, and R. W. Heath, “Beam switching for millimeter wave communication to support high speed trains,” in 2015 IEEE 82nd Vehicular Technology Conference (VTC2015-Fall). IEEE, 2015, pp. 1–5. [117] X. Hu, Y. Zhang, X. Liao, Z. Liu, W. Wang, and F. M. Ghannouchi, “Dynamic beam hopping method based on multi-objective deep reinforcement learning for next generation satellite broadband systems,” IEEE Transactions on Broadcasting, vol. 66, no. 3, pp. 630–646, 2020. [118] Z. Lin, Z. Ni, L. Kuang, C. Jiang, and Z. Huang, “Dynamic beam pattern and bandwidth allocation based on multi-agent deep reinforcement learning for beam hopping satellite systems,” IEEE Transactions on Vehicular Technology, vol. 71, no. 4, pp. 3917–3930, 2022. [119] R. Siraskar, S. Kumar, S. Patil, A. Bongale, and K. Kotecha, “Application of the Nadaraya-Watson estimator based attention mechanism to the field of predictive maintenance,” MethodsX, vol. 12, p. 102754, 2024. [120] Y. Y. Linke, “Towards insensitivity of Nadaraya–Watson estimators to design correlation,” Theory of Probability & Its Applications, vol. 68, no. 2, pp. 198–210, 2023. [121] S. Chaudhari, V. Mithal, G. Polatkan, and R. Ramanath, “An attentive survey of attention models,” ACM Transactions on Intelligent Systems and Technology, vol. 12, no. 5, pp. 1–32, 2021. [122] J. Hu, B. Xia, B. Chen, W. Yang, and L. Zhang, “Gaussiansr: High fidelity 2D Gaussian splatting for arbitrary-scale image super-resolution,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 4, 2025, pp. 3554–3562. [123] H. Li, J. Liu, M. Sznaier, and O. Camps, “3D-HGS: 3D half-Gaussian splatting,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 10 996–11 005. [124] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015. [125] I. Goodfellow, Y. Bengio, A. Courville, and Y. Bengio, Deep learning. MIT Press Cambridge, 2016, vol. 1, no. 2. [126] Y. Bengio, I. Goodfellow, A. Courville et al., Deep learning. MIT Press Cambridge, MA, USA, 2017, vol. 1. [127] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Advances in Neural Information Processing Systems, vol. 25, 2012. [128] M. Ilse, J. Tomczak, and M. Welling, “Attention-based deep multiple instance learning,” in International Conference on Machine Learning. PMLR, 2018, pp. 2127–2136. [129] G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis, “Efficient streaming language models with attention sinks,” arXiv preprint arXiv:2309.17453, 2023. [130] A. Gu, C. Gulcehre, T. Paine, M. Hoffman, and R. Pascanu, “Improving the gating mechanism of recurrent neural networks,” in International conference on machine learning. PMLR, 2020, pp. 3800–3809. [131] Z. Lin, E. Nikishin, X. O. He, and A. Courville, “Forgetting transformer: Softmax attention with a forget gate,” arXiv preprint arXiv:2503.02130, 2025. [132] Z. Qiu, Z. Wang, B. Zheng, Z. Huang, K. Wen, S. Yang, R. Men, L. Yu, F. Huang, S. Huang et al., “Gated attention for large language models: Non-linearity, sparsity, and attention-sink-free,” arXiv preprint arXiv:2505.06708, 2025. [133] R. Csordás, P. Pi˛ekos, K. Irie, and J. Schmidhuber, “Switchhead: Accelerating transformers with mixture-of-experts attention,” Advances in Neural Information Processing Systems, vol. 37, pp. 74 411–74 438, 2024. [134] L. Xue, X. Li, and N. L. Zhang, “Not all attention is needed: Gated attention network for sequence data,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 04, 2020, pp. 6550– 6557. [135] J. Yuan, H. Gao, D. Dai, J. Luo, L. Zhao, Z. Zhang, Z. Xie, Y. Wei, L. Wang, Z. Xiao et al., “Native sparse attention: Hardware-aligned and natively trainable sparse attention,” in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, pp. 23 078–23 097.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

[136] W. Hua, Z. Dai, H. Liu, and Q. Le, “Transformer quality in linear time,” in International Conference on Machine Learning. PMLR, 2022, pp. 9099–9117. [137] S. Yang, B. Wang, Y. Shen, R. Panda, and Y. Kim, “Gated linear attention transformers with hardware-efficient training,” arXiv preprint arXiv:2312.06635, 2023. [138] L. Zhu, Z. Huang, B. Liao, J. H. Liew, H. Yan, J. Feng, and X. Wang, “Dig: Scalable and efficient diffusion models with gated linear attention,” in Proceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 7664–7674. [139] Y. Bondarenko, M. Nagel, and T. Blankevoort, “Quantizable transformers: Removing outliers by helping attention heads do nothing,” Advances in Neural Information Processing Systems, vol. 36, pp. 75 067–75 096, 2023. [140] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga et al., “PyTorch: An imperative style, high-performance deep learning library,” Advances in Neural Information Processing Systems, vol. 32, 2019. [141] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in Pytorch,” 2017. [142] L. Tao, X. Tong, and C. W. Tan, “Learning to optimize by differentiable programming,” arXiv preprint arXiv:2601.16510, 2026. [143] J. Ansel, E. Yang, H. He, N. Gimelshein, A. Jain, M. Voznesensky, B. Bao, P. Bell, D. Berard, E. Burovski et al., “PyTorch 2: Faster machine learning through dynamic Python bytecode transformation and graph compilation,” in Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2, 2024, pp. 929–947. [144] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard et al., “TensorFlow: A system for large-scale machine learning,” in 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), 2016, pp. 265–283. [145] N. Shukla and K. Fricklas, Machine learning with TensorFlow. Manning Greenwich, 2018, vol. 7, no. 06. [146] J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. WandermanMilne, and Q. Zhang, “JAX: Composable transformations of Python+NumPy programs,” 2018. [Online]. Available: http://github. com/jax-ml/jax [147] R. Frostig, M. J. Johnson, and C. Leary, “Compiling machine learning programs via high-level tracing,” in SysML Conference 2018, Stanford, United States, 2019. [148] S. Van Der Walt, S. C. Colbert, and G. Varoquaux, “The NumPy array: A structure for efficient numerical computation,” Computing in Science & Engineering, vol. 13, no. 2, pp. 22–30, 2011. [149] G. Sapunov, Deep learning with JAX. Simon and Schuster, 2024. [150] J. B. Lee, R. A. Rossi, S. Kim, N. K. Ahmed, and E. Koh, “Attention models in graphs: A survey,” ACM Transactions on Knowledge Discovery from Data, vol. 13, no. 6, pp. 1–25, 2019. [151] D. Soydaner, “Attention mechanism in neural networks: Where it comes and where it goes,” Neural Computing and Applications, vol. 34, no. 16, pp. 13 371–13 385, 2022. [152] K. Cho, B. van Merriënboer, D. Bahdanau, and Y. Bengio, “On the properties of neural machine translation: Encoder–decoder approaches,” in Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, 2014, pp. 103–111. [153] G. Brauwers and F. Frasincar, “A general survey on attention mechanisms in deep learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 3279–3298, 2023. [154] Y. Tang, K. Han, J. Guo, C. Xu, C. Xu, and Y. Wang, “GhostNetv2: Enhance cheap operation with long-range attention,” Advances in Neural Information Processing Systems, vol. 35, pp. 9969–9982, 2022. [155] A. Hassani, S. Walton, J. Li, S. Li, and H. Shi, “Neighborhood attention transformer,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6185–6194. [156] F. D. Keles, P. M. Wijewardena, and C. Hegde, “On the computational complexity of self-attention,” in International Conference on Algorithmic Learning Theory. PMLR, 2023, pp. 597–619. [157] J. Qu, S. Hou, W. Dong, Y. Li, and W. Xie, “A multilevel encoder– decoder attention network for change detection in hyperspectral images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–13, 2022. [158] Y. Xu and H. Chen, “Multimodal optimal transport-based co-attention transformer with global structure consistency for survival prediction,”

29

in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 21 241–21 251. [159] J. Li, X. Wang, Z. Tu, and M. R. Lyu, “On the diversity of multi-head attention,” Neurocomputing, vol. 454, pp. 14–24, 2021. [160] Y. Wang and H. Zhang, “HARP: A novel hierarchical attention model for relation prediction,” ACM Transactions on Knowledge Discovery from Data, vol. 15, no. 2, pp. 1–22, 2021. [161] P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, Y. Bengio et al., “Graph attention networks,” Stat, vol. 1050, no. 20, pp. 10– 48 550, 2017. [162] J. N. Yan, J. Gu, and A. M. Rush, “Diffusion models without attention,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 8239–8249. [163] Z. Shen, M. Zhang, H. Zhao, S. Yi, and H. Li, “Efficient attention: Attention with linear complexities,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 3531–3539. [164] H. Wang, D. Liu, Y. Kang, Y. Li, Z. Lin, N. K. Jha, and Y. Liu, “Attention-driven training-free efficiency enhancement of diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 16 080–16 089. [165] B. L. Edelman, S. Goel, S. Kakade, and C. Zhang, “Inductive biases and variable creation in self-attention mechanisms,” in International Conference on Machine Learning. PMLR, 2022, pp. 5793–5831. [166] Y. Zhao, Z. Dai, F. Li, X. Zhu, and C. Ran, “A new attention-based neural network for the identification of non-line-of-sight signals in data from global navigation satellite systems,” Engineering Letters, vol. 32, no. 10, 2024. [167] G. Huang and T. Shu, “A global-local probsparse self-attention transformer for LEO satellite orbit prediction,” in 2024 International Conference on Machine Learning and Applications (ICMLA). IEEE, 2024, pp. 91–98. [168] I. Yaman, G. Tian, D. Pjanic, F. Tufvesson, O. Edfors, Z. Zhang, and L. Liu, “Adaptive attention-based model for 5G radio-based outdoor localization,” arXiv preprint arXiv:2503.23810, 2025. [169] K. Cai, Z. Di, Y. Zhu, P. Zhao, and C. Shi, “An attention-based convolutional network framework for detection and localization of GNSS interference sources,” IEEE Transactions on Aerospace and Electronic Systems, vol. 60, no. 3, pp. 2995–3011, 2024. [170] B. Kumar, Y. Chen, N. G. Prelcic, T. Shimizu, and A. Ganlath, “DeepSatLoc: A multimodal fusion strategy for enhanced localization in urban canyons exploiting GPS and LEO satellite communication signals,” in 2024 58th Asilomar Conference on Signals, Systems, and Computers. IEEE, 2024, pp. 842–849. [171] C. Cai, M. Liu, P. Li, Z. Li, and K. Lv, “Enhancing satellite clock bias prediction in BDS with LSTM-attention model,” GPS Solutions, vol. 28, no. 2, p. 92, 2024. [172] N. P. Ré, M. Popplewell, M. Caudill, T. Sullivan, T. Hanf, B. Tatman, K. Parmar, T. Presser, S. Chikine, M. Grant et al., “Transformers for orbit determination anomaly detection and classification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 6819–6827. [173] S. Yuan and K. Zhang, “Research on low earth orbit satellite orbit prediction based on a two-stage integrated model,” in Proceedings of the 3rd International Conference on Signal Processing, Computer Networks and Communications, 2024, pp. 153–157. [174] S. S. Hassan, Y. M. Park, Y. K. Tun, W. Saad, Z. Han, and C. S. Hong, “Satellite-based ITS data offloading & computation in 6G networks: A cooperative multi-agent proximal policy optimization DRL with attention approach,” IEEE Transactions on Mobile Computing, vol. 23, no. 5, pp. 4956–4974, 2024.