A Composition-Aware Pretraining Framework for Geospatial Foundation Models

A Composition-Aware Pretraining Framework for Geospatial Foundation Models

arXiv:2608.30817v1 [cs.CV] 31 Aug 2026

Aryan Kashyap Naveen

sriabhi2407@gmail.com Visa Inc Bangalore, Karnataka, India

Pranav Moothedath

Shrutilipi Bhattacharjee

pranavbijum@gmail.com Greenlight Bangalore, Karnataka, India

Abstract Geospatial and remote sensing technologies provide an unprecedented volume of data for earth observation, driving critical applications such as zero-shot image retrieval, object detection, land cover classification, and change detection. Recently, geospatial foundation models have emerged as state-of-the-art methods for these tasks. However, existing pretraining methodologies, predominantly based on patch reconstruction via masked autoencoders or global contrastive learning, process imagery through a single-concept lens. This aggregation fails to capture the highly compositional nature of complex satellite scenery. We propose a compositionaware pretraining framework that explicitly encodes these fractional land-cover mixtures. An offline visual vocabulary of fundamental geospatial textures is constructed by clustering patch embeddings extracted from a frozen vision transformer. Each satellite image cell is then mapped to a histogram denoting its exact fractional makeup, a representation we term the “composition target”. These targets serve as the primary prediction objective, distilled into the backbone via the Sinkhorn Earth Mover’s Distance, which penalizes predictions in proportion to the semantic distance of their error. A Bidirectional Multi-Instance Learning (MIL) objective provides complementary categorical feature alignment. Experimental evaluation reveals that composition-aware pretraining yields substantial gains on region-level understanding tasks requiring semantic similarity judgment (zero-shot image retrieval, scene classification), while remaining competitive on tasks requiring fine-grained spatial precision (segmentation, object detection). Operating with a 36.8M parameter backbone, our framework surpasses SatMAE and Prithvi-EO-2.0 (303M and 600M parameters) on region-level understanding tasks. On the fine-grained ForestNet12 dataset, a rigorous testbed for compositional discrimination, our method nearly doubles the baseline mAP@10 (0.279 to 0.434), directly validating the effectiveness of explicit composition modeling. The code implementation can be found here.

Keywords Geospatial Foundation Models, Remote Sensing, Content-Based Image Retrieval, Visual Representation Learning, Land Cover Analysis

1

Abhishek Srinivas

aryankashyapnaveen@gmail.com National Institute of Technology, Karnataka Surathkal, Karnataka, India

Introduction

Satellite imagery contains rich information to improve geospatial understanding. Recently, geospatial Foundation Models (FMs)

shrutilipi@nitk.edu.in National Institute of Technology, Karnataka Surathkal, Karnataka, India have achieved state-of-the-art performance in generating the highdimensional embeddings required for various tasks such as zero shot image retrieval, change detection, and object detection [4, 6, 17, 22]. They are not, however, without their limitations [20]. Geospatial FMs suffer from a fundamental limitation rooted in their pretraining paradigms: they process imagery through a singleconcept lens. Unlike standard computer vision datasets, such as ImageNet, which predominantly feature single, object-centric subjects, satellite imagery is inherently compositional. A standard satellite image cell rarely contains a monolithic concept. Instead, it is a complex mixture of multiple types of land cover. A coastal port scene, for example, is a fractional mixture of calm water, concrete docks, ship hulls, road markings, and warehouse roofs. Figure 1 illustrates this dichotomy between single-subject natural images and multi-concept geospatial images.

Figure 1: Contrast between the highly compositional nature of satellite imagery and object-centric datasets (e.g., ImageNet). While natural images typically feature a distinct primary subject, geospatial cells represent a mixture of multiple distinct ground-cover textures. Existing pretraining paradigms for geospatial FMs rely on patch reconstruction [6, 17] (masked autoencoders) or global contrastive learning [4, 12, 22]. However, these approaches fail to explicitly model the compositional nature of satellite scenery. Patch reconstruction methods force the model to waste capacity memorizing low-level and redundant background textures instead of learning semantic relationships. Meanwhile, the global contrastive approaches average heterogeneous land-cover mixtures into a single, holistic embedding. A scene containing 60% water and 40% buildings would

Naveen et al.

produce a pooled embedding that semantically represents neither water nor buildings in isolation. Consequently, two entirely different scenes can yield nearly identical pooled vectors if their average spectral content happens to align. In this paper, we propose a composition-aware pretraining framework that shifts the representational focus from whole cells to localized patches. We treat each image cell as a distribution over a finite vocabulary of visual textures. In our framework, first an expressive offline visual vocabulary is computed of k fundamental geospatial building blocks by clustering high-dimensional patch embeddings extracted from a frozen self-distilled vision transformer. Every cell in the training corpus is then mapped to a target histogram using soft assignments, explicitly capturing what individual visual textures are present and in what exact proportions. We call this the "composition-target". While predicting Bag-of-Visual-Words (BoVW) representations has been explored in general computer vision [8] and classic remote sensing [23], we propose a novel integration tailored for modern geospatial foundation models. We train a highly efficient, Mamba-based backbone (DynamicVis) to predict these patch-level target histograms. By explicitly modeling fractional composition as a target, our framework enables the foundation model backbone to generate highly discriminative embeddings that effectively capture the heterogeneity of the Earth’s surface. Our main contributions are summarized as: • We identify the single-concept pooling bottleneck in current geospatial foundation models and introduce a compositionaware pretraining strategy based on predicting visual texture histograms. • We design a robust loss landscape combining EMD distillation with Multi-Instance Contrastive (MIL) auxiliary losses to teach a Mamba-based [9] architecture [4] to understand semantic distances between ground-cover mixtures. • The proposed method demonstrates substantial improvements across tasks that require region-level understanding. The zero-shot image retrieval and scene classification tasks see improvements of up to 56% mAP@10 scores and 7% accuracy, respectively.

2

Background and Related Work

Remote sensing research has been fundamentally transformed by the evolution of embedding techniques and the increase in data scale. Works have transitioned from task-specific architectures [1] to generalized foundation models. Foundation models have brought a notable shift in how remote sensing tasks are approached, making it possible to learn generalizable representations from large volumes of remote sensing data. Most of these models are built on transformer or state-space architectures and are pre-trained at a massive scale to capture the semantics of geospatial imagery. The resulting representations transfer well across downstream tasks, such as classification, segmentation, retrieval, and change detection, often without requiring extensive fine-tuning. The sections below trace this development across popular pretraining paradigms in the field.

2.1

Masked Autoencoder Based Models

The dominant pretraining paradigm for early vision foundation models heavily relied on Masked Image Modeling (MIM). As one of the pioneering foundational models for Earth observation, Prithvi [17] demonstrated the potential of this approach by pretraining a Vision Transformer (ViT) using a masked autoencoding objective on the Harmonized Landsat Sentinel-2 (HLS) dataset. Similarly, models like SatMAE [6] extended this reconstructive framework to process temporal and multi-spectral satellite imagery, proving that reconstructing masked patches could yield enhanced generalizable multi-modal representations. Despite their success in capturing generic textural patterns, MAEbased models suffer from a critical misalignment with the inherent physical properties of Earth observation imagery. Unlike objectcentric natural images, satellite scenes are fundamentally characterized by massive, homogeneous backgrounds. Consequently, pixellevel reconstructive pretraining inadvertently forces the network’s capacity to over-index on memorizing these redundant background statistics, rather than extracting highly discriminative, high-level semantic features for complex land-cover mixtures. While a thorough treatment of this limitation is provided by [4, 16], we briefly restate it here as it directly motivates our composition-aware pretraining objective.

2.2

Contrastive Learning Based Models

To circumvent the limitations of pixel reconstruction, researchers explored contrastive learning approaches, which optimize the latent space by pulling semantically similar views together while pushing dissimilar ones apart. Early adaptations, such as SeCo [12], SkySense [22] and RS-M-CLIP [14] focused on aligning global, imagelevel representations. However, this coarse semantic alignment tends to dilute localized, fine-grained details, effectively collapsing complex, multi-class spatial compositions into a single, diluted holistic embedding. Addressing the severe target sparsity and spatial redundancy of remote sensing, DynamicVis [4] introduced a highly efficient state-space model backbone governed by an adaptive token routing mechanism. To train this dynamic router, the authors proposed a Region-Level Meta-Embedding Multi-Instance Learning (MIL) pretraining paradigm. By treating high-resolution scenes as "bags" of instances and aligning regional visual embeddings with language-guided categorical meta-embeddings, DynamicVis successfully learnt to decouple heterogeneous foreground instances from expansive backgrounds. While the bidirectional MIL approach in DynamicVis provides categorical separation, it still processes regions through a discrete, single-concept classification lens. It does not explicitly quantify the proportional distribution of land-cover categories within a complex patch.

3

Methodology

This section details the proposed composition-aware pretraining framework. First, we explore the creation of a patch-level pseudo dataset; then we discuss the generation of quantitative composition targets, and finally, the geospatial foundation model pretraining framework is explained.

A Composition-Aware Pretraining Framework for Geospatial Foundation Models

3.1

Pretraining Dataset Creation

To capture the high variance of textures within a single geospatial scene, we shift the representational focus from whole images to localized visual patches. We first process the large-scale pretraining dataset containing raw satellite imagery by dividing it into standard 512 × 512 pixel cells. Each patch of these cells is embedded using a ViT encoder to generate expressive descriptions for the composition of each cell. Then the embeddings are clustered, assigned to a visual vocabulary, and soft-assigned per cell to generate composition targets. 3.1.1 Patch Representation. Each cell is passed through a frozen, pretrained DINOv3 (ViT-L/16) Vision Transformer [15] to extract dense representations (Figure 2). Internally, the DINOv3 architecture natively patches the 512 × 512 cell into a grid of 16 × 16 pixel patches. This results in 1024 patches; each of these patches is projected to a 1024 dimension spatial patch token. During the forward pass, these tokens are processed alongside a class (CLS) token and four register tokens. The register tokens serve as a scratch space, absorbing high-norm global artifacts that would otherwise pollute the spatial representations. To maintain strict spatial awareness throughout the forward pass, 2D Rotary Position Embeddings (RoPE) are added to the token sequence. The spatial tokens pass through the 24 transformer layers. The self-attention mechanism ensures that every token attends to every other token in the cell. Consequently, the resulting feature vectors are context-aware embeddings. For example, the token for a patch of water inherently encodes its surrounding semantic context. This could help distinguish open ocean from water adjacent to a harbor. After the transformer layers, the intermediate outputs are passed through a feed forward network (FFN); the final output dimension is 1029 × 1024 (1029 tokens of 1024 dimensions). The CLS and the four register tokens are discarded, leaving only the 1024 contextualized spatial patch tokens. To improve computational efficiency and reduce storage overhead during the subsequent clustering phase, we apply a spatial downsampling step. The 1024 patch tokens are average-pooled in 2×2 spatial groups. This yields 256 dense embeddings per cell, where each resulting token effectively represents a 32 × 32 pixel region. This approach preserves the discriminative spatial semantics of the scene while significantly compressing the data, producing highly expressive and context-aware patch token embeddings across the pretraining corpus. 3.1.2 Composition Targets Generation. The patch embeddings generated in the previous step are used to quantify the compositional makeup of each cell (Figure 3). To achieve this, each patch must be mapped to a “concept” or “visual texture”. We perform K-Means clustering on the extracted patch embeddings to discover the fundamental visual texture of our dataset, defining 𝑘 = 512 distinct concepts (𝐶 1 − 𝐶𝑘 ). The earlier collected patch embeddings were roughly 93 million individual 1024-dimensional vectors across the full pretraining dataset (363,000 cells × 256 embeddings each). Clustering all 93 million embeddings simultaneously was computationally infeasible. Instead, a random subsample of 5 million embeddings was drawn uniformly across cells. Before clustering, every token is 𝐿2 -normalized. Without normalization, K-Means would

Figure 2: Generate embeddings for each patch in a satellite image cell utilizing average-pooling.

cluster by vector magnitude rather than by semantic direction, and tokens representing “dark water” and “bright water” would end up in different clusters despite being semantically similar simply because one has higher pixel intensity. The normalized 5 million tokens are clustered using FAISS GPU-accelerated K-Means with K=512. These 512 cluster centroids form our global visual vocabulary, where each centroid represents a recurring and meaningful geospatial texture. After clustering, a 512 × 512 ground cost matrix is precomputed. The matrix encodes how semantically similar every pair of vocabulary textures is via cosine similarity. This matrix is used later during training as the geometric substrate for the Earth Mover’s Distance loss [7]. To compute the histogram targets, we start by iterating over each cell; the 256 patch tokens are loaded and 𝐿2 -normalized. For each token, its cosine similarity to every one of the 512 centroids is computed as their dot product. Rather than assigning each token to its single nearest centroid, we use a soft assignment. The assignment weight of token a to centroid is computed using a Radial Basis Function (RBF) kernel. The effect is that each token distributes its probability mass across multiple nearby centroids rather than committing entirely to one. We opt for soft rather than hard assignment because deep embedding spaces are continuous and forcing a binary boundary between semantically adjacent clusters introduces a discretization error that could propagate into the training signal.

Naveen et al.

Figure 3: The patch embeddings are clustered to create a visual vocabulary. Then the patch tokens for each cell are assigned to the vocabulary. All assignments are summed and normalized to generate the final composition target. Once every patch token in the cell has been soft-assigned to the 512 centroids, the assignments are aggregated into a single histogram. For each centroid k, all 256 per-patch weights are summed. This gives a 512-dimensional vector where each entry represents the total soft-assignment mass that flowed to that centroid across all patches in the cell. Finally, the vector is then 𝐿1 -normalized. Ultimately, this 𝐿1 -normalized histogram we term the "composition target", provides a precise and fractional description of the fundamental visual textures that construct the scene.

3.2

Foundation Model Training

During the online pretraining phase (Figure 4), the objective is to teach an efficient foundation model (FM) backbone to intrinsically understand both the composition and semantic categorization of an input scene. The training utilizes three inputs: the raw pre-training dataset images, the offline-generated histogram targets, and weak image-level annotations. 3.2.1 Backbone Architecture. To process the complex, multi-scale nature of remote sensing imagery, we employ the DynamicVis-base architecture [4] as our foundation model backbone. DynamicVis is available in two variants, Base (36.8M parameters) and Large (91.3M parameters). Both variants process 3-channel RGB imagery. We adopt the Base variant to prioritize parameter efficiency. Unlike traditional Vision Transformers that rely on uniform dense processing and allocate equal computational resources across vast, redundant backgrounds, DynamicVis introduces a Dynamic Region-Aware State Space Model (SSM) explicitly tailored for spatial sparsity. The architecture utilizes a Spatial-Preserving Hierarchical Patch Merger. Instead of employing aggressive patch embeddings that irreversibly compromise fine-grained details, this module uses a progressive, small-stride down-sampling strategy across hierarchical stages to

strictly preserve the granular geometric configurations necessary for resolving tiny targets. To efficiently process the resulting highresolution token sequences without immense computational waste, the backbone integrates a Dynamic Sparse Mixer composed of Adaptive Token Routing and Incremental Modeling (ATRIM) units. The ATRIM unit operates through three sequential mechanisms. The module partitions the input sequence by isolating dense global semantics through 1D pooling, while simultaneously utilizing a top-𝑘 routing mechanism to explicitly identify a sparse subset of highly salient regional tokens. The pooled global context and the selected salient regional tokens are concatenated and processed through a streamlined, bidirectional (forward and reverse) Mamba [9] block. This focuses the intensive 𝑂 (𝑁 ) state-space computation exclusively on the most informative spatial elements. Crucially, to prevent the fragmentation of spatial topology, unselected background tokens are not permanently discarded. They bypass the heavy SSM computation and are reintegrated into the sequence via parameter-free residual connections, preserving the macroscopic structural continuity necessary for pixel-level interpretation. The input cell is passed through the FM Backbone, which outputs a continuous 768-dimensional latent feature tensor. The embedding is simultaneously routed through three distinct parallel branches. 3.2.2 Branch A: Histogram Distillation via Sinkhorn EMD. The latent feature is passed through a Multi-Layer Perceptron (MLP) head and a softmax activation function to project the 768-dimensional embedding into a 512-bin predicted probability distribution, denoted as 𝑝. This prediction is compared against the ground-truth Composition Target histogram, denoted as 𝑞. We utilize the Earth Mover’s Distance (EMD), or Wasserstein-1 distance, rather than standard Mean Squared Error. Given a cost matrix 𝐶 ∈ R𝑘 ×𝑘 , where 𝐶𝑖,𝑗 represents the pairwise semantic distance between vocabulary centroids 𝐶𝑖 and 𝐶 𝑗 , the EMD objective is formulated as finding the optimal transport plan 𝑇 :

L𝐸𝑀𝐷 =

min

𝑘 ∑︁ 𝑘 ∑︁

𝑇 ≥0,𝑇 1=𝑝,𝑇 𝑇 1=𝑞 𝑖=1 𝑗=1

𝑇𝑖,𝑗 𝐶𝑖,𝑗

(1)

We approximate this using the Sinkhorn-Knopp algorithm [7] to ensure computational efficiency during training. The optimal transport problem is transformed into a strictly convex formulation solvable via highly parallelizable, iterative matrix multiplications. The approximation scales efficiently on GPU hardware while providing the continuous gradients necessary for backpropagation. Because this loss incorporates the ground distance 𝐶 between vocabulary clusters, it penalizes the network proportionally to the severity of its error. This allows it to differentiate between mild misclassifications and severe ones. 3.2.3 Branch B: Feature Alignment via Bidirectional MIL. The parallel branch processes the raw 768-dimensional embedding and routes it to a Multi-Instance Learning (MIL) branch to explicitly structure the latent space. Utilizing the million-scale dataset with its weak, region-level annotations, we treat each massive image as a “bag” of instances. The model explicitly learns to decouple heterogeneous foregrounds from the background by contrasting region specific visual representations against categorical meta-embeddings.

A Composition-Aware Pretraining Framework for Geospatial Foundation Models

Figure 4: The proposed Composition-Aware Pretraining Architecture. The input satellite cell is processed by the DynamicVis backbone to extract a dense feature representation. This feature tensor is then routed through three parallel objectives: (A) A BoVW head predicting a fractional texture histogram optimized via Sinkhorn EMD, (B) a Bidirectional MIL branch aligning raw features with learnable category embeddings, and (C) an Auxiliary classification head operating on detached features for stabilization. To achieve this, we apply the Multi-Instance Learning Noise Contrastive Estimation (MIL-NCE) loss, formulated as: Í (𝑣,𝑡 ) ∈𝑃 𝑓 (𝑣, 𝑡) Í L𝑀𝐼 𝐿−𝑁𝐶𝐸 = − log Í (2) 𝑓 (𝑣, 𝑡) + (𝑣 ′ ,𝑡 ′ ) ∈𝑁 𝑓 (𝑣 ′, 𝑡 ′ ) (𝑣,𝑡 ) ∈𝑃 where 𝑓 (𝑥, 𝑦) = exp(⟨𝑥, 𝑦⟩/𝜏). Here, ⟨𝑥, 𝑦⟩ denotes cosine similarity, 𝑃 represents a set of positive matches between regional visual embeddings (𝑣) and their corresponding categorical metaembeddings (𝑡), and 𝑁 comprises negative pairs sampled from nonassociated categories within the mini-batch. The parameter 𝜏 acts as a learnable temperature scalar. This bidirectional MIL loss ensures that the feature is pulled toward its correct category embedding, and conversely, that the category embedding is pulled toward all batch instances sharing that label. 3.2.4 Branch C: Auxiliary Classification (CLS). To further stabilize the training dynamics and provide a grounded discriminative signal, we implement a standard auxiliary classification branch. Crucially, before the latent feature 𝑓 enters the auxiliary classification head, we apply a stop-gradient (detach) operation. This ensures that the classification loss only trains the auxiliary mapping weights and does not backpropagate gradients into the DynamicVis backbone, preventing the single-label classification objective from dominating or overwriting the fine-grained compositional features learned by the EMD and MIL branches. 3.2.5 Total Objective Formulation. The complete pretraining objective is a weighted sum of the compositional distillation, feature alignment, and stabilization branches. The total loss L𝑡𝑜𝑡𝑎𝑙 is formulated as: L𝑡𝑜𝑡𝑎𝑙 = 𝜆𝑒𝑚𝑑 · L𝑒𝑚𝑑 + 𝜆𝑚𝑖𝑙 · L𝑚𝑖𝑙 + 𝜆𝑐𝑙𝑠 · L𝑐𝑙𝑠

This weighting scheme is designed to prioritize the fine-grained compositional EMD loss while allowing the MIL and CLS losses to gently constrain the global topology of the embedding space. The empirical justification for the chosen values of 𝜆𝑒𝑚𝑑 , 𝜆𝑚𝑖𝑙 , and 𝜆𝑐𝑙𝑠 is provided in Section 4.4.2.

4

Results and Analysis

We evaluate the proposed framework across region-level understanding, instance-level perception, and pixel-level dense prediction tasks, preceded by an analysis of the pretraining pipeline and followed by ablation studies on vocabulary size and loss formulation. For all downstream evaluations, backbone weights are kept strictly frozen and only lightweight task-specific heads are trained, ensuring that reported performance reflects representational quality rather than fine-tuning capacity.

4.1

Pretraining Pipeline Analysis

Before evaluating on downstream tasks, we first analyze the internal dynamics of our composition-aware pretraining pipeline to validate the visual vocabulary and training stability. 4.1.1 Pretraining Dataset. We utilize the Functional Map of the World (fMoW) dataset in our pretraining framework. fMoW is a large-scale remote sensing dataset designed to facilitate the understanding of complex spatial patterns across the globe. It comprises more than one million high-resolution satellite images spanning more than 200 countries. The dataset encompasses a highly diverse range of human-made structures, land-use patterns, and natural environments, categorized into 62 classes. To adapt this raw imagery for our patch-level process, the data is processed into a uniform grid structure. First, the imagery is

Naveen et al.

Figure 5: Spatial attention diversity across multiple cells. Overlaying the top diverse centroid activations reveals that specific clusters focus on distinct, localized geographical features. tiled into cells of size 512 × 512 pixels. These cells serve as the raw inputs for the DynamicVis backbone. Next, to explicitly capture the compositional information within each scene, every 512 × 512 cell is further subdivided into 256 individual 32 × 32 pixel patches. This decomposition ensures that the massive variance of localized geospatial textures present within the satellite imagery is physically isolated. These small patches are embedded using DINOv3 to construct the global visual vocabulary and generate the fractional histogram targets that drive our composition-aware pretraining objective. 4.1.2 Composition Target Generation. The foundation of our approach relies on extracting a robust, offline visual vocabulary. By embedding individual 32 × 32 patches using a frozen DINOv3 encoder, we captured a highly expressive latent space of localized textures. Applying K-Means clustering (𝐾 = 512) to these patch embeddings successfully grouped the visual primitives into distinct semantic concepts. Figure 5 highlights the spatial attention diversity inherent in our fractional targets. The figure overlays the top diverse centroid activations onto the original cells; it is evident that individual clusters accurately ground themselves to specific regions and features within the image. Consequently, scenes with disparate land-cover mixtures produce orthogonal probability distributions, providing a rich, localized, and high-variance target for the backbone to learn from. 4.1.3 Foundation Model Training. We trained the DynamicVis backbone till convergence on the fMoW dataset with a batch size of 256 across two Nvidia H100 GPUs. We found that the multibranch loss framework was necessary for stability, as the Sinkhorn Earth Mover’s Distance (EMD) loss smoothly converged with the presence of Multi-Instance Learning (MIL) and Classification (CLS) losses. To evaluate the out-of-the-box performance of the trained foundation model, we apply it to various downstream tasks detailed below.

4.2

Evaluation on Region-Level Understanding

For each downstream task, we compare our proposed compositionaware approach against the vanilla DynamicVis base model, the large variant, and the large-scale SatMAE and Prithvi-EO-2.0 foundation models. To strictly evaluate the quality of the learned representations, the foundation model backbones were kept completely frozen during all downstream task evaluations. Only the lightweight, task-specific heads were trained (Figure 6).

Figure 6: The FM can be applied to a variety of downstream tasks out of the box by utilizing the backbone feature maps. Region-level understanding encompasses image classification, region classification, and zero-shot image retrieval. For image classification, category logits are generated by applying average pooling to the highest-resolution semantic feature map, followed by a linear projection layer. Scene classification is achieved by extracting the pooled features and passing them through a lightweight classification head. Zero-shot image retrieval involves using the high-level semantic features extracted by the backbone to retrieve similar images from the satellite imagery dataset. 4.2.1 Zero-Shot Image Retrieval. The goal of zero-shot image retrieval is to accurately retrieve semantically similar scenes from a large, unindexed database given a query image. The datasets used to evaluate the method are detailed below. The Aerial Image Dataset (AID) [18] is a large-scale dataset for aerial scene classification. We evaluate image retrieval on this dataset due to the presence of scenes that are compositional in nature. For example, a harbor scene is composed not only of the water body but also surrounding ships, docks, and piers. AID comprises 10,000 images across 30 classes, ranging from dense urban areas to natural landscapes. To assess generalization, we adopt a cross-validation strategy, splitting the data into 5 folds. We also use the ForestNet dataset [10] for this task. This dataset was originally designed to classify drivers of deforestation in the Amazon basin. We employ it here to evaluate the model’s ability to retrieve fine-grained land-cover mixtures. ForestNet imagery captures complex transitions between primary forest, smallholder agriculture, infrastructure, and commercial clearing. We evaluate the proposed composition-aware approach on both the fine-grained 12-class variant and the broader 4-superclass variant. A single cell often contains a mixture of intact forest and various stages of clearing, so

A Composition-Aware Pretraining Framework for Geospatial Foundation Models

Figure 7: Top 20 retrieved results across the AID and Forestnet datasets. Correct matches are delineated by green frames and incorrect matches by red ones. this dataset serves as a rigorous testbed for composition-aware embeddings. Table 1: Retrieval Performance (Recall@𝐾 and mAP@𝐾) on AID and ForestNet Datasets Model

Params

@1 R

@5

mAP

R

@10

mAP

R

mAP

0.704 0.746 0.819 0.676 0.831

0.911 0.936 0.962 0.907 0.960

0.670 0.713 0.782 0.638 0.808

0.267 0.279 0.419 0.433 0.447

0.759 0.732 0.803 0.852 0.833

0.279 0.282 0.410 0.427 0.434

0.489 0.495 0.617 0.616 0.666

0.942 0.949 0.954 0.961 0.956

0.467 0.475 0.589 0.590 0.621

AID Dataset (5-Fold Cross Validation) DynamicVis-B [4] DynamicVis-L [4] SatMAE [6] Prithvi-EO-2.0 [17] Proposed Method

36.8M 91.3M 303M 600M 36.8M

DynamicVis-B [4] DynamicVis-L [4] SatMAE [6] Prithvi-EO-2.0 [17] Proposed Method

36.8M 91.3M 303M 600M 36.8M

DynamicVis-B [4] DynamicVis-L [4] SatMAE [6] Prithvi-EO-2.0 [17] Proposed Method

36.8M 91.3M 303M 600M 36.8M

0.653 0.699 0.786 0.625 0.802

0.653 0.699 0.786 0.625 0.802

0.854 0.887 0.930 0.841 0.934

ForestNet-12 (Fine-Grained) 0.148 0.152 0.317 0.321 0.347

0.148 0.152 0.317 0.321 0.347

0.509 0.543 0.639 0.706 0.683

ForestNet-4 (Superclass) 0.332 0.351 0.501 0.507 0.579

0.332 0.351 0.501 0.507 0.579

0.799 0.820 0.867 0.896 0.892

As shown in Table 1, our composition-aware pretraining yields significant performance gains across all retrieval cutoffs (@𝐾). On the AID dataset, we observe a 20.5% relative improvement in mAP@10 (0.670 to 0.808). The performance gap is even more pronounced on the highly compositional ForestNet dataset. For the fine-grained ForestNet-12 variant, our method almost doubles the baseline mAP@10 (0.279 to 0.434), proving that explicitly modeling fractional land-cover significantly enhances the model’s ability to distinguish subtle differences in complex environments. Qualitative retrieval results (Figure 7) demonstrate the success of the composition-aware pretraining approach. We observe excellent retrieval performance on the AID dataset. The visualised retrieval examples demonstrate the model sucessfully learnt the compositional makeup for the "Port" and "Park" classes, evidenced by more than half of the retrieved results being relevant. We observe similar results on the fine-grained ForestNet dataset; in the example shown, the model was able to sucessfully distinguish the specifics of an "Oil Palm Plantation" from regular ground-cover by modeling the compositional makeup. 4.2.2 Scene Classification. This task requires assigning a single label to an entire aerial or satellite image patch based on its dominant land-cover characteristics. This task evaluates the model’s capacity for global semantic understanding and its ability to aggregate complex local textures into a single categorical representation.

Naveen et al.

We evaluate whole-image semantic understanding using two datasets. The first is the UC Merced Land Use dataset [21]. This benchmark contains 2,100 high-resolution aerial images extracted from the USGS National Map Urban Area Imagery collection. The dataset is divided into 21 specific land-use classes, with exactly 100 images per class. The second dataset we use is the NWPU RESISC45 dataset [5]. It contains 31,500 images with 45 scene categories. For this task, we extracted pooled features from the frozen backbones and trained a simple Multi-Layer Perceptron (MLP) head to predict the class label. Despite the presence of highly overlapping visual features between classes (such as “freeway” vs. “intersection”) on both datasets, our composition-aware backbone outperforms the DynamicVis-B baseline. On the UC Merced dataset, our method achieves an Accuracy of 0.928, significantly outperforming the DynamicVis baseline (0.864), as shown in Table 2. The bidirectional MIL alignment during pretraining ensures that these global scene representations remain categorically separated, while the compositional histogram awareness provides highly discriminative features for accurate class determinations. On the NWPU-RESISC45 dataset, the proposed method achieves 81.60% accuracy, which represents a substantial improvement over the DynamicVis-B baseline (0.692) but falls marginally below the SatMAE (0.829) model. Table 2: Scene Classification Performance on the UC Merced and NWPU-RESISC45 Datasets Model

Params

Accuracy

Precision

Recall

F1 Score

0.864 0.864 0.881 0.902 0.928

0.863 0.863 0.882 0.901 0.927

0.692 0.776 0.807 0.829 0.816

0.692 0.776 0.807 0.829 0.815

UC Merced Dataset DynamicVis-B [4] DynamicVis-L [4] Prithvi-EO-2.0 [17] SatMAE Proposed Method

36.8M 91.3M 600M 303M [6] 36.8M

0.864 0.864 0.881 0.902 0.928

0.866 0.866 0.885 0.903 0.930

NWPU-RESISC45 Dataset DynamicVis-B [4] DynamicVis-L [4] Prithvi-EO-2.0 [17] SatMAE [6] Proposed Method

4.3

36.8M 91.3M 600M 303M 36.8M

0.692 0.776 0.807 0.829 0.816

0.700 0.779 0.811 0.834 0.818

Evaluation on Instance-Level Perception and Pixel-Level Dense Prediction

To assess the versatility of the learned representations, we evaluate our model on dense prediction and instance-level tasks, specifically change detection, tiny object detection, and semantic segmentation. While our composition-aware pretraining explicitly optimizes for region-level semantic mixtures, it remains critical that the foundation model preserves localized spatial features for exact pixel-level tasks. A Feature Pyramid Network is coupled with a Faster R-CNN head for instance detection [13]. Pixel-level Interpretation targets dense prediction tasks, such as change detection and segmentation. The feature differences between bi-temporal inputs are computed in high-dimensional semantic space, followed by an MLP head to produce the final change prediction. We compare our approach with

only Dynamicvis-B and L on these tasks due to the presence of an FPN that performs top-down cross-scale feature fusion. This mechanism reconstructs multi-scale feature maps with a uniform channel dimension. This ensures compatibility with standard prediction heads, such as the Mask R-CNN [13] and UperNet[19].

Figure 8: Qualitative results on the LEVIR-CD dataset. From left to right: Pre-event image, Post-event image, Ground Truth mask, and Model Prediction. 4.3.1 Change Detection. Change detection is the process of identifying meaningful structural or land-cover modifications across bi-temporal image pairs of the same geographic region. This task requires the FM to isolate localized physical changes while remaining invariant to superficial differences such as seasonal variations, atmospheric conditions, or lighting. For the downstream task of change detection, we utilize the LEVIR-CD (LEVIR Change Detection) dataset [2]. LEVIR-CD is a large-scale dataset specifically curated for identifying building construction and demolition. It consists of over 600 bi-temporal high-resolution Google Earth images, capturing the same geographical regions across temporal spans of 5 to 14 years. The dataset provides pixel-level binary annotations. On the LEVIR-CD dataset, the foundation model must identify structural changes across bi-temporal image pairs. To evaluate this, we trained a lightweight prediction head for 50 epochs while keeping the respective backbones strictly frozen. As detailed in Table 3, our composition-aware method achieves an F1-score of 0.806 and an Intersection over Union (IoU) of 0.680. While this demonstrates robust sensitivity to fine-grained structural evolution, the vanilla DynamicVis achieves slightly higher spatial precision on this specific task (F1: 0.805, IoU: 0.690). Figure 8 illustrates a sample prediction, highlighting the model’s ability to localize modifications. The marginal gap relative to the vanilla DynamicVis-B baseline shows a trade-off inherent to compositionaware pretraining. The composition objective trains the backbone to encode fractional distributions over entire cells, reinforcing holistic mixture representations at the expense of the sharp, high-frequency spatial gradients that precise boundary localization demands. Consequently, the backbone learns to subordinate precise boundary

A Composition-Aware Pretraining Framework for Geospatial Foundation Models

information in favor of semantically richer, cell-level compositional summaries. This trade-off yields significant gains on region-level tasks while incurring a modest cost on pixel-level ones. Table 3: Change Detection Performance on the LEVIR-CD Dataset. The proposed framework maintains highly competitive structural sensitivity using only 36.8M parameters. Model

Params

Precision

Recall

F1 Score

IoU

DynamicVis-B [4] DynamicVis-L [4] Proposed Method

36.8M 91.3M 36.8M

0.813 0.838 0.801

0.780 0.824 0.822

0.805 0.830 0.806

0.690 0.712 0.680

4.3.2 Tiny Ship Detection. Tiny object detection involves localizing and classifying extremely small targets within expansive backgrounds. In remote sensing, this task evaluates whether a foundation model preserves high-resolution, fine-grained spatial activations rather than washing them out through pooling operations, which is critical for identifying minimal objects like vessels or vehicles. The LEVIR-Ship dataset [3] presents a severe challenge for foundation models, as tiny objects occupy only a fraction of the spatial resolution. This dataset is designed specifically for tiny object detection in optical remote sensing imagery. It features thousands of high-resolution scenes with heavily clustered and multi-scale ship instances. Because the targets often occupy only a few pixels relative to a massive background of water or port infrastructure, this task heavily relies on the foundation model’s ability to preserve high-resolution, localized spatial features rather than collapsing them into a holistic background embedding. To assess localization capabilities, we trained a detection decoder for 50 epochs over the frozen backbones. As reported in Table 4, our compositionaware model achieves an mAP@0.50 of 0.619. While the model successfully localizes targets, the proposed approach demonstrates a stronger performance across both IoU thresholds compared to the DynamicVis baseline but is consistently outperformed by the large variant. Table 4: Tiny Ship Detection Performance on the LEVIR-Ship Dataset. Model

Params

mAP@0.10

mAP@0.50

Precision

Recall

DynamicVis-B [4] DynamicVis-L [4] Proposed Method

36.8M 91.3M 36.8M

0.730 0.759 0.742

0.552 0.643 0.619

0.135 0.162 0.140

0.870 0.904 0.875

4.3.3 Semantic Segmentation. We evaluate semantic segmentation performance on the single-category WHU building dataset [11], comprising discrete, localized instance-like structures. UperNet [19] is used as the prediction framework. It couples a Feature Pyramid Network with a Unified Perceptual Parsing head that aggregates multi-scale feature maps from the frozen backbone across hierarchical stages, enabling dense per-pixel predictions without requiring any modification to the backbone architecture. We observe that the model does learn to successfully delineate structures, but it falls

Figure 9: Qualitative results on the WHU building dataset show that the model successfully learnt to delineate structures. The overlay demonstrates the true positives in green, false negatives in blue, and false positives in red. Table 5: Semantic Segmentation Performance on the WHU Building Dataset. Our method delivers a competitive IoU while baseline preserves slightly better sub-pixel boundary delineations due to its lack of cell-level pooling constraints. Model

Params

Precision

Recall

F1 Score

IoU

DynamicVis-B [4] DynamicVis-L [4] Proposed Method

36.8M 91.3M 36.8M

0.936 0.964 0.934

0.914 0.941 0.908

0.925 0.953 0.921

0.860 0.886 0.853

slightly short of the DynamicVis-B baseline. As visible in Figure 9 and Table 5, the DynamicVis baseline produces fewer false positives, consistent with its stronger preservation of fine-grained spatial topology. This result corroborates the resolution-semantics tradeoff identified earlier. The composition-aware pretraining framework optimizes representations for distributional similarity over entire cells, and this bias, while highly effective for semantic retrieval and scene classification, modestly reduces the sub-pixel boundary precision that dense prediction tasks require.

4.4

Architecture Analysis

We perform ablation studies on the number of vocabulary classes and different loss configurations. The models were trained with half the training budget of the final models reported in the earlier results. 4.4.1 Vocabulary Size. The visual vocabulary size 𝑘 governs the granularity of the composition targets. A vocabulary that’s too small collapses semantically distinct textures into the same centroid, producing low-variance targets that provide a weak training signal. An excessively large vocabulary over-partitions the embedding space and creates sparse distributions.

Naveen et al.

To determine the optimal vocabulary size, we trained the full pretraining pipeline with 𝑘 ∈ {32, 64, 128, 256, 512, 1024} and evaluated each configuration on the zero-shot image retrieval task with the AID dataset. Table 6 reports the full retrieval metrics with standard deviations across folds. Table 6: Vocabulary size ablation on the AID dataset. 𝑘 32 64 128 256 512 1024

R@1

mAP@1

R@5

mAP@5

R@10

mAP@10

0.735±.007 0.727±.009 0.744±.008 0.734±.008 0.751±.013 0.707±.012

0.735±.007 0.727±.009 0.744±.008 0.734±.008 0.751±.013 0.707±.012

0.902±.002 0.914±.006 0.913±.006 0.906±.009 0.919±.011 0.900±.008

0.774±.002 0.774±.007 0.784±.006 0.774±.008 0.794±.010 0.756±.008

0.943±.002 0.950±.003 0.953±.006 0.946±.009 0.952±.009 0.943±.004

0.740±.002 0.738±.006 0.750±.006 0.739±.008 0.761±.008 0.721±.006

𝑘 = 512 achieves the highest mAP@10 of 0.761 ± 0.008, outperforming both smaller and larger vocabularies. The performance drop at 𝑘 = 1024 is the most pronounced, with a −0.040 reduction in mAP@10 relative to 𝑘 = 512. We hypothesize that this degradation is due to vocabulary over-partitioning. The clear advantage of 𝑘 = 512 over this range is the primary basis for our vocabulary size selection. 4.4.2 Loss Formulation Ablation. To validate each component of the multi-branch training objective, we conduct a systematic ablation study over subsets of the three loss terms. The relative weighting of 𝜆𝑚𝑖𝑙 and 𝜆𝑐𝑙𝑠 preserves the ratio established in the DynamicVis pretraining objective [4]. The compositional distillation weight 𝜆𝑒𝑚𝑑 is then set at a 2:1 ratio to 𝜆𝑚𝑖𝑙 . Seven configurations are evaluated on AID mAP@10 and ForestNet-4 mAP@10. Table 7 summarises the results. Table 7: Loss component ablation on AID and ForestNet-4. Configuration

AID mAP@10

FN-4 mAP@10

EMD only EMD + CLS EMD + MIL EMD + MIL + CLS

0.750±.007 0.805±.006 0.759±.006 0.812±.004

0.598 0.617 0.578 0.621

CLS only MIL only MIL + CLS

0.646±.006 0.611±.005 0.719±.008

0.516 0.572 0.476

The EMD loss is the primary driver of representation quality. The performance gap between EMD-inclusive and EMD-absent configurations is substantial and consistent across both datasets. Removing EMD entirely while retaining MIL and CLS yields a drop of −0.148 in AID mAP@10. This confirms that the histogram distillation objective is the core source of compositional discriminability in the learned representations, and that the auxiliary branches cannot substitute for it.

5

Conclusions and Future Work

This paper introduces a composition-aware pretraining framework that resolves the single-concept bottleneck in modern geospatial foundation models. This is achieved by explicitly encoding the heterogeneous land-cover mixtures fundamental to Earth observation data. Evaluations across region-level understanding, instance-level perception, and pixel-level dense prediction highlight a distinct representational trade-off. The EMD objective trains the backbone to encode distributional summaries over entire cells, naturally subordinating precise boundary information in favor of holistic mixture representations. As a result, this composition-aware prior dramatically benefits tasks requiring region-level understanding, such as zero-shot image retrieval and scene classification, while incurring only a minimal performance cost on sub-pixel tasks like change detection and semantic segmentation. The strength of explicitly modeling fractional composition is most evident in zero-shot image retrieval. On the fine-grained ForestNet-12 dataset, designed to capture subtle transitions between intact forests and agricultural clearing, our method almost doubles the DynamicVis-B baseline mAP@10 (0.279 to 0.434). By forcing the model to represent not only which land-cover types are present, but in what exact proportions, the framework develops the representational capacity required to distinguish highly nuanced scenes, such as separating an oil palm plantation from adjacent secondary growth. Furthermore, on the Aerial Image Dataset (AID), our method yields a 20.5% relative improvement in mAP@10, jumping from 0.670 to 0.808. The results on tasks requiring instance and pixel level prediction validate that although the model sacrifices a marginal degree of sub-pixel boundary precision, it successfully preserves the localized spatial features necessary for these tasks. There are several promising directions for further exploration. First, the pretraining methodology could be extended to a broader range of architectures, including standard Vision Transformers and modern CNNs, to establish its universality beyond the DynamicVis state-space model. Second, because the current framework strictly processes 3-channel RGB imagery, expanding the formulation to accommodate multispectral and hyperspectral data could unlock even richer compositional insights. Third, replacing the current offline K-Means clustering with end-to-end learnable vocabularies would allow the model to dynamically discover task-relevant textures during pretraining, rather than relying on static initialization. Finally, to address the resolution-semantics trade-off observed in our pixel-level evaluations, applying histogram targets at multiple spatial resolutions simultaneously could recover fine-grained spatial precision. This would bridge the performance gap in dense prediction tasks without compromising the highly discriminative region-level representations that define this framework.

References [1] Shrutilipi Bhattacharjee and Jia Chen. 2020. Prediction of Satellite-Based Column CO2 Concentration by Combining Emission Inventory and LULC Information. IEEE Transactions on Geoscience and Remote Sensing 58, 12 (2020), 8285–8300. doi:10.1109/TGRS.2020.2985047 [2] Hao Chen and Zhenwei Shi. 2020. A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection. Remote Sensing 12, 10 (2020). doi:10.3390/rs12101662 [3] Jianqi Chen, Keyan Chen, Hao Chen, Zhengxia Zou, and Zhenwei Shi. 2022. A Degraded Reconstruction Enhancement-based Method for Tiny Ship Detection in Remote Sensing Images with A New Large-scale Dataset. IEEE Transactions on

A Composition-Aware Pretraining Framework for Geospatial Foundation Models

Geoscience and Remote Sensing 60 (2022), 1–14. doi:10.1109/TGRS.2022.3180894 [4] Keyan Chen, Chenyang Liu, Bowen Chen, Wenyuan Li, Zhengxia Zou, and Zhenwei Shi. 2025. DynamicVis: An Efficient and General Visual Foundation Model for Remote Sensing Image Understanding. arXiv:2503.16426 [cs.CV] https://arxiv.org/abs/2503.16426 Available at: https://arxiv.org/abs/2503.16426. Accessed on: April 28, 2026. [5] Gong Cheng, Junwei Han, and Xiaoqiang Lu. 2017. Remote Sensing Image Scene Classification: Benchmark and State of the Art. Proc. IEEE 105, 10 (2017), 1865–1883. doi:10.1109/JPROC.2017.2675998 [6] Yezhen Cong, Samar Khanna, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, and Stefano Ermon. 2022. SatMAE: Pretraining Transformers for Temporal and Multi-Spectral Satellite Imagery. In Advances in Neural Information Processing Systems, Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.). https://openreview.net/forum? id=WBhqzpF6KYH [7] Marco Cuturi. 2013. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems, Vol. 26. 2292– 2300. [8] Spyros Gidaris, Andrei Bursuc, Gilles Puy, Nikos Komodakis, Matthieu Cord, and Patrick Pérez. 2021. Obow: Online bag-of-visual-words generation for selfsupervised learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 6830–6840. [9] Albert Gu and Tri Dao. 2024. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. In First Conference on Language Modeling. https: //openreview.net/forum?id=tEYskw1VY2 [10] Jeremy Irvin, Hao Sheng, Neel Ramachandran, Sonja Johnson-Yu, Sharon Zhou, Kyle Story, Rose Rustowicz, Cooper Elsworth, Kemen Austin, and Andrew Y. Ng. 2020. ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery. CoRR abs/2011.05479 (2020). arXiv:2011.05479 https://arxiv.org/abs/2011.05479 [11] Shunping Ji, Shiqing Wei, and Meng Lu. 2019. Fully Convolutional Networks for Multisource Building Extraction From an Open Aerial and Satellite Imagery Data Set. IEEE Transactions on Geoscience and Remote Sensing 57, 1 (2019), 574–586. doi:10.1109/TGRS.2018.2858817 [12] Oscar Mañas, Alexandre Lacoste, Xavier Giró-i Nieto, David Vazquez, and Pau Rodríguez. 2021. Seasonal Contrast: Unsupervised Pre-Training From Uncurated Remote Sensing Data. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9414–9423. [13] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2016. Faster RCNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv:1506.01497 [cs.CV] https://arxiv.org/abs/1506.01497 [14] João Daniel Silva, João Magalhães, Devis Tuia, and Bruno Martins. 2024. Multilingual Vision-Language Pre-training for the Remote Sensing Domain. In Proceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems (Atlanta, GA, USA) (SIGSPATIAL ’24). Association for Computing Machinery, New York, NY, USA, 220–232. doi:10.1145/3678717.3691318 [15] Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. 2025. DINOv3. arXiv:2508.10104 [cs.CV] https://arxiv.org/abs/2508.10104 [16] Xian Sun, Peijin Wang, Wanxuan Lu, Zicong Zhu, Xiaonan Lu, Qibin He, Junxi Li, Xuee Rong, Zhujun Yang, Hao Chang, Qinglin He, Guang Yang, Ruiping Wang, Jiwen Lu, and Kun Fu. 2023. RingMo: A Remote Sensing Foundation Model With Masked Image Modeling. IEEE Transactions on Geoscience and Remote Sensing 61 (2023), 1–22. doi:10.1109/TGRS.2022.3194732 [17] Daniela Szwarcman, Sujit Roy, Paolo Fraccaro, Þorsteinn Elí Gíslason, Benedikt Blumenstiel, Rinki Ghosal, Pedro Henrique de Oliveira, Joao Lucas de Sousa Almeida, Rocco Sedona, Yanghui Kang, Srija Chakraborty, Sizhe Wang, Carlos Gomes, Ankur Kumar, Vishal Gaur, Myscon Truong, Denys Godwin, Sam Khallaghi, Hyunho Lee, Chia-Yu Hsu, Ata Akbari Asanjan, Besart Mujeci, Disha Shidham, Rufai Omowunmi Balogun, Venkatesh Kolluru, Trevor Keenan, Paulo Arevalo, Wenwen Li, Hamed Alemohammad, Pontus Olofsson, Timothy Mayer, Christopher Hain, Robert Kennedy, Bianca Zadrozny, David Bell, Gabriele Cavallaro, Campbell Watson, Manil Maskey, Rahul Ramachandran, and Juan Bernabe Moreno. 2026. Prithvi-EO-2.0: A Versatile Multitemporal Foundation Model for Earth Observation Applications. IEEE Transactions on Geoscience and Remote Sensing 64 (2026), 1–20. doi:10.1109/TGRS.2025.3642610 [18] Gui-Song Xia, Jingwen Hu, Fan Hu, Baoguang Shi, Xiang Bai, Yanfei Zhong, and Liangpei Zhang. 2016. AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification. CoRR abs/1608.05167 (2016). arXiv:1608.05167 http://arxiv.org/abs/1608.05167 [19] Tete Xiao, Yingcheng Liu, Bolei Zhou, Yuning Jiang, and Jian Sun. 2018. Unified Perceptual Parsing for Scene Understanding. In Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part V (Munich, Germany). Springer-Verlag, Berlin, Heidelberg, 432–448. doi:10.

1007/978-3-030-01228-1_26 [20] Yiqun Xie, Zhaonan Wang, Gengchen Mai, Yanhua Li, Xiaowei Jia, Song Gao, and Shaowen Wang. 2023. Geo-Foundation Models: Reality, Gaps and Opportunities. In Proceedings of the 31st ACM International Conference on Advances in Geographic Information Systems (Hamburg, Germany) (SIGSPATIAL ’23). Association for Computing Machinery, New York, NY, USA, Article 66, 4 pages. doi:10.1145/ 3589132.3625616 [21] Yi Yang and Shawn Newsam. 2010. Bag-of-visual-words and spatial extensions for land-use classification. In Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems (San Jose, California) (GIS ’10). Association for Computing Machinery, New York, NY, USA, 270–279. doi:10.1145/1869790.1869829 [22] Yingying Zhang, Lixiang Ru, Kang Wu, Lei Yu, Lei Liang, Yansheng Li, and Jingdong Chen. 2025. SkySense V2: A Unified Foundation Model for Multimodal Remote Sensing. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9136–9146. [23] Yasen Zhang, Xian Sun, Hongqi Wang, and Kun Fu. 2013. High-Resolution Remote-Sensing Image Classification via an Approximate Earth Mover’s Distance-Based Bag-of-Features Model. IEEE Geoscience and Remote Sensing Letters 10, 5 (2013), 1055–1059. doi:10.1109/LGRS.2012.2228625

Related documents

• MoRAX: Mobility-based Representation Augmentation for Geospatial Foundation Models#463229
• SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Models#997525
• Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP#175229
• From Synthetic Priors to Model Behavior: Structural Coverage in Tabular Foundation Models#974991
• No Free Efficiency: Revisiting the Trade-off Between Training Efficiency and Model Vulnerability#1118971