C ARBON CLIP: E NHANCE C ARBON P REDICTION FROM S ATELLITE I MAGERY VIA I NTEGRATED S TREET-V IEW S EMANTICS AND T EMPORAL C ONTEXT T RAINING
arXiv:2607.07292v1 [cs.CV] 8 Jul 2026
1
Zeru Yang1,2 , Fang-Ying Gong3 , Steve H.L. Yim4,5 , Chau Yuen2,5 Energy Research Institute at NTU, Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore 2 School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 3 School of Public Administration and Policy, Renmin University of China, Beijing, China 4 Asian School of the Environment, Nanyang Technological University, Singapore 5 Center for Climate Change and Environmental Health, Nanyang Technological University, Singapore [email protected], [email protected], [email protected], [email protected]
A BSTRACT Accurately estimating urban carbon emissions is critical for sustainable urban planning, yet many existing approaches remain difficult to apply consistently across cities due to data-source heterogeneity and the lack of fine-grained semantic-temporal context in remote sensing data. We propose CarbonCLIP, a task-oriented multimodal distillation framework that improves satellite-based carbon emission prediction by transferring contextual knowledge into a unified satellite representation through dual-branch contrastive learning. Unlike conventional methods that rely on static visual features, CarbonCLIP explicitly bridges the gap between top-down satellite views and ground-level human activities. Specifically, the spatial branch uses fine-grained textual descriptions automatically generated from street-view images by Large Multimodal Models (LMMs) to provide semantic priors reflecting building functions, infrastructure, and urban activities, while the temporal branch employs a month encoder to encode temporal priors associated with monthly emission variation. CarbonCLIP requires multimodal data only during the pretraining phase; during inference, it relies solely on satellite imagery, thereby supporting scalable deployment when ground-level data are unavailable at inference. Experiments on Beijing and Singapore demonstrate that CarbonCLIP outperforms baselines in both study cities. The results validate that our method effectively transfers multimodal knowledge into satellite representations, offering a robust solution for satellite-based urban carbon modeling.
1
Introduction
With the rapid urbanization occurring worldwide, cities have become the dominant sources of anthropogenic carbon emissions, accounting for more than one-third of global totals [1, 2]. Accurately quantifying and estimating urban carbon emissions is essential for developing sustainable and climate-resilient cities [3]. Urban sustainability requires not only reducing carbon footprints but also integrating intelligent monitoring tools that can inform policy-making and promote equitable environmental outcomes. A core challenge in achieving this goal lies in how we perceive and analyze cities at scale. Carbon emissions in urban areas are deeply intertwined with spatial patterns such as land use and infrastructure, as well as human-scale factors like greenery, density, and activity levels. In recent years, visual data have become a cornerstone of urban analytics and environmental monitoring, providing unprecedented means of observing cities at multiple scales. Satellite imagery offers a comprehensive overview of urban environments, capturing spatial structure and physical layout [4]. As a scalable and consistent data source, it has been widely applied in air pollution mapping [5], agricultural monitoring [6], and carbon stock estimation [7]. Advances in remote sensing, including high-resolution Planet imagery at 3 m resolution [8], enable detailed detection of urban expansion and environmental change [9], shown in Fig. 1(a). Yet, despite its broad coverage, satellite imagery lacks
CarbonCLIP
human-centric information such as facade features and street-level greenery, limiting its ability to capture fine-grained drivers of emissions and livability.
Image Encoder
Image Embedding
(a) Simple Deep Learning Methods Carbon Prediction
MLP
Satellite Image
(b) Enhanced Feature Encoder Method (CarbonCLIP)
LMMs
Image Encoder
Street View Descriptions Spatial Embedding Temporal Embedding
Enhance the capacity for human-centric information retrieval
MLP
Carbon Prediction
Satellite Image
Month
Month Encoder
Month embedding
Enhance the capacity for temporal context retrieval
Figure 1: Comparison of frameworks for carbon emission prediction: (a) Simple transfer learning framework based on visual embeddings; (b) The proposed CarbonCLIP method enhanced via LMM-generated human-centric information from street-view imagery and temporal context with month indication. In contrast, street-level imagery from platforms like Google Maps and Baidu Maps provides a complementary groundlevel perspective [10], revealing visual details often missed from above, including building facades, signage, sidewalks, and vegetation [11–13]. These human-scale semantics relate closely to livability, mobility, and micro-environmental quality, which influence emissions at local scales [14–16]. However, their spatial and temporal sparsity restricts large-scale and continuous monitoring. The advancement of Large Multimodal Models (LMMs) provides a good foundation for extracting semantic information from visual data, and by utilizing LMMs to abstract visual street scenes into structured textual descriptions, we bypass the need for precise pixel-level geometric alignment, enabling robust knowledge transfer from the human-scale perspective to the satellite domain. Furthermore, urban carbon emissions exhibit temporal volatility associated with seasonal energy consumption cycles (e.g., heating and cooling) and phenological changes [17–19]. While satellite imagery provides periodic revisits (e.g., monthly observations) that preserve month-level temporal variation, street-view imagery is typically collected at sparse, fixed time points and represents a static moment of the urban fabric. Consequently, directly aligning satellite imagery with static street-view data may suppress temporal information in the satellite observations. This limitation motivates a dedicated mechanism for incorporating temporal context alongside spatial semantics. Recent studies have pursued multi-source data fusion, combining points of interest (POIs), socio-demographic statistics, transportation facilities, and remote sensing data [20–22]. Others have begun coupling satellite and street-view imagery to enrich urban representation for downstream tasks such as carbon emission prediction [23–25]. Despite these advances, data-source heterogeneity, arising from inconsistent spatial, temporal, and measurement standards, hampers 2
CarbonCLIP
harmonization, and reliance on region-specific auxiliary data limits scalability. Many cities, especially in developing regions, lack reliable or frequently updated datasets, making model deployment outside the original data-rich setting uncertain. To reduce dependence on region-specific auxiliary sources in urban carbon emission prediction, a paradigm shift is needed, from aggregating local auxiliary datasets to distilling multimodal knowledge into a satellite-centered representation. Building on this insight, we propose CarbonCLIP, a framework that enhances carbon prediction from satellite imagery by fusing street-view semantics and temporal context during training, depicted in Fig. 1(b). Through dual-branch contrastive learning, CarbonCLIP transfers fine-grained spatial semantics and temporal priors into the satellite domain, enabling a satellite-only model to retain multimodal awareness during inference. Specifically, CarbonCLIP employs a pretrained image encoder for satellite features, uses automatically generated textual descriptions as semantic anchors, and incorporates temporal embeddings to capture monthly temporal correlations. The final predictor is trained for monthly emission regression using only satellite imagery, forming a unified and scalable modeling paradigm. CarbonCLIP differs from previous multi-source or satellite-based methods in three key aspects. First, it bridges the gap between top-down and ground-level perspectives by aligning satellite imagery with textual descriptions automatically generated from street-view images, thereby transferring authentic human-centric semantics into a scalable satellite representation. Second, CarbonCLIP introduces a cyclic temporal alignment mechanism through month embeddings, enabling the model to encode temporal priors associated with monthly emission variation and dependencies often ignored in static visual models. Third, unlike conventional multi-source frameworks that rely on region-specific auxiliary data, CarbonCLIP distills multimodal contextual knowledge during training but performs inference using only satellite imagery, supporting deployment when auxiliary ground-level observations are unavailable at inference. In summary, our main contributions are: • We propose a contrastive learning framework that distills street-level semantics and temporal knowledge into a unified satellite representation, achieving multimodal awareness while maintaining single-modality inference. • We construct a dataset by generating fine-grained textual descriptions from street-view imagery and plan to make it publicly available, facilitating future research on scalable urban understanding. • We validate CarbonCLIP on two cities with distinct spatial and climatic patterns, Beijing and Singapore, showing substantial improvements in prediction accuracy in both study cities.
2
Related Work
2.1
Carbon Emission Prediction
Early studies predominantly relied on bottom-up inventory methods [26, 27], which estimate emissions by aggregating energy consumption statistics with predefined emission factors. Although these approaches are widely regarded as the most reliable benchmarks, they suffer from inherently coarse spatiotemporal resolution, typically annual and city-level, and substantial reporting delays. These limitations significantly restrict their applicability for fine-grained analysis and timely urban carbon management. To overcome these constraints, recent research has explored data-driven models that fuse multiple urban data sources, including point of interest (POI) distributions [20, 28], vehicle trajectories [21, 29, 30], and energy metering records [22, 31]. By capturing human activity patterns and socioeconomic conditions, those methods achieve improved spatial granularity. However, their reliance on region-specific data severely limits scalability and makes deployment beyond the original data-rich setting uncertain. The availability, quality, and definition of fine-grained socioeconomic datasets vary substantially across regions, making these models difficult to apply consistently, especially to data-scarce or developing regions. Remote sensing offers a promising alternative due to its global coverage, consistency, and independence from local statistical infrastructures. Initial remote-based studies employed night-time light (NTL) imagery as a proxy for economic activity to spatially disaggregate carbon emissions [28, 32, 33]. While effective at large scales, NTL data are prone to saturation effects in dense urban cores [14, 34, 35] and lack the spatial resolution needed to distinguish detailed emission sources. With advances in deep learning, more recent work has shifted toward high-resolution daytime optical satellite imagery (e.g., Sentinel-2, Planet) [36, 37], leveraging urban morphological information (such as building density, vegetation coverage, and road networks) [18, 38, 39] for carbon emission prediction. Despite these advances, remote-based approaches face a fundamental limitation: optical satellite imagery primarily captures spectral reflectance and surface texture, which are often insufficient to distinguish functionally distinct urban zones with similar visual appearances from a top-down perspective (i.e., iso-spectral objects) [25]. Moreover, satellite 3
CarbonCLIP
Table 1: Descriptive comparison of representative CLIP-style geospatial representation methods. The table summarizes each method by the representation it learns and the training signal it uses rather than ranking them by a single criterion. Method Learned representation Training signal / objective SatCLIP [44]
Geographic location representation Contrastively matches Sentinel-2 satellite imagery with geographic coordinates to learn a location encoder RemoteCLIP [47] Remote-sensing image–text repre- Continual CLIP pretraining on image-caption pairs consentation structed from detection, segmentation, and existing imagetext datasets UrbanCLIP [48] Text-enhanced satellite representa- Uses LLM-generated satellite-image descriptions and option for urban regions timizes image-text contrastive plus language-modeling losses CarbonCLIP Satellite representation for monthly Distills street-view-generated text and month context into (ours) carbon emission prediction satellite features through spatial-semantic and temporal contrastive alignment
observations inherently lack vertical and ground-level information, such as building facade characteristics, business types inferred from signage, and street-level traffic intensity, which are factors that are strongly correlated with energy consumption and carbon emissions [10, 13, 30, 40]. These challenges motivate a satellite-centered method that can inject fine-grained ground-level semantics and temporal contextual knowledge into satellite-based representations, while keeping inference independent of region-specific auxiliary data. 2.2
Cross-Modal Geospatial Representation Learning
Recent geospatial representation learning increasingly uses contrastive alignment to connect visual observations with external contextual signals [41, 42]. One common strategy is to use geographic coordinates as dense and easily available supervision. GeoCLIP [43] aligns natural images with GPS locations for image geo-localization, whereas SatCLIP [44] contrastively matches Sentinel-2 satellite observations with coordinates to learn a global geographic location encoder. Coordinate supervision is scalable because location labels are widely available, but it mainly tells the model where an observation is located, not what urban functions or emission-related activities are present there. This distinction is important for carbon emission modeling: nearby or geographically similar areas may contain different land uses, infrastructure densities, and activity intensities, whereas the same urban function may appear in different districts [45, 46]. This motivates semantic supervision that describes what is present in the scene, not only where the scene is located. To mitigate these limitations, vision–language geospatial models introduce explicit semantic supervision into remotesensing and urban representations. RemoteCLIP [47] addresses remote-sensing pretraining-data scarcity by converting heterogeneous annotations into image-caption pairs through box-to-caption (B2C) and mask-to-box (M2B) conversions, then continually pretraining CLIP for remote-sensing vision-language representation. UrbanCLIP [48] instead targets urban region profiling: it generates satellite-image descriptions with an image-to-text LLM and trains image-text representations with contrastive and language-modeling losses for urban indicator prediction. Earlier RS-CLIP-style models [49] and remote-sensing multimodal large language models such as GeoChat [50], EarthGPT [51], and EarthGPT-X [52] further show that language supervision can support diverse remote-sensing understanding tasks, including scene classification, captioning, visual question answering, visual grounding, and object detection. More recent advances further enrich semantic representations by incorporating ground-level street-view imagery [23, 24, 53]. By bridging overhead satellite observations with human-scale perspectives, these cross-view frameworks aim to capture fine-grained physical and functional attributes that are invisible from a top-down view alone. Nevertheless, most existing approaches still rely on retrieval-based supervision or static feature alignment, and thus fail to fully exploit generative semantics, wherein LMMs can synthesize dense and structured descriptions of urban scenes. Table 1 summarizes these representative methods descriptively according to the representation they learn and the training signal they use. Moreover, across both coordinate-based and vision–language alignment paradigms, the temporal dimension remains largely overlooked. Urban representations are typically learned from static snapshots, neglecting seasonal variations in vegetation, energy demand, and human activity that fundamentally shape urban dynamics [17, 19]. This limitation is particularly restrictive for applications such as carbon emission modeling, where temporal patterns play a central and indispensable role. 4
CarbonCLIP
(Section III.A)
Large Multimodal Model
The panoramic image depicts an urban residential area in Beijing, characterized …
Street View Descriptions
Street View Images Google/Baidu Map
Spatial Projection
Spatial embedding
Carbon Emission Predictor (Section III.D) Spatial embedding
Temporal embedding
Concatenation
Temporal embedding
(Pretrained) Satellite Image Encoder Month embedding
Temporal Projection
Monthly Carbon Emission
Alignment Strategies (Section III.C)
Satellite Imagery
Step 1: Pretrain Phase
Projection
Text embedding
Image Encoder
Text Encoder
Step 2: Train Predictor
Month Encoder
(Section III.B)
Month Frozen weight
Training
Figure 2: Overall framework of our proposed CarbonCLIP.
3
Proposed CarbonCLIP Methodology
The proposed CarbonCLIP architecture is shown in Fig. 2, which enhances carbon emission prediction from satellite imagery by fusing street-view semantics and temporal context during training. An automatic pipeline first generates fine-grained textual descriptions of street-view images using both geographic and visual prompts, powered by the multimodal large language model Qwen2.5-VL [54]. In parallel, a month encoder is designed to capture temporal representations of emission patterns. During the pretraining stage, CarbonCLIP employs dual contrastive objectives to distill multimodal knowledge into the satellite representation: one objective transfers detailed spatial semantics from street-view descriptions to satellite images, while the other injects temporal variation by associating month embeddings with satellite imagery. After pretraining, the satellite image encoder and projection layers are frozen, and a lightweight multi-layer perceptron (MLP) is trained for monthly carbon emission regression using only satellite imagery, ensuring an efficient and scalable inference pipeline. 3.1
Street-View Semantic Representation
Providing detailed descriptions for large-scale collections of street-view images poses a significant challenge, as traditional methods such as manual annotation are labor-intensive, time-consuming, and require substantial human resources. With the rapid advancement of LMMs, however, it is now possible to automatically generate rich and context-aware textual descriptions that capture both visual and geographical semantics. In this study, we leverage Qwen2.5-VL [54], an advanced, stable, and open-source LMM developed by Alibaba, which demonstrates capability in visual understanding tasks. To further enhance the quality and specificity of the generated street-view descriptions, we incorporate prompt strategies inspired by the UrbanCLIP model [48]. Specifically, we design structured prompts as follows that include not only visual information but also geographic metadata, such as city names and precise longitude and latitude coordinates. This integration of spatial context into the prompting process enables the model to generate more accurate, detailed, and geographically grounded descriptions. "You are a helpful assistant to analyze street-view images. Analyze the features about urban structure and environment from the panoramic image of street-view in [city] in a comprehensive and detailed manner and summarize it into one paragraph: The coordinate of the street-view image is [longitude], [latitude]. Note that ignore the weather and other details that change over a short period of time on the image. Note that ignore the street-view vehicle and other details but focus on the overall features. Note that try to avoid inferences and focus on the content of the image." 5
CarbonCLIP
Following the generation process, the descriptions are processed by a frozen text encoder from the CLIP framework [55] to obtain high-dimensional semantic embeddings. Specifically, urban 1 km2 spatial tiles can contain heterogeneous land-use components, such as residential blocks, commercial streets, transport corridors, and green spaces. We therefore do not use a single street-view image as a direct tile-level label. Instead, all retained street-view panoramas whose GPS coordinates fall inside the same 1 km2 spatial tile are associated with that tile. Each panorama is first converted into a textual description and encoded by the Transformer-based text encoder. The encoder’s pooled text embedding is used as the semantic representation for the corresponding panorama, and the tile-level street-view semantic representation is then constructed at the embedding level from all panoramas matched to that tile. These text embeddings serve as ground-level semantic anchors, enabling the subsequent alignment with satellite image features within a shared contrastive latent space. 3.2
Month-Level Temporal Context Encoding
To incorporate temporal information into CarbonCLIP without attributing it to a causal seasonal emission mechanism, we propose a month encoder that maps each observation month to a learnable temporal context embedding. This embedding provides a coarse statistical cue about recurring month-level conditions associated with the training labels. Given a month index m ∈ {1, 2, ..., 12}, we first normalize it to the range [0, 1] and convert it into a radian angle θ = 2π · m−1 12 to reflect its position in the annual cycle. To model continuous temporal transitions, we adopt a multi-frequency sinusoidal encoding strategy inspired by positional encodings in Transformers [56]. Specifically, for a set of n exponentially scaled base frequencies {fi }ni=1 , defined as follows: i−1 · γ , i = 1, 2, ..., n (1) fi = exp n−1 where n represents the number of base frequencies, and γ is a hyperparameter that controls the maximum log-frequency (i.e., fn = exp(γ)). We compute the sine and cosine of each frequency-scaled angle and concatenate the results to form the base temporal feature vector: vbase = [sin(f1 θ), cos(f1 θ), . . . , sin(fn θ), cos(fn θ)] (2) This representation provides a multi-frequency encoding of the month index. To project the features into a temporal embedding space, we apply a non-linear transformation followed by normalization: ebase = LayerNorm(ReLU(W · vbase + b))
(3)
where W and b are learnable parameters. While ebase provides a continuous encoding of month position, it may not fully reflect coarser temporal groups that recur within a year. We therefore incorporate a learnable coarse calendar-group embedding. For implementation, the 12 months are partitioned into predefined coarse temporal groups, and each group is represented using a learnable embedding from a trainable embedding table: egroup = Embedding[g(m)]
(4)
where Embedding is a trainable embedding table that maps each group index to a corresponding d-dimensional embedding vector. The index g(m) is determined based on the given month m. In our experiments, Beijing uses four calendar groups and Singapore uses two rainy/dry groups, which serve as coarse temporal partitions rather than causal emission categories. To achieve a smooth blend between fine-grained and coarse-grained temporal features, we define a cosine-based transition weight: α = 0.5 · | cos(T θ − ϕ)| (5) where T controls the periodicity, ϕ denotes the phase shift, and | · | represents the absolute value. By adjusting T and ϕ, we can flexibly align the transition curve with city-specific temporal groupings, allowing the model to encode month-level context in a continuous manner. The final month embedding is then computed by weighted interpolation: emonth = (1 − α) · ebase + α · egroup
(6)
This design allows the month encoder to represent both continuous month position and coarser calendar-group context in a unified embedding. 6
CarbonCLIP
3.3
Dual-Branch Contrastive Pretraining
To align modality-specific representations from satellite imagery, street-view textual descriptions, and temporal information, CarbonCLIP adopts two separate contrastive learning strategies for spatial-semantic and temporal alignment. This decoupled design avoids manual tuning of loss weights and reduces conflicts between modality-specific objectives, enabling the model to learn complementary spatial-temporal representations more effectively. 3.3.1
Image-Text Contrastive Alignment
To enforce semantic consistency between satellite imagery and their corresponding street-view textual descriptions, we adopt a symmetric contrastive learning objective inspired by contrastive language-image pretraining (CLIP) [55]. Specifically, we utilize a visual encoder (e.g., ViT [57]) followed by a trainable projection layer to extract satellite image spatial embeddings. Meanwhile, descriptive captions are generated from street-view images using LMMs. These captions are further transformed into feature embeddings through a separate trainable projection layer, enabling both modalities to be mapped into a shared embedding space suitable for contrastive learning. For each image-text pair (Isi , Ti ) in a mini-batch of size N , where Isi denotes the spatial feature embedding of the i-th satellite image and Ti denotes its corresponding tile-level street-view semantic embedding, we contrast the aligned pair against all mismatched pairs within the batch. This encourages the model to maximize the mutual information between positive image-text pairs while minimizing it for negative pairs. We define a global image-text contrastive loss LIT as the average of the two symmetric objectives: " N 1 X exp (S(Isi , Ti )/τ ) LIT = − log PN s 2N j exp (S(Ii , Tj )/τ ) i # N X exp (S(Ti , Isi )/τ ) + log PN s j exp S(Ti , Ij )/τ i
(7)
where S(·, ·) denotes the cosine similarity between normalized embeddings, and τ is a temperature parameter that scales the logits to control the sharpness of the probability distribution. The loss encourages each satellite image to be most similar to its corresponding street-view description and vice versa, forming a bidirectional alignment in the shared embedding space. This symmetric contrastive formulation improves multimodal fusion by leveraging both image-to-text and text-to-image associations, which is critical for learning robust joint representations in carbon emission prediction tasks. 3.3.2
Temporal Alignment with Month Embeddings
To incorporate temporal awareness into the learned representations, we develop a symmetric contrastive alignment strategy between satellite image temporal features and their corresponding month embeddings. This alignment helps CarbonCLIP learn temporally smooth and cyclically aware representations that are beneficial for tasks with recurrent month-level variation. Given a mini-batch of size N , let Iti denote the temporal projection of the i-th satellite image feature, and Mi denote the corresponding embedding from the month encoder based on its month index mi ∈ {1, 2, . . . , 12}. To softly reflect the cyclic temporal structure of months, we define the circular month distance of month mi and mj as ∆ij = min(|mi − mj |, 12 − |mi − mj |) and convert it into a soft alignment weight using a Gaussian kernel: ! ∆2ij wij = exp − 2 (8) 2σ where σ is a hyperparameter to determine the width of the similarity window. By adjusting σ, the framework can be adapted to different month-level temporal similarity patterns. The final temporal alignment loss is the average of two symmetric soft contrastive objectives: " N N 1 X X exp (S(Iti , Mj )/τ ) LIM = − wij · log PN t 2N i k exp (S(Ii , Mk )/τ ) j (9) # N X exp S(Mi , Itj )/τ + wij · log PN t k exp (S(Mi , Ik )/τ ) j 7
CarbonCLIP
Table 2: Dataset details. Data
Resolution / Spatial granularity
Source
Satellite Imagery Planet [8] Street-View Singapore [58] / Imagery Beijing [59] Carbon ODIAC [27, 60] Emissions
3m 200 m 1 km
where S(·, ·) denotes the cosine similarity between normalized embeddings, and τ is a temperature parameter that scales the logits to control the sharpness of the probability distribution. This symmetric contrastive formulation encourages temporally similar satellite images to lie closer to their corresponding month embeddings in the shared space, while cyclic month distance ensures smooth alignment across year-end boundaries (e.g., December and January). By softly weighting the alignment based on temporal proximity, the model learns temporally aware representations that are useful for monthly environmental prediction tasks. 3.4
Satellite-Only Carbon Emission Predictor
To evaluate the effectiveness of the learned satellite image representations for real-world downstream tasks, we design a carbon emission prediction module that estimates monthly carbon emissions at the region level. In this phase, we freeze the vision encoder and feature projection layers pretrained on image-text and temporal alignment tasks, and train a lightweight MLP predictor with satellite image and carbon emission data pairs. For each satellite image tile, we extract two types of features using the pretrained model: a spatial embedding that captures spatial semantics and urban morphology, and a temporal embedding that encodes month-level temporal context. These features are generated by feeding satellite images into a frozen vision encoder followed by two separate projection layers, a spatial projection and a temporal projection. The outputs are normalized and concatenated to form the final feature vector: zi = Norm (Isi ) ∥ Norm Iti
(10)
where Isi and Iti denote the spatial and temporal feature embeddings of the i-th image, and ∥ indicates feature concatenation. Then, we use a simple MLP as the carbon emission predictor, which maps the concatenated feature vector zi to a scalar emission value ŷi : ŷi = fθ (zi )
(11)
where fθ (·) denotes the learnable MLP with ReLU activations. This design enables fast convergence and avoids overfitting, as the upstream encoders are already well-trained. The predictor is trained to minimize the mean squared error (MSE) loss between the predicted and ground-truth emissions: N
Lreg =
M
1 XX (yi,m − ŷi,m )2 N M i=1 m=1
(12)
where yi,m denotes the ground-truth carbon emission for the corresponding region i and month m. During training, each emission label is divided by a fixed constant of 1000 for numerical stability. This predictor serves as a carbon emission estimation validation step for our multimodal pretraining framework, demonstrating that the learned image features encode task-relevant spatial-temporal knowledge for urban-scale carbon emission estimation. 8
CarbonCLIP
4
Performance Evaluation
4.1
Experimental Setup
4.1.1
Dataset
The datasets used in this study comprise satellite imagery, street-view imagery, and carbon emission data for two representative Asian cities with different climatic and temporal regimes: Beijing and Singapore (see Table 2 for detailed information). We utilize high-resolution satellite imagery provided by Planet [8] for constructing monthly satellite observations in both Beijing and Singapore, which provides high-resolution imagery at a resolution of 3 meters per pixel. Both the satellite imagery and carbon emission data used in this study correspond to the year 2022. To ensure temporal consistency and minimize the influence of atmospheric conditions, we manually select one representative image for each month by using online filtering and preview tools in the Planet platform. Specifically, for each city and month, we prioritize images with less than ten percent cloud coverage, verified via both automated cloud masks and manual inspection of RGB composites. This selection strategy balances image availability and quality, allowing us to observe monthly surface changes with minimal cloud artifacts. For Singapore, no suitable September image with sufficiently low cloud coverage could be identified because of frequent tropical cloud cover; therefore, September was excluded from the Singapore dataset to maintain data quality. To align with the spatial resolution of the emission dataset and facilitate regional analysis, we aggregate and clip the Planet imagery into non-overlapping 1 km2 spatial tiles. After monthly quality filtering, the remaining satellite image-label pairs form the candidate pool before applying street-view coverage constraints. Street-view imagery was systematically collected to provide rich ground-level semantic context for multimodal urban representation learning. Due to regional accessibility constraints, imagery was obtained from two platforms: Google Maps Street View for Singapore [58] and Baidu Maps Street View for Beijing [59]. To ensure comprehensive, spatially uniform, and semantically diverse coverage, while avoiding biases introduced by population density or road hierarchy, we adopted a fixed-interval sampling strategy based on road network geometry. Specifically, we extracted complete drivable road networks for both cities from OpenStreetMap (OSM). Along each road segment, virtual sampling points were placed at regular 200-meter intervals. This step size was empirically chosen to balance wide geographic coverage with redundancy control, particularly in areas with dense road meshes. The strategy ensures equitable sampling across diverse urban forms, including central business districts, residential neighborhoods, and industrial zones. At each sampling point, the nearest available panoramic street-view image was queried using the corresponding API. In total, this process yielded 45,408 street-view images in Beijing and 20,415 images in Singapore. To fuse street-level semantics with satellite imagery, each street-view image was mapped to the 1 km2 spatial tile containing its GPS coordinates. After this spatial matching step, 45,398 panoramas in Beijing and 15,976 panoramas in Singapore were retained for tile-level semantic representation. Because street-view coverage is spatially static in this study, the spatial tile counts are computed once per city rather than expanded into month-level satellite observations. The resulting cross-modal linkage enables robust semantic enrichment and supervision during multimodal pretraining, with street-view imagery subsequently used as input to Qwen2.5-VL [54] for dense text generation. Table 3: Street-view coverage over 1 km2 spatial tiles counted once per city. Statistic Beijing Singapore 1 km2 spatial tiles Matched panoramas Panoramas/tile (mean) Panoramas/tile (median)
1,021 45,398 44.46 42
512 15,854 30.96 29.0
The statistics in Table 3 show that most tile-level semantic anchors are supported by multiple panoramas, which motivates constructing tile-level semantic anchors from the matched panoramas rather than treating any single panorama as the complete semantic label of a 1 km2 area. We use monthly gridded carbon dioxide emission estimates from the Open-source Data Inventory for Anthropogenic CO2 (ODIAC) dataset [27, 60]. ODIAC provides globally consistent fossil-fuel carbon dioxide emissions at a high spatial resolution of 1 km2 and a monthly temporal frequency. Owing to its fine granularity and observation-constrained construction, ODIAC has been widely adopted for urban-scale carbon monitoring, emission estimation, and policy evaluation [24, 48, 61, 62]. To construct image-label pairs, each study region was partitioned into non-overlapping 1 km2 9
CarbonCLIP
tiles consistent with the spatial resolution of both the satellite imagery and the ODIAC grids. For every tile and each month, a corresponding satellite image was independently extracted and paired with the exact ODIAC carbon emission value for the same spatial tile and temporal interval, forming a one-to-one image-label correspondence. To ensure strict temporal consistency and high-quality supervision, we adopted a conservative quality-control strategy. If a satellite image corresponding to a given tile-month pair was missing or deemed invalid due to cloud contamination, sensor noise, or other acquisition issues, the associated carbon emission label for that month was discarded. This filtering step inevitably leads to variability in the number of valid samples across months and locations. 4.1.2
Model Settings
We detail the key model components and configurations used, including the street-view description generator and the multimodal encoders. • Street-View Description Generator. To balance semantic fidelity and computational efficiency, we adopt the Qwen2.5-VL-7B model to generate fine-grained street-view descriptions. Table 4 provides a qualitative comparison of descriptions generated by Qwen2.5-VL models with different parameter sizes. As a small-scale generator-selection check, we manually score twenty held-out panoramic street-view images, ten from each city, under the same prompt. Each generated description is rated on a 1–10 scale according to three criteria: factual grounding, which measures whether visual claims are supported by the image; carbon relevance, which measures whether the description focuses on emission-related urban factors such as road infrastructure, land use, building density, traffic facilities, greenery, and industrial or commercial activity; and hallucination-free quality, which measures whether unsupported visual claims are avoided. As summarized in Table 5, Qwen2.5VL-7B improves over the 3B model in factual grounding and hallucination control, while remaining close to the 72B model across all three criteria. We therefore choose the 7B model because it provides a favorable tradeoff between semantic quality and computational cost for large-scale street-view description generation. • Image and Text Encoders. For contrastive pretraining, we employ a ViT-B/32 architecture as the satellite image encoder and a Transformer-based architecture as the text encoder. Both encoders are initialized with pretrained weights from the official openai/clip-vit-base-patch32 model [55]. 4.1.3
Baselines
We compare the performance of CarbonCLIP with the following baselines in carbon emission prediction: • ResNet [63]. ResNet is a classic and widely used convolutional neural network architecture composed of residual blocks. We adopt a ResNet-18 model pretrained on ImageNet and fine-tune it on the carbon emission prediction task using satellite imagery as input. This serves as a strong convolutional baseline to evaluate the effectiveness of spatial feature extraction. • Vision Transformer (ViT) [57]. ViT is a transformer-based architecture that divides an image into patches and applies self-attention mechanisms to model global dependencies across them, offering a powerful alternative to convolutional networks for visual tasks. In our baseline, we use ViT to extract spatial representations from satellite images and train a lightweight regression head to predict carbon emissions. This allows us to evaluate the effectiveness of transformer-based spatial encoding under a purely image-based setting. • UrbanCLIP [48]. UrbanCLIP is a multimodal urban representation model that generates textual descriptions from satellite images and enhances visual features through CLIP-style contrastive learning. In our adaptation, we follow its unimodal inference setup by using only satellite images to extract pretrained embeddings, which are then fed into a regression head for carbon emission prediction. All methods are evaluated under a controlled protocol using the same train, validation, and test partitions, ODIAC emission labels, fixed label-scaling procedure, and evaluation metrics. After each representation is obtained, the downstream carbon-emission regressor is trained using the same downstream training and evaluation procedure. This design keeps the comparison focused on the learned representation rather than differences in data preparation or evaluation settings. All downstream regressors use the same three-layer regression head, with 512 neurons in the first hidden layer, 256 neurons in the second hidden layer, and a final scalar output layer. We do not set a fixed number of training epochs; instead, each model is trained with early stopping and terminated when the validation performance does not improve for the most recent 10 validation steps. Therefore, the exact number of epochs can vary across methods and runs. For UrbanCLIP, we follow its satellite-only inference setting by extracting pretrained satellite embeddings and feeding them into the same downstream regressor. No baseline receives street-view imagery, street-view text, or month embeddings during inference. Table 6 summarizes the encoder, auxiliary supervision used during representation learning, and test-time input. 10
CarbonCLIP
Table 4: Comparison of generated descriptions from Qwen2.5-VL models with different parameter sizes (3B, 7B, and 72B) given the same panoramic street-view image and prompt. We highlight accurate details, factual errors, and redundant information to illustrate the impact of model scaling on semantic quality.
Model Size
Generated Description
3B
The panoramic street-view image of Singapore displays an urban scene featuring a large apartment building with multiple balconies arranged in a curved profile, bordered by trees with dense foliage on the left side and a solid fence on the right. This building suggests a residential area. The two-way roads curving around the buildings are lined with green lawns and spaces where sidewalk pedestrian traffic is expected. Administrative structures are likely within or adjacent to this building complex. There is also some public transport visible as red buses parked in designated lanes. The presence of fenceline and open space indicates provisions for both residential and commuter-friendly infrastructure.
7B
The panoramic image showcases an urban environment with a large multi-story residential building prominently situated in the center-right portion of the frame. The building exhibits a uniform architectural style with multiple balconies and windows, suggesting a planned residential area. Surrounding the building are lush green trees lining both sides of a wide road, indicating a well-maintained urban landscape. The road appears to be divided into lanes, with a median strip separating the traffic flow. On the left side of the image, there is a red bus partially visible, hinting at public transportation services available in this area. The overall scene reflects a clean and organized urban setting, likely part of a planned community or estate in Singapore.
72B
The panoramic street-view image at coordinates (103.833802, 1.367105) showcases a well-organized urban environment in Singapore characterized by a wide, multi-lane road flanked by lush greenery and mature trees providing ample shade. On either side of the road, there are sidewalks bordered by black and white striped curbs, with a bus stop visible on the right side indicating public transportation accessibility. The presence of a large, curved residential building with multiple stories suggests a high-density living area designed for efficiency and space optimization. Adjacent to the building, there is an open grassy area with scattered trees, possibly serving as a communal park or recreational space. The overall layout reflects a balance between urban infrastructure and natural elements, emphasizing pedestrian-friendly pathways and green spaces integrated within the built environment. The image also captures another bus on the left side of the road, reinforcing the area’s connectivity and reliance on public transit. The architectural style of the buildings and the structured urban planning suggest a modern and planned cityscape.
Table 5: Small-scale manual quality assessment of generated street-view descriptions of 20 samples (10 from each city). Scores range from 1 to 10, and higher scores are better for all metrics. Factual Carbon HallucinationModel Grounding Relevance free Qwen2.5-VL-3B Qwen2.5-VL-7B Qwen2.5-VL-72B
7.1 8.4 8.5
7.1 7.8 8.2
11
7.0 7.9 8.0
CarbonCLIP
Table 6: Controlled evaluation settings for baselines and CarbonCLIP. All methods use the same split, emission labels, fixed label scaling, and metrics. Method
Backbone
ResNet ResNet-18 ViT ViT-B/32 UrbanCLIP CoCa-ViTL/14 CarbonCLIPViT-B/32
Aux. Test-time Supervision Input None None Satellitederived text Street-view text and month context
Satellite tile Satellite tile Satellite embedding Satellite embedding
Table 7: Performance comparison of different methods on Beijing and Singapore across seasons in 2022. The best results are in bold and the second-best results are underlined. All results are averaged over three runs, with CarbonCLIP exhibiting consistently strong performance across both cities. Method
Beijing
Metric
Singapore
Spring
Summer
Autumn
Winter
Year
Rainy
Dry
Year
2
ResNet [63]
R ↑ RMSE↓ MAE↓
0.451 0.843 0.657
0.356 1.164 0.879
0.168 1.126 0.940
0.474 1.056 0.827
0.386 1.053 0.824
0.575 0.555 0.431
0.552 0.650 0.503
0.569 0.599 0.463
ViT [57]
R2 ↑ RMSE↓ MAE↓
0.463 0.834 0.601
0.588 0.931 0.660
0.488 0.883 0.661
0.604 0.917 0.671
0.559 0.892 0.648
0.517 0.591 0.469
0.558 0.645 0.497
0.544 0.616 0.482
UrbanCLIP [48]
R2 ↑ RMSE↓ MAE↓
0.628 0.694 0.540
0.582 0.938 0.674
0.703 0.672 0.510
0.640 0.873 0.617
0.642 0.803 0.586
0.537 0.579 0.460
0.602 0.613 0.469
0.576 0.594 0.464
CarbonCLIP (Proposed)
R2 ↑ RMSE↓ MAE↓
0.716 0.606 0.449
0.696 0.800 0.580
0.737 0.632 0.478
0.737 0.747 0.550
0.728 0.701 0.514
0.687 0.476 0.349
0.712 0.521 0.379
0.704 0.496 0.362
4.1.4
Evaluation Metrics
To evaluate prediction performance, we adopt three widely used metrics: the coefficient of determination (R2 ), root mean squared error (RMSE), and mean absolute error (MAE). A higher R2 and lower RMSE and MAE indicate better predictive accuracy. 4.2
Performance Evaluation
To evaluate CarbonCLIP, we compare it with representative baselines on the collected datasets. Table 7 and Fig. 3 present the overall results, from which we obtain the following three main findings: 1) CarbonCLIP achieves the best overall performance and temporal consistency across both cities. As reported in Table 7, CarbonCLIP consistently outperforms all baseline methods in terms of R2 , RMSE, and MAE for both Beijing and Singapore. In Beijing, CarbonCLIP attains an annual R2 of 0.728, substantially exceeding UrbanCLIP (0.642), ViT (0.559), and ResNet (0.386). Likewise, in Singapore, CarbonCLIP achieves an R2 of 0.704, outperforming UrbanCLIP (0.576), ViT (0.544), and ResNet (0.569). The baseline models exhibit relatively similar performance in Singapore, suggesting that existing approaches may share limitations in feature extraction. These results indicate that CarbonCLIP provides a stronger satellite-based representation for monthly urban carbon emission prediction. 2) CarbonCLIP consistently achieves top performance throughout the year, showing stable monthly prediction. As shown in Fig. 3, CarbonCLIP maintains stronger monthly performance than the baselines across Beijing and the available Singapore observations. ResNet and ViT suffer from noticeable degradation during several monthly periods, while UrbanCLIP is more stable but still fluctuates. In contrast, CarbonCLIP achieves consistently high R2 values with lower variation, reflecting more stable monthly prediction under the satellite-only inference setting. 3) These results highlight the benefit of fusing spatial and temporal priors through contrastive pretraining. The advantage of CarbonCLIP stems from its ability to integrate spatial semantics and month-level temporal context by 12
CarbonCLIP