MotifGen: Spatiotemporal interpolation of misaligned satellite images via multi-source generative modeling, in an application to tropical cyclones
arXiv:2606.24263v1 [cs.CV] 23 Jun 2026
Clément Dauvilliers1
and Claire Monteleoni1,2
INRIA, Paris, Île-de-France, France University of Colorado Boulder, Boulder, CO, USA. 1
2
Abstract. Microwave satellite imagery plays a crucial role in monitoring tropical cyclone precipitation and intensity worldwide, but suffers from long revisit times, potentially missing rapid storm evolution phases. While this raises the need for an interpolation method, it is made challenging by the high level of heterogeneity of microwave data coming from different instruments. In this work, we introduce the first generative model that can be applied to multiple geospatial sources that change across samples, occur at irregular time intervals, are misaligned geographically, and come from instruments with varying characteristics. We apply this model to the case of spatio-temporal interpolation of tropical cyclone microwave images from other microwave and infrared instruments. We train using a self-supervised task in which a random source is masked and reconstructed, and show that it leads to a significant decrease in Continuous Ranked Probability Score over supervised training. We show a further improvement by combining infrared and microwave data compared to microwave only. Using these improvements, the generative model produces an ensemble mean on par with that of a deterministic model, while generating a power spectrum significantly closer to that of true observations. To the best of our knowledge, this is the first generative model that interpolates microwave images of cyclones by combining multiple microwave instruments and infrared observations at irregular time intervals. Keywords: Multi-source machine learning · Satellite imagery · Generative models · Tropical cyclones.
1
Introduction
Tropical cyclones (TCs) are among the most destructive extreme weather events globally, both in terms of human fatalities and economic loss. Satellite imagery is a core ingredient in their monitoring, including in estimating and forecasting their intensities, sizes, and trajectories [31,8,6,33,35]. Among the products used to observe tropical cyclones, only geostationary satellites can provide near-continuous observations of cyclones, but these are limited to visible and infrared imaging
2
C. Dauvilliers and C. Monteleoni
[8]. Other types of measurement, such as microwave images, are only available when a satellite equipped with an adapted sensor orbits over a storm, leading to high revisit times that can often miss crucial evolution phases [13,20]. This raises the need for methods that can interpolate observations of tropical cyclones for non-stationary satellites, in particular microwave images [8]. Input sources
SSMIS F16 t = 35 min
SSMIS F17 t = 02h 40 min
-17.01
-17.25
-9.19
-21.46
-21.70
GMI GPM (2)
260
GMI GPM (3)
220
4
66
Temperature (K)
GMI GPM (1)
280
240
Flow matching
GMI GPM
70 .
9
6
Longitude
Deterministic
66 .9
50
55 .
80
78 .
74 .
6
64 69 .
7
82
65 .0
62 .
08
-4.04
-3.89
79 .0
.3 73
.5 67
.8
.1 56
61
68 66 .
66 .
64 .
59 .
2
-13.42
-27.61
7
-23.00
2
-12.90
1
-8.42
-12.65
32
-8.28
-17.58
32
-21.64
-18.47
51
-13.93
12
-25.55
61 .7
Infrared t = -01h 38 min
-9.38
61 .
Latitude
GMI GPM t = 0 min
57 .2
Groundtruth observation
(a) Inputs, target and reconstructions of the 37GHz microwave channel. Top left: target observation. Top right group: input sources, with δt indicating the time difference between the target and each input source. Bottom left: reconstruction from a deterministic model trained to optimize the RMSE. Bottom right: reconstructions generated by a flow matching model with different input noises. Source Footprint Coverage
50°E 60°E 70°E 80°E 0°
0°
10°S
10°S
20°S
20°S
30°S
Infrared GMI GPM SSMIS F16 SSMIS F17
30°S
50°E 60°E 70°E 80°E (b) Footprint of the satellite images (inputs and target) in the sample displayed in Figure 1a.
Fig. 1: Example of interpolation for a deterministic model versus a generative one. However, satellite imagery is a particular modality in machine learning, as the large diversity of instruments leads to highly heterogeneous data [26].
Generative multi-source spatiotemporal interpolation of tropical cyclones
3
Interpolating microwave images of tropical cyclones is no exception, mainly as microwave images are provided by a constellation of satellites rather than a lone instrument. The consequence is that, for a given cyclone, microwave observations occur at unpredictable times, depending on when one of the constellation’s satellites orbits over the storm’s location. This implies that the time interval between two successive observations is irregular, and that within any given time window the number of available microwave images isn’t known in advance. In addition, the observations are generally geographically misaligned, as tropical cyclones most often travel significant distances between two successive microwave images. Finally, the microwave sensors within the constellation present important differences in their characteristics, such as the measured frequency, viewing angle, spatial geometry, resolution, and swath width, among others. This means that models trained on data from a single sensor do not generally transfer well to other instruments [23,27]. All of these aspects (irregular time intervals, geographical misalignment, and varying characteristics) make microwave images of tropical cyclones a particularly heterogeneous type of data, thus requiring adapted machine learning methods and architectures. In addition to the challenge of heterogeneity, real observations do not generally contain the full information required to interpolate images of a cyclone at another time and place, making the task inherently probabilistic. Consequently, deterministic models for example trained to optimize the Root Mean Square Error (RMSE) output blurry, nonphysical predictions [6] - an example is shown in Figure 1a. This raises the need to switch to generative methods such as flow matching [17], which allow to sample from an approximation of the target distribution. In this work, we address these two challenges by proposing a generative framework to perform spatio-temporal interpolation of heterogeneous satellite observations. We use this framework to interpolate high-resolution satellite images of tropical and extra-tropical cyclones: our model receives as input observations from a large pool of microwave and infrared sensors, and reconstructs images from the GPM Microwave Imager (GMI), the reference instrument for the GPM constellation. As the amount of GMI images to use as training target is limited, we train our model using a self-supervised reconstruction task, and show that it leads to a significant improvement in Continuous Ranked Probability Score (CRPS) over training in a supervised manner. We show a further gain by including infrared images on top of microwave images in the model’s input. We validate our generative model by comparing its ensemble mean against a deterministic version adapted from [6]. Our model obtains comparable results on deterministic metrics (RMSE, MAE, MAPE), with a power spectrum significantly closer that of the groundtruth. The summary of our contributions is the following: – We introduce a generative model that can be applied to multiple geospatial sources that change across samples, occur at irregular time intervals, are misaligned geographically, and come from instruments with varying characteristics.
4
C. Dauvilliers and C. Monteleoni
– We apply this model to perform spatio-temporal interpolation of highresolution microwave images of tropical and extra-tropical cyclones. To the best of our knowledge, this is the first generative model that produces microwave images of cyclones by combining multiple microwave instruments and infrared observations. – We show that by leveraging self-supervised training and including infrared data in the input, our model reaches a better Root Mean Squarer Error than a deterministic model trained in the same conditions, while producing a significantly more realistic power spectrum. This paper is organized as follows: Section 2 introduces the related work; Section 3 details the multi-source generative framework; Section 4 describes the experiments and associated results; finally 6 concludes the paper.
2
Related Work
In this section, we discuss recent works that specifically study geospatial data through the use of generative modeling and multi-source machine learning. Generative models for meteorological data The success of generative models in recent years in natural images has naturally led to their widespread adoption for geospatial data in turn. In the context of meteorological data, generative models have been used with great success for global weather forecasting [5,24], atmospheric downscaling [29,19,21], and precipitation nowcasting [1,16], among others. In the specific case of microwave data, [27] used a CycleGAN to perform domain adaptation of microwave images from the SSMIS F18 sensor satellite to GMI images; [11] used a diffusion model to estimate precipitation maps from SSMIS F17 and infrared observations. In the specific case of tropical cyclones, [12] use a GAN to generate GMI images from infrared data. While these papers highlight the usefulness of generative models to generate microwave and precipitation data, they only generate images over the same area as their inputs. In contrast, we perform spatio-temporal interpolation, meaning we train a model to reconstruct images over a different area and at different times from the input sources, which is required to track a cyclone whose location changes with time. Besides, [27,11,12] use a single, fixed source as input, while we suggest to use a flexible number of input sources from multiple different instruments. Multi-source self-supervised models for geospatial data Multi-modal machine learning for geospatial and especially remote sensing data is an active field. A large part of it is foundation models that are trained in a self-supervised manner to take advantage of the large amount of publicly available unlabeled data [4,15,2]. Self-supervised learning pairs well with multi-modal geospatial problems: by gathering multiple sources over a common geographical area, one can exploit their correlations to train a latent representation [14]. In the case of meteorological data, MODS [29] recently introduced a model to downsample meteorological states, using multiple sources as input and a self-supervised reconstruction task.
Generative multi-source spatiotemporal interpolation of tropical cyclones
5
While MODS relies on a fixed set of sources, MoTiF [6] propose a self-supervised architecture and self-supervised training task adapted to multi-source geospatial data, and use those to interpolate microwave images of tropical cyclones. This work is essentially an extension of [6], with the following differences: (1) we modify the framework to train a flow matching model instead of optimizing the RMSE; (2) we use a combination of microwave and infrared as input data.
3
Methods
This section presents the details of our methodology. Subsection 3.1 first formulates the problem; then 3.2 gives an overall view of flow matching and how we employ it the context of this work. Finally, 3.3 details the architecture we use to process multi-source data. 3.1
Problem Formulation
Definition of an observation from a source Let Si be a source (e.g. one the microwave instruments). An element x from Si includes the following components: the pixels, as an image vi ∈ RC×H×W ; the latitude and longitude at each pixel, as an array of shape c ∈ R2×H×W ; the time t associated with the element; and a vector of characteristic variables s ∈ R5 . These characteristics are the instantaneous field-of-view (IFOV) at nadir along and across track, at the edges along and across track, and the observing frequency in GHz. Finally, x also includes a land-sea mask and a binary availability mask valued at zero at missing pixels. In practice, the dimensions H and W of the pixels, coordinates and masks are specific to each observation. Definition of a sample Let S = {S1 , S2 , ..., S|S| } be the set of all sources in the dataset. A sample X is defined as a set X = (x0 , x1 , x2 , ..., xK ) of observations from a subset of S. A source Si may be present or not in X, and can appear multiple times. We only consider a sample for training or evaluation if it includes at least two observations, so that one can be used as target (either for training or evaluation) and the other as conditioning for the generation. Every sample is defined around its reference observation x0 , which itself defines the sample’s reference time t0 . Using the observation times t0 , t1 , t2 , ..., tK , we compute the relative time differences ti − t0 ∆ti = ∆tmax where ∆tmax is the maximum time difference between two observations that are within the same sample, and which is a hyperparameter of the experiment. The intuition behind this choice is that the time difference between the input sources and between the input and output sources is valuable for the interpolation task, while absolute times would be over-specific and could lead to overfitting. Dividing by the model’s assimilation window normalizes the time differences to stay within [0, 1].
6
C. Dauvilliers and C. Monteleoni
Reconstruction task Let X = (x0 , x1 , x2 , ..., xK ) be a sample defined as above, with x0 being the reference observation. Our objective is to learn a model that reconstructs the reference image’s pixels, conditioned on the other sources and the reference source’s coordinates, time and characteristics: Fθ (x1 , x2 , ..., x3 , c0 , s0 , ∆t0 ) ≈ v0 In other words, the model generates a synthetic image with the characteristics of the reference source, at a requested time and over a requested geographic area, conditioned on a set of input sources to guide its generation. 3.2
Flow matching
Flow matching [17] is a form of generative modeling in which a starting distribution pinit that can be easily sampled (often a standard Gaussian) is transported to a data distribution pdata via a flow function ψr : Rd × [0, 1] → Rd . To do so, flow matching defines the flow via an ordinary differential equation (ODE): d ψ(x0 ) = ur (ψ(x0 )) for r ∈ [0, 1], dr
s.t. ψ0 (x0 ) = x0
(1)
where ur : R × [0, 1] → Rd is a marginal velocity field such that if X0 ∼ p0 = pinit and Xr is a solution of Equation 1, then X1 ∼ p1 = pdata . The model is trained to approximate the velocity field, i.e. uθr ≈ ur . Since ur is intractable, flow matching relies on the marginalization trick (Theorem 1 in [17]), which states that one can equivalently learn conditional velocity fields ur (x|x0 , x1 ). This is done by optimizing the conditional flow matching loss: LCF M (θ) = Er∼pr ,X0 ∼pinit ,X1 ∼pdata ||uθr (Xr ) − ur (Xr |X0 , X1 )||2 In this work, we use the CondOT path [17], which simply defines the conditional flow as a linear interpolation between a noise point and a data point: ψr (x0 |x1 ) = rx1 + (1 − r)x0 which corresponds to the conditional velocity field ur (xr |x0 , x1 ) = x1 − x0 . In our case, we replace the target source’s pixels with a version corrupted by standard Gaussian noise, while leaving the input sources unchanged. The training process is described in Figure 2. During training, we sample r ∼ pr following a lognormal distribution lognormal(0.0, 1.0), following the recommendations in [9]. Inferring with a trained model is done by sampling x0 ∼ Ndpixels (0, I) and numerically solving Equation 1 via the 1-step Euler method: xθr+h = xθr + huθr (xθr ) where h = N1 is the solving step size. We set N = 25 in our experiments.
Generative multi-source spatiotemporal interpolation of tropical cyclones
3.3
7
Multi-source architecture
We use an architecture based on MoTiF [6], which we adapt for flow matching. It is similar to the widely used Diffusion Transformer [22], with significant differences designed to handle multiple misaligned geospatial sources. The architecture uses three separate embedding spaces for the pixels, spatio-temporal coordinates and characteristics of the source. The coordinates embedding serves as positional encoding: since the sources correspond to different geographical areas, a traditional encoding based on the position of a patch within the image would loose any sense when computing the attention across sources. For example, the center of the cyclone could be in the top-left corner of one observation but in the bottom-right of another, depending on the satellites’ orbits. Instead, we use the spatio-temporal coordinates (latitudes, longitudes, time) as positional encoding, as was done in [6], since those have a common meaning across sources. ...
(target)
... Separately embed pixels, coords and characteristics
Noise the pixels (linear interpolation)
...
S1 ... SK
Backbone
S0
Decoders
Conditional Flow Matching loss
Fig. 2: Overall training pipeline.
Embedding layers The model possesses two embedding layers, for microwave images and for infrared data, which have separate weights the same functioning. To begin with, the channels are concatenated with the land-sea mask and the availability mask. The result v ∈ R(C+2)×H×W is then cut into square patches and embedded to the pixel embedding dimension dpixels following the common ViT patchifying process [7]. Regarding the coordinates, the latitudes, longitudes and time are concatenated, and then converted into sine-cosine features before being embedded as patches to a dimension dcoords . This follows the common usage in (vision) transformers for positional embedding [30,7,9]. Besides, this transformation accounts for the periodicity of latitudes and longitudes.
8
C. Dauvilliers and C. Monteleoni
Finally, the embedding layers also produce a conditioning sequence, which is also spatial and has the same shape as the embedded pixels and coordinates. It is built by first embedding the land-sea mask via patches to a dimension dcond . Then, the source characteristics vector and an availability flag are projected to dcond via a linear layer. Finally, the flow matching time step r ∈ [0, 1] is embedded via sine-cosine features, as is commonly done for diffusion / flow matching models [9]. The embedded land-sea mask, availability flag, characteristics and flow matching step are then summed together to obtain the final conditioning sequence. The availability flag is a scalar whose value is 1.0 for input sources, 0.0 for the target source whose pixels are masked, and -1.0 for missing sources. As a source may be present in one sample but absent from another one within the same mini-batch, empty tensors are inserted to maintain the same structure across samples. The availability flag is added in the conditioning to carry the information over whether each source is available or missing. Backbone The backbone is similar to that of DiT [22], in the sense that it is a chain of blocks that each contain attention layers, followed by a Multi-Layer Perceptron (MLP), each wrapped in an adaptive conditional layer normalization (adaLN). However, instead of using a single self-attention layer across all sources, each block contains a source-wise spatial self-attention layer followed by a crosssource attention layer. Source-wise self-attention The self-attention layers compute the attention within each source individually using 2D Swin windows [18]. However, as mentioned in 3.3, we use the embedded spatio-temporal coordinates as relative positional encoding [28]. For each source Si , the embedded patches of pixels are projected to a triplet (Qpixels , Kpixels , Vpixels ) of queries, keys and values of dimension dpixels . The spatio-temporal coordinates are in turn projected to another pair of queries and keys (Qcoords , Kcoords ), of dimension dcoords . The attention-weighted values are then computed as ! T T Qpixels Kpixels Qcoords Kcoords p √ Vout = Softmax +α Vpixels (2) dcoords dpixels where α is a learnable parameter that lets the model adapt the importance of the coordinates term. Cross-source attention As the sources are geographically misaligned, 3D Swin attention [3] or axial attention methods [5] cannot be applied to let information travel across sources. On the other hand, directly computing an attention matrix over the full sequence of all patches from all sources would be prohibitively costly. While [6] proposed a cross-source attention attention based on anchor points for a deterministic model, we found that it impaired performance for our generative task, likely due to compressing the embeddings within the attention layer. For this reason, we design a new cross-source attention layer, which lets information
Generative multi-source spatiotemporal interpolation of tropical cyclones
9
travel between any areas of any source, while reducing the cost by a constant factor, and without compressing the values. Its process is described in Figure 3. First, the projected queries and keys are split into 2D spatial square windows of side l, and averaged to obtain one query and key per window. The values are also split into the same windows, then stacked along the feature dimension. This process is repeated for each source. The queries, keys and values thus obtained are concatenated into sequences of length N l2 for queries and keys, and N for the values, where N is the total number of patches across all sources. The same is done to convert the embedded coordinates into keys and queries, resulting in two pairs (Qpixels , Kpixels ), (Qcoords , Kcoords ). The attention-weighted values are then computed using Equation 2, resulting in a set of updated values of shape (N, l2 dpixels ). These then go through the reverse process of being unstacked and reshaped into the original shape of each source. Through the averaging of the keys and queries, this layer has a memory cost of O(N 2 /l2 ), which while it remains quadratic, still reduces by the cost by 16 times with windows of length l = 4, compared to full attention. In practice, the computation of the attention matrix is done using multiple attention heads, as is commonly done in transformers [30]. xN blocks Shift Scale
features
LN
Cross-source attention
Shift Scale
Scale
coordinates
Skip
LN
Spatial Swin self-attention
LN = Layer normalization Shift Scale
Scale Skip
LN
Scale
MLP
Skip
characteristics
Source Si Project to
queries, keys and values
Average over spatial windows Stack windows over features dim
Shape
• Apply to all sources • Flatten spatial dimensions • Concatenate across all sources
• Apply for pixels and coordinates • Compute the updated values:
Shape • Split back the sources • Reshape to
Fig. 3: Diagram of the architecture. Top: overall view of the backbone. Bottom: scheme of the cross-source attention.
4
Experiments
This section describes the details of the experiments. We first present the dataset and data preprocessing; we then detail the experiments that were conducted on the training strategy and inclusion of infrared data. 4.1
Dataset
We use the TC-PRIMED v01r01 dataset [25], which is publicly available. We extract microwave images from 11 microwave sensors (AMSR2 GCOMW1, TMI
10
C. Dauvilliers and C. Monteleoni
TRMM, GMI GPM, SSMI F11-13-14-15, and SSMIS F16-F19). We use the near37GHz and near-89GHz channels, with both horizontal and vertical polarizations for each. This choice of frequencies has two motivations: their combination gives valuable information onto precipitation within the cyclone [32,27], and they are available on many satellites across the GPM constellation. We also extract near11µm infrared observations, which are pre-centered around the cyclones’ centers in TC-PRIMED. We use observations spanning from 1987 to 2024, of which the 2005-07-14-21-23 seasons are reserved for validation, 2006-08-15-22-24 for testing, and all other seasons are used for training. The dataset includes images over all major basins. Each source is normalized individually by subtracting the mean of each channel and dividing its by its standard deviation. We add a land-sea mask obtained via the global-land-mask Python package. The characteristics (IFOV and frequency) described in Section 3.1 are each normalized jointly across all microwave sources by subtracting the minimum and dividing by the maximum minus the minimum. 4.2
Training strategies
In this section, we describe two strategies to train the model. These differ in the way the training samples are defined, and by which source may be used as target. Both strategies rely on a hyperparameter ∆tmax , as explained in 3.1, which is the maximum time difference between two input sources. In this sense, ∆tmax can be thought of as an assimilation window. Supervised strategy Since our objective is to learn a model that generates images from the GMI instrument, a natural way to train the model is to train in a supervised manner, using GMI images as target and all other sources as inputs. In this context, we define the samples by the following process, schematized in Figure 4: each GMI observation x0 in the dataset defines a time window ω = [t0 − ∆tmax , t0 + ∆tmax ] centered around the time t0 of the GMI image. 2 2 Then, all observations xi of the same cyclone in the dataset from sources other GMI that are within ω are gathered into a set of input sources (x1 , x2 , ..., xK ). The model is then tasked with reconstructing the GMI image x0 , conditioned on (x1 , x2 , ..., xK ). We only consider samples for which at least one input source is available, i.e. there is at least one image from another source that is temporally separated from the GMI image by ∆tmax or less. This strategy results in 5,876 2 training samples. Self-supervised strategy While the supervised strategy is a natural approach, the quantity of training samples is limited to the number of GMI images which are close enough in time to an observation of the same cyclone from another source. For this reason, we use a self-supervised strategy proposed in [6], to increase the amount of training data. The process is the following: every observation x in the dataset (as opposed to GMI images only) defines a time window
Generative multi-source spatiotemporal interpolation of tropical cyclones Supervised strategy
input input
11
Self-supervised strategy
Reference instrument (GMI) target input
One source randomly selected as target, others are input
Time
Time Sample 1
Sample 1 Sample 2
Fig. 4: Scheme of the training strategies. Left: supervised strategy, in which the samples are defined as time windows centered around GMI observations. Right: self-supervised strategy, where every observation in the dataset (not only GMI) defines a sample. In this case, the target source is randomly selected among all sources within the time window.
ω = [t0 − ∆tmax , t0 ]. Then again, all observations of the same cyclones that are within ω are gathered to form a sample X = (x0 , x1 , x2 , ..., xK ). Instead of always using the reference image x0 as target as in the supervised strategy, one of the observations (x0 , x1 , x2 , ..., xK ) is randomly selected as the target with uniform probability. This augments the training data in two ways compared to the supervised strategy: first, we keep all samples for which at least any two sources are temporally separated by at most ∆tmax , instead of necessarily including a GMI image. This raises the number of training samples from 5,876 with the supervised strategy to 47,793. While most of these samples do not include a GMI observation, which is the source that we eventually aim to interpolate and that we evaluate our model on, we make the hypothesis that the tasks are still correlated enough that the model will benefit from the gain in training samples. Second, the random target selection means that for a given sample, the task differs between two training epochs, virtually further increasing the diversity of the training task.
4.3
Impact of including infrared data in the input sources
After assessing the impact of the training strategy, we also compare the performances of two models respectively trained with and without including infrared images in the input sources. As, to our knowledge, no previous work has used both microwave and infrared to interpolate microwave images of tropical cyclones, the aim is to assess the usefulness of combining these two modalities. This comparison is done on models trained with the self-supervised strategy. The training parameters used across the experiments are listed in Appendix 1.
12
C. Dauvilliers and C. Monteleoni
5
Results
5.1
Evaluation metrics
Continous Ranked Probability Score (CRPS) The CRPS measures the quality of an ensemble of predictions against a single observation, and is only commonly used in numerical weather forecasting to evaluate probabilistic models [24,5]. It rewards both sharpness (error in the ensemble mean) and calibration, by comparing the predicted distribution with the marginal single-step distribution defined by the single true observation: θ FairCRPS((y1θ , y2θ , ...yM ), y) =
M M X 1 X θ 1 |yi − y| − |y θ − yjθ | M i=1 2M (M − 1) i,j=1 i
where M is the ensemble size. We note that we use the Fair version of the CRPS, which is unbiased for finite ensembles [34], as we only use M = 10 members. The CRPS is computed at each pixel, then averaged over the image. Skill-Spread ratio The Skill-Spread Ratio (SSR) measures the dispersion of an ensemble as the ratio between the average standard deviation of the members and the ensemble mean RMSE: r σspread M SSR = M + 1 RMSEensemble mean A well-calibrated ensemble should have an SSR close to 1 [10]. An SSR below 1 means the ensemble is underdispersive, while an SSR above 1 means it is overdispersive. We note that we use the MM+1 correction as we use a relatively low number of members (10). Deterministic metrics We use the Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE) and to compare the performances of a deterministic model versus the ensemble mean of our flow matching model. 5.2
Evaluation setting
The evaluation samples are defined following the same process as in the supervised strategy: for all cyclones within the test seasons, we gather every available GMI image. For each such GMI image x0 , we define a time window ω = [t0 − ∆tmax , t0 + 2 ∆tmax ], and gather all observations of the same storm that occur within ω. These 2 sources are used as input for the reconstruction of the GMI image, which is used as groundtruth to evaluate the metrics. For all flow matching models, 10 random realizations are generated for each sample, with the same starting noise points across models.
Generative multi-source spatiotemporal interpolation of tropical cyclones
5.3
13
Impacts of adding self-supervised training and infrared input
8
1.2
6
1.0
SSR
CRPS
In this section, we ablate over the following choices: (a) training with the supervised strategy (described in Section 4.2), and including only microwave data as input sources; (b) training with the self-supervised strategy, still only with microwave inputs; and (c) training with the self-supervised strategy, including both microwave and infrared sources as input. For this evaluation, all models are trained with exactly the same architecture and training hyperparameters, and are evaluated over the same test samples. The distributions of CRPS over the test samples for the three models are presented in Figure 5a. Training the model with the self-supervised strategy over the supervised improves the average CRPS by 13.2%, with a further improvement of 8% gained by combining infrared and microwave data instead of microwave only. That conclusion is also valid according to the RMSE, MAE and MAPE, as displayed in Table 1. On the other hand, Figure 5b displays the Spread-skill ratio for each model. While all models are underdispersive (SSR below 1.0), the addition of self-supervised training and infrared inputs worsen that caveat. We conclude from this that the gap in CRPS between the models stems from the improvement in accuracy of the ensemble mean (as shown in Figure 5c), compensating the lower spread.
4 2
0.8 0.6
Supervised Microwave only
Self-supervised Microwave only
Self-supervised Microwave+Infrared
Model
Supervised Microwave only
MAPE (%)
(a) Distributions of CRPS on the test samples for three models with different training configurations and input sources. Lower is better.
Self-supervised Microwave only
Model
Self-supervised Microwave+Infrared
(b) Distribution of Spread-Skill Ratio (SSR) on the test samples for each flow matching model. All models are below 1.0, indicating underdispersion.
6 5 4 3 2 1 Supervised Microwave only
Self-supervised Microwave only
Model
Self-supervised Microwave+Infrared
(c) Distribution of the ensemble mean Mean Absolute Percentage Error (MAPE) on the test samples for each flow matching model.
Fig. 5: Comparison of CRPS, SSR and MAPE between flow matching models.
14
5.4
C. Dauvilliers and C. Monteleoni
Comparing generative models with a deterministic baseline
In this section, we compare our best flow matching model (self-supervised, using both microwave and infrared) to a deterministic baseline. The deterministic model uses exactly the same architecture as the flow matching model, and is trained with the Root Mean Square Error (RMSE) as loss function. This deterministic baseline is an adaptation of the MoTiF architecture proposed in [6], which is to our knowledge the only model in the literature that can be applied to data with the level of heterogeneity faced in this work. Compared to the original MoTiF, the following changes have been applied: first, the multi-source cross-attention layer has been replaced with the cross-source attention layer described in 3.3; and second, an embedding layer has been added to let the model use infrared observations as input.
Table 1: Quantitative evaluation comparing the flow matching ensemble means and a deterministic baseline. The values are presented as the average metric over the test samples, followed by a 95% confidence interval in brackets. The MAE and RMSE are in Kelvin. "MW" stands for Microwave, while "IR" stands for Infrared. Lower is better for all metrics. Model type
Training strategy Input sources
CRPS
-
RMSE
MAE
MAPE (%)
9.32 5.15 2.26 [9.19, 9.45] [5.07, 5.22] [2.22, 2.30]
Deterministic Self-supervised
MW+IR
Flow matching
Supervised
MW only
4.08 10.92 6.29 2.79 [4.02, 4.14] [10.77, 11.07] [6.19, 6.38] [2.73, 2.84]
Flow matching
Self-supervised
MW only
3.54 9.72 5.31 2.34 [3.49, 3.60] [9.58, 9.85] [5.23, 5.39] [2.30, 2.39]
Flow matching
Self-supervised
MW+IR
3.26 9.01 4.76 2.09 [3.21, 3.31] [8.89, 9.13] [4.69, 4.83] [2.05, 2.13]
Quantitative comparison The quantitative comparison of the deterministic model against the flow matching models is presented in Table 1. As the deterministic model only outputs a single prediction, we do not compare its CRPS, which inherently advantages probabilistic models. The deterministic metrics (RMSE, MAE and MAPE) are computed against the ensemble mean of the flow matching models. When using self-supervised training and including infrared in the inputs to both our flow matching model and the deterministic MOTIF baseline, the generative model produces an ensemble mean that is on par (and actually marginally better) than the deterministic baseline. This result validates that the switch to generative modeling does not come at the cost of a worse estimation of the conditional expectation. Power spectrum comparison The main aim of training a generative model is its supposed ability to generate realistic predictions, notably by producing a power
Generative multi-source spatiotemporal interpolation of tropical cyclones
15
spectrum close to that of true data. Mid and high frequencies are especially important in the context of tropical cyclones microwave images, as extreme precipitation and winds occur in small-scale spatial features. In this regard, the flow matching model reproduces the target spectrum more accurately, as shown in Figure 6. The mid and high frequencies are particularly better conserved for the 89GHz channels, which have a higher spatial frequency than the 37GHz ones. Additional visualizations are included in Appendix C. 106
PSD - Channel: 89.0H
PSD - Channel: 89.0V
104
102
102
102
10 2
10 1
100
100
10 2
10 2
100 10 1 10 2
Deterministic Flow matching 10 2
10 1
Frequency
PSD Gain
Frequency PSD Gain - Channel: 37.0H
10 2
10 1
Frequency PSD Gain - Channel: 37.0V 100 10 1 10 2
10 2
10 1
Frequency
100 10 2
10 2
10 1
Frequency PSD Gain - Channel: 89.0H 100
PSD Gain
10 2
Deterministic Flow matching target
PSD Gain
102
PSD
104
PSD
104
104
100
PSD Gain
PSD - Channel: 37.0V
106
PSD
PSD
PSD - Channel: 37.0H 106
10 1 10 2
10 2
10 1
Frequency
10 2
10 1
Frequency PSD Gain - Channel: 89.0V 100 10 1 10 2
10 2
10 1
Frequency
Fig. 6: Comparison of the radially-averaged Power Spectral Density (PSD), averaged over all test samples, between a flow matching and a deterministic model (self-supervised, using both microwave and infrared as input). The PSD Gain is computed by dividing the predictions’ PSD by that of the target at each frequency. We attribute the spikes at the very highest frequencies in the deterministic spectrum to artifacts due to the patching process.
6
Discussion and Conclusion
To summarize, this work proposes the first generative model to interpolate microwave images of tropical cyclones conditioned on data from multiple microwave and infrared instruments. To deal with the high heterogeneity of the data, we use an architecture specifically designed for spatio-temporally misaligned geospatial sources. We show that our model benefits from being trained in a self-supervised manner, as well as combining both infrared and microwave data. Using these benefits, our model retains an ensemble mean accuracy comparable to that of a deterministic baseline, while better reproducing high-frequency features. One of the direct directions of improvement is to correct the underdispersivity, which is increased by self-supervised training and using infrared data. A plausible way to increase the spread is to calibrate the input noise variance, by multiplying the noise vector by a constant factor. Besides, the self-supervised model presented
16
C. Dauvilliers and C. Monteleoni
in this work is not fine-tuned on GMI targets specifically, meaning it is calibrated to fit the spread of the full ensemble of microwave sensors used during training. At longer term, future work could include for example looking at other types of weather events such as extreme precipitation. Besides, while this is to our knowledge the only deep learning architecture that can be applied to such heterogeneous data, there could most definitely be other approaches to test in the future. Still, we hope this work can serve as inspiration for further works on highly heterogeneous geospatial data. Reproducibility The data used is publicly available through the TCPRIMED website.The code is available at https://github.com/dauvillc/ motif. Use of Generative AI Generative AI was used to format the tables and figures, as well as for coding. AI was not used to write or rephrase content in this paper.
7
Author contributions
C.D.: Conceptualization (lead), Methodology (lead), Software, Validation, Investigation, Data Curation, Writing - Original Draft, Visualization. C.M.: Conceptualization (support), Methodology (support), Writing - Review & Editing, Supervision, Project Administration, Resources, Funding Acquisition.
8
Acknowledgments
We thank Nicolas Viltard, Cécile Mallet and Laurent Barthès for sharing their knowledge regarding microwave imagery and tropical cyclones. Many thanks to Anastase Charantonis for the fruitful discussions regarding the methodology. C.D. and C.M. were supported by the French government, via the Choose France Chair in AI. This work was granted access to the HPC resources of IDRIS under the allocation AD011014682R2 made by GENCI.
References 1. Asperti, A., Merizzi, F., Paparella, A., Pedrazzi, G., Angelinelli, M., Colamonaco, S.: Precipitation nowcasting with generative diffusion models (Sep 2023). https: //doi.org/10.48550/arXiv.2308.06733 2. Astruc, G., Gonthier, N., Mallet, C., Landrieu, L.: OmniSat: Self-supervised Modality Fusion for Earth Observation. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds.) Computer Vision – ECCV 2024, vol. 15086, pp. 409–427. Springer Nature Switzerland, Cham (2025). https: //doi.org/10.1007/978-3-031-73390-1_24 3. Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., Tian, Q.: Accurate medium-range global weather forecasting with 3D neural networks. Nature 619(7970), 533–538 (Jul 2023). https://doi.org/10.1038/s41586-023-06185-3
Generative multi-source spatiotemporal interpolation of tropical cyclones
17
4. Cong, Y., Khanna, S., Meng, C., Liu, P., Rozi, E., He, Y., Burke, M., Lobell, D.B., Ermon, S.: SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery 5. Couairon, G., Singh, R., Charantonis, A., Lessig, C., Monteleoni, C.: ArchesWeather & ArchesWeatherGen: A deterministic and generative model for efficient ML weather forecasting (Dec 2024). https://doi.org/10.48550/arXiv.2412.12971 6. Dauvilliers, C., Monteleoni, C.: MoTiF: A self-supervised model for multi-source forecasting with application to tropical cyclones. Environmental Data Science 4, e36 (Jan 2025). https://doi.org/10.1017/eds.2025.10014 7. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Jun 2021). https://doi.org/10.48550/arXiv.2010.11929 8. Duong, Q.P., Wimmers, A., Herndon, D., Tan, Z.M., Zhuo, J.Y., Knaff, J., Al Abdulsalam, I., Horinouchi, T., Miyata, R., Avenas, A.: Objective satellite methods including AI algorithms reviewed for the tenth International workshop on tropical cyclones (IWTC-10). Tropical Cyclone Research and Review 12(4), 259–266 (Dec 2023). https://doi.org/10.1016/j.tcrr.2023.11.001 9. Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., Podell, D., Dockhorn, T., English, Z., Lacey, K., Goodwin, A., Marek, Y., Rombach, R.: Scaling Rectified Flow Transformers for High-Resolution Image Synthesis (Mar 2024). https://doi.org/10.48550/arXiv. 2403.03206 10. Fortin, V., Abaza, M., Anctil, F., Turcotte, R.: Why Should Ensemble Spread Match the RMSE of the Ensemble Mean? Journal of Hydrometeorology 15(4), 1708–1713 (Aug 2014). https://doi.org/10.1175/JHM-D-14-0008.1 11. Guilloteau, C., Kerrigan, G., Nelson, K., Migliorini, G., Smyth, P., Li, R., FoufoulaGeorgiou, E.: A Generative Diffusion Model for Probabilistic Ensembles of Precipitation Maps Conditioned on Multisensor Satellite Observations (Sep 2024). https://doi.org/10.48550/arXiv.2409.16319 12. Han, K.H., Jo, S., Hong, S.: GeoGMI: A generative adversarial framework for virtual 89 GHz microwave brightness temperature retrieval from geo-kompsat-2A infrared observations for tropical cyclone monitoring. Atmospheric Research 336, 108855 (Jun 2026). https://doi.org/10.1016/j.atmosres.2026.108855 13. Haynes, K., Slocum, C., Knaff, J., Musgrave, K., Razin, M.N., Ebert-Uphoff, I.: Aiding Tropical Cyclone Forecasting by Creating Synthethic 89- and 37- GHz Imagery from Operational Geostationary Satellites. In: 36th Conference on Hurricanes and Tropical Meteorology. AMS (May 2024) 14. Jakubik, J., Yang, F., Blumenstiel, B., Scheurer, E., Sedona, R., Maurogiovanni, S., Bosmans, J., Dionelis, N., Marsocci, V., Kopp, N., Ramachandran, R., Fraccaro, P., Brunschwiler, T., Cavallaro, G., Bernabe-Moreno, J., Longépé, N.: TerraMind: Large-Scale Generative Multimodality for Earth Observation (Sep 2025). https: //doi.org/10.48550/arXiv.2504.11171 15. Klemmer, K., Rolf, E., Robinson, C., Mackey, L., Rußwurm, M.: SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery (Apr 2024). https: //doi.org/10.48550/arXiv.2311.17179 16. Li, C., Ling, X., Xue, Y., Luo, W., Zhu, L., Qin, F., Zhou, Y., Huang, Y.: Precipitation Nowcasting Using Diffusion Transformer with Causal Attention (Oct 2024). https://doi.org/10.48550/arXiv.2410.13314 17. Lipman, Y., Chen, R.T.Q., Ben-Hamu, H., Nickel, M., Le, M.: Flow Matching for Generative Modeling (Feb 2023). https://doi.org/10.48550/arXiv.2210.02747
18
C. Dauvilliers and C. Monteleoni
18. Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 9992–10002. IEEE, Montreal, QC, Canada (Oct 2021). https://doi.org/10.1109/ICCV48922. 2021.00986 19. Mardani, M., Brenowitz, N., Cohen, Y., Pathak, J., Chen, C.Y., Liu, C.C., Vahdat, A., Nabian, M.A., Ge, T., Subramaniam, A., Kashinath, K., Kautz, J., Pritchard, M.: Residual corrective diffusion modeling for km-scale atmospheric downscaling. Communications Earth & Environment 6(1), 124 (Feb 2025). https://doi.org/ 10.1038/s43247-025-02042-5 20. Meng, F., Song, T., Xu, D.: Simulating Tropical Cyclone Passive Microwave Rainfall Imagery Using Infrared Imagery via Generative Adversarial Networks. IEEE Geoscience and Remote Sensing Letters 19, 1–5 (2022). https://doi.org/10.1109/ LGRS.2022.3152847 21. Merizzi, F., Asperti, A., Colamonaco, S.: Wind speed super-resolution and validation: From ERA5 to CERRA via diffusion models. Neural Computing and Applications 36(34), 21899–21921 (Dec 2024). https://doi.org/10.1007/s00521-024-10139-9 22. Peebles, W., Xie, S.: Scalable Diffusion Models with Transformers (Mar 2023). https://doi.org/10.48550/arXiv.2212.09748 23. Pfreundschuh, S., Brown, P.J., Kummerow, C.D., Eriksson, P., Norrestad, T.: GPROF-NN: A neural-network-based implementation of the Goddard Profiling Algorithm. Atmospheric Measurement Techniques 15(17), 5033–5060 (Sep 2022). https://doi.org/10.5194/amt-15-5033-2022 24. Price, I., Sanchez-Gonzalez, A., Alet, F., Andersson, T.R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., Lam, R., Willson, M.: Probabilistic weather forecasting with machine learning. Nature 637(8044), 84–90 (Jan 2025). https://doi.org/10.1038/s41586-024-08252-9 25. Razin, M.N., Slocum, C.J., Knaff, J.A., Brown, P.J., Bell, M.M.: Tropical Cyclone Precipitation, Infrared, Microwave, and Environmental Dataset (TC PRIMED). Bulletin of the American Meteorological Society 104(11), E1980–E1998 (Nov 2023). https://doi.org/10.1175/BAMS-D-21-0052.1 26. Rolf, E., Klemmer, K., Robinson, C., Kerner, H.: Position: Mission Critical – Satellite Data is a Distinct Modality in Machine Learning. In: Forty-First International Conference on Machine Learning (Jun 2024) 27. Sambath, V., Dubois-Quilici, N., Viltard, N., Martini, A., Mallet, C.: Unsupervised Domain Adaptation to Mitigate Out-of-Distribution Problem of Spatial Radiometer Images: Application to Quantitative Precipitation Estimation. IEEE Transactions on Geoscience and Remote Sensing 62, 1–14 (2024). https://doi.org/10.1109/ TGRS.2024.3403373 28. Shaw, P., Uszkoreit, J., Vaswani, A.: Self-Attention with Relative Position Representations (Apr 2018). https://doi.org/10.48550/arXiv.1803.02155 29. Tu, S., Xu, J., Yang, W., Bai, L., Fei, B.: MODS: Multi-source Observations Conditional Diffusion Model for Meteorological State Downscaling (Jun 2025). https://doi.org/10.48550/arXiv.2506.14798 30. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention Is All You Need (Aug 2023). https://doi.org/10. 48550/arXiv.1706.03762 31. Velden, C.S., Herndon, D.: A Consensus Approach for Estimating Tropical Cyclone Intensity from Meteorological Satellites: SATCON (Aug 2020). https://doi.org/ 10.1175/WAF-D-20-0015.1
Generative multi-source spatiotemporal interpolation of tropical cyclones
19
32. Viltard, N., Sambath, V., Lepetit, P., Martini, A., Barthès, L., Mallet, C.: Evaluation of drain, a deep-learning approach to rain retrieval from gpm passive microwave radiometer (Mar 2023). https://doi.org/10.48550/arXiv.2303.01220 33. You, S., Zhu, P., Guzman, O., Jiang, H.: Predicting Tropical Cyclone Intensity Using a Convolutional Neural Network and 20 Years of IMERG Satellite Rainfall Data. Weather and Forecasting 40(11), 2317–2331 (Oct 2025). https://doi.org/ 10.1175/WAF-D-24-0196.1 34. Zamo, M., Naveau, P.: Estimation of the Continuous Ranked Probability Score with Limited Information and Applications to Ensemble Weather Forecasts. Mathematical Geosciences 50(2), 209–234 (Feb 2018). https://doi.org/10.1007/ s11004-017-9709-7 35. Zhuo, J.Y., Tan, Z.M.: Physics-Augmented Deep Learning to Improve Tropical Cyclone Intensity and Size Estimation from Satellite Imagery. Monthly Weather Review 149(7), 2097–2113 (Jul 2021). https://doi.org/10.1175/MWR-D-20-0333. 1
20
C. Dauvilliers and C. Monteleoni
Appendix A
Training details
Table 1 list the architectural and optimization parameters used for all flow matching and deterministic experiments.
Table 2: Training settings used for the experiment. Architecture
Optimization
Number of blocks in the backbone
12 Optimizer
AdamW
Pixels embedding dim. dpixels
512 β1 , β2
0.9, 0.999
Coordinates embedding dim. dcoords 256 Weight decay
0.05
Conditioning embedding dim. dcond 256 LR Schedule
Cosine annealing
Number of attention heads
8
Min LR, Max LR
10−6 , 3 × 10−4
Cross-source att. window size
4
LR warmup
1 epoch, linear from 0 to 3 × 10−4
Source-wise att. window size
8
Epochs
MLP inner expansion ratio
2.0 Checkpoint selection
B
100 Best val. loss
Impact of sources availability
This section displays the variation in performance depending on which input sources are available. Figure 7 shows the variations in CRPS, MAPE and SSR depending on the time between the target image and the input source that’s closest to it in time. We observe that, as expected, the metrics generally improve as the model gets a source that’s closer in time to its target. Figure 8 displays the metrics marginalized over each input source specifically.
Generative multi-source spatiotemporal interpolation of tropical cyclones
21
CRPS
5 4 3 3
2
1
0
1
2
3
Signed time difference to closest input (hours)
Self-supervised Microwave only
Self-supervised Microwave+Infrared
Supervised Microwave only
MAPE
3.0 2.5 2.0 3
2
1
0
1
2
3
Signed time difference to closest input (hours)
Self-supervised Microwave only
Self-supervised Microwave+Infrared
Supervised Microwave only
SSR
0.9 0.8 3
2
1
0
1
2
3
Signed time difference to closest input (hours)
Number of samples
Self-supervised Microwave only
Self-supervised Microwave+Infrared
Supervised Microwave only
200 100 0
3 2 1 0 1 2 3 Min. signed dt to source (hours) AMSR2 GCOMW1 SSMIS F16 SSMIS F18 Infrared SSMIS F17 SSMIS F19
TMI TRMM
Fig. 7: CRPS, MAPE and Skill-Spread ratio against time delta (dt) between the target and the source that is temporally closest to it. THe time delta is binned in 30min intervals, centered on dt=0. Shaded areas indicate 95% confidence intervals.
C. Dauvilliers and C. Monteleoni
Input source
22
AMSR2 GCOMW1 Infrared SSMIS F16 SSMIS F17 SSMIS F18 SSMIS F19 TMI TRMM 2
4
Input source
Self-supervised Microwave only
Self-supervised Microwave+Infrared
2
3
Self-supervised Microwave only
Input source
8 Supervised Microwave only
AMSR2 GCOMW1 Infrared SSMIS F16 SSMIS F17 SSMIS F18 SSMIS F19 TMI TRMM 1
4
MAPE
5
6
7
Self-supervised Microwave+Infrared
Supervised Microwave only
AMSR2 GCOMW1 Infrared SSMIS F16 SSMIS F17 SSMIS F18 SSMIS F19 TMI TRMM 0.4
0.6
Self-supervised Microwave only
Input source
6
CRPS
0.8
SSR
1.0
1.2
Self-supervised Microwave+Infrared
1.4 Supervised Microwave only
AMSR2 GCOMW1 Infrared SSMIS F16 SSMIS F17 SSMIS F18 SSMIS F19 TMI TRMM 0 Self-supervised Microwave only
200
400
600
Number of samples Self-supervised Microwave+Infrared
800 Supervised Microwave only
Fig. 8: For each input source, values of CRPS, MAPE and Skill-Spread ratio averaged over the test samples containing that source.
Generative multi-source spatiotemporal interpolation of tropical cyclones
Additional visualizations
Groundtruth observation GMI GPM t = 0 min
-21.05
150
250 200
8 -17 1.4 -17 4 5. 17 93 8.2 17 5 2.9 16 0 9.6 8
7
-17
3
5.2
9.5
16
17
8
16
3.7
6 8.0
8.9
3
-13.59
15
16
SSMIS F17 t = 02h 45 min
-18.49
Longitude
Flow matching
Deterministic GMI GPM
-32.90 -28.20 -23.36
6.3
4
-26.01 -22.07 -18.02 -13.88
15
15
9.0
3
3
4.0
16
17
1 2.8
1
16
3.2
5 5.2
16
16
8.0
6
3.6
7
-29.48
16
-25.11
9.5
-20.77
Infrared t = 02h 19 min
AMSR2 GCOMW1-29.78 t = -58 min
-32.12 -27.82 -23.42 -18.99 -14.54
-16.37
GMI GPM (3)
Input sources
t = 0 min
0
-17
GMI GPM (2)
Groundtruth observation GMI GPM
0.5
3
-17
GMI GPM (1)
GMI GPM (1)
GMI GPM (2)
Temperature (K)
GMI GPM
17
8.2
6.0
17
0.2
17
4.5
8.8
16
15
5.2
5.6
16
7.6
16
2.7
17
17
16
Flow matching
Deterministic
-11.93
5. -17 68 9.7 17 3 4.8 16 6 9.8 16 9 6.9 8
-5.00
2
-4.33
7
-21.12
3
-9.82
1
-8.61
3
200
-16.62
2
250
-14.63
2
-19.42
-12.84
8
-17.01
4
-7.49
Longitude
Latitude
t = 01h 05 min
-24.18
-12.07
0.3
Latitude
-2.87
Input sources SSMIS F16
Infrared t = -01h 48 min
Temperature (K)
C
23
GMI GPM (3)
C. Dauvilliers and C. Monteleoni
Groundtruth observation GMI GPM 26.17
SSMIS F18 t = -01h 33 min
7.10
10.49
25.34
26.91
11 4.5
11 4.7
11 6.7
11 9.9
250 200
11 0.6 3 11 6.3 4 12 2.0 9 12 7.8 4 13 3.5 9
23.00
12 7.9 12 5 4.4 11 9 9.3 11 9 4.7 11 0 2.2 3
20.69
0
14.48
0
18.97
9
14.85
16.16
1
11.62
18.35
12 2.6
t = 44 min
10.68
22.32
2
Latitude
Input sources Infrared
t = 0 min
Temperature (K)
24
Longitude
Flow matching GMI GPM (1)
Input sources Infrared
SSMIS F18 t = -5 min
7.58 11.89
15.23
4
9
9.5
14
4
3.7
14
9
6.5
8.2 12
12
8.4 12
14
0.5
3.6 13
13
6.3
200 8.0
27.26
250
8
23.36
25.15
9.6 14 0 6.1 14 4 1.0 13 0 6.2 13 9 3.8 2
20.73
10.82
9
14.59
9
19.35
8
16.31
9
18.35
13
t = 01h 12 min
11.07
2.2
25.73 22.10
GMI GPM (3)
13
t = 0 min
8
Latitude
Groundtruth observation GMI GPM
GMI GPM (2)
13
GMI GPM
Temperature (K)
Deterministic
Longitude
Flow matching
GMI GPM
GMI GPM (1)
Groundtruth observation GMI GPM
250
18.46
200 150 9
5
8.3
2.6
-11
4 4.1
-11
9 9.8
-12
-12
1 5.6
2 -13
3.5
2 1.5
-12
6
8
-12
7.1 -11
9.1
1
-10
1
1.6 -13
2
1.4 -13
0
9.3 -12
6.2 -12
0 3.5
14.33
26.43
5
10.03
t = 02h 17 min
10.15
22.50
2.3
14.13
-11
18.12
-12
Latitude
6.85 11.90 17.04 22.10 27.25
22.09
GMI GPM (3)
Input sources Infrared
AMSR2 GCOMW1 t = -01h 21 min
t = 0 min
26.05
GMI GPM (2)
Longitude
Flow matching
Deterministic GMI GPM
GMI GPM (1)
Temperature (K)
Deterministic
GMI GPM (2)
GMI GPM (3)
Groundtruth observation GMI GPM t = 0 min
33.16
20.23
15.66
15.82
6
SSMIS F18 t = -48 min
Infrared t = -48 min 33.41 29.09
250
24.67
200
20.25 15.84
-15
5.4
7.8
-14
0.6
-14
-15
3.1
-15
11.77 15.92 20.03 24.02 27.90
7.7 -15 8 5.0 -15 0 0.0 -14 0 4.4 -13 0 9.9 1 -15 4.4 -15 0 1.6 -14 7 6.6 -14 5 1.0 -13 1 6.4 -16 6 0.1 -15 1 4.4 -14 0 8.6 -14 5 2.9 -13 0 7.1 -16 5 2.9 -16 9 0.1 -15 4 5.0 -14 1 9.3 -14 0 4.7 2
20.09
31.46
7
26.88
6
24.54
8
28.95
24.41
0
28.84
22.27
-15
SSMIS F19 t = -02h 11 min
33.26
17.63
2.6
Latitude
12.98
Input sources
SSMIS F17 t = -02h 20 min
25
Temperature (K)
Generative multi-source spatiotemporal interpolation of tropical cyclones
Longitude
Flow matching
4
9
2.9 12
13
3.3
12
5.7
12
8.5
12
0.5
13
0.9
220 5.9
26.46
14
24.90
4
24.89
240
0.1
22.54
9
20.26
280 260
7
20.32
8.2 13 7 4.8 12 8 9.8 12 4 5.1 12 8 2.7 1
18.50
9
15.74
7
15.80
6
14.36
6
11.22
13
t = 02h 56 min
10.19
4.4
6.70
Input sources Infrared
11.13
1
Latitude
6.53
SSMIS F19 t = 3 min
13
t = 0 min
GMI GPM (3)
8.6
Groundtruth observation GMI GPM
GMI GPM (2)
Temperature (K)
GMI GPM (1)
12
GMI GPM
14
Deterministic
Longitude
Flow matching GMI GPM (1)
0
5
4.8
-13
0
0.5
-14
7.7
5
200
-15
-14
6.1 -14
8.5 -14
1.3 -15
3.4 -15
250
6
24.87
4. -14 72 2. -13 04 7. -13 14 1. -12 69 7.3 8
20.90
9.63
5
14.50
28.70
7
24.76
7
16.82
1
12.65
19.37
2
24.25
20.91
3.8
t = -34 min
8.45
6.3
29.12
17.04
-15
Latitude
13.17
Input sources Infrared
SSMIS F18 t = -02h 46 min
-14
t = 0 min
GMI GPM (3)
2.0
Groundtruth observation GMI GPM
GMI GPM (2)
-15
GMI GPM
Longitude
Flow matching
Deterministic GMI GPM
GMI GPM (1)
GMI GPM (2)
Temperature (K)
Deterministic
GMI GPM (3)
Input sources
-0.65
SSMIS F18 t = 1 min
3.85 8.46 12.98 17.60
.78 52 .66 58 .37 64 .12 69 .87 75 .62 71 .21 67 .69 62 .59 57 .90 55 .34
.23
Infrared t = 0 min
54
.16
.78
.68
70
.19
1.40 5.67 9.94 14.15 18.29
60
.78
63
.73
67
.12
72
60
62
76
17.52
.80
16.80
.06
2.53
59
12.98
.11
8.44
12.38
.08
8.08
7.30
.64
11.95
65
SSMIS F17 t = -01h 01 min
-0.49 3.92
57
3.67
16.72
67
Latitude
SSMIS F19 t = -01h 04 min
-0.61
.72
t = 0 min
21.36
61
Groundtruth observation GMI GPM
Longitude
Flow matching
Deterministic GMI GPM
GMI GPM (1)
GMI GPM (2)
GMI GPM (3)
Temperature (K)
C. Dauvilliers and C. Monteleoni
66
26
280 260 240 220