Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Zhonghao Li∗ 1 Chaoyu Liu∗ 2 Qian Zhang 1
arXiv:2605.13790v1 [cs.LG] 13 May 2026
Abstract
tinuous spatial–temporal domains. Codes are available at https://github.com/Maxwell996/Di-BiLPS.
Partial differential equations (PDEs) are fundamental for modeling complex natural and physical phenomena. In many real-world applications, however, observational data are extremely sparse, which severely limits the applicability of both classical numerical solvers and existing neural approaches. While neural methods have shown promising results under moderately sparse observations, their inference efficiency at high resolutions is limited, and their accuracy degrades substantially in the extremely sparse regime. In this work, we propose the Di-BiLPS, a unified neural framework that effectively handle both forward and inverse PDE problems under extremely sparse observations. DiBiLPS combines a variational autoencoder to compress high-dimensional inputs into a compact latent space, a latent diffusion module to model uncertainty, and contrastive learning to align representations. Operating entirely in this latent space, the framework achieves efficient inference while retaining flexible input–output mapping. In addition, we introduce a PDE-informed denoising algorithm based on a variance-preserving diffusion process, which further improves inference efficiency. Extensive experiments on multiple PDE benchmarks demonstrate that Di-BiLPS consistently achieves SOTA performance under extremely sparse inputs (as low as 3%), while substantially reducing computational cost. Moreover, Di-BiLPS enables zero-shot superresolution, as it allows predictions over con-
1. Introduction In the natural sciences, a wide range of physical, chemical, and biological processes can be modeled by partial differential equations (PDEs) (Roubíček, 2013; Wight & Zhao, 2020). In both academic research and industrial applications, numerical methods for PDEs are primarily concerned with simulating system states (the forward problem) and inferring unknown properties or parameters from experimental or observational data (the inverse problem). Conventional numerical solvers are typically handcrafted and restricted to specific classes of PDEs, requiring customization for each new formulation. This task-specific nature often results in limited generality, reduced flexibility, and significant computational overhead, especially when dealing with complex geometries or repeated simulations across varying conditions. To fill the gap, a number of learning-based approaches (Li et al., 2020b; 2024b) are proposed to learn the mappings between coefficient space and state space directly. These approaches differ mainly in how the solution operators are parameterized, leading to a variety of neural architectures, including graph-based methods (Li & Farimani, 2022), physics-informed networks (Raissi et al., 2019), and convolution-based models (Thuerey et al., 2020). While neural PDE methods significantly improve generality and computational efficiency, they typically rely on access to dense observations of coefficients or system states to learn accurate solution operators. In real-world applications, however, such complete observations are rarely available. Instead, measurements are often extremely sparse across both spatial and temporal domains, which poses substantial challenges for accurate and reliable modeling. Under such extreme sparsity (e.g., observation rates below 3%), neither classical numerical solvers nor state-of-the-art neural
* 1 Equal contribution School of Science, Harbin Institute of Technology, Shenzhen, China, [email protected] 2 Department of Theoretical Physics and Applied Mathematics, University of Cambridge, Cambridge, United Kingdom, [email protected]. Correspondence to: Qian Zhang <[email protected]>.
Preprint. May 14, 2026.
1
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Figure 1. Overview of the architecture of Di-BiLPS : (a) Compression module via a pre-trained VAE (left), (b) Diffusion module with proposed PDE-Guided denoising algorithm(middle), and (c) Contrastive learning module (right). Proposed design leverages module (a) to extract informative and robust representations from extremely sparse inputs and performs high-speed denoising algorithm within the compressed latent space.
PDE models (Wu et al., 2024; Hao et al., 2023) remain effective. This gap highlights the need for models that can robustly operate under severely limited observational data.
work that jointly addresses forward and inverse PDE problems under extremely sparse observations. By reformulating PDE learning in a compressed latent space, Di-BiLPS achieves improved computational efficiency and enhanced input– output flexibility compared to existing methods.
To this end, several diffusion-based (Huang et al., 2024; Shu et al., 2023) and graph-based (Zhao et al., 2022; Li et al., 2025b) methods have been proposed in recent years. However, these methods often suffer from high computational costs during inference, particularly when applied to densely discretized domains. Furthermore, for PDEs exhibiting drastic variations (Cen et al., 2024; Ma et al., 2024), interpolation-based methods are generally inadequate to reduce the reliance on dense discretization. To address this limitation, several latent neural PDE methods (Li et al., 2025a;c) have been proposed to learn solution mappings within compressed latent spaces.
(2) We develop a PDE-informed denoising algorithm based on a variance-preserving diffusion process, which effectively integrates sparse observations with physical constraints to enable accurate bidirectional inference. (3) Extensive experiments on five PDE benchmark datasets demonstrate that Di-BiLPS consistently outperforms state-of-the-art baselines in both prediction accuracy and computational efficiency. Benefitting from the nature of GINO, Di-BiLPS supports zero-shot resolution generalization, enabling inference on unseen spatial resolutions without retraining.
Motivated by the aforementioned challenges, we propose Di-BiLPS, a unified and computationally efficient framework for learning and inference of PDEs under extreme data sparsity. As illustrated in Fig. 1, DiBiLPS consists of three key components: (i) a contrastive learning module that aligns representations between sparse and full observations; (ii) a pre-trained variational autoencoder that encodes inputs into a compact latent space; and (iii) a latent diffusion model that enables bidirectional inference for PDE solutions within the latent domain.
(4) We show that the proposed contrastive alignment and latent compression modules provide effective representations under extreme sparsity, suggesting their applicability to a broader class of latent neural PDE frameworks.
2. Related Work Diffusion informed Operator Learning: In the context of operator learning, diffusion models were first introduced for high-fidelity flow field reconstruction (Shu et al., 2023) and were subsequently extended
Our main contributions can be summarized as follows: (1) We propose Di-BiLPS, a scalable neural frame2
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
to address ill-posed sparse prediction problems (Huang et al., 2024). In addition, governing-equation-driven denoising strategies have been explored to accelerate the solution of PDE-based diffusion models (Gao et al., 2024; Huang et al., 2024). ECI sampling(Cheng et al., 2025) also presents a novel zero-shot framework that adapts pre-trained flow-matching models to strictly satisfy physical hard constraints without gradient computations or fine-tuning for PDE-related scientific generation tasks. Compared with other machine learning approaches, diffusion models exhibit an inherent ability to capture rich high-frequency components (Xu et al., 2025; Karumuri et al., 2026), making them particularly well suited for problems characterized by complex high-frequency dynamics, such as the Navier– Stokes equations(Molinaro et al., 2024). Further discussions on related work are provided in Appendix A.
Similarly, we consider dynamic systems, which can be formulated as follows: F (P, t; A, U ) = 0,
in Ω × (0, ∞),
U (P, t) = g(P, t),
on ∂Ω × (0, ∞),
U (P, t) = A,
in Ω × {0}.
(2) (3)
Here, t denotes the temporal coordinate, A := U (·, 0) ∈ A is the initial condition, and U is the solution. The boundary constraint is given by U |∂Ω×(0,∞) = g. Our objective is to simultaneously recover both the initial condition A and the solution at a specific time ti , denoted by U ti := U (·, ti ) ∈ U, from sparse observations available on either A or U ti . Here, we rewrite U ti as U for convenience in following parts since we don’t care about U at other time. In our formulation, we denote the constraints F defined in Eqs. (1) and (2) as the measurement condition M : F (·) = 0, while the initial and boundary conditions, A and g, are collectively represented as the conditioning C.
3. Preliminaries 3.1. Problem Setup Our study considers two categories of partial differential equations (PDEs): static PDEs and timedependent(dynamic) PDEs. Static PDEs, such as Darcy flow and Poisson equations, describe equilibrium states of physical systems and are characterized by time-invariant parameters and solutions. These equations typically govern phenomena like fluid flow in porous media, heat conduction at steady state, and electrostatic potential distributions.
4. Methodology In this section, we present the compressed sensing module, the condition learning module and the latent diffusion model that constitute Di-BiLPS, as detailed in following subsections. 4.1. Variational Auto-Encoder for Irregular PDE Data
In contrast, time-dependent PDEs model the evolution of physical systems over time, such as Navier–Stokes equations for fluid dynamics. These PDEs incorporate temporal derivatives and require initial conditions in addition to boundary conditions. By addressing both static and dynamic PDEs, our framework aims to provide a unified solution approach capable of handling a broad class of forward and inverse problems arising in computational physics, engineering, and scientific machine learning.
The successful application of Variational Autoencoders (VAEs) (Kingma et al., 2013) in Latent Diffusion Models (LDMs) (Rombach et al., 2022) has advanced the generation of high-resolution images. Performing machine learning in the compressed latent space not only accelerates the convergence of neural networks when fitting complex data, but also enhances their generalization capabilities.
Static systems are governed by time-independent coefficients and boundary conditions. They are typically defined by a function as follows:
In contrast to image processing, operator learning poses greater challenges in terms of data requirements. Models are required to handle inputs defined on irregular grids and perform prediction tasks over continuous solution spaces. To address these challenges, we propose a novel framework designed to meet both demands effectively.
F (P ; A, U ) = 0 in Ω ⊂ Rd , U (P ) = g(P ) on ∂Ω. (1) where Ω is a bounded spatial domain, P ∈ Ω denotes a spatial coordinate, A ∈ A represents the PDE coefficient, and U ∈ U is the corresponding solution. The boundary of the domain is denoted by ∂Ω, with the boundary condition given by U |∂Ω = g. Our objective is to recover both the coefficient A and the solution U from sparse observations available on either A or U .
Building upon the U-Net architecture (Ronneberger et al., 2015), we incorporate a GINO-based mesh encoder and decoder(see Appendix C) into the left and right sides of U-Net, respectively. Suppose the input data V ∈ RM ×(dim(Ω)+f ) has f features defined 3
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Algorithm 1 Training Algorithm Input: VAE-Encoder E, Ge , NumSample N LearnedCondition CA and CU , JointInput V , DenoisingSampler S, MaxTimeStep τ 1: {α0 , · · · , ατ } ←− S 2: repeat 3: i ∼ Uniform({1, · · · , N }) 4: t ∼ Uniform({1, · · · , τ }) 5: c ∼ Uniform({ciA , ciU }) 6: Zi = E(Ge (Vi )) 7: Sample ϵ ∼ N (0, √ I) √ 8: Zt = αt Zi + 1 − αt ϵ 9: Taking gradient descent step on (t)
4.2. Condition Learning under Sparse Observations In the operator learning benchmarks (Boullé & Townsend, 2024), neural PDE methods are designed to solve the forward process (F : A → U) and the inverse process (I : U → A) separately. In practice, samples in A is discretized as a set {Ai } defined over either a structured or irregular grid, with the grid points sampled from domain Ω. The same discretization strategy is applied to samples in U, yielding Ui . Although finer discretizations encode more detailed information, much of it may be irrelevant to the objective mappings F and I.
2
∇θ ϵθ (Zt , c) − ϵ 2 10: Update parameters θ 11: until converged
In many practical scenarios, due to limitations such as instrument precision, we can typically observe only partial measurements of physical properties A or system states U. Since U and A lie in a continuous function spaces, we believe that appropriately sparse partial observations over a discretized grid can still retain most of the interactions relevant to the underlying mappings F and I. To formalize this, we define P A and P U as the sparse subspaces of A and U, respectively.
in domain Ω. In our setting, we compress A and U jointly in order to model the underlying interactions. As shown in Eq. (4), through the GINO-based mesh encoder, irregular input data V is first embedded into a structured w-resolution grid representation dim(Ω) Ge (V ) ∈ Rh×w , enabling it to be processed by the encoder E of U-Net. Afterward, we obtain the prodim(Ω) cessed latent embeddings Z ∈ Rl×wl with l latent channels and wl resolution. Subsequently, for a given query point q ∈ Ω, the GINO-based decoder Gd infers the solution based on neighboring points of decoded embeddings D(Z). Z = E(Ge (V )) = UnetL GINOEnc(V ) ,
vq = Gd (q, D(Z)) = GINODec q, UnetR (Z) ,
To effectively capture interactions, we propose a contrastive learning framework with a novel encoder. As illustrated in Fig. 2, the encoder integrates GINObased mesh encoder (Li et al., 2023) with a Vision Transformer (ViT) encoder (Dosovitskiy et al., 2020). Suppose PiA and PiU are sparse point clouds, downsampled from the discretizations Ai and Ui 1 , respectively. Within the proposed contrastive learning framework, our goal is to learn the interactions between the sparse observations and their corresponding original spaces. In fact, we seek to improve the effectiveness of learning the mappings F S : P A → U and I S : P U → A.
(4)
where UnetL and UnetR are the down-convolution and up-convolution parts of U-net, respectively.
Let Ai and Ui denote the discretized representations in A and U, respectively. Correspondingly, the sparse observations PiA ∈ RM ×(dim(Ω)+fa ) and PiU ∈ RM ×(dim(Ω)+fu ) consist of M points sampled from Ai and Ui , respectively. fa and fu are the feature quantity of A and U . The associated boundary condition is denoted as Bi . By applying the encoder defined in Eq. (7), the i-th sparse observations Pij are encoded into an h-dim conditioning space cij ∈ Rh , while the full observations Ai and Ui are encoded into an h-dim key spaces kiA ∈ Rh and kiU ∈ Rh , respectively. cij = ViTj GINOEncj (Pij ), Bi , j = A, U, (7) kiA = ViTA ResNetA (Ai ), Bi , kiU = ViTU ResNetU (Ui ), Bi ,
Let Vp denote the positional coordinates of the input data V . The restructured data V̂ at the original input points Vp can be expressed as follows: V̂ = Gd (Vp , D(Z)).
(5)
Here, MSE loss, Perceptual loss (Johnson et al., 2016) and KL loss (Kingma et al., 2013) are adopted to train the GiT as follows: Loss = MSE(V, V̂ ) + PercLoss(V, V̂ ) + KL(Z).
(6)
The design of our VAE is also shown in Fig. 1 and the detailed implementation of GINO (Li et al., 2023) is deferred to Sec. C.
1
4
i.e., Sampled from subspaces P A and P U , respectively.
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
4.3. PDE-Guided Denoising Algorithm for Diffusion Models Algorithm 2 Guided Diffusion Sampling Algorithm Input: GuidedDiffusionModel ϵθ , VAE-Decoder D, Gd , LearnedCondition C, PDEConstraint M, Qm , Observations P = {Pp , Pv }, Weights ζobs , ζpde , DenoisingSampler S, NumTimestep T , Query positions: Q, Output: X 1: {t0 , · · · , tT }, {αt0 , · · · , αtT } ←− S(T ) // Get α from sampler 2: Sample ZtT from N (0, I) 3: for i = T to 1 do 4: αi ← αti , αi−1 ← αti−1 // Get variance at t (t ) 5: Ncond ← ϵθ i (Zti , C) √// Estimate conditional noise 6: Zˆ0 ← √1αi (Zti − 1 − αi Ncond ) // Estimate Zˆ0 7: XˆP ← Gd Pp , D Zˆ0
Figure 2. Illustration of contrastive learning module. Proposed GINO-ViT framework encodes both sparse and full observations into a unified latent space. By optimizing the match score within this space, framework is designed to capture informative and robust representations between sparse and full observations. Here, we emphasize that ViT modules with the same color share weights in this figure.
where modules GINOEncA and ResNetA embed PiA and Ai into a shared structured grid format, thereby enabling their outputs to be processed by a shared encoder ViTA . The same procedure is applied to Ui . Let CA , CU , KA , and KU ∈ RN ×h denote the matrices formed by stacking and normalizing the vectors ciA , ciU , kiA , and kiU as rows, respectively. Here N is the batch size. To capture the underlying interactions between C and K, we compute the pairwise cosine similarity matrix CK T as matching scores, as described in Eq. (8). In our contrastive learning setup, matching scores S F and S I along the main diagonal represent the similarity between matched pairs, reflecting how well each condition in C aligns with its corresponding key in K. S F = CA KUT ,
T S I = CU KA ,
9:
// Decode latent embeddings at grid positions. 2 Npde ← −ζpde ∇Z F (XˆF ) ti
2
// Calculate Eq. (15) 10:
obs Nobs ← − ζM ∇Zˆ0 XˆP − Pv
2 2
// Calculate Eq. (17) 11:
cond S ← − √N1−α − Npde i
// Calculate q Eq. (16)
√ α √ i−1 Zt − i αi
1−αi−1 Zti−1 ← − αi−1 q p 1−αi αi−1 (1 − αi )S αi // Do denoising step as Eq. (13) 13: Zti−1 ← Zti−1 + Nobs // Guidance to Sparse Observations 14: end for 15: X ← Gd D Z0 , Q // t0 = 0 // Decode latent embeddings on query positions. 16: return X
12:
(8)
T are the transposes of KU and KA where KUT and KA respectively. The loss function can be expressed as follows:
Loss = CE(S F , I) + CE(S I , I)+ CE (S F )T , I + CE (S I )T , I ,
8:
// Decodelatent embeddings at observed positions. ˆ ˆ XF ← Gd Qm , D Z0
(9)
Diffusion models comprise a predefined forward process that gradually corrupts the data by adding Gaussian noise, and a learnable reverse process that iteratively denoises the noisy inputs to recover the original data distribution. As described in DDIM (Song et al., 2020a), the mathematical formulations of both the forward and reverse processes are given as follows: √ q(Zt |Z0 ) = N ( αt Z0 , (1 − αt )I), (10)
where CE is the cross-entropy loss function and I ∈ RN ×N is the identity matrix. The pseudocode for the core of an implementation of GiT is shown in D. After training GiT, we obtain the conditioning embeddings CA and CU from the sparse observations P A and P U . These embeddings facilitate more efficient mappings CU,A → U , A, compared to the original mappings F S, IS. 5
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
√ q(Zt−1 |Zt , Z0 ) = N ( αt−1 Z0 + p √ (11) 1 − αt−1 − σt2 (Zt − αt Z0 ) 2 √ , σt I). 1 − αt Here, we consider the case where σt = 0, corresponding to a deterministic denoising process. According to Eq. (11), the latent variable Zt−1 can be obtained from Zt as follows: p √ (t) (12) Zt−1 = αt−1 Z0 + 1 − αt−1 ϵθ (Zt ) ,
rewrite δ12 as ζpde for convenience. To summarize Eqs. (14) and (15), the ∇Zt log pθ (Zt | C, M) can be estimated as follows: S := ∇Zt log pθ (Zt |C, M)
=∇Zt log pθ (Zt |C)+∇Zt log pθ M|Z0 (Zt , C) (16) (t) 2 ϵθ (Zt , C) ˆ ≈− √ −ζpde ∇Zt F Gd [Qm , D Z0 ] , 2 1 − αt
where Zˆ0 (Zt , C) =
where Z0 can be reconstructed from Zt via Z0 = √ Zt −
. Under the framework of stochastic differential equations (SDEs) (Song et al., 2020b), Eq. (12) can be reformulated by leveraging the idenαt
(t)
(Z )
tity ∇Zt log pθ (Zt ) = − √θ 1−αt . To facilitate controlt lable denoising, classifier-free guided diffusion models (Liu et al., 2023) design a noise estimation network based on conditional probabilistic formulation ϵ
(t)
(Z ,C)
t pθ (Zt |C) = − θ√1−α .
Nobs =−
t
In the context of our problem, we employ the guided diffusion approach with the formulation pθ (Zt | C, M), where C and M represent the conditioning information defined in Sec. 3.1. Under the assumptions of Eqs, (10) and (11), the accelerated sampling method proposed by DDIM (Song et al., 2020a) can be formulated as follows: Zτ Zt =√ − √ ατ αt
r
1−ατ − ατ
r
1−αt αt
√
1−αt
4.4. Architecture of Diffusion Model As discussed in Sec. 4.2, our approach enables more efficient mappings from C A to U and from C U to A, compared to the original mappings F S and I S . In many practical scenarios, however, full observations are unavailable due to measurement limitations, making sparse recovery in the observation space equally important. To address this, we propose a novel latent diffusion model that learns the joint mappings C A → {U , A} and C U → {U , A} within a unified generative framework as shown in Fig. 1.
(13)
where τ ∈ {0, 1, · · · , t−1} denotes a previous timestep. Since the condition M acts as a measurement operator F (·) applied to the embeddings D(Z0 ), the conditional probability can be decomposed into the following components: = pθ (Zt |C) ∗ pθ (M|Z0 (Zt , C)).
2 ζobs ∇Zˆ0 Gd (D Zˆ0 (Zt , C), Pp ) −Pv , (17) M 2
where Pv is the function value at the observed points Pp .
∇Zt log pθ (Zt |C, M),
pθ (Zt |C, M) ∝ pθ (Zt |C) ∗ pθ (M|Zt , C)
In addition,
since the latent variable Ẑ0 has been estimated at each denoising step, we introduce an additional ℓ2 penalty term that measures the discrepancy between the observed points P = {Pp , Pv } and their corresponding reconstructions in Ẑ0 . As shown in Eq. (17), this penalty is incorporated into the denoising process to further guide Zˆ0 (Zt , C) toward sparse observations.
(t) 1−αt ϵθ (Zt ) √
ϵ
√ (t) Zt − 1−αt ϵθ (Zt ,C) √ . αt
Here, we aim to model the joint distribution over X = A × U using VAE. Given i-th joint input observation Vi ∈ RM ×(dim(Ω)+f ) in irregular grids2 , where f = fa + fu , the VAE encodes Vi into a latent representation Zi , as illustrated in Eq. (4). Therefore, regardless of whether the diffusion module is conditioned on ciA or ciU to predict the latent representation Zi , both Ai and Ui can be subsequently reconstructed from Zi .
(14)
According to DPS (Chung et al., 2022) and diffusionPDE (Huang et al., 2024), if measurement operator F (·) is corrupted by Gaussian noise with some standard deviation δ (i.e., M|Z0 ∼ N (F D(Z0 ) , δ 2 I)), then the log-likelihood function can be approximated accordingly as follows: Npde = ∇Zt log pθ M|Z0 (Zt , C) ≈ ∇Zt log pθ M|Ẑ0 (Zt , C) (15) 2 1 , ≈ − 2 ∇Zt F Gd [Qm , D Zˆ0 (Zt , C) ] δ 2
Our noise estimation network ϵθ is built upon the UNet architecture (Ronneberger et al., 2015). Given an input sample Vi and a randomly selected timestep t, the training procedure is summarized in Algorithm 1, where the conditioning embeddings CA and CU are precomputed using the model described in Eq. (7). dim(Ω)
Without loss of generality, if Vi ∈ Rf ×w , we employ a ResNet-based encoder Ge to map Vi into a latent representation of the same shape as GINO. 2
where Qm typically denotes a discretized grid over the domain Ω, enabling the computation of F . Here, we 6
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Once the noise estimation network ϵθ has been trained, it can be used to perform inference on any point set Q within the domain Ω. The detailed inference procedure is outlined in Algorithm 2. We emphasize that Pp and Pv denote the positions and values of the sparse observations. In the forward and inverse problems, sparse observations P = {Pp , Pv } are defined over A and U, respectively. Accordingly, the way C is precomputed from P in Eq. (7) also depends on the space of P in Algorithm 2.
2022) in huggingface. All experiments were run on 8 Nvidia RTX4090 GPUs with 24GB memory. 5.2. Training Pipeline In the standard operator learning benchmark, paired data (Ai , Ui ) is provided to learn the mappings Ai ↔ Ui . In our setting, however, only 500 points(3%) in Ai or Ui are sampled to learn these bidirectional mappings. First, as outlined in Sec. 4.2, we train a condition learning module(GiT) to extract robust message from 3% available observations. Second, we train a GINO-VAE to encode the paired data into a latent space, with a compression rate of 6.25%. Third, the noise estimation network ϵθ is trained using compressed latent representations from the GINO-VAE, together with corresponding conditioning information derived from the trained conditional learning module.
5. Experiments In this section, we evaluate Di-BiLPS on five PDE benchmarks under sparse observational settings, covering both forward and inverse operator learning tasks. Results demonstrate consistent improvements over existing operator learning and diffusion-based baselines in terms of accuracy and efficiency. 5.1. Experiment Settings Dataset Preparation: In our experimental setup, PDEs are formulated on the unit square domain Ω = (0, 1)2 , represented by a uniform 128 × 128 grid. The training dataset is obtained through numerical simulation using the Finite Element Method (FEM). To enable fair and consistent comparison with prior work, our pipeline Di-BiLPS: is evaluated on five standard PDE benchmark datasets(Darcy Flow, Helmholtz, Poisson, Bounded Navier-Stokes, and NonBounded Navier-Stokes) proposed by DiffusionPDE (Huang et al., 2024) and FNO (Li et al., 2020b). Detailed information of datasets above is shown in B.1.
. Figure 3. Comparison between our model and DiffusionPDE on the Darcy Flow (top) and inhomogeneous Helmholtz equation (bottom) tasks.
Baseline Models: We evaluate Di-BiLPS by comparing it against both operator learning baselines and representative models on uncertain prediction in operator learning. For operator learning, we consider PINO (Li et al., 2024b), DeepONet (Lu et al., 2021), PINNs (Raissi et al., 2019), and FNO (Li et al., 2020b). For uncertain prediction in operator learning, we include DiffusionPDE (Huang et al., 2024) and GraphPDE (Zhao et al., 2022) as comparative baselines. We evaluate Di-BiLPS on five benchmark PDE datasets under sparse observational settings, covering both forward and inverse operator learning tasks.
5.3. Main Evaluation Results Table 1 presents the results of our model and other baseline models over 10 runs. Since the coefficients of Darcy Flow are binary, we evaluate the error rates of our predictions. Across five PDE benchmark datasets, our model consistently outperforms existing operator learning approaches. It surpasses the diffusion-based approach DiffusionPDE on 8 out of 10 tasks, while achieving a substantial reduction in prediction error and a 90% decrease in inference time, highlighting both its accuracy and efficiency. In the inverse problems of the Poisson and Helmholtz equations, the performance of DiffusionPDE deteriorates due to inadequate constraints in the coefficient space arising from randomly generated fields. Our model addresses this limitation effectively by incorporating the proposed condition learning module, resulting a 47% error re-
Metrics: In our experiments, we use the relative ℓ2 error to evaluate model performance, defined as 2 ℓ2 (x, x̂) = ∥x−x̂∥ ∥x∥2 , where x is the ground truth and x̂ is the prediction. Setup: We conducted our experiments within an open source framework diffusers(von Platen et al., 7
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations Table 1. Comparison of the relative ℓ2 error of various methods on forward and inverse PDE tasks. The runtime performance (in seconds) of our model and DiffusionPDE is also reported. The best results in each task are highlighted in bold.
Task
Forward
Inverse
PDE Func Darcy Flow Poisson Helmholtz Non-B NS Bounded NS Darcy Flow Poisson Helmholtz Non-B NS Bounded NS
Ours(ℓ2 | RunTime) 0.021(↓16%) | 35.45(↓86%) 0.038(↓16%) | 19.69(↓92%) 0.025(↓72%) | 20.52(↓92%) 0.043(↓38%) | 19.82(↓92%) 0.021(↓46%) | 19.08(↓93%) 0.013(↓59%) | 36.04(↓86%) 0.123(↓38%) | 19.59(↓92%) 0.120(↓47%) | 20.56(↓92%) 0.116(↑12%) | 19.52(↓92%) 0.032(↑19%) | 19.80(↓92%)
DiffPDE 0.025 | 255.74 0.045 | 255.37 0.088 | 249.81 0.069 | 256.40 0.039 | 256.54 0.032 | 255.74 0.200 | 254.22 0.226 | 252.72 0.104 | 254.18 0.027 | 253.41
PINO 0.352 1.071 1.065 1.014 0.811 0.492 2.319 2.169 0.960 0.695
DeepONet 0.383 1.555 1.231 1.032 0.977 0.411 1.058 1.328 0.972 0.919
PINNs 0.488 1.281 1.423 1.427 1.001 0.597 1.300 1.600 1.468 1.055
FNO 0.282 1.009 0.982 1.014 0.828 0.493 2.327 2.182 0.960 0.696
Table 2. Relative ℓ2 error for ablation study on Di-BiLPS. This table illustrates the effect of removing key components in both forward and inverse PDE tasks. The best results in each task are highlighted in bold.
Method Direction Di-BiLPS w/o PDE Guidance w/o Observation Guidance w/o Condition Guidance
Darcy Flow For Inv 0.021 0.013 0.021 0.013 0.042 0.040 0.237 0.272
Poisson For Inv 0.038 0.123 0.038 0.123 0.108 0.450 1.185 0.292
Helmholtz For Inv 0.025 0.120 0.026 0.120 0.140 0.339 0.573 1.288
Non-Bounded NS For Inv 0.043 0.116 0.044 0.116 0.274 0.414 0.790 1.429
Bounded NS For Inv 0.021 0.032 0.021 0.032 0.109 0.074 0.189 0.978
duction. Throughout all tasks, the GPU memory consumption of our model was 8.6 GB, compared to 4.5 GB for DiffusionPDE. Fig. 3 presents a comprehensive comparison between our model and DiffusionPDE, covering both forward and inverse problems on the Darcy Flow and Helmholtz equations. The results in Fig. 3 indicate that our model achieves superior performance in predictive accuracy, whereas DiffusionPDE demonstrates a stronger ability to recover the original coefficient space.
Figure 5. Comparison among our model, GraphPDE and DiffusionPDE(Huang et al., 2024) on the inverse problem of Bounded Navier-Stokes equations.
Fig. 5 presents a comparison among our model, GraphPDE and DiffusionPDE on the inverse problem of the bounded Navier-Stokes equation. The results in Fig. 5 indicate that our model more accurately captures the dynamical states around the cylinder, resulting in a lower overall prediction error. We also conducted additional experiments to explore the generalization of sparsity settings. We retrained the contrastive learning module at 1% and 10% observation densities with the encoder of full observations frozen, and reused VAE and diffusion modules. As shown in Appendix G, the performance degrades gracefully as sparsity increases, indicating that our method remains stable across a range of observation regimes and easy to generalize to different sparsity regimes.
Figure 4. Comparison of all baseline models(Huang et al., 2024) listed in Table 1 on bidirectional problems of Nonbounded Navier–Stokes equations.
Fig. 4 presents a comprehensive comparison of all baseline models listed in Table 1, encompassing both forward and inverse problems for the non-bounded Navier–Stokes equations. The results indicate that most existing methods are unable to effectively solve PDEs under sparse observational settings. In contrast, our model achieves accurate and efficient predictions.
5.4. Ablation Studies As shown in Table 2, we conduct an ablation study to evaluate the contribution of each component in our model across five benchmark PDEs in both forward 8
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Second, our model deteriorates drastically when the observation rate decreases to 0.1%. Exploring the minimal sparsity threshold for valid model performance remains a valuable avenue for future work.
7. Conclusion
Figure 6. Zero-shot super-resolution results on the forward problem of the inhomogeneous Helmholtz equations.
In this work, we propose Di-BiLPS, a unified and effective neural framework for solving bidirectional PDE problems under highly sparse settings. Our results highlight the application potential of latent PDE solvers through leveraging carefully constructed lowrank representations. Moreover, our findings in this work demonstrate that diffusion-based models exhibit robust predictive performance in sparse data scenarios of industrial applications by directly learning the underlying distribution of data. We anticipate that both Di-BiLPS and the corresponding proposed PDE denoising algorithm will offer valuable insights and serve as a foundation for future research on more complex PDE challenges.
and inverse settings. In this table, PDE Guidance corresponds to the term Npde as defined in Eq. (15), Observation Guidance refers to Nobs defined in Eq. (17), and Condition Guidance denotes the conditional input C to the noise estimation network ϵθ . Removing the PDE guidance results in negligible performance degradation, suggesting its auxiliary role. However, excluding the observation guidance leads to a significant increase in relative ℓ2 error across all tasks, highlighting its critical role in leveraging sparse observations. Furthermore, removing the condition guidance causes severe performance deterioration, particularly in complex scenarios such as the Non-Bounded Navier–Stokes equations, indicating its essential role in capturing contextual dependencies.
Acknowledgement The authors would like to thank the editor and the anonymous reviewers for their valuable comments and constructive suggestions, which have helped improve the quality of this manuscript. The authors are also grateful to Professor Zhonghua Qiao from The Hong Kong Polytechnic University and Professor Dong Wang from The Chinese University of Hong Kong, Shenzhen, for their helpful discussions and support. Qian Zhang acknowledges the support from the National Natural Science Foundation of China (Grant No. 12401480) and the Shenzhen Talent Start-up Fund (Grant No. ZX2023295). Chaoyu Liu acknowledges the support from the Maths4DL program under grant EP/V026259/1.
5.5. Zero-shot Super-Resolution As discussed in Sec. 4.1, our VAE model incorporates the encoder and decoder from GINO. As a result, our model is inherently capable of performing predictions over a continuous spatial domain. This property enables seamless super-resolution inference in the solution space by querying the decoder of GINO at arbitrary spatial locations. Notably, although the training data contains only low-resolution samples, our model can generate high-resolution predictions in a zero-shot manner by specifying query points at a finer resolution. Fig. 6 illustrates the zero-shot super-resolution results on the forward inhomogeneous Helmholtz equations.
Impact Statement 6. Limitations
This paper presents work whose goal is to advance the field of Machine Learning by developing more efficient and flexible learning-based methods for scientific computing problems governed by partial differential equations. The techniques introduced in this work are methodological in nature and are evaluated on standard benchmark datasets. By improving computational efficiency and robustness under limited observations, such methods may support more efficient modeling and inference in real-world scientific and engineering systems. We do not anticipate any direct negative societal or ethical consequences arising from this work.
Despite these encouraging results, several aspects of Di-BiLPS deserve further discussion. First, within the scope of diffusion-informed operator learning, developing governing equation-driven denoising algorithms is a vital paradigm for incorporating physical priors. However, our ablation studies demonstrate that directly performing gradient descent with the loss derived from governing equations(i.e., PDE guidance) barely improves model performance, especially for diffusion models in the latent space. Our current implementation of operator learning tailored for latent diffusion models still has certain limitations. 9
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
References
Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A general neural operator transformer for operator learning, 2023. URL https://arxiv.org/abs/2302.14376.
Ali, A., Bai, J., Bala, M., Balaji, Y., Blakeman, A., Cai, T., Cao, J., Cao, T., Cha, E., Chao, Y.-W., et al. World simulation with video foundation models for physical ai. arXiv preprint arXiv:2511.00062, 2025.
Huang, J., Yang, G., Wang, Z., and Park, J. J. Diffusionpde: Generative pde-solving under partial observation. arXiv preprint arXiv:2406.17763, 2024.
Alkin, B., Fürst, A., Schmid, S., Gruber, L., Holzleitner, M., and Brandstetter, J. Universal physics Transformers. arXiv preprint arXiv:2402.12365, 2024.
Johnson, J., Alahi, A., and Fei-Fei, L. Perceptual losses for real-time style transfer and superresolution, 2016. URL https://arxiv.org/abs/ 1603.08155.
Boullé, N. and Townsend, A. A mathematical guide to operator learning. Handbook of Numerical Analysis, 25:83–125, 2024.
Karumuri, S., Graham-Brady, L., and Goswami, S. Physics-informed latent neural operator for realtime predictions of time-dependent parametric pdes. Computer Methods in Applied Mechanics and Engineering, 450:118599, 2026.
Cao, S. Choose a transformer: Fourier or Galerkin. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Kingma, D. P., Welling, M., et al. Auto-encoding variational bayes, 2013.
Cen, D., Liang, H., and Vong, S. Pointwise error estimates of l1 method for multi-singularity problems arising from delay fractional equations. East Asian J. Appl. Math., 14:820–840, 2024.
Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., and Anandkumar, A. Neural operator: learning maps between function spaces with applications to PDEs. Journal of Machine Learning Research, 24(89):1–97, 2023.
Cheng, C., Han, B., Maddix, D., Ansari, A. F., Stuart, A., Mahoney, M. W., and Wang, B. Gradient-free generation for hard-constrained systems. In International Conference on Learning Representations, volume 2025, pp. 100510–100539, 2025.
Li, Z. and Farimani, A. B. Graph neural networkaccelerated lagrangian fluid simulation. Computers & Graphics, 103:201–211, 2022.
Chung, H., Kim, J., Mccann, M. T., Klasky, M. L., and Ye, J. C. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687, 2022.
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Neural operator: graph kernel network for partial differential equations. arXiv preprint arXiv:2003.03485, 2020a.
Dhatt, G., Lefrançois, E., and Touzot, G. Finite element method. John Wiley & Sons, 2012.
Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020b.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.
Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations’ operator learning. arXiv preprint arXiv:2205.13671, 2022.
Gao, H., Kaltenbach, S., and Koumoutsakos, P. Generative learning for forecasting the dynamics of highdimensional complex systems. Nature Communications, 15(1):8904, 2024.
Li, Z., Kovachki, N., Choy, C., Li, B., Kossaifi, J., Otta, S., Nabian, M. A., Stadler, M., Hundt, C., Azizzadenesheli, K., et al. Geometry-informed neural operator for large-scale 3d pdes. Advances in Neural Information Processing Systems, 36:35836– 35854, 2023.
Goswami, S., Kontolati, K., Shields, M. D., and Karniadakis, G. E. Deep transfer operator learning for partial differential equations under conditional shift. Nature Machine Intelligence, 4(12):1155–1164, 2022. 10
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Li, Z., Shu, D., and Barati Farimani, A. Scalable Transformer for PDE surrogate modeling. Advances in Neural Information Processing Systems (NeurIPS), 2024a.
Computer Vision and Pattern Recognition Conference, pp. 22381–22391, 2025. Raissi, M., Perdikaris, P., and Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686– 707, 2019.
Li, Z., Zheng, H., Kovachki, N., Jin, D., Chen, H., Liu, B., Azizzadenesheli, K., and Anandkumar, A. Physics-informed neural operator for learning partial differential equations. ACM/JMS Journal of Data Science, 1(3):1–27, 2024b.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684–10695, 2022.
Li, Z., Patil, S., Ogoke, F., Shu, D., Zhen, W., Schneier, M., Buchanan Jr, J. R., and Farimani, A. B. Latent neural pde solver: A reduced-order modeling framework for partial differential equations. Journal of Computational Physics, 524: 113705, 2025a.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. Medical image computing and computerassisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234–241, 2015.
Li, Z., Shi, J., Zhang, X., Zhang, M., and Li, B. Sfi-former: Sparse flow induced attention for graph transformer. Proceedings of the 2025 International Conference on Multimedia Retrieval, pp. 760–769, 2025b.
Roubíček, T. Nonlinear partial differential equations with applications, volume 153. Springer Science & Business Media, 2013.
Li, Z., Zhou, A., and Farimani, A. B. Generative latent neural pde solver using flow matching. arXiv preprint arXiv:2503.22600, 2025c.
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. The graph neural network model. IEEE Transactions on Neural Networks, 20 (1):61–80, 2008.
Liu, X., Park, D. H., Azadi, S., Zhang, G., Chopikyan, A., Hu, Y., Shi, H., Rohrbach, A., and Darrell, T. More control for free! image synthesis with semantic diffusion guidance. Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 289–299, 2023.
Shu, D., Li, Z., and Farimani, A. B. A physicsinformed diffusion model for high-fidelity flow field reconstruction. Journal of Computational Physics, 478:111972, 2023.
Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218– 229, 2021.
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020a. Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b.
Ma, B., Li, H., Li, D., He, X., and Yang, H. Numerical investigation on cooperative evaporation from microdroplet array on heated substrate. Physics of Fluids, 36(2):022011, 02 2024. ISSN 1070-6631. doi: 10.1063/5.0188170. URL https://doi.org/ 10.1063/5.0188170.
Team, M. L., Cai, X., Huang, Q., Kang, Z., Li, H., Liang, S., Ma, L., Ren, S., Wei, X., Xie, R., et al. Longcat-video technical report. arXiv preprint arXiv:2510.22200, 2025.
Molinaro, R., Lanthaler, S., Raonić, B., Rohner, T., Armegioiu, V., Simonis, S., Grund, D., Ramic, Y., Wan, Z. Y., Sha, F., et al. Generative ai for fast and accurate statistical computation of fluids. arXiv preprint arXiv:2409.18359, 2024.
Thuerey, N., Weißenow, K., Prantl, L., and Hu, X. Deep learning methods for reynolds-averaged navier–stokes simulations of airfoil flows. AIAA journal, 58(1):25–36, 2020.
Ni, J., Guo, Y., Liu, Y., Chen, R., Lu, L., and Wu, Z. Maskgwm: A generalizable driving world model with video mask reconstruction. Proceedings of the
von Platen, P., Patil, S., Lozhkov, A., Cuenca, P., Lambert, N., Rasul, K., Davaadorj, M., Nair, D., 11
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Paul, S., Berman, W., Xu, Y., Liu, S., and Wolf, T. Diffusers: State-of-the-art diffusion models. https: //github.com/huggingface/diffusers, 2022. Wang, T. and Wang, C. Latent neural operator for solving forward and inverse pde problems. Advances in Neural Information Processing Systems, 37:33085–33107, 2024. Wen, J., Zhu, Y., Li, J., Tang, Z., Shen, C., and Feng, F. Dexvla: Vision-language model with plugin diffusion expert for general robot control. arXiv preprint arXiv:2502.05855, 2025. Wight, C. L. and Zhao, J. Solving allen-cahn and cahn-hilliard equations using the adaptive physics informed neural networks. arXiv preprint arXiv:2007.04542, 2020. Wu, H., Hu, T., Luo, H., Wang, J., and Long, M. Solving high-dimensional PDEs with latent spectral models. arXiv preprint arXiv:2301.12664, 2023. Wu, H., Luo, H., Wang, H., Wang, J., and Long, M. Transolver: A fast transformer solver for pdes on general geometries, 2024. URL https://arxiv. org/abs/2402.02366. Xiao, Z., Hao, Z., Lin, B., Deng, Z., and Su, H. Improved operator learning by orthogonal attention. arXiv preprint arXiv:2310.12487, 2023. Xu, Z.-Q. J., Zhang, L., and Cai, W. On understanding and overcoming spectral biases of deep neural network learning methods for solving pdes. Journal of Computational Physics, pp. 113905, 2025. Zhao, Q., Lindell, D. B., and Wetzstein, G. Learning to solve pde-constrained inverse problems with graph networks. arXiv preprint arXiv:2206.00711, 2022. Zheng, Y., Liang, R., Zheng, K., Zheng, J., Mao, L., Li, J., Gu, W., Ai, R., Li, S. E., Zhan, X., et al. Diffusion-based planning for autonomous driving with flexible guidance. arXiv preprint arXiv:2501.15564, 2025.
12
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
A. Related Work A.1. Operator Learning Operator learning aims to approximate mappings between infinite dimensional function spaces, typically from coefficients, source terms, or initial conditions to the corresponding solution functions. Instead of explicitly solving the underlying partial differential equations, these methods train neural networks to learn such functional correspondences directly from data. A pioneering framework in this direction is DeepONet (Lu et al., 2021; Goswami et al., 2022), which establishes the feasibility of approximating nonlinear operators using neural networks. Subsequent research has developed a variety of neural operators based on kernel integral formulations (Li et al., 2020a;b; Kovachki et al., 2023). Within this family, the Galerkin Transformer (Cao, 2021) introduces an attention mechanism inspired by Galerkin projections to realize kernel integral operators. Building upon this idea, OFormer (Li et al., 2022) separates observation and query locations through a cross-attention design, enabling more flexible operator approximation. GNOT (Hao et al., 2023) further enhances adaptability to heterogeneous inputs by employing normalized cross-attention and reduces computational overhead through a linear-time attention mechanism. FactFormer (Li et al., 2024a) factorizes kernel integrals along axial directions, decomposing multi-dimensional functions into multiple 1D components, thereby lowering computational complexity. ONO (Xiao et al., 2023) improves generalization by incorporating orthogonality regularization into its attention layers. Another line of work focuses on constructing operators in latent spaces. Transolver (Wu et al., 2024) alternates between geometric and physical representations using physics-attention, enabling expressive latent modeling. LNO(Wang & Wang, 2024) adopts an encoder-decoder architecture built upon physics-attention to tackle heterogeneous geometric problems. LSM (Wu et al., 2023) encodes input functions via cross-attention into a latent space where a set of orthogonal bases is learned, and then reconstructs outputs back to the geometric domain via a second cross-attention module. UPT (Alkin et al., 2024) introduces additional encoding and decoding objectives and compresses geometric information into super-nodes with graph neural networks (Scarselli et al., 2008), enabling more compact latent representations. In contrast to these approaches, our method learns the bidirectional mapping through latent representations in a fully end-to-end manner. Leveraging cross-attention, our framework avoids hand-crafted feature construction or auxiliary loss terms, while effectively capturing heterogeneous functional interactions. A.2. Latent Diffusion Models Latent Diffusion Models(LDMs) (Rombach et al., 2022), introduced in 2021, have rapidly emerged as a transformative framework within the field of generative modeling. By performing the diffusion process in a compact latent space rather than the original high-dimensional domain, LDMs substantially reduce computational cost while preserving expressive representational capacity. This design leads to remarkable performance in image synthesis and establishes a general architecture capable of learning highly nonlinear and structured mappings. Building upon their success, a series of commercial-scale foundation models have adopted LDMs as core building blocks, achieving state-of-the-art results across a wide spectrum of applications, including video generation(Team et al., 2025; Ali et al., 2025), autonomous driving perception(Ni et al., 2025; Zheng et al., 2025), and robotic manipulation(Wen et al., 2025). These developments demonstrate that diffusion-based priors provide strong inductive biases that allow the models to capture multimodal, physically plausible, and semantically coherent structures. Importantly, diffusion models exhibit an ability to handle otherwise intractable inverse or ill-posed problems by leveraging their robust generative priors. This capability makes them particularly promising for learning operators in sparse or partially observed PDE settings(Huang et al., 2024), where traditional machine learning architectures(Li et al., 2020b; Kovachki et al., 2023; Li et al., 2020a) and classical numerical solvers(Dhatt et al., 2012) often fail or become unstable. Thus, diffusion-driven operator learning provides a new direction for addressing complex PDE-related tasks under data scarcity and strong irregularity.
13
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
B. Dataset Description This section details the benchmark datasets employed in our experimental evaluations. We selected these datasets to cover a diverse range of partial differential equations (PDEs), addressing both forward and inverse problems. They provide a rigorous and standardized basis for evaluating the predictive accuracy and generalization capabilities of our proposed framework against established PDE solvers. B.1. Classic PDE Datasets To systematically evaluate the performance of our model, we benchmark against several established methods using the datasets detailed below. Note that the variables a and u in the subsequent problem descriptions correspond to A and U in Eqs. (1), (2), and (3). Darcy Flow The Darcy flow equations describe fluid dynamics within porous media under steady-state conditions. Our experiments utilize a static formulation of the Darcy flow problem, subject to homogeneous Dirichlet (no-slip) boundary conditions on the domain boundary ∂Ω: −∇ · (a(p)∇u(p)) = q(p),
p ∈ Ω,
p ∈ ∂Ω.
u(p) = 0,
Here, the function a(p) represents the heterogeneous permeability field taking binary values, and q(p) = 1 is a constant forcing term. The corresponding PDE guidance function is formulated as F (p) = ∇ · (a(p)∇u(p)) + q(p). Inhomogeneous Helmholtz Equation To model wave scattering in media with varying properties, we incorporate the static inhomogeneous Helmholtz equation, defined with homogeneous Dirichlet boundaries: ∇2 u(p) + k 2 u(p) = a(p), u(p) = 0,
p ∈ Ω,
p ∈ ∂Ω.
In this context, a(p) serves as a piecewise constant source term, and k denotes a real-valued constant. (Note that reducing k to zero recovers the standard Poisson equation). For our experiments, the wavenumber is fixed at k = 1. The associated PDE guidance is therefore: F (p) = ∇2 u(p) + k 2 u(p) − a(p). Unbounded Navier-Stokes Equation We simulate the dynamics of an unbounded, incompressible fluid using the vorticity-velocity formulation of the Navier-Stokes equations: ∂τ w(p, τ )+v(p, τ ) · ∇w(p, τ ) = ν∆w(p, τ ) + q(p), p ∈ Ω, τ ∈ (0, T ], ∇ · v(p, τ ) = 0,
p ∈ Ω, τ ∈ [0, T ].
Here, w = ∇×v defines the scalar vorticity, v(p, τ ) is the velocity vector field, and q(p) acts as the external forcing. The kinematic viscosity is set to ν = 10−3 , corresponding to a Reynolds number of Re = 1000. Our model aims to learn the joint distribution linking the initial state w0 and the state wT at time T = 10 (equivalent to one physical second). Because the extended temporal horizon makes exact computation of the full PDE residual computationally prohibitive, we leverage the vector calculus identity ∇ · (∇ × v) = 0 to construct a simplified guidance function: F (p, τ ) = ∇ · w(p, τ ). Bounded Navier-Stokes Equation Finally, we evaluate our framework on the 2D incompressible NavierStokes equations within a bounded physical domain, formulated using primitive variables (velocity v and pressure p): 1 ∂τ v(p, τ ) + v(p, τ ) · ∇v(p, τ )+ ∇p = ν∇2 v(p, τ ), p ∈ Ω, τ ∈ (0, T ], ρ ∇ · v(p, τ ) = 0,
p ∈ Ω, τ ∈ (0, T ].
For these simulations, we set ν = 0.001 and assume a constant fluid density of ρ = 1.0. The geometry includes randomly sized cylindrical obstacles, with a turbulent inflow driven from the upper boundary. No-slip conditions 14
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
(v = 0) are strictly enforced on the lateral walls (∂Ωleft , ∂Ωright ) and the boundaries of the internal cylinders (∂Ωcylinder ). The objective is to capture the joint distribution between v0 and vT over a timeframe of T = 4 (0.4 physical seconds). Consistent with the unbounded scenario, we utilize the divergence-free condition to define the PDE guidance function as F (p, τ ) = ∇ · v(p, τ ).
C. GINO Architecture In this section, we present the architectural design of GINO (Li et al., 2023), which serves as the foundational model for the subsequent analyses in this paper. GINO Encoder Following the problem setup 3.1, we consider a set of M spatial coordinates xm ⊂ Ω, where m ∈ {0, 1, . . . , M }. At each location xm , a solution vector u(xm ) ∈ Rh is defined. This framework allows for the representation of solutions across arbitrary points in space. To facilitate modeling on a uniform spatial grid, we map the input coordinates xm and corresponding solutions u(xm ) onto a set of uniformly sampled grid points xl ⊂ Ω for l ∈ {0, 1, . . . , Ml }, with associated latent vectors q(xl ) ∈ Rh defined at each grid location. This mapping is performed via the following kernel-based integral operator. Z q(xl ) =
κ(xl , x) u(x) dx.
(18)
Ω
In practice, following the approach (Li et al., 2023), the integration domain is restricted to a spatial ball of radius r, centered at xl , denoted as Br (xl ). This locality constraint ensures that each latent vector depends only on nearby solution values and spatial coordinates. The kernel function κ is parameterized by a neural network that takes as input a physical coordinate x and a latent coordinate xl , and outputs a scalar kernel value. To compute the integral efficiently, it is approximated by a Riemann sum over Mb < M physical coordinates yb ⊂ Ω that lie within the ball Br (yl ), where b ∈ {0, 1, . . . , Mb }. This yields the following approximation: Z q(xl ) = ≈
Br (xl ) Mb X
κ(xl , x) u(x) dx (19)
κ(xl , xb ) u(xb ) µ(xb ).
b=1
Here, µ(xb ) denotes the Riemann weight associated with each physical point xb within the spatial ball Br (xl ). This kernel integral effectively aggregates neighboring physical solutions for each latent location, and acts as an approximation of a spatial operator mapping the input field u(x) to the latent representation q(xl ). GINO Decoder To map from the latent spatial grid to the physical domain, the kernel integral can be applied in reverse. Specifically, given a decoded latent representation qd (xl ) defined on the latent grid, the corresponding decoded physical solution ud (x) can be computed via a Riemann sum:
ud (x) =
Mb X
κ(x, xb ) qd (xb ) µ(xb ).
(20)
b=1
In this case, the summation is taken over the spatial ball Br (x), and the coordinates xb denote the latent grid points that lie within this ball. This reverse aggregation process gathers information from neighboring latent vectors to reconstruct the solution at each physical location, while maintaining the property of discretization-invariance. Notably, this allows the 15
Di-BiLPS: Denoising induced Bidirectional Latent-PDE-Solver under Sparse Observations
Algorithm 3 Pseudocode of Contrastive Learning Input: Dataloader D , Full observations U, A 1: repeat 2: Select minibatch U· , A· from dataloader D. 3: Down-Sample P·A , P·U from A· , U· . 4: C·A = SparseEncoderA(P·A ) 5:
C·U = SparseEncoderU(P·U )
6:
K·A = GridEncoderA(A· )
7:
K·U = GridEncoderU(U· )
# [N, M, fa ] → [N, h] # [N, M, fu ] → [N, h] # [W, W, fa ] → [N, h] # [W, W, fu ] → [N, h]
8: L2-Normalize C·A , C·U , K·A , K·U T 9: logitsF = C·A × K·U T 10: logitsI = C·U × K·A 11: labels = range(N) 12: Loss = Cross-Entropy-Loss(logitsF,labels,axis = 1) 13: Loss += Cross-Entropy-Loss(logitsF,labels,axis = 0) 14: Loss += Cross-Entropy-Loss(logitsI,labels,axis = 1) 15: Loss += Cross-Entropy-Loss(logitsI,labels,axis = 0) 16: Loss.backward() 17: Update parameters 18: until converged
# [N, N ] # [N, N ]
latent representation to be decoded onto arbitrary spatial meshes by evaluating the Riemann sum at any query point x.
D. Pseudocode of Contrastive Learning The pseudocode outlining the contrastive learning algorithm is provided in Algorithm 3, offering a step-by-step description of the training procedure.
E. Experiments Information We conducted our experiments within an open source framework diffusers (von Platen et al., 2022) in huggingface. All experiments were run on 8 Nvidia RTX4090 GPUs with 24GB memory.
F. (Hyper-) Parameter Information See github project.
G. Additional Experiment Results Table 3. Relative ℓ2 errorRelative ℓ2 error on various observation rate
Dataset Darcy Flow Poisson
Task Fwd Inv Fwd Inv
1% 0.025 0.019 0.045 0.145
16
3% 0.021 0.013 0.038 0.123
10% 0.020 0.012 0.035 0.117