An adaptive wavelet-based PINN for problems with localized high-magnitude source Himanshu Pandey1 and Ratikanta Behera1
arXiv:2604.28180v1 [cs.LG] 30 Apr 2026
1
Department of Computational and Data Sciences, Indian Institute of Science, Bangalore, 560012, India
Abstract. In recent years, physics-informed neural networks (PINNs) have gained significant attention for solving differential equations, although they suffer from two fundamental limitations, namely, spectral bias inherent in neural networks and loss imbalance arising from multiscale phenomena. This paper proposes an adaptive wavelet-based PINN (AW-PINN) to address the extreme loss imbalance characteristic of problems with localized high-magnitude source terms. Such problems frequently arise in various physical applications, such as thermal processing, electro-magnetics, impact mechanics, and fluid dynamics involving localized forcing. The proposed framework dynamically adjusts the wavelet basis function based on residual and supervised loss. This adaptive nature makes AW-PINN handle problems with high-scale features effectively without being memory-intensive. Additionally, AW-PINN does not rely on automatic differentiation to obtain derivatives involved in the loss function, which accelerates the training process. The method operates in two stages, an initial short pre-training phase with fixed bases to select physically relevant wavelet families, followed by an adaptive refinement that adapts scales and translations without populating high-resolution bases across entire domains. Theoretically, we show that under certain assumptions, AW-PINN admits a Gaussian process limit and derive its associated NTK structure. We evaluate AW-PINN on several challenging PDEs featuring localized high-magnitude source terms with extreme loss imbalances having ratios up to 1010 : 1. Across these PDEs, including transient heat conduction, highly localized Poisson problems, oscillatory flow equations, and Maxwell’s equations with a point charge source, AW-PINN consistently outperforms existing methods in its class. Keywords: Deep learning, Physics-informed machine learning, Loss balancing, Wavelet bases
1
Introduction
Recent years have witnessed significant progress in learning-based methods, mainly due to enhanced computational capabilities, advanced optimization algorithms, and the development of automatic differentiation techniques [1, 2]. These advances have facilitated the widespread adoption of deep learning approaches within the scientific community. Among these, physicsinformed neural networks (PINNs) [3] have emerged as a promising tool for solving partial differential equations (PDEs) in both forward and inverse settings. PINNs provide a meshless framework that optimizes the loss function by incorporating physical information through the residuals of differential equations. This methodology offers a robust alternative to conventional numerical methods, especially for tackling high-dimensional PDEs [4] and problems defined on complex geometries [5]. In particular, the capability of PINNs stands out when addressing inverse problems and generating highly resolved solutions by generalizing solutions across a complete computational domain rather than discrete points. Additionally, the PINN framework has been successfully extended to address integro-differential equations [6], stochastic differential equations [7, 8], and fractional differential equations [9, 10]. For a comprehensive overview of the theory and applications of PINNs, refer to [11, 12, 13] and the references therein.
2 Although PINNs offer notable advantages, several limitations have been extensively discussed and addressed in the literature. One key limitation arises from the spectral bias inherent in neural networks [14, 15], which causes PINNs to preferentially learn low-frequency components of the solution. Jacot et al. [16] formulated the neural tangent kernel (NTK) theory, demonstrating that the training process of an infinite-width neural network is equivalent to kernel regression with the NTK. Their analysis shows that loss function components associated with larger eigenvalues converge more rapidly, and the eigenvalues decrease sharply as the frequency of the target function increases. Building on NTK theory, Wang et al. [17] investigated the training dynamics of PINNs and confirmed the presence of spectral bias in PINNs. Additionally, the work by Wang et al. [18] revealed that the gradient flow in PINN models becomes stiff for PDEs with highfrequency components, resulting in imbalanced gradients during backpropagation. To overcome the challenge of learning high-frequency components, Tancik et al. [19] proposed Fourier feature mapping, which enhances the network’s ability to represent high-frequency functions. Building on this approach, Wang et al. [20] demonstrated that Fourier feature mapping modulates the frequency of NTK eigenvectors and developed an improved PINN architecture by integrating Fourier features before the fully connected layers. This architecture enhances the capacity of PINNs to solve PDEs with pronounced high-frequency characteristics. Further advancements include the work of Liu et al. [21], which used multiple Fourier feature encoding and adaptively adjusted encoding frequencies based on the loss function. In addition to spectral bias, PINNs also encounter poor training dynamics when there is a significant magnitude disparity between different loss terms. This loss imbalance is prominent in multiscale PDEs, where the optimizer tends to minimize the most dominant loss term at the expense of others. Such an imbalance hinders the ability of the model to accurately learn and represent all relevant features of the solution. Several studies have been conducted to address the loss balancing issue in PINNs. McClenny et al. proposed Self-adaptive PINNs (SA-PINNs) [22] that use trainable self-adaptive weights in loss terms at each training point. During optimization, the loss is minimized with respect to the network weights and maximized with respect to the self-adaptive weights. A non-negative, strictly increasing, and differentiable mask function, which takes self-adaptive weights as an argument, is used to make the network pay more attention to training points that are hard to fit. To further address the issue of loss term imbalance, Wang et al. [23] developed a modified loss function by imposing different power operations on each loss term to make them of the same order of magnitude. Further, Bischof et al. [24] developed relative loss balancing with random look-back, in which loss weights are updated based on loss statistics from previous training steps using exponential decay. Additionally, a Bernoulli random variable is used to decide whether relative improvements since the beginning of the training should be carried forward. This random look-back helps the model escape local minima by changing the loss space. These loss balancing techniques require careful selection of hyperparameters or prior knowledge of the solution to appropriately scale the loss terms for optimal performance. Alternatively, rather than explicitly balancing the loss function, Pandey et al. introduced wavelet-based PINNs (W-PINNs) [25], which train the model in wavelet space and bring it back to physical space using precomputed wavelet matrices. In wavelet space, the multiscale nature of the problem is smoothed out, making the optimization effective. Eventually, it transforms the optimization process to learn weights corresponding to wavelet bases composed of various scales and translates, inherently addressing both loss balancing and the spectral bias nature of the PINN. Furthermore, W-PINNs do not rely on automatic differentiation for derivatives involved in the loss function, which significantly accelerates the training process. However, this spectral approach is promising, but managing wavelet matrices, particularly for problems involving extremely high-scale features, becomes memory-intensive. Moreover, the exponential growth in the number of wavelet basis functions with increasing
3 scales further increases optimization challenges. In the present study, we propose an adaptive wavelet-based PINN (AW-PINN), which extends the W-PINN framework by enabling the network to dynamically adapt the wavelet family, rather than relying on a fixed basis. This adaptive nature enhances the effectiveness of the method in resolving problems with high-scale features. In particular, we focus on problems with localized high-magnitude source terms. Such problems frequently arise in various physical contexts, including heat conduction in materials with spatially localized sources, like in welding, electromagnetic wave propagation in inhomogeneous media, and fluid dynamics involving localized forcing. The rest of the manuscript is organized into four sections. Starting with Section 2, which reviews several related methodologies, we later compare the performance of our method against these. Followed by Section 3 describes the proposed AW-PINN framework in detail, and subsequently, the testing over a couple of challenging problems in Section 4. Finally, Section 5 summarizes the main findings along with the method’s limitations, and potential directions for future research. 2 2.1
Related Works PINN framework for problems with multi-magnitude loss terms
Wang et al. [23] developed a PINN framework for multiscale problems with multi-magnitude loss terms (MMPINN). To balance the magnitude of different loss terms, they proposed a regularization strategy that applies power operations on each loss term. To further refine their strategy, they incorporate multi-level training by adjusting the power exponent at each level and using a grouping regularization for different subdomains. The effectiveness of this regularization scheme is demonstrated through several PDE examples exhibiting loss imbalance, including a few with strong localized source terms. This makes it a fair comparison benchmark for our proposed method. Consider the following general form of PDE P[u(x)] = f (x), x ∈ Ω, (1) B[u(x)] = g(x), x ∈ ∂Ω, where P[·] represents a differential operator on the d-dimensional domain Ω ⊂ Rd , which also includes the time variable as a component, f is the source term, and B corresponds to the given initial and boundary conditions. Let û(x; θ) represent the PINN-based approximation to the PDE 1, with θ denoting the trainable parameters of the model. These parameters, which include weights and biases of the network, are obtained by optimizing the loss function. For the traditional PINN, the loss function can be expressed as follows L(θ) = ωr Lres (θ) + ωs Lsup (θ), Nres 1 X 2 P[û(xri ; θ)] − f (xri ) , xri ∈ Ω, Lres (θ) = (2) Nres i=1 N sup 1 X 2 L (θ) = B[û(xsi ; θ)] − g(xsi ) , xsi ∈ ∂Ω, sup Nsup i=1
where the total loss L is composed of the residual loss, Lres , and the supervised loss Lsup . Nres and Nsup are the numbers of residual and supervised training points in Ω and ∂Ω, respectively. This loss function is generally minimized using the most popular optimization algorithms, namely, Adam [26] and L-BFGS [27].
4 For highly imbalanced loss terms, MMPINN proposes an exponent regularized loss function, for which the loss function takes the following form 1
1
p q L̄(θ) = Lres (θ) + Lsup (θ),
p, q ∈ Z+ ,
(3)
where p and q are positive integer regularization parameters, which are chosen in a way that balances the order of magnitude of different loss terms. However, the effectiveness of this regularization depends on choosing p and q within an optimal range that is problem-specific. 2.2
Wavelet-based PINN
The Wavelet-based PINN (W-PINN) [25] learns the solution in wavelet space spanned by wavelet basis functions at multiple scales and translations that cover the entire domain. The learned solution is then mapped back to physical space using predefined wavelet basis transformation Q matrices. Let Ω = dn=1 [an , bn ] ⊂ Rd be a compact domain. For a mother wavelet ψ on R, the scaled and translated wavelets can be defined as ψj,k (x) = 2j/2 ψ 2j x − k , j, k ∈ Z, (4) where j and k are scale and translation parameters. This can be extended in d-dimensions by using a separable tensor-product basis in the following manner Ψj,k (x) =
d Y
2jn /2 ψ 2jn xn − kn ,
x ∈ Ω, j = (j1 , . . . , jd ), k = (k1 , . . . , kd ) ∈ Zd .
(5)
n=1
The scale parameters are chosen from a finite resolution-level sets Jn = {Jn,1 , . . . , Jn,Nn } ⊂ Z for each coordinate n = 1, . . . , d. For each jn ∈ Jn , define the integer translation range n o Kn (jn ) := k ∈ Z : (an − γ) 2jn ≤ k ≤ (bn + γ) 2jn , (6) which ensures coverage of Ω at each resolution. Here γ > 0 is a fixed translation hyperparameter. For this hierarchical multi-resolution basis, we can define a multi-index set as n o I = (j, k) ∈ Zd × Zd : jn ∈ Jn , kn ∈ Kn (jn ) for n = 1, . . . , d . (7) Consider a bijection mapping M : I → {1, . . . , Nfam }, and define Ψi (x) := Ψj,k (x) for i = M(j, k). Let ûψ (x; θ) is a W-PINN approximation to the PDE 1 such that Nf am ψ
û (x; θ) =
X
ci (θ)Ψi (x) + B,
x ∈ Ω ∪ ∂Ω,
(8)
i=1 fam where θ∪{B} are the trainable parameters of W-PINN network for learnable coefficients {ci }N i=1 , fam and the fixed basis {Ψi }N i=1 . Training minimizes the loss in Equation (2), with û replaced by ûψ from (8). Instead of using autograd, derivatives required in the loss function are obtained by analytically differentiating the basis functions Ψi , i.e, Y jn /2 ∂xm Ψj,k (x) = 2jm 2jm /2 ψ ′ 2jm xm − km × 2 ψ 2j n x n − k n , (9)
n̸=m
and higher-order derivatives follow similarly. The W-PINN effectively addresses multiscale PDEs because the localized wavelet basis transforms the problem into wavelet space, where multiscale features become smoother, eventually facilitating the training process.
5 2.3
Neural Tangent Kernel theory for PINN
The neural tangent kernel (NTK) [16, 17] provides a theoretical perspective for training dynamics of neural networks. Consider an infinite width PINN approximating the PDE (1) with û(x; θ). For gradient descent algorithm with a sufficiently small learning rate, Wang et al. [17] showed that, in the infinite-width limit, the training dynamics converge to independent, identically distributed Gaussian processes with zero mean. Based on this, NTK for PINN is defined as
Kpp (t) Kpb (t) K(t) = , Kbp (t) Kbb (t)
(10)
where t is the training step. For given training points {xri , f (xri )} and {xsi , g(xsi )}, {i, j}-th element of submatrices of the NTK matrix K is obtained by * (Kpp )ij (t) =
∂P [û (xri ; θ(t))]
, ∂θ h i + * s ; θ(t) ∂B û x r j ∂P [û (xi ; θ(t))] (Kpb )ij (t) = , , ∂θ ∂θ h i + * s ; θ(t) ∂B û x s j ∂B [û (xi ; θ(t))] (Kbb )ij (t) = , , ∂θ ∂θ ∂θ
,
h i + ∂P û xrj ; θ(t)
where ⟨·, ·⟩ denotes the inner product over all trainable parameters. Further, using the NTK matrix, training a PINN through gradient descent can be expressed as
∂P [û (xr ; θ(t))] r r ∂t = −K(t) P [û (x ; θ(t))] − f (x ) . ∂B [û (xs ; θ(t))] B [û (xs ; θ(t))] − g (xs ) ∂t
(11)
Under the assumption of training a sufficiently large-width PINN with a small enough learning rate, Wang et al. [17] showed the NTK remains constant, ie, K(t) ≈ K(0). Using this solution of the Equation (11) becomes
r P [û (xr ; θ(t))] − f (xr ) −K(0)t f (x ) , ≈ −e g (xs ) B [û (xs ; θ(t))] − g (xs )
(12)
which provides error evolution during training, which can be further simplified by spectral decomposition of the NTK: K(0) = QT ΛQ, here Q and Λ are orthogonal matrix and corresponding diagonal eigenvalue matrix. Then the Equation (13) take the form
P [û (xr ; θ(t))] − f (xr ) f (xr ) T −Λt ≈ −Q e Q . B [û (xs ; θ(t))] − g (xs ) g (xs )
(13)
This formulation suggests that the ith error component exhibits exponential decay at a rate determined by the corresponding NTK eigenvalue; hence, larger eigenvalues yield faster decay of the error and, consequently, faster convergence of the PINN.
6 3
Adaptive Wavelet-based PINN
The Adaptive Wavelet-based PINN (AW-PINN) extends the W-PINN framework by making the wavelet family adaptive, allowing scales and translates to be learned as parameters, unlike the fixed values in W-PINN. The method consists of two stages: a short pre-training stage using the W-PINN formulation 2.2, followed by an adaptive stage that refines the selected wavelet families obtained from the pre-training stage. We discuss both stages in detail below. • Pre-training and family selection res Consider W-PINN pre-training for the PDE (1) with residual points {xri }N i=1 and superNsup vised points {xsi }i=1 . Evaluate the right side of PDE (1) at the residual and supervised training points in vector form
f = (f (xr1 ), . . . , f (xrNres ))⊤ ∈ RNres , g = (g(xs1 ), . . . , g(xsNres ))⊤ ∈ RNsup ,
(14)
where f and g denote the PDE residual source term and supervised condition values, respectively. For each wavelet family index i with basis Ψi and pre-trained coefficient ci , define the family-induced residual and boundary response vectors ⊤ Ri = P[ci Ψi (xr1 ), . . . , P[ci Ψi (xrNres ) ∈ RNres , (15) ⊤ Bi = B[ci Ψi (xs1 ), . . . , B[ci Ψi (xsNsup ) ∈ RNsup , then we get the alignment measure between the family i and the PDE right side via the following similarity scores ScoreR i =
⟨Ri , f ⟩ , |⟨Ri , f ⟩|max
ScoreB i =
⟨Bi , g⟩ , |⟨Bi , g⟩|max
(16)
When the right-hand side of the PDE is identically zero, the above similarity becomes ill-defined. In such cases, mean of the normalized response vector, R̄i /∥Ri ∥max and B̄i /∥Bi ∥max , are used as respective scores. Wavelet families exhibiting a higher similarity score or lower normalized response magnitudes are deemed more physically relevant. Additionally, wavelet families corresponding to the top κ largest coefficient magnitudes |ci | are also included. This similarity-based selection works because the wavelet bases exhibit minimal overlap. The final set of active families is denoted as n o IA = (j, k) ∈ Zd × Zd ⊂ I, (17) A along with their corresponding coefficients cA = {ci }N i=1 , NA denotes number of selected wavelet basis function.
• Adaptive stage and network formulation The selected coefficients and scale-translate parameters are utilized to initialize the AWPINN network. For each selected family i ∈ IA associated with scale–translate pair (j i , ki ), an adaptive submodule Wi (x; θ i ) is constructed as a parametric wavelet. For d-dimensional input x = (x1 , . . . , xd ), we define Wi (x; θ i ) =
d Y n=1
ψ(wi,n xn + bi,n ) ,
θ i = {(wi,n , bi,n )}dn=1 ,
(18)
7 Backpropogation
ψ ⋯
ψ ψ ψ N ⋯
x2
wi,1
ψ
L = Lres + Lsup W1 W2 W3
c1 ∂xm ûAψ =
⋯
⋯
NA X
′
ci wi,m ψi,m
i=1
ûAψ
ψ
∂x2m ûAψ =
NA X
ψi,n
Lres =
Nres 1 X 2 P[ûAψ ] − f Nres i=1
ψi,n
Lsup =
Nsup 1 X 2 B[ûAψ ] − g Nsup i=1
n̸=m ′′
2 ci wi,m ψi,m
i=1
cNA
Y
Y n̸=m
ψ
xd
ψ ⋯
W-PINN Pre-Training
x1
WA wi,d
ψ Fig. 3.1. A schematic architecture of AW-PINN. Parameters for purple wavelet units and final linear layer are initialized using Iadapt and c⊗ adapt , respectively. The symbol ψ denotes the wavelet activation function and ψi,m = ψ(wi,m xm + bi,m ), while represents element-wise multiplication.
where ψ(·) is the same mother wavelet used in the Equation (4). The initial parameters are (0) (0) transferred from IA as, wi,n = 2ji,n , bi,n = −ki,n , enabling continuous adaptation of scale and translation during optimization. This adaptive wavelet layer is achieved by using ψ(·) as activation function. A final linear layer, initialized with cA , is imposed to get AW-PINN approximation
û
Aψ
(x; θ) =
NA X
ci (θ) Wi (x; θ i ) + B,
(19)
i=1
where B denotes a trainable bias term. The optimization process minimizes the same Algorithm 1 Adaptive Wavelet-based PINN (AW-PINN) Input: P, B, domain Ω, points {xri }, {xsi }, wavelet resolutions {Jn }, translation hyperparameter γ, threshold κ. Stage 1: Pre-training and family selection 1: Build fixed family I = {(j, k) : jn ∈ Jn , kn ∈ Kn (jn )}, as described in [25]. P 2: Pre-train W-PINN ûψ (x; θ) = ci (θ)Ψi (x) + B via loss (2). 3: For each i ∈ I compute responses Ri = P[ci Ψi ], Bi = B[ci Ψi ]. R B 4: Compute similarity scores Scorei = ⟨Ri , f ⟩/|⟨Ri , f ⟩|max , Scorei = ⟨Bi , g⟩/|⟨Bi , g⟩|max . 5: Select active families IA := selection based on score threshold and top κ coefficients |ci |. Stage 2: Adaptive stage 6: Initialize adaptive parameters for i ∈ IA : w1i,n = 2ji,n , bi,n = −ki,n . Qd 7: Define adaptive wavelet unit Wi (x; θ i ) = n=1 ψ(wi,n xn + bi,n ). P 8: Initialize ci from pre-training and construct ûAψ (x; θ) = i∈IA ci (θ) Wi (x; θ i ) + B. 9: Compute analytic derivatives of Wi for P and B. 10: Train model parameters θ = {ci , θ i , B} by minimizing loss in Eq. (2) using L-BFGS until convergence. Output: Trained solution ûAψ (x; θ ∗ )
8 physics-informed loss as in the Equation (2), and the autograd is avoided by taking the analytical derivative of the Equation (18), similar to the Equation (9). A schematic diagram of the AW-PINN architecture is shown in Fig. 3.1, and the AW-PINN algorithm is summarized in Algorithm 1. We now demonstrate that, as the adaptive family grows to an infinitely large size, under certain assumptions, the AW-PINN training converges to a zero-mean Gaussian process Gp . Theorem 1. For bias B = 0, as NA → ∞, the random functions ûAψ N (·) converge in finitedimensional distributions to a centered Gaussian process ûAψ ∞ (·)
NA 1 X =√ ci Wi ∼ Gp 0, k(·, ·) NA i=1
Proof. Fix m ∈ N and inputs X = {x(1) , . . . , x(m) } ⊂ Rd . Define fNA =
1 Aψ (1) (m) ⊤ ûAψ (x ), . . . , û (x ) =√ NA NA
NA X
NA i=1
c i Φi ,
where the random feature vector Φi =
Wi (x(1) ; θ i ), . . . , Wi (x(m) ; θ i )
⊤
∈ Rm . i.i.d.
i.i.d.
Assume parameters and coefficients are initialized i.i.d. normal distribution, ci ∼ N (0, σc2 ), θi ∼ N (0, σθ2 ), where σc and σθ are fixed constants, then Cov [ci Φi ] = σc2 Eθ [Φi Φ⊤ i ], whose (m, n)-th entry is σc2 Eθ [W(x(m) ; θ)W(x(n) ; θ)]. By the multivariate central limit theorem, fNA converges in distribution to a multivariate Gaussian with mean zero and covariance matrix equal to that limit covariance. Because the finite dimensional distributions converge to Gaussian ones with the process the above covariance, ′ 2 ′ converges to the centered Gp with kernel K(x, x ) = σc Eθ W(x; θ) W(x ; θ) . Although, for practical purposes, the AW-PINN parameters are transferred effectively from W-PINN pre-training, the above theorem applies under random initialization as stated. The above theorem suggests the learning dynamics of AW-PINN in the infinite family regime may be analyzed via kernel regression and neural tangent kernel theory as in Section 2.3. For a scalar output ûAψ , the empirical NTK between inputs x, x′ is K(x, x′ ) =
∇θ ûAψ (x; θ), ∇θ ûAψ (x′ ; θ) ,
(20)
where the inner product is over all trainable scalar parameters. Decompose the gradient into derivatives with respect to the linear coefficients ci and the wavelet parameters θi . Using the √ model and the 1/ NA scaling, ∂uAψ 1 =√ Wi (x; θi ), ∂ci NA 1 ∂uAψ =√ ci ∇φ Wi (x; θi ). ∂θi NA
(21) (22)
9 Hence, the NTK decomposes as a sum over units K(x, x′ ) =
A A 1 X 1 X Wi (x; θi )Wi (x′ ; θi ) + c2i ∇θ Wi (x; θi ), ∇θ Wi (x′ ; θi ) , NA NA
N
N
i=1
i=1
(23)
we denote the two contributions by Kc (from linear weights ci ) and Kθ (from adaptive parameters): K(x, x′ ) = Kc (x, x′ ) + Kθ (x, x′ ). 4
Results
This section evaluates the performance of AW-PINN on several challenging problems and compares with baseline PINN, W-PINN, and MMPINN, all of which are designed to address the loss imbalance issue in PINNs. For W-PINN and MMPINN, training starts with a few epochs of the ADAM optimizer, followed by the LBFGS optimizer until convergence. In the case of AW-PINN, the first phase is done via ADAM, and the subsequent adaptive phase employs the LBFGS optimizer. In this study, we use the Gaussian wavelet, which is the first derivative of the Gaussian function. The smooth regularity and localization property of the Gaussian wavelet make it one of the optimal choices. All model parameters are initialized using Xavier’s technique [28], which prevents vanishing gradients and provides stable and efficient training. The hyperparameters used for each problem are tabulated in Appendix A 5. For quantitative comparison, we use the relative L2 -Error metric over uniformly distributed test samples across the domain, which is defined as ||u(xt ) − û(xt ; θ)||2 Relative L2 -Error = , (24) ||u(xt )||2 where u and û are the exact/reference solution and the model’s prediction over test samples xt , respectively. Further, to average out the model’s sensitivity over parameter initialization, each experiment is repeated five to ten times independently, and reported results correspond to the average outputs of these runs. Training is carried out using PyTorch 2.6.0 + CU12.4 on an NVIDIA RTX A6000 GPU. All source code to reproduce the results of this study will be made available on request. 4.1
Heat conduction problem with extreme heat source
Heat conduction problems with a localized, strong heat source are often encountered in various practical applications, such as material processing, geophysics, nuclear engineering, and many others. For such problems, challenges lie in the transient nature and the intense gradients generated in the process. This problem is chosen to test the effectiveness of the proposed method for such an extreme source. Consider the following heat conduction model ∂u ∂2u ∂t = ∂x2 + h(x, t), (x, t) ∈ (−1, 1) × (0, 1), (25) u(x, 0) = (1 − x2 ) exp 1/(1 + ϵ) , x ∈ (−1, 1), u(−1, t) = 0, u(1, t) = 0, t ∈ (0, 1], the exact solution is given as u(x, t) = (1 − x2 ) exp 1/((2t − 1)2 + ϵ) , for which the source term becomes h i 2 −1) 1 h(x, t) = 2 1 + 2 (2t−1)(x exp . 2 2 2 ((2t−1) +ϵ) (2t−1) +ϵ
10 Table 4.1 Comparison of various methods for solving the heat conduction problem 4.1.
ϵ
0.12
0.11
0.10
Method
NI
NB
NR
Relative L2 -Error
Avg. Training Time
Baseline PINN W-PINN MMPINN AW-PINN
5000 1000 3000 1000
10000 2000 6000 2000
50000 20000 90000 20000
8.1 ± 1.32 × 10−1 4.05 ± 0.38 × 10−5 1.71 ± 0.53 × 10−5 3.52 ± 0.81 × 10−6
15.35 min 91.21 min 24.83 min
Baseline PINN W-PINN MMPINN AW-PINN
5000 1000 4000 1000
10000 2000 8000 2000
50000 20000 100000 20000
9.13 ± 0.82 × 10−1 3.11 ± 0.63 × 10−5 2.44 ± 0.61 × 10−5 8.15 ± 1.27 × 10−6
16.79 min 208.36 min 24.14 min
Baseline PINN W-PINN MMPINN AW-PINN
5000 1000 7000 1000
10000 2000 14000 2000
100000 20000 200000 20000
9.61 ± 0.76 × 10−1 3.10 ± 0.86 × 10−5 8.46 ± 0.68 × 10−1 8.86 ± 0.87 × 10−6
16.28 min 25.52 min
For the lower ϵ, the source term as well as the solution exhibit transient behavior with a high gradient near t = 0.5. Additionally, due to smooth initial and boundary conditions, it poses a highly loss imbalance. For instance, at ϵ = 0.1, the initial loss ratio is Lb : Li : Lr = 1 : 10 : 109 , which makes it challenging for any PINNs-based approaches. In our experiments, we employ 5000 Adam iterations (approximately two minutes of pretraining) using the W-PINN for pre-training, followed by L-BFGS optimization with AW-PINN until convergence. Table 4.1 lists comparative results for several small ϵ values. It includes the relative L2 -error, average training time, and number of collocation points used in training. Training times are omitted for methods with ineffective convergence. The table demonstrates AW-PINN achieves the lowest relative L2 -error for all ϵ values, with substantially reduced training time compared to MMPINN. Specifically for ϵ = 0.1, only AW-PINN could solve the problem satisfactorily. Another interesting observation is that, unlike MMPINN, AW-PINN does not require additional collocation points with a decrease in ϵ values. Furthermore, for ϵ = 0.1, extremely transient behavior of the heat source is evident from Fig. 4.2, and the consistently low point-wise absolute error across the domain demonstrates the effectiveness of the AW-PINN. Moreover, Fig. 4.3 shows the loss curve of AW-PINN and also compares the relative L2 -error with other methods for ϵ = 0.12. The narrow shaded regions for each loss term suggest low variance in the AW-PINN model, while the relative L2 -error comparison clearly demonstrates the faster convergence achieved by AW-PINN. Additionally, the non-oscillatory error curve indicates Heat source
×105 3.5
x
1
0
−1 0.0
0.0
0.5
t
1.0
AW-PINN prediction
×104 2.1
1
0
−3.0 −1 0.0
1.0
AW-PINN point-wise error 1 0.2
0
0.1
0.1 0.5
t
1.0
−1 0.0
0.0 0.5
1.0
t
Fig. 4.2. Left to Right: Heat source function, the prediction using AW-PINN and respective absolute point-wise error for problem 4.1 with ϵ = 0.1.
11 Relative L2-Error for different methods
AW-PINN Loss 0
IC Loss BC Loss PDE Loss
Log10 Relative L2 Error
Log10 Loss
8
4
0
−4
0 1000
5000
10000
MMPINN W-PINN AW-PINN
−1
−3
−5
14000
1000
Iteration
10000
20000
Iteration
Fig. 4.3. Left: Average log loss plots for each loss terms in problem 4.1 with ϵ = 0.12 using AW-PINN. Shaded regions denote the standard deviation across 10 independent runs. The vertical dashed line marks the completion of pre-training with W-PINN using 1000 Adam iterations. Right: Log relative L2 -error plots of different methods for problem 4.1 with ϵ = 0.12. The vertical dashed line indicates the end of the pre-training phase with 1000 Adam iterations.
that AW-PINN provides more stable training dynamics than MMPINN. 4.2
Poisson problem with highly localized source term
In this test case, we examine a two-dimensional Poisson equation with an extremely localized source term. Such equations frequently arise in modeling electrostatic potentials with point charges and concentration profiles in chemical diffusion processes. They are also relevant in fields such as materials science and geophysics, where localized phenomena must be accurately resolved. Consider the following Poisson model: 2 ∂ u2 + ∂ 2 u2 = f (x, y), (x, y) ∈ (0, 1) × (0, 1), ∂x ∂y (26) B(x, y) = g(x, y), (x, y) ∈ ∂Ω, with the exact solution as 2 u(x, y) = 1 + (y 2 + 103 ) exp − (x−0.5) , 2 2ϵ for small ϵ values, both the source function and the solution are highly localized, as illustrated in the top panel of Fig. 4.4 for ϵ = 0.02. This strong localization results in a huge loss imbalance. For example, at ϵ = 0.02 the initial loss ratio is Lb : Lr = 104 : 1011 , making it a challenging scenario for the PINN-based methods. Table 4.2 compares the performance of various methods for ϵ = 0.05 and ϵ = 0.02. It demonstrates that the AW-PINN achieves relative L2 -error that is one and two orders of magnitude lower than those obtained by MMPINN and W-PINN, respectively. The same can be observed Table 4.2 Comparison of various methods for solving the Poisson problem 4.2.
ϵ
Method
NB
NR
Relative L2 -Error
Avg. Training Time
0.05
W-PINN MMPINN AW-PINN
4000 6000 4000
10000 30000 10000
3.55 ± 0.26 × 10−4 5.71 ± 1.53 × 10−4 3.42 ± 0.13 × 10−5
4.87 min 7.17 min 5.14 min
0.02
W-PINN MMPINN AW-PINN
4000 8000 4000
15000 50000 10000
9.81 ± 1.72 × 10−3 3.25 ± 1.16 × 10−3 2.68 ± 0.53 × 10−4
4.21 min 9.11 min 6.41 min
12
y
Source term
Exact solution
AW-PINN prediction
1.0
×106 1.0
1.0
×102 9.5
1.0
×102 9.5
0.5
−0.5
0.5
5.0
0.5
5.0
0.1
0.0 0.0
−2.2 0.0 0.0
0.5
0.0 0.0
1.0
W-PINN point-wise error
0.5
1.0
MMPINN point-wise error
1.0
25
0.1 0.5
1.0
AW-PINN point-wise error
1.0
1.0
0.45
0.5
0.24
y
3.0
0.5
12
1
0.0 0.0
0.5
1.0
0.5
2.2
1.5 0.0 0.0
0.5
x
0.02
0.0 0.0
1.0
0.5
x
1.0
x
Fig. 4.4. Top - Left to Right: The source function, exact solution, and prediction using AW-PINN for problem 4.2 with ϵ = 0.02. Bottom - Left to Right: Absolute point-wise error of W-PINN, MMPINN, and AW-PINN, respectively.
Scale adaptation via AW-PINN W-PINN scales AW-PINN scales
30
y-scale values
x-scale values
45
30
15
10
W-PINN scales AW-PINN scales
−10
0 0
500
1000
1500
2000
0
Wavelet family index
500
1000
1500
2000
Wavelet family index
Fig. 4.5. Plot of x-scale (left) and y-scale (right) adaptation for problem 4.2. Here, solid lines denote dyadic scales and dashed lines represent scales after adaptation.
in Fig. 4.4 bottom panel, where AW-PINN exhibits the lowest absolute point-wise error across the domain. Moreover, Fig. 4.5 illustrates the scale adaptation capability of AW-PINN. In the case of W-PINN, it requires a highly resolved wavelet basis and corresponding matrices to handle such problems with localized high-scale features, which often need intensive memory and lead to highly non-convex optimization. Whereas AW-PINN efficiently adapts to higher scales only in the region where needed, without populating the entire domain with a high-resolution basis function. This not only improves computational efficiency but also enhances training stability and accuracy. 4.3
Flow Equation with a strong oscillating source term
The flow equation arises in a wide range of practical applications. For example, conservation laws in non-inertial reference frames, such as flows around airfoils, two-phase flow models incor-
13 Table 4.3 Comparison of various methods for solving the flow equation 4.3.
Method
NI
NB
NR
Relative L2 -Error
Avg. Training Time
W-PINN MMPINN AW-PINN
1000 2000 1000
1000 2000 1000
20000 50000 20000
1.27 ± 0.15 × 10−2 8.35 ± 0.97 × 10−2 5.17 ± 0.61 × 10−4
4.17 min 7.21 min 6.17 min
Source term
x
1
90
−90 0.5
1.0
0.8
0
0.5
x = −0.9
0.01
−1 0.0
0.5
x = 0.8 3
u
1
−1 0.0
0.5
0.0
1.0
0.5
t
1.0
t
x = −0.5
3
u
u
1.0
t
1
−1
×10−2
1.50
0
−0.8
−1 0.0
t 3
1
2.5
2.70
0
0
−1 0.0
AW-PINN point-wise error
AW-PINN prediction
1
1.0
1
−1
t Exact solution
0.0
0.5
1.0
t Prediction
Fig. 4.6. Top - Left to Right: The source function, the prediction using AW-PINN, and the corresponding absolute point-wise error for problem 4.3. Bottom: Cross-section comparison of the prediction with the exact solution at various x-domain snapshots.
porating momentum and energy exchange, turbulence models governed by two-equation formulations, and reactive flows involving chemical processes. For this example, we are interested in the following flow model with a large oscillating source term ∂u + ∂u = A sin 2π t , ∂t
∂x
u(x, 0) = sin(πx),
Ts
(x, t) ∈ (−1, 1) × (0, 1),
x ∈ (−1, 1),
(27)
with exact solution t s , u(x, t) = sin(π(x − t)) + AT 2π 1 − cos 2π Ts here we take A = 100 and Ts = 0.05. A small Ts makes the source scale much smaller than the mean flow time scale, and a large A strengthens the oscillations of the source term. These characteristics make this flow equation a challenging problem. The comparison Table 4.3 suggests that AW-PINN achieves accuracy superior to other methods by two orders of magnitude. The top panel of Fig. 4.6 shows a highly oscillatory source term, AW-PINN prediction, and the corresponding absolute point-wise error, demonstrating the model’s efficiency. The bottom panel further validates the accuracy of the prediction through multiple spatial snapshot plots.
14 4.4
Maxwell’s equation for a point charge source
In this example, we apply AW-PINN on a three-dimensional (one temporal and two spatial) electromagnetic problem. The fundamental principles of electromagnetism are governed by Maxwell’s equations. It models the electric and magnetic fields through a set of coupled partial differential equations. These equations are widely used in various fields of physics and engineering, including wireless communications and antenna design, microwave devices, radar and remote sensing, photonics and optical waveguides, and biomedical imaging. To validate the robustness of the proposed method, we consider the TEz formulation of Maxwell’s equations in a rectangular cavity with perfectly electrically conducting (PEC) boundary conditions ∂Ex 1 ∂Hz = , ∂t ϵo ∂y ∂Ey 1 ∂Hz =− , ∂t ϵo ∂x ∂Hz 1 ∂Ey ∂Ex =− − +S , ∂t µo ∂x ∂y
(28)
where Ex and Ey denote the in-plane electric field components and Hz denotes the out-of-plane magnetic field component. The constants ϵo and µo are the permeability and permittivity of the space, respectively. The source function S represents a Gaussian pulse emitted by a point source located at (xo , yo ), which can be expressed as t − τ 2 , S(x, y, t) = δ(x − xo )δ(y − yo ) · exp − ω
(29)
with temporal delay τ and pulse width ω. The spatial point source is approximated using the Gaussian distribution centered at (xo , yo ) with kernel width σ = 0.01. 1 (x − x0 )2 + (y − y0 )2 δ(x − x0 ) δ(y − y0 ) ≈ exp − . 2πσ 2 2σ 2
(30)
To solve this system of equations, all physical quantities are non-dimensionalized such that ϵo and µo become unity, and take τ = 0.25 and ω = 0.25. We solve the problem for the domain Ω = [0, 1]2 with the point source located at (xo , yo ) = (0.5, 0.5) and starting with a null field condition. The simulation is run for the non-dimensional time T = 0.5. For a reference solution, we employ the finite-difference time-domain (FDTD) method with spatial resolution as ∆x = ∆y = 5 × 10−3 and time step ∆t = 1.5 × 10−3 , chosen to satisfy the CFL stability condition under the non-dimensional wave speed. W-PINN and MMPINN are unable to solve this problem satisfactorily, which is evident from high relative L2 −error values in the Table 4.4. Further, the point-wise error plot shown in Fig. 4.7 demonstrates higher accuracy of AW-PINN prediction. Table 4.4 Comparison of various methods for solving Maxwell’s equation 4.4.
Method
Relative L2 -Error Ex
Relative L2 -Error Ey
Relative L2 -Error Hz
Avg. Training Time
W-PINN MMPINN AW-PINN
8.52 ± 0.21 × 10−2 4.33 ± 0.17 × 10−2 5.31 ± 0.61 × 10−3
8.94 ± 0.13 × 10−2 3.91 ± 0.21 × 10−2 6.54 ± 0.47 × 10−3
4.52 ± 0.21 × 10−1 2.31 ± 0.18 × 10−1 9.01 ± 0.83 × 10−3
42.71 min 182.16 min 76.57 min
15 Ex prediction
1.0
Ey prediction
1.0
0.0
0.5
Hz prediction 0.0
6.0
y
y
6.0
0.0
0.5
y
1.0
0.5 −0.5
−6.0 0.0 0.0
0.5
−6.0 0.0 0.0
1.0
0.5
0.0 0.0
1.0
x
x 1.0
0.23
−0.9 0.5
1.0
x
1.0
0.10
1.0
y
0.06
0.12
0.5
0.0 0.0
0.00 0.5
x
1.0
0.5
0.0 0.0
0.5
x
1.0
0.05
0.5
0.00
0.0 0.0
0.03
0.00 0.5
1.0
x
Fig. 4.7. The electromagnetic field prediction using AW-PINN for TEz Maxwell’s equations 4.4 at the top panel and their respective point-wise absolute error at the bottom panel.
5
Conclusion
In this study, we addressed the challenges encountered during the training of PINNs due to the severe loss imbalance between different loss components. In particular, we considered problems with localized high-magnitude source terms. For such problems, there is an extreme loss imbalance, and it requires special treatment. To tackle these issues, we proposed an extension of W-PINN that dynamically adapts the wavelet family based on residual and supervised loss functions. This adaptive mechanism enables the model to effectively capture fine-scale features without populating the entire domain with higher-resolution wavelets. Through various challenging problems, we have demonstrated significant improvement over W-PINN and the well-known MMPINN. Although the method is promising and has achieved impressive accuracy with highly competitive training time, there is still room for improvement. The similarity-based selection of wavelet bases involves manual tuning partially and depends on the quality of the pre-training phase, suggesting that more automated selection schemes could improve robustness. Additionally, systematically investigating the performance of AW-PINN using various wavelet bases could be an interesting comparison. Moreover, future work could involve evaluating the AW-PINN across a broader class of multiscale problems to further assess its general applicability and performance. Acknowledgment The authors would like to thank Mr. Anirudha Sen, Department of Mechanical Engineering, Indian Institute of Technology Bhilai, India, for his assistance in computations related to Neural Tangent Kernel theory for Physics-Informed Neural Networks. Funding details No funding source is available for this research.
16 Data availability statements All source code to reproduce the results of this study will be made available on request. Conflicts of interest and declarations The authors declare that they do not have any conflicts of interest. In addition, they also declare that this work is not under consideration anywhere. References [1] A. G. Baydin, B. A. Pearlmutter, A. A. Radul, J. M. Siskind, Automatic differentiation in machine learning: a survey, J. Mach. Learn. Res. 18 (153) (2018) 1–43. URL http://jmlr.org/papers/v18/17-468.html [2] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, Pytorch: An imperative style, high-performance deep learning library, in: Advances in Neural Information Processing Systems, Vol. 32, Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/ bdbca288fee7f92f2bfa9f7012727740-Paper.pdf [3] M. Raissi, P. Perdikaris, G. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378 (2019) 686–707. doi:10.1016/j.jcp.2018.10.045. [4] Z. Hu, K. Shukla, G. E. Karniadakis, K. Kawaguchi, Tackling the curse of dimensionality with physics-informed neural networks, Neural Networks 176 (2024) 106369. doi:10.1016/j.neunet. 2024.106369. [5] F. Sahli Costabal, S. Pezzuto, P. Perdikaris, Δ-pinns: Physics-informed neural networks on complex geometries, Eng. Appl. Artif. Intell. 127 (2024) 107324. doi:10.1016/j.engappai.2023.107324. [6] L. Yuan, Y.-Q. Ni, X.-Y. Deng, S. Hao, A-pinn: Auxiliary physics informed neural networks for forward and inverse problems of nonlinear integro-differential equations, J. Comput. Phys. 462 (2022) 111260. doi:10.1016/j.jcp.2022.111260. [7] D. Zhang, L. Guo, G. E. Karniadakis, Learning in modal space: Solving time-dependent stochastic pdes using physics-informed neural networks, SIAM J. Sci. Comput. 42 (2) (2020) A639–A665. doi:10.1137/19M1260141. [8] Tushar, S. Chakraborty, Deep physics corrector: A physics enhanced deep learning architecture for solving stochastic differential equations, J. Comput. Phys. 479 (2023) 112004. doi:10.1016/j.jcp. 2023.112004. [9] G. Pang, L. Lu, G. E. Karniadakis, fpinns: Fractional physics-informed neural networks, SIAM J. Sci. Comput. 41 (4) (2019) A2603–A2626. doi:10.1137/18M1229845. [10] S. S. M., P. Kumar, V. Govindaraj, A novel optimization-based physics-informed neural network scheme for solving fractional differential equations, Engineering with Computers 40 (2) (2024) 855– 865. doi:10.1007/s00366-023-01830-x. [11] S. Cuomo, V. S. D. Cola, F. Giampaolo, G. Rozza, M. Raissi, F. Piccialli, Scientific machine learning through physics–informed neural networks: Where we are and what’s next, J. Sci. Comput. 92 (3) (2022) 88. doi:10.1007/s10915-022-01939-z. [12] W. Zhang, W. Suo, J. Song, W. Cao, Physics informed neural networks (pinns) as intelligent computing technique for solving partial differential equations: Limitation and future prospects, arXiv preprint arXiv:2411.18240 (2024). doi:10.48550/arXiv.2411.18240. [13] C. Zhao, F. Zhang, W. Lou, X. Wang, J. Yang, A comprehensive review of advances in physicsinformed neural networks and their applications in complex fluid dynamics, Phys. Fluids. 36 (10) (2024) 101301. doi:10.1063/5.0226562. [14] N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, A. Courville, On the spectral bias of neural networks, in: K. Chaudhuri, R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning, Vol. 97 of Proceedings of Machine Learning
17 Research, PMLR, 2019, pp. 5301–5310. URL https://proceedings.mlr.press/v97/rahaman19a.html [15] Z.-Q. J. Xu, Y. Zhang, T. Luo, Overview frequency principle/spectral bias in deep learning, Commun. Appl. Math. Comput. 7 (3) (2025) 827–864. doi:10.1007/s42967-024-00398-7. [16] A. Jacot, F. Gabriel, C. Hongler, Neural tangent kernel: Convergence and generalization in neural networks, in: Advances in Neural Information Processing Systems, Vol. 31, Curran Associates, Inc., 2018. URL https://proceedings.neurips.cc/paper_files/paper/2018/file/ 5a4be1fa34e62bb8a6ec6b91d2462f5a-Paper.pdf [17] S. Wang, X. Yu, P. Perdikaris, When and why pinns fail to train: A neural tangent kernel perspective, J. Comput. Phys. 449 (2022) 110768. doi:https://doi.org/10.1016/j.jcp.2021.110768. [18] S. Wang, Y. Teng, P. Perdikaris, Understanding and mitigating gradient flow pathologies in physics-informed neural networks, SIAM J. Sci. Comput. 43 (5) (2021) A3055–A3081. doi: 10.1137/20M1318043. [19] M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (Eds.), Advances in Neural Information Processing Systems, Vol. 33, Curran Associates, Inc., 2020, pp. 7537–7547. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/ 55053683268957697aa39fba6f231c68-Paper.pdf [20] S. Wang, H. Wang, P. Perdikaris, On the eigenvector bias of fourier feature networks: From regression to solving multi-scale pdes with physics-informed neural networks, Comput. Methods Appl. Mech. Eng 384 (2021) 113938. doi:https://doi.org/10.1016/j.cma.2021.113938. URL https://www.sciencedirect.com/science/article/pii/S0045782521002759 [21] Y. Liu, H. Gu, X. Yu, P. Qin, Diminishing spectral bias in physics-informed neural networks using spatially-adaptive fourier feature encoding, Neural Networks 182 (2025) 106886. doi:https://doi. org/10.1016/j.neunet.2024.106886. URL https://www.sciencedirect.com/science/article/pii/S0893608024008153 [22] L. D. McClenny, U. M. Braga-Neto, Self-adaptive physics-informed neural networks, J. Comput. Phys. 474 (2023) 111722. doi:10.1016/j.jcp.2022.111722. [23] Y. Wang, Y. Yao, J. Guo, Z. Gao, A practical pinn framework for multi-scale problems with multimagnitude loss terms, J. Comput. Phys. 510 (2024) 113112. doi:10.1016/j.jcp.2024.113112. [24] R. Bischof, M. A. Kraus, Multi-objective loss balancing for physics-informed deep learning, Comput. Methods Appl. Mech. Engrg. 439 (2025) 117914. doi:10.1016/j.cma.2025.117914. [25] H. Pandey, A. Singh, R. Behera, An efficient wavelet-based physics-informed neural networks for singularly perturbed problems (2025). arXiv:2409.11847. URL https://arxiv.org/abs/2409.11847 [26] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization (2017). arXiv:1412.6980. URL https://arxiv.org/abs/1412.6980 [27] J. Nocedal, Updating quasi-newton matrices with limited storage, Math. Comp. 35 (151) (1980) 773–782. URL https://courses.grainger.illinois.edu/ece544na/fa2014/nocedal80.pdf [28] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward neural networks, in: Y. W. Teh, M. Titterington (Eds.), Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, Vol. 9 of Proceedings of Machine Learning Research, PMLR, Chia Laguna Resort, Sardinia, Italy, 2010, pp. 249–256. URL https://proceedings.mlr.press/v9/glorot10a.html
18 Appendix A: Hyperparameters
Table A.1 Hyperparameters used for MMPINN Hyperparameter Hidden layers Neurons per layer Residual points Nres Supervised points Nsup Exponent p Exponent q ADAM iterations
Heat Conduction 4.1
Poisson Problem 4.2
ϵ = 0.12
ϵ = 0.11
ϵ = 0.10
ϵ = 0.05
ϵ = 0.02
5 400 90000 9000 3.0 1.0 1000
5 400 100000 12000 4.0 1.0 2000
6 500 200000 21000 4.0 1.0 4000
5 200 30000 6000 3.0 1.0 2000
5 300 50000 8000 4.0 1.0 2000
Flow Eq. 4.3
Maxwell’s Eq. 4.4
5 300 50000 4000 3.0 1.0 4000
8 400 150000 20000 2.0 1.0 10000
Flow Eq. 4.3
Maxwell’s Eq. 4.4
6 100 20000 2000 1.0 1.0 [-6,6] [-6,6] 0.3 3000
10 200 40000 10000 0.1 1.0 [-5,4] [-5,4] [-5,4] 0.1 10000
Flow Eq. 4.3
Maxwell’s Eq. 4.4
20000 2000 1000 1.0 1.0 > 0.5 > 0.95 < 0.25 2
40000 10000 10000 0.1 1.0 < 0.95 < 0.95 < 0.91 5
Table A.2 Hyperparameters used for W-PINN Hyperparameter
Heat Conduction 4.1
Poisson Problem 4.2
ϵ = 0.12
ϵ = 0.11
ϵ = 0.10
ϵ = 0.05
ϵ = 0.02
6 100 20000 3000 0.01 1.0 [-6,5] [-6,6] 0.3 1000
6 100 20000 3000 0.01 1.0 [-6,5] [-6,6] 0.3 2000
6 100 20000 3000 0.01 1.0 [-6,5] [-6,6] 0.3 2000
9 50 10000 4000 0.01 1.0 [-6,6] [-6,6] 0.1 1000
9 50 15000 4000 0.01 1.0 [-6,6] [-6,6] 0.2 2000
Hidden layers Neurons per layer Residual points Nres Supervised points Nsup Residual loss weight ωr Supervised loss weight ωs Resolution Jx Resolution Jy Resolution Jt Translation parameter γ ADAM iterations
Table A.3 Hyperparameters used for AW-PINN Hyperparameter Residual points Nres Supervised points Nsup Pre-trainingADAM iterations Residual loss weight ωr Supervised loss weight ωs Threshold IC Threshold BC Threshold PDE Top κ - %
Heat Conduction 4.1
Poisson Problem 4.2
ϵ = 0.12
ϵ = 0.11
ϵ = 0.10
ϵ = 0.05
ϵ = 0.02
20000 3000 1000 0.01 1.0 > 0.5 < 0.002 < 0.980 8
20000 3000 2000 0.01 1.0 > 0.2 < 0.02 < 0.988 10
20000 3000 3000 0.01 1.0 > 0.4 < 0.003 < 0.984 8
10000 4000 1000 0.01 10.0 > 0.2 < 0.850 2
10000 4000 2000 0.01 10.0 > 0.2 < 0.975 1