Target-Guided Selective Reweighting for Physics-Informed Neural Network Inverse Problems: A Transfer Learning Approach Qian Hua , Bin Fana,∗, Yao Xiaoa , Zhicheng Lina , Meixin Xionga a School of Computing and Data Science, Fujian University of Technology, 350118 Fuzhou, Fujian, China
arXiv:2607.05271v1 [cs.LG] 6 Jul 2026
Abstract Physics-informed neural networks (PINNs) encounter ill-posed optimization, loss competition, and parameter compensation in partial differential equation (PDE) inverse problems. Transfer learning can reuse representations from source tasks, but direct fine-tuning may introduce negative transfer when dominant physical mechanisms, governing parameters, or observation noise differ between source and target domains: the model achieves low field error yet recovers incorrect target physical parameters. To mitigate, we propose Target-Guided Selective Reweighting PINN (TGSR-PINN), a target-evidence-driven representation correction method for PINN inverse transfer learning. TGSR-PINN transfers only the weights and biases from the source PINN, while target physical parameters are independently initialized; after a short target-adaptation phase, the method computes neuron target scores using first-order Taylor sensitivity and pre-activation variance on fixed scoring batches, and converts evidence associated with low-scoring neurons into continuous weak-adaptation signals via a Gaussian mixture model (GMM) with rank fallback. TGSRPINN then applies selective soft decay to input weight rows and biases of low-scoring neurons instead of hard pruning or random resetting. In experiments, TGSR-PINN improves target parameter recovery while maintaining comparable field accuracy in the high-Péclet 2D advection-diffusion task and in the Allen–Cahn to Burgers cross-PDE-family transfer task; a 5%-noise reaction– diffusion case provides supplementary evidence under milder source-target mismatch. Ablation studies suggest that neuron target scoring, weak-adaptation signal estimation, layer protection, and selective soft decay jointly contribute to the benefits. Keywords: Physics-informed neural networks, Partial differential equation inverse problems, Transfer learning, Negative transfer, Selective soft decay, Parameter inversion
1. Introduction Partial differential equations (PDEs) are fundamental mathematical tools for describing continuous physical systems, widely used in fluid mechanics, heat and mass transfer, elasticity, electromagnetic fields, reaction–diffusion systems, and biomedical engineering. Classical numerical methods, such as finite difference, finite element, and finite volume methods, have established mature theoretical and engineering frameworks for solving PDE forward problems. These methods typically require known governing equations, boundary conditions, and physical parameters, and solve for the target physical field through mesh discretization. However, in many practical engineering scenarios, researchers face not only forward problems but also inverse problems that require inferring unknown material properties, diffusion coefficients, velocity parameters, source terms, or boundary conditions from limited, sparse, or noisy observation data. Inverse problems are typically ill-posed and exhibit parameter correlations, where small errors in observational data may lead to significant fluctuations in inversion results, and different parameter combinations may produce similar physical field responses. Consequently, PDE inverse problems are not merely numerical solution tasks but also parameter ∗ Corresponding author
Email address: [email protected] (Bin Fan)
estimation problems influenced by observation noise, physical priors, and optimization procedures. Physics-informed neural networks provide a unified differentiable modeling framework for both forward and inverse PDE problems [1, 2]. PINNs approximate the unknown physical field using a neural network uθ (x, t) and compute PDE residuals through automatic differentiation, incorporating governing equations, initial conditions, boundary conditions, and observational data into the loss function. For inverse problems, unknown physical parameters can serve as trainable variables, optimized simultaneously with network weights and biases, thereby completing field reconstruction and parameter inversion within the same framework. Compared with purely data-driven models, PINNs leverage physical constraints to reduce reliance on large-scale labeled data and are applicable to scenarios with sparse or incomplete observations; meanwhile, the comparative advantages of PINNs over traditional methods such as finite elements in terms of accuracy, computational cost, and problem settings differ [3]. Subsequent research has further integrated PINNs into physics-informed machine learning and scientific machine learning frameworks, advancing their application to complex engineering problems [2, 4, 5]. Recent surveys have also summarized the development and challenges of PINNs from perspectives including loss function design, geometric modeling, and structural engineering applications [6, 7].
Despite their concise formulation and unified modeling advantages, PINN training is not inherently stable. The PINN loss function typically comprises multiple components—PDE residuals, boundary conditions, initial conditions, and observational data errors—which may differ significantly in numerical scale, gradient direction, and convergence speed. Existing studies have analyzed PINN training difficulties and applicability conditions from perspectives of gradient pathology, neural tangent kernel, and theoretical convergence [8, 9, 10], and further identified that training failure modes and complex loss landscapes increase optimization difficulty [11, 12]. In inverse problems, these difficulties are further amplified. On one hand, observational data are often limited and noisy; on the other hand, the parameters to be inverted may suffer from insufficient identifiability or mutual compensation. Even when a model can reduce PDE residuals or field prediction errors, it may not accurately recover the target physical parameters. Particularly in multiparameter inversion tasks, the network representation may compensate for incorrect parameters by adjusting field predictions, yielding seemingly reasonable field results while physical parameters remain far from their true values. To address PINN training difficulties, numerous improvement directions have been proposed, including gradient enhancement and adaptive loss weighting [13, 14, 15], domain decomposition [16, 17], frequency-domain modeling and augmented Lagrangian training [18, 19], and residual decay or loss-attention-based weight balancing methods [20, 21]. These methods primarily focus on improving convergence stability and prediction accuracy on a single task. In contrast, PINN transfer learning, especially for inverse problems, must additionally consider the impact of source task representations on target parameter recovery. For inverse problems, the value of transfer cannot be judged solely by training speed or field error; it must also be assessed by whether the transferred representation affects the quality of target physical parameter recovery. The fundamental idea of transfer learning is to leverage knowledge learned from a source task to assist target task training. When source and target tasks share partial physical structures, geometric features, or solution-space patterns, the network weights and hidden representations from the source model can serve as initialization for the target task, reducing training cost from scratch and potentially improving convergence speed and training stability [22, 23, 24]. Transfer learning has been applied to PINN inverse problems and data-guided inverse problems [23, 24], and extended to structural mechanics simulation, vortex-induced vibration, and complex engineering systems [25, 26]; recent work has also compared full fine-tuning and lightweight fine-tuning PINN transfer strategies [27]. These studies demonstrate that when source and target tasks exhibit strong relevance, the source model can provide an effective starting point for target training. However, in PINN inverse transfer learning, source model representations are not uniformly beneficial. The source model may contain physical structures useful for the target task, but may also carry source-task biases inconsistent with the target physics. When source and target tasks differ in governing parameters, boundary conditions, observation layouts, noise
levels, or dominant physical mechanisms, directly reusing the source model may steer target optimization toward regions detrimental to parameter recovery, thereby creating negative transfer risk [28, 29]. This risk may not be directly observable from field errors. The source model may accelerate target loss reduction and even achieve low field prediction errors, but simultaneously affect parameter recovery through compensation between network representations and physical parameters. Engineering inverse problems such as material characterization, nondestructive testing, and structural load identification all demonstrate that physical parameter recovery is often as important as field reconstruction, or even a more direct task objective [30, 31]. Existing PINN transfer strategies mostly operate at the layer or parameter-block level. For example, full fine-tuning typically copies all network weights and biases from the source model and continues optimization on the target task; frozenlayer methods assume that shallow representations are more general while deep representations are more task-specific; partial-layer transfer selects certain network layers for copying or updating based on preset rules. These strategies are simple to implement and can constitute effective baselines, but their transfer scope is typically predetermined by human judgment before target training, and rarely reassesses whether hidden neurons remain suitable for the current target inverse problem after the model has been exposed to target data. In other words, existing methods more often address “which layers to transfer” or “which parameter blocks to update,” but less often address the finer-grained question after transfer: which hidden neurons receive high values of the neuron target score under current target evidence, and which low-scoring neurons require selective soft decay? Therefore, from a machine learning perspective, PINN inverse transfer is not merely an initialization problem but a representation adaptation problem under target evidence. For inverse problems, coarse-grained transfer is particularly prone to masking negative transfer: network representations can compensate for incorrect physical parameters through field function fitting, causing field errors and parameter errors to decouple. This paper formulates the research problem as “target-evidence-driven representation correction”: without inheriting source-task physical parameters, the method reassesses transferred hidden representations using target loss, target responses, and neuron sensitivity, and reduces the influence of transfer units with insufficient target support on subsequent target optimization through continuous reweighting. This formulation distinguishes TGSRPINN from standard PINN fine-tuning, pruning-based compression, and random resetting methods. This problem is related to neuron importance estimation and pruning research. Taylor saliency and related methods approximate neuron importance through first-order or second-order loss changes and are widely used for model pruning and compression [32, 33, 34]; selective pruning has also been applied to weaken noise-affected representations in PINN inverse problems [35]. However, our scenario differs: low-scoring neurons after neuron target scoring are not necessarily entirely useless, and hard pruning, sparse subnetwork search (e.g., the lot2
tery ticket hypothesis [36]), or random reinitialization may all destroy reusable source representations, introducing additional perturbation to subsequent optimization. Therefore, this paper adopts selective soft decay—using target-side evidence to estimate weak-adaptation signals and progressively reducing the influence of low-scoring neurons, rather than hard removal or resetting. Based on these considerations, we propose Target-Guided Selective Reweighting PINN (TGSR-PINN), a target-side representation correction method for PINN inverse transfer learning. The term “Reweighting” in the method name refers to continuous adjustment of hidden representations through targetside evidence assessment, rather than random reinitialization or hard pruning. The core idea is: the source model transfers only neural network representation parameters; the physical parameters to be inverted are not inherited from the source task but are initialized by the target task protocol and updated jointly with network parameters under the target loss. After representation transfer, TGSR-PINN first performs target short adaptation, allowing the transferred representation to be exposed to target observation data, boundary/initial conditions, and PDE constraints. Subsequently, the method performs neuron target scoring based on target loss, combining Taylor sensitivity, pre-activation variance, and multiple fixed scoring batches to obtain neuron target scores. Further, TGSR-PINN converts neuron target scores into weak-adaptation signals through a GMM with rank fallback. Finally, the method maps weakadaptation signals to continuous decay factors and applies selective soft decay to low-scoring neurons under layer protection constraints. Note: selective soft decay is not hard pruning or random reinitialization—it preserves network topology and only reduces the influence of low-scoring neurons on subsequent target optimization. Through these designs, TGSR-PINN transforms coarsegrained weight reuse in PINN inverse transfer into a targetevidence-driven representation diagnosis and soft correction process. Throughout this paper, the core terminology follows the chain “neuron target score → weak-adaptation signal → low-scoring neuron → selective soft decay.” The framework focuses not only on final field error but also on physical parameter error, and records target-side adaptation dynamics, parameter changes, layer-wise selective soft decay statistics, and weak-adaptation signal diagnostic information, thereby providing a basis for analyzing where and how transfer corrections occur and their impact on parameter recovery. This paper focuses on inverse transfer scenarios with significant source– target differences, strong parameter coupling, and field errors insufficient to fully represent parameter recovery quality, with particular attention to multi-parameter inversion performance in high-Péclet number two-dimensional advection–diffusion inverse problems. The main contributions of this paper are as follows:
biases, while target physical parameters are independently initialized by the target task and updated under the target loss. This design focuses the investigation on the impact of transferred representations on parameter recovery. 2. A neuron-level target scoring mechanism is designed. This mechanism combines first-order Taylor sensitivity and pre-activation variance under the target loss to compute empirical neuron target scores for hidden neurons on fixed scoring batches, providing target-side evidence for subsequent weak-adaptation signal estimation. 3. A weak-adaptation signal estimation procedure combining GMM and rank fallback is developed. This procedure identifies low-scoring neurons from the intra-layer neuron target score distribution, falls back to rank-based estimation when GMM evidence is insufficient, and maps weakadaptation signals to selective soft decay factors. Unlike hard pruning, random resetting, or pure magnitude-based pruning, TGSR-PINN preserves network topology and applies selective soft decay only to the weight rows and biases corresponding to low-scoring neurons. 4. The proposed method is evaluated on high-Péclet number two-dimensional advection–diffusion, Allen–Cahn to Burgers cross-PDE-family transfer, and a 5%-noise reaction–diffusion inverse problem. Mechanism ablation, scoring component scanning, GMM/rank fallback diagnosis, layer-protection analysis, overall intensity matching, selective soft decay mapping ablation, and computational overhead evaluation are further conducted to clarify the empirical advantages and applicability boundaries of TGSR-PINN. The remainder of this paper is organized as follows. Section 2 presents the PDE inverse problem formulation and the overall framework of TGSR-PINN. Section 3 presents the experimental design and results, including high-Péclet advection– diffusion, cross-PDE-family transfer, 5%-noise reaction– diffusion experiments, and high-Péclet mechanism ablation and layer-wise diagnosis. Section 4 discusses the relationship between field and parameter errors, the selective soft decay mechanism, computational overhead, and applicability boundaries. Section 5 concludes the paper and outlines future research directions.
2. Method Figure 1 illustrates the overall pipeline of TGSR-PINN. The method first copies the network weights and biases from the source model and independently initializes target physical parameters; then performs short adaptation on the target task, computing neuron target scores using Taylor sensitivity and pre-activation variance under the target loss; estimates weakadaptation signals via GMM with rank fallback and performs selective soft decay under layer protection constraints; and finally continues from the corrected state to complete target main training.
1. A target-evidence-driven representation correction framework is proposed for PINN inverse transfer learning. Unlike directly transferring the complete source-task state, TGSR-PINN explicitly transfers only network weights and 3
parameters are denoted θT . Together they constitute the trainable variables ΘT = {θT , λT } in the target task and are simultaneously updated under the target loss function. Substituting the neural network prediction uθT (x, t) into the governing equation yields the PDE residual for the target task: r(x, t; ΘT ) = N uθT (x, t); λT − f (x, t). (4) The total loss for the target inverse problem is a weighted sum of components: LT (ΘT ) = ω pde L pde + ωic Lic + ωbc Lbc + ωdata Ldata .
(5)
where ω pde , ωic , ωbc , and ωdata are the weights for each loss term. The focus of this paper is on how the transferred network representation is continuously corrected based on targettask evidence. Section 2.2 will describe how TGSR-PINN reduces the influence of low-scoring neurons on target optimization through selective soft decay, thereby avoiding the excessive perturbation caused by random resetting. 2.2. Overall Framework of TGSR-PINN To avoid terminological confusion, we define TGSR-PINN as Target-Guided Selective Reweighting PINN. Here, TargetGuided means that scoring evidence comes from the target task’s loss, sampling batches, gradients, and pre-activation responses; Selective means that only low-scoring neurons are processed; and selective soft decay / reweighting means that the corresponding weight rows and biases are continuously scaled, rather than randomly reset, hard-pruned, or neurons removed. For consistency, the following sections use “neuron target score” for the target-side evidence obtained from Taylor sensitivity and pre-activation variance, “weak-adaptation signal” for the continuous decay signal derived from neuron target scores, “low-scoring neurons” for the neuron objects to be weakened, and “selective soft decay” for the final weight-row and bias scaling operation.
Figure 1: TGSR-PINN method pipeline.
2.1. PINN Inverse Problem Formulation We first present the general PINN formulation for PDE inverse problems. Consider a physical system defined on a spatial domain Ω and a temporal interval [0, T ]. We directly use spatial coordinates x and time t as network inputs; for steady-state problems, the time variable t can be omitted. Let u(x, t) denote the physical field to be solved and λ denote the physical parameters to be inverted. The general governing equation, boundary conditions, and initial conditions can be written as: N[u(x, t); λ] = f (x, t),
(1)
B[u(x, t); λ] = g(x, t),
(2)
u(x, 0) = u0 (x),
(3)
Notation Summary Numerical stability constants are defined when introduced: ε = 10−6 is used in Eq. (11) and layer-wise normalization; ε s , ε p , and εr are set to 10−12 in Eqs. (13), (16), and (17), respectively. The GMM variance floor is 10−10 . TGSR-PINN starts from the following transfer setting: the source model provides only neural network representation parameters, and target physical parameters are independently initialized by the target task. This setting follows the fundamental principle in transfer learning of reusing transferable representations while avoiding direct inheritance of mismatched task states. Source-task physical parameters, optimizer states, training history, and sampling batches are all not transferred. Let θS∗ denote the network parameters after source training; then the target network is initialized as
where Eq. (1) holds on Ω × (0, T ], Eq. (2) holds on ∂Ω × [0, T ], and Eq. (3) gives the initial state. N[·] denotes the PDE differential operator, B[·] denotes the boundary operator, f (x, t) is the source term, and g(x, t) and u0 (x) denote boundary conditions and initial conditions, respectively. In inverse problems, λ is the unknown parameter to be identified from limited observational data and physical constraints, which may represent diffusion coefficients, reaction coefficients, velocity parameters, material properties, or other physical quantities. PINNs approximate the unknown physical field u(x, t) using a fully connected neural network uθ (x, t), where θ denotes the network parameters including weight matrices and bias vectors for each layer. For the target inverse problem, the physical parameters to be inverted are denoted λT , and the target network
θTtr ← θS∗ ,
Θ0T = {θTtr , λ0T }.
(6)
where λ0T is the initial value of the target physical parameter. This setting ensures that the comparison focuses on transferred 4
representations and their correction mechanisms, rather than direct inheritance of source-task physical parameters. Subsequently, starting from Θ0T , the model performs Ead steps of target short adaptation under the target loss LT . This stage simultaneously updates the transferred network parameters θTtr and the target physical parameters λ0T , yielding the postadaptation state: ad ad 0 Θad T = {θT , λT } = Train ΘT , LT , E ad .
The target short adaptation is positioned as a short training phase before neuron target scoring, whose role is to expose the copied network weights and biases to target observation data, boundary/initial conditions, and PDE constraints. Thus, subsequent neuron target scoring is based on the state Θad T after the model has acquired preliminary target responses, rather than directly on the source-task state. In implementation, the shortadaptation step count is chosen by the target training budget proportion and a short-adaptation upper bound. If this phase is too long or the initial physical parameters deviate severely from true values, the scoring signal may also be contaminated by early erroneous parameter compensation; therefore, we treat the short-adaptation length as a proportional control and report its sensitivity analysis in experiments. After completing target short adaptation, TGSR-PINN temporarily fixes K scoring batches from the target training sampler:
Table 1: Summary of main notation
Symbol
Meaning
θS∗
Source network representation parameters (weights and biases) after source training Target initial network parameters copied from the source model Target physical parameter initial values, independently initialized Target initial state: transferred network parameters and target parameter init State after target short adaptation, containing θTad and λad T Target state after selective soft decay Final target state after main training Target inverse loss (PDE, BC/IC, and data error) Target short adaptation steps and subsequent main training steps Number of scoring batches and k-th scoring batch Pre-activation of neuron j in layer ℓ on batch k Taylor sensitivity and pre-activation variance Per-batch and multi-batch averaged neuron target score Compressed intra-layer neuron target score Posterior responsibility of neuron to GMM low-mean component Weak-adaptation signal (before/after layer protection) GMM two-component separation confidence Shallow protection strength, layer protection coefficient, layer position factor Diagnostic set of neurons effectively affected by selective soft decay in layer ℓ Neuron scaling factor, intermediate and minimum scaling factors Input weight row and bias for neuron j in layer ℓ Base fusion weight and layer-wise increment for Taylor/variance Rank fallback max signal, rank power exponent, selective soft decay threshold
θTtr λ0T Θ0T Θad T ΘTsr Θ∗T LT Ead , Emain K, Bkscore z(k) ℓ, j T ℓ,(k)j , Vℓ,(k)j s(k) ℓ, j , sℓ, j xℓ, j qℓ,L j pℓ, j , e pℓ, j cℓ βh , η ℓ , d ℓ Mℓ γℓ, j , γm , γmin Wℓ, j: , bℓ, j α s , ∆α ξ, ρr , δg
(7)
B score = {B1score , B2score , . . . , BKscore }.
(8)
Each scoring batch is consistent with the sample types in the target loss, including PDE collocation points, boundary condition points, initial condition points, and observation data points. If a task does not include a certain constraint type, the scoring batch does not include the corresponding samples either. Unless overridden by configuration, the experimental implementation defaults to K = 3. During the scoring phase, the model state is fixed at Θad T , i.e., the post-adaptation network parameters θTad and target physical parameters λad T are frozen. This phase performs only forward computation and gradient backpropagation to collect hidden neuron pre-activation responses, target loss gradients, and pre-activation variance information; no optimizer updates are executed. After scoring, these cached batches no longer participate as fixed constraints in subsequent training, and the model continues sampling and optimizing according to the original target training protocol. Consider the j-th hidden neuron in the ℓ-th layer. In our implementation, hooks are registered at the linear output of each fully connected layer, so we denote its pre-activation response on the k-th scoring batch as z(k) ℓ, j (x, t). To approximately measure the neuron’s local contribution to the target loss, we introduce a virtual gating variable gℓ, j , writing the pre-activation as gℓ, j z(k) ℓ, j (x, t). During normal forward computation, gℓ, j = 1; perturbing it to 0 can be approximately understood as suppressing that neuron’s input response. This scoring idea is consistent with Taylor-expansion-based neuron importance estimation for pruning [32]. Based on the first-order Taylor expansion, the 5
Taylor sensitivity on the k-th scoring batch is defined as T ℓ,(k)j = E(x,t)∈Bkscore z(k) ℓ, j (x, t)
∂L(k) T ∂z(k) ℓ, j (x, t)
.
A higher sℓ, j indicates stronger target evidence supporting retention of the neuron; a lower neuron target score indicates weaker target evidence support, and the subsequent continuous weak-adaptation signal will determine the selective soft decay intensity. After obtaining neuron target scores sℓ, j , TGSR-PINN avoids fixed-threshold binary classification because score scales and distribution shapes vary significantly across layers. We convert neuron target scores to weak-adaptation signals pℓ, j : higher sℓ, j means the current target evidence better supports retaining the neuron, while higher pℓ, j means the neuron requires more weakening. To reduce the influence of extreme high scores on distribution estimation, we first apply a compression transform to each layer’s neuron target scores: n o rℓ = median sℓ, j : sℓ, j > 0 + ε s , ! (13) sℓ, j . xℓ, j = log 1 + rℓ
(9)
Taylor sensitivity reflects the first-order sensitivity of the target loss to this neuron’s pre-activation response. A larger value indicates that changes in the neuron’s pre-activation have a more pronounced effect on the current target loss; the absolute value in Eq. (9) prevents positive and negative gradient directions from canceling each other. Relying solely on Taylor sensitivity may be affected by local gradient fluctuations. Some neurons may have large gradients on specific batches but weak response variation on target samples. Therefore, TGSR-PINN simultaneously computes pre-activation variance: Vℓ,(k)j = Var(x,t)∈Bkscore z(k) (x, t) . (10) ℓ, j Here, the variance specifically refers to the pre-activation variance of the linear layer output zℓ, j , not the variance of the postactivation value tanh(zℓ, j ). We use pre-activation statistics because the hooks in our implementation are registered at the linear layer output, and subsequent selective soft decay directly modulates the pre-activation magnitude by scaling Wℓ, j: and bℓ, j . Deep network initialization and normalization research has long focused on the mean and variance of layer inputs or neuron summed inputs to maintain signal propagation and training stability [37, 38, 39]. Thus, pre-activation variance characterizes the linear response variation range of the neuron on target samples and serves as an auxiliary statistic for target sample response activity in neuron target scoring. Since numerical scales may differ across layers, we first normalize T (k) and V (k) within each layer separately, obtaining Te(k) ℓ, j
ℓ, j
where rℓ is the robust scale for layer ℓ and ε s = 10−12 is a numerical stability term. Subsequently, a one-dimensional two-component GMM is fit on each layer’s xℓ, j , with parameters estimated via the EM algorithm [40]: fℓ (x) = πℓ,L N x; µℓ,L , σ2ℓ,L (14) + πℓ,H N x; µℓ,H , σ2ℓ,H . where the lower-mean component L represents the candidate low-scoring-neuron component and the higher-mean component H represents the relatively high-scoring-neuron component. For neuron j, its posterior responsibility to the low-mean component is πℓ,L N xℓ, j ; µℓ,L , σ2ℓ,L L qℓ, j = . (15) fℓ xℓ, j
ℓ, j
e(k) . The implementation uses layer-wise z-score, negativeand V ℓ, j value truncation, and maximum rescaling to bring both types of evidence into [0, 1]. Let Lh denote the number of hidden layers. Before geometric fusion, we define αℓ = α s + ∆α (ℓ − 1)/(Lh − 1) to control the relative weight of Taylor sensitivity and preactivation variance in layer ℓ; for a single hidden layer, we set αℓ = α s + ∆α /2. We then compute the neuron target score for this batch using geometric fusion: αℓ (k) 1−αℓ (k) e +ε e V . (11) s(k) ℓ, j ℓ, j = T ℓ, j + ε
To make it more suitable for subsequent selective soft decay, we calibrate it to obtain the GMM branch’s weak-adaptation signal: L qℓ, j − πℓ,L . pgmm = c · clip , 0, 1 (16) ℓ ℓ, j 1 − πℓ,L + ε p
where ε = 10−6 is the numerical stability term in Eq. (11). The experimental defaults are α s = 0.5 and ∆α = 0.3, so deeper neurons score more toward Taylor sensitivity. Compared with arithmetic averaging, this geometric fusion is more sensitive to the smaller of the two evidence terms: if either Taylor sensitivity or pre-activation variance is weak, the neuron target score is suppressed. This prevents a neuron from being judged strongly target-relevant solely because of a single large gradient or a single large response magnitude. Averaging over K scoring batches yields the neuron target score for neuron j in layer ℓ:
where cℓ ∈ [0, 1] is the separation confidence derived from the two-component overlap and ε p = 10−12 is a numerical stability term; in implementation, cℓ = 1 − BCℓ , where BCℓ is the Bhattacharyya coefficient [41] between the two Gaussian components. The stronger the overlap between the two components, the smaller cℓ becomes, making the weak-adaptation signal from the GMM branch more conservative. In Eq. (16), qℓ,L j −πℓ,L denotes the posterior excess of the low-mean component responsibility over its layer-wise prior weight. Only posterior evidence exceeding this prior is converted into a weak-adaptation signal, which avoids interpreting a large low-scoring component as an anomalous subgroup requiring strong selective soft decay. The GMM branch uses distributional information when
K
sℓ, j =
1 X (k) s . K k=1 ℓ, j
(12) 6
where δg = 0.02 is only a reporting threshold. The actual operation is still the continuous scaling by γℓ, j , rather than hard deletion or random resetting based on Mℓ . Selective soft decay is written as
the intra-layer neuron target score distribution contains a clear low-scoring-neuron cluster, while rank fallback provides a conservative alternative when GMM evidence is insufficient. For implementation, TGSR-PINN fits the GMM to the compressed scores log(1 + s/rℓ ) in Eq. (13). The two component means are initialized at the 25th and 75th percentiles of the compressed scores, the mixture weights are initialized to 0.5, the variance floor is set to 10−10 , and EM runs for at most 100 iterations with convergence declared when the log-likelihood change is below 10−8 . We define ∆BIC = BIC1G −BIC2G , where BIC1G is the BIC of the single-Gaussian model and BIC2G is the BIC of the two-component GMM. Since lower BIC is better, ∆BIC > 0 indicates that the two-component model is preferred over the single-Gaussian model. In implementation, we use ∆BIC > 10 and πℓ,L ≤ 0.5 as the primary conditions for enabling the GMM branch [42]. The constraint πℓ,L ≤ 0.5 ensures that the low-mean component represents a minority group of low-scoring neurons within the layer rather than the majority component. If the conditions are not met, the GMM branch is abandoned and rank fallback is activated: ! nℓ − rankℓ (sℓ, j ) ρr rank . (17) pℓ, j = ξ · nℓ − 1 + εr
Wℓ, j: ← γℓ, j Wℓ, j: ,
ΘTsr = S(Θad T ; {γℓ, j }),
(22)
the model finally continues training to obtain:
dℓ = 1 −
Θ∗T = Train(ΘTsr , LT , Emain ).
(18)
(23)
This completes the full TGSR-PINN pipeline from source representation transfer, target short adaptation, neuron target scoring, weak-adaptation signal estimation, to selective soft decay and target main training.
where βh = 0.55 is the shallow protection strength [29]. For a single-hidden-layer network, we set dℓ = 1. Based on the adjusted weak-adaptation signal e pℓ, j , TGSRPINN maps it directly to a continuous scaling coefficient: e pℓ, j < 0.5, 1 − Aℓ, j (1 − γm ), γℓ, j = γm − Bℓ, j (γm − γmin ), e pℓ, j ≥ 0.5, 3 (19) Aℓ, j = 2e pℓ, j , 3 Bℓ, j = 2e pℓ, j − 1 .
2.3. Algorithm Overview Algorithm 1 summarizes the main steps of TGSR-PINN. 3. Experimental Design and Results 3.1. Overall Protocol, Evaluation Metrics, and Figure Arrangement The baseline methods compared in this paper include:
where 0 ≤ e pℓ, j ≤ 1, γm = 0.85, and γmin = 0.4. This mapping is used as a monotone, continuous, and lower-bounded selective scaling rule; it provides mild selective soft decay for moderate weak-adaptation signals and stronger but still recoverable selective soft decay for pronounced weak-adaptation signals. Its motivation is consistent with soft pruning and soft-threshold sparsification, which preserve network capacity and avoid irreversible deletion [43, 44, 45]. Section 3.5 further compares this default mapping with linear, sigmoid, and hard-threshold alternatives. For reporting and visualization, we define the diagnostic set of effectively softened neurons as Mℓ = { j : 1 − γℓ, j ≥ δg },
(21)
Let zℓ, j = Wℓ, j: hℓ−1 + bℓ, j . At the instant when Eq. (21) is applied, for a fixed previous-layer input hℓ−1 , the updated preactivation satisfies z′ℓ, j = γℓ, j zℓ, j . Therefore, Eq. (21) is consistent with the virtual pre-activation gate used for scoring in Eq. (9) at the pre-activation level: both correspond to multiplicative modulation of zℓ, j . It should be distinguished from a post-activation gate γℓ, j tanh(zℓ, j ) and from outgoing weight scaling in the next layer. Under nonlinear activations such as tanh, scaling the pre-activation is not exactly equivalent to linearly scaling the activation output. We adopt weight-row and bias scaling because it requires no additional network modules or permanent masks and matches the parameterization used in the implementation. Since γmin > 0, Eq. (21) does not remove neurons or alter network topology. The selectively decayed weights and biases remain trainable in subsequent main training, so low-scoring neurons can still recover if later target optimization makes them useful. Denoting the post-decay target state as
where ξ = 0.75, ρr = 1.5, and εr = 10−12 . To prevent excessive intervention on shallow input representations, we first apply layer protection to the weak-adaptation signals. With Lh hidden layers: ℓ−1 , Lh − 1 ηℓ = 1 − βh dℓ , e pℓ, j = clip ηℓ pℓ, j , 0, 1 .
bℓ, j ← γℓ, j bℓ, j .
• PINNs: No source model; target task trained from random initialization; • Full Fine-Tuning: All source weights and biases copied and updated; • Lightweight Fine-Tuning: Earlier layers frozen, only later layers updated; • Partial Transfer: Only selected network layers transferred; • TL-gPINN: Transfer learning combined with gradientenhanced PINN;
(20) 7
• BitFit: Only bias terms are trained. Evaluation metrics include relative L2 field error and average parameter error, both reported as percentages. The former represents the relative L2 error of the predicted physical field with respect to the true solution, while the latter denotes the mean of the relative errors of the physical parameters to be inverted. For multi-parameter inversion tasks, the average parameter error is the arithmetic mean of the absolute relative errors of each physical parameter and is therefore non-negative. Since field error and parameter error may not be consistent in PINN inverse problems, we report both metrics simultaneously and focus on parameter recovery quality in the main experiments. All main experimental tasks employ multiple random repetition protocols. The main text reports aggregated results and key representative repetitions; complete settings and raw results are preserved as reproduction materials. During training, the LBFGS stage uses a fixed-sampling-batch early stopping rule: if the relative improvement in total loss falls below 10−8 for consecutive rounds, that repetition terminates early. This early stopping condition depends only on training loss and does not use true physical parameter errors or test field errors as stopping criteria, thus avoiding leakage of true label information into the training process. To reduce the influence of random initialization, observation sampling, and optimization path differences on conclusions, we adopt a paired comparison protocol based on matched repetitions wherever possible. For different transfer strategies within the same task, if they share the matched initialization protocol, observation data pool, collocation points, and training budget, the final field and parameter errors are treated as paired samples. The main text uses the mean ± standard deviation across repetitions as the primary evidence; the best repetition is used for representative mechanism diagnosis and parameter recovery potential analysis, while core conclusions still rely on aggregated statistics from multiple repetitions.
Algorithm 1: TGSR-PINN Target-Guided Selective Reweighting Pipeline Input: source network θS∗ , target parameter init λ0T , target loss LT , Ead , K, Emain Output: final target state Θ∗T ∗ 0 tr 1 Copy network weights only: θ ← θ ; initialize λ independently S T T (Eq. 6); ad tr 0 2 Train {θ , λ } for E ad target short-adaptation steps to obtain Θ T T T (Eq. 7); ad during scoring 3 Fix K target scoring batches B score and freeze Θ T (Eq. 8); 4 foreach hidden neuron (ℓ, j) do 5 Compute T ℓ,(k)j and Vℓ,(k)j on each scoring batch (Eqs. 9–10); 6
Normalize and fuse them to obtain s(k) , then average over K ℓ, j batches to obtain sℓ, j (Eqs. 11–12);
end foreach hidden layer ℓ do 9 Compress intra-layer scores sℓ, j to xℓ, j and fit a two-component GMM (Eqs. 13–14); 10 if ∆BIC > 10 and πℓ,L ≤ 0.5 then gmm 11 Compute GMM weak-adaptation signals pℓ, j (Eqs. 15–16); 12 else 13 Compute rank-fallback weak-adaptation signals prank ℓ, j (Eq. 17); 14 end 15 Set pℓ, j from the active branch and apply layer protection to obtain e pℓ, j (Eq. 18); 16 Map e pℓ, j to γℓ, j and scale Wℓ, j: and bℓ, j (Eqs. 19–21); 17 end sr ∗ 18 Continue target main training from Θ for E main steps to obtain Θ T T (Eqs. 22–23); ∗ 19 return Θ ; T 7 8
3.2. High-Péclet Number 2D Advection–Diffusion Inverse Problem The source task is a 2D diffusion inverse problem (infer α, 0.5% noise, α init 0.1, true 0.001): ut − α(u xx + uyy ) = QS (x, y, t).
(24)
The target task introduces directional advection on the diffusion structure: ut − α(u xx + uyy ) + v x u x + vy uy = QT (x, y, t), infer
α, v x , vy .
(25)
TheqPéclet number is defined as Pe = UL/α, with L = 1, √ U = v2x + v2y = 5, and α = 0.001, yielding Pe ≈ 2.24 × 103 , indicating a strongly advection-dominated regime. Table 3 shows that TGSR-PINN achieves the lowest average field error and average parameter error among all compared methods in the high-Péclet setting. Compared with Full FineTuning, TGSR-PINN reduces the average parameter error from 8
advection-dominated inverse problems. BitFit’s average parameter error of 7.137% is close to Full FT’s 6.582%, indicating that bias-only adjustment can partially mitigate transfer error; however, its best α repetition is 4.09%, still significantly higher than TGSR-PINN’s 0.38%. This comparison suggests that for physical parameters like α that are deeply coupled with network representation structure, bias-only updates have limited recovery capability, while target-side neuron-level representation correction can more effectively improve parameter inversion quality.
Table 2: High-Péclet 2D advection–diffusion target task settings
Item
Setting
PDE Domain Reference Boundary / IC
ut − α(u xx + uyy ) + v x u x + vy uy = QT Ω = [0, 1]2 , t ∈ [0, 1] Manufactured solution u = e−t sin(πx) sin(πy) Homogeneous Dirichlet boundaries induced by the manufactured solution; u(x, y, 0) = sin(πx) sin(πy) α, v x , vy α = 0.001, v x = 2.0, vy = 1.0 α = 0.005, v x = 1.0, vy = 0.5 0.5% on target observations PDE 10000, BC 2500, IC 2500, data 1000 ω pde = 10, ωic = 10, ωbc = 10, ωdata = 50 MLP [3, 1006 , 1], tanh, LBFGS 300 epochs Pe ≈ 2.24 × 103
Unknown params True params Init params Noise Sampling / batches Loss weights Network Péclet
Multiple random repetition results show that the high-Péclet task is TGSR-PINN’s most concentrated advantage scenario. Among the methods fully comparable in the main experiment, TGSR-PINN achieves the lowest relative L2 field error and lowest average parameter error (L2 0.080%, param 4.426%). BitFit’s field error of 0.184% is also relatively low, and its parameter mean is close to Lightweight FT, indicating that parameterefficient fine-tuning can be competitive in some inverse transfer scenarios; however, its larger cross-repetition fluctuation in α recovery further shows that inverse transfer cannot be evaluated solely by field error, nor by comparing only means while ignoring parameter-level stability. Table 4 further reveals TGSRPINN’s α recovery advantage (best repetition: 0.38%). To supplement the tabular statistics, Figure 2 shows training loss evolution, Figure 3 presents convergence of average parameter error and three individual parameter errors, and Figure 4 compares the true field and absolute error maps at the t = 1 crosssection.
Table 3: High-Péclet 2D advection–diffusion inverse problem results
Method
n
L2 error (%)
Param error (%)
Time (s)
PINNs Full FT Lightweight FT Partial Transfer TL-gPINN BitFit TGSR-PINN
7 7 7 7 7 7 7
0.134 ± 0.038 0.088 ± 0.018 0.087 ± 0.011 0.129 ± 0.027 0.091 ± 0.022 0.184 ± 0.035 0.080 ± 0.016
14.825 ± 6.021 6.582 ± 3.314 4.825 ± 2.384 13.906 ± 4.332 7.430 ± 3.320 7.137 ± 4.754 4.426 ± 3.019
252 235 232 242 552 160 266
6.582% to 4.426% while maintaining a slightly lower field error. Compared with Lightweight FT, the field errors are close, but TGSR-PINN still obtains a lower average parameter error. These results indicate that the proposed target-side representation correction is more effective for parameter recovery than direct fine-tuning or simple layer freezing in this inverse transfer setting. Table 4: Best-repetition α error for each method on the high-Péclet task
Method PINNs Full FT Lightweight FT Partial Transfer TL-gPINN BitFit TGSR-PINN
Best α error (%) 14.63 10.68 4.06 24.25 11.63 4.09 0.38
Figure 2: High-Péclet task training loss curves
The advection velocities v x and vy are non-discriminative for all methods (<0.03%); the real challenge lies in the diffusion coefficient α. Therefore, Table 4 reports only the best α error across random repetitions for each method, to show parameter recovery potential, while the overall performance ranking is based on the multi-repetition mean ± standard deviation in Table 3. TGSR-PINN achieves the best α error of 0.38%, while the best non-TGSR baseline (Lightweight FT) achieves 4.06%. This result indicates that target-side representation correction can help recover key physical parameters in challenging
Figure 2 shows that TGSR-PINN reaches a low training loss while avoiding the unstable plateau observed in TL-gPINN. Although the final training loss alone does not fully determine parameter accuracy, the curve indicates that the proposed softdecay correction does not destabilize the target optimization process. 9
The source task is a 1D Allen–Cahn inverse problem: ut = ε2 u xx + u − u3 ,
infer ε.
(26)
The target task is a 1D Burgers inverse problem: ut + uu x = νu xx ,
infer ν.
(27)
The Allen–Cahn source task uses ε = 0.85 as the initial value, ε = 0.8 as the true value, and 1% observation noise. The Burgers target task uses ν = 0.05 as the initial value, ν = 0.01 as the true value, and 3% observation noise. We use 11 random repetitions with LBFGS training and fixed-batch early stopping. BitFit is included as an additional parameter-efficient baseline to assess whether bias-only adaptation is sufficient under crossPDE-family mechanism mismatch. Figure 3: High-Péclet task parameter error convergence curves Table 5: Allen–Cahn → Burgers cross-PDE transfer results
Figure 3 further shows that TGSR-PINN reduces the mean parameter error and the key diffusion-parameter error more consistently during the later training stage. The velocity parameters are recovered accurately by most methods, whereas the diffusion coefficient remains the main source of difficulty, confirming that the high-Péclet task mainly challenges parameter recovery rather than field reconstruction.
Method PINNs Full FT Lightweight FT Partial Transfer BitFit TGSR-PINN
L2 error (%)
ν error (%)
Time (s)
0.300 ± 0.080 0.260 ± 0.050 0.800 ± 0.440 0.290 ± 0.060 69.010 ± 13.080 0.268 ± 0.058
0.781 ± 0.755 0.568 ± 0.399 12.565 ± 27.800 0.832 ± 0.621 4891 ± 13133 0.408 ± 0.260
95 81 101 92 87 92
Table 5 shows that TGSR-PINN and Full Fine-Tuning achieve the lowest relative L2 field errors at (0.268% ± 0.058%) and (0.260% ± 0.050%), respectively. The small field error difference demonstrates that relying solely on field reconstruction accuracy is insufficient to distinguish parameter recovery quality in cross-PDE transfer. In contrast, TGSR-PINN’s average ν error of (0.408% ± 0.260%) is lower than Full FineTuning’s (0.568% ± 0.399%), a relative reduction of approximately 28.2%. This indicates that neuron target scoring and selective soft decay can improve target physical parameter inversion while maintaining comparable field accuracy. Meanwhile, Lightweight FT and Partial Transfer show significantly increased parameter errors, suggesting that simply freezing partial layers or transferring only selected layers may struggle with deep representation mismatch in cross-PDE-family scenarios. BitFit’s relative L2 field error in Table 5 is (69.010% ± 13.080%), significantly higher than other stable methods. All 11 BitFit repetitions produce finite final field and ν errors; the repetition-level field error range is 47.78%–99.95% (median 69.16%), and the ν error range is 13.88%–46355.98% (median 511.89%). Therefore, BitFit’s CrossPDE results should be interpreted as large-scale training instability or representation adaptation failure. This also indicates that while bias-only updates can be competitive in high-Péclet tasks, in cross-PDEfamily transfer with stronger dominant mechanism differences such as Allen–Cahn to Burgers, bias degrees of freedom are insufficient to stably reconstruct the representations required by the target equation. Figure 5 presents the repetition-level distribution of final ν relative errors across 11 random repetitions. Each scatter
Figure 4: High-Péclet task t = 1 cross-section: true field and absolute error maps
Figure 4 indicates that the absolute error of TGSR-PINN is more concentrated near low-error regions than those of the competing transfer baselines. This visual comparison is consistent with the quantitative results in Table 3, where TGSR-PINN achieves the lowest average field error and parameter error. 3.3. Cross-PDE-Family Transfer: Allen–Cahn → Burgers Inverse Problem The high-Péclet experiment primarily examines transfer from a 2D diffusion source task to a strongly advectiondominated target task. To further test whether the method remains effective when the dominant physical mechanisms differ between source and target, we construct an Allen–Cahn– Burgers cross-PDE-family transfer task. This task requires the model to transfer from a reaction–diffusion-dominated source model to a nonlinear advection and viscous diffusion target inverse problem, making it more likely to expose source representation bias and target parameter compensation issues. 10
point corresponds to one repetition; boxes indicate interquartile ranges, and values in the figure are medians. Due to BitFit’s extremely large ν errors in some repetitions, the vertical axis uses a logarithmic scale. Together with Table 5, this figure shows that TGSR-PINN’s ν errors are more concentrated in the low-error interval, indicating more stable parameter recovery in cross-PDE-family transfer.
Table 6: 5%-noise 2D reaction–diffusion inverse problem (7 repetitions) Method PINNs Full FT Lightweight Partial TL-gPINN TGSR-PINN
Figure 5: Repetition-level distribution of ν relative errors in Allen–Cahn → Burgers cross-PDE transfer
Following the high-Péclet main experiment and cross-PDEfamily transfer, we further examine a dual-parameter inverse problem with milder source–target differences but noisy observation data. The 2D reaction–diffusion target task is:
α, ρ.
Param mean (%)
Param std (%)
Time (s)
0.021 0.033 0.030 0.035 0.034 0.026
1.151 1.378 1.329 1.606 1.308 1.042
1.393 1.341 0.849 1.621 1.589 1.311
192 155 104 176 378 194
Table 6 shows that TGSR-PINN achieves the lowest average field error and average parameter error in this 5%-noise setting, but the field error gap relative to the best non-TGSR baseline is small, and the parameter error standard deviation is relatively large. Since average parameter error is a non-negative absolute relative error, Table 6 reports mean and standard deviation in separate columns to avoid misinterpreting the standard deviation as an error range. This result should therefore be interpreted as supplementary evidence in a noisy dual-parameter scenario rather than as the primary basis for the method’s robustness claim. Figure 6 presents the absolute error maps for the 5%-noise reaction–diffusion task under the same conditions. Figure 6 shows that the general reaction–diffusion field structure can be reconstructed under 5% observation noise, while TGSR-PINN produces a relatively lower absolute error pattern. This supports the quantitative observation in Table 6 that the improvement in this milder noisy setting is positive but less pronounced than in the high-Péclet and cross-PDE transfer cases.
3.4. 2D Reaction–Diffusion Inverse Problem: 5% Noise
infer
L2 std (%)
0.229 0.228 0.238 0.237 0.231 0.219
Figure 6: Reaction–diffusion 5%-noise task: ground-truth field and per-method absolute error maps
Figure 5 reveals that TGSR-PINN produces a more compact low-error distribution of the recovered viscosity parameter. In contrast, BitFit exhibits extremely large outliers, suggesting that bias-only adaptation is insufficient when the source and target PDEs have substantially different dominant mechanisms.
ut − α(u xx + uyy ) + ρu2 = QT (x, y, t),
L2 mean (%)
(28)
3.5. High-Péclet Mechanism Ablation, Scoring Batch Count, and Layer-Wise Diagnosis
This task transfers from a 2D diffusion source task to a target task with a reaction term, inferring diffusion coefficient α and reaction coefficient ρ. Compared to the high-Péclet task, the source–target difference is milder: both tasks are based on diffusion-type dynamics, but the target task additionally includes a nonlinear reaction term, and 5% noise is added to the target observation data only (not to PDE collocation, initial condition, or boundary condition points). Table 6 reports the mean and standard deviation over 7 random repetitions, presenting them in separate columns. Since the source–target difference is mild and the performance improvement is smaller than in the high-Péclet and cross-PDE-family main experiments, we position this as a noisy dual-parameter auxiliary case.
These ablation experiments adopt the same high-Péclet task settings as Section 3.2. Considering that different ablation experiments vary in computational purpose and repetition count, we distinguish their interpretation levels by evidence function and avoid direct mixing with the main experiment results in Table 3. The first category comprises early mechanism observations and stress tests, including target short adaptation/random reset comparison, scoring batch count, data sparsity, and target short adaptation length analysis, primarily used to assess whether components exhibit reasonable trends. The second category consists of paired mechanism diagnoses. The core counterfactual, layer-protection, and selective soft decay mapping ablations use 10 paired repetitions with the same source 11
± 2.377% and Score-Shuffled TGSR’s 5.907% ± 3.574%. In the paired repetition comparison, TGSR-PINN is better in 8/10 pairs in both comparisons; the median paired parameter-error increases of the two counterfactuals are +1.065 percentage points and +2.278 percentage points, with two-sided Wilcoxon p = 0.037 and p = 0.014, respectively. These results indicate that the benefit cannot be attributed to arbitrary soft perturbation alone: target-informed neuron selection and preserving the score–neuron correspondence are both important.
model, target data settings, and training budget. These paired comparisons provide the main ablation evidence reported in this section. Other diagnostics, including the GMM/rank fallback check and the α s scoring-component sweep, are treated as sensitivity or mechanism observations rather than primary evidence for performance gain. 3.5.1. Mechanism ablation: target short adaptation, random reset, and selective soft decay In the mechanism ablation experiment, we compare three settings: Target-Adaptation Only (only target short adaptation, no neuron target scoring or subsequent parameter reorganization), Random Reset (retains neuron target scoring and weakadaptation signal estimation but replaces selective soft decay with random initialization-style resetting), and TGSR-PINN (complete method).
3.5.2. Scoring batch count ablation This subsection presents early diagnostic results on scoring batch count and data sparsity, examining the sensitivity of neuron target scoring to variations in sampling batches and observation data volume. These results serve as mechanism observation and stress testing; the main robustness evidence comes from multi-repetition experiments.
Table 7: Early mechanism ablation on high-Péclet task (3 reps, mean±std)
Method
L2 error (%)
Param error (%)
Target-Adaptation Only Random Reset TGSR-PINN
0.144 ± 0.010 0.168 ± 0.009 0.069 ± 0.006
19.577 ± 1.689 19.457 ± 0.979 2.555 ± 2.024
Table 9: Scoring batch count ablation (single-run diagnostic)
L2 (%)
Param (%)
α (%)
Wilcoxon
TGSR-PINN Random Soft Decay Score-Shuffled
10 10 10
0.074 ± 0.015 0.084 ± 0.012 0.084 ± 0.023
3.782 ± 3.068 5.216 ± 2.377 5.907 ± 3.574
11.312 ± 9.205 15.613 ± 7.122 17.688 ± 10.727
— p = 0.037 p = 0.014
Param error (%)
1 3
0.0693 0.0693
2.662 2.555
Table 10: Data sparsity stress test (single-run, α error / L2 error)
Nd = 1000
Nd = 50
19.1% / 0.09% 19.7% / 0.10% 2.3% / 0.09%
30.6% / 0.12% 16.8% / 0.11% 11.4% / 0.10%
Method PINNs Full FT TGSR-PINN
Table 10 presents a single-run data sparsity stress test. When the observation count drops from Nd = 1000 to Nd = 50, TGSR-PINN’s α error increases from 2.3% to 11.4%, indicating that extreme sparsity weakens parameter recovery; however, compared to PINNs (30.6%) and Full FT (16.8%), TGSRPINN still maintains lower error in this stress test. Table 11: Target short-adaptation length ablation (2 random reps, sensitivity analysis)
Table 8: Paired counterfactual ablation (10 repetitions, mean±std) n
L2 error (%)
Table 9 shows that increasing the scoring batch count from K = 1 to K = 3 leaves the L2 error essentially unchanged while the parameter error decreases slightly from 2.662% to 2.555%. This indicates that multi-batch scoring helps reduce single-batch randomness, yielding a modest improvement under the current protocol.
Table 7 shows that while Target-Adaptation Only exposes the transferred model to the target loss, the average parameter error remains at 19.577%, indicating that target short adaptation alone is insufficient to eliminate the adverse effects of source-side representations. Random Reset’s parameter error of 19.457% is close to Target-Adaptation Only, suggesting that even when low-scoring neurons are located, random initialization-style resetting may destroy partially reusable representations and introduce additional optimization perturbation. In contrast, the full TGSR-PINN reduces the average parameter error to 2.555% while lowering the relative L2 field error to 0.069%. This early diagnostic result indicates that neither target short adaptation nor random resetting can explain the full TGSR-PINN benefit. The subsequent paired ablations further examine specific components including neuron target scoring, neuron correspondence, layer protection, and selective soft decay mapping. To further test whether the full method’s benefit comes from neuron target scoring and neuron correspondence rather than general weight perturbation, we run paired counterfactual ablations over 10 paired repetitions under the same source model, target task, noise level, target data settings, and optimization budget. Results are shown in Table 8.
Method
K
Table 8 shows that TGSR-PINN’s average parameter error is 3.782% ± 3.068%, lower than Random Soft Decay’s 5.216%
Ead
Rep A α error
Rep B α error
Fraction of main
10 20 30 (default) 60
14.6% 15.7% 9.2% 11.4%
12.5% 6.3% 6.2% 6.9%
6.7% 13.3% 20% 40%
Table 11 presents a preliminary sensitivity analysis of target short-adaptation length over 2 random repetitions. Ead = 30 12
(20% of main training) achieves the lowest or comparable α errors in both repetitions, and is therefore adopted as the fixed default setting for subsequent diagnostic experiments. This result indicates that adaptation length affects parameter recovery and provides empirical justification for the default setting; finer task-adaptive step selection is left for future work.
estimation. The core idea is: when the intra-layer neuron target score distribution exhibits a clear separation between lowscoring and high-scoring neuron clusters, the GMM branch exploits distributional information for finer weak-adaptation signals; when the distribution is relatively uniform or twocomponent evidence is insufficient, rank fallback provides a stable conservative alternative.
3.5.3. Scoring component ablation: α s parameter sweep To address the question “why Taylor sensitivity and preactivation variance must be used jointly,” we ablate the scoring components on the high-Péclet task. We sweep α s from 0 to 1 (step 0.1), with 5 random repetitions per setting.
Table 13: GMM and rank fallback ablation (5 reps, mean±std)
α mean
Best
Worst
All-param mean
0.0 0.1 0.2 0.3 0.4 0.5 (default) 0.6 0.7 0.8 0.9 1.0
17.9% 14.4% 29.2% 15.9% 17.4% 13.8% 16.2% 15.2% 13.2% 18.1% 14.6%
1.6% 5.3% 3.2% 0.4% 2.4% 2.3% 1.3% 7.8% 1.6% 8.3% 1.9%
34.4% 30.3% 78.2% 40.7% 29.1% 38.1% 28.0% 35.7% 38.5% 40.5% 33.3%
6.0% 4.8% 9.7% 5.3% 5.8% 4.6% 5.4% 5.1% 4.4% 6.1% 4.9%
n
L2 (%)
Param (%)
α (%)
Wilcoxon
TGSR (GMM+rank) TGSR (rank only)
5 5
0.071 ± 0.007 0.069 ± 0.007
2.944 ± 2.348 3.090 ± 2.380
8.805 ± 7.044 9.239 ± 7.140
— p = 0.593
Table 13 shows that the rank-only version produces field and parameter errors very close to the full TGSR-PINN, with a Wilcoxon test p = 0.593. In the current high-Péclet diagnostic task, most hidden layers’ neuron target score distributions do not exhibit strongly separated bimodal structure (the ∆BIC condition is not met), so the automatic switching mechanism activates the rank fallback path for most layers. Thus, rank fallback is the primary active route in this diagnostic setting, while retaining the GMM branch does not introduce additional perturbation or instability. We therefore do not treat the GMM branch as an independent source of the observed performance gain in this task; instead, its value lies in conditional enhancement— when the target task or network layer’s neuron target score distribution exhibits a clear low-scoring-neuron cluster, the GMM branch can activate and provide finer distributional diagnosis than ranking alone.
Table 12: α s parameter sweep (5 random repetitions)
αs
Method
The α s = 0.5 (joint default) setting achieves a mean α error of 13.8%, falling within the low-error plateau from 0.3 to 0.8; both endpoints perform systematically worse. The α s ∈ [0.3, 0.8] range forms a stable low-error plateau, with the default α s = 0.5 located in its center. Although α s = 0.8 yields a slightly lower mean (13.2%) in this 5-repetition sweep, the difference is within one standard deviation. Combined with the 7-repetition main experiment results for α s = 0.5 (mean α error 13.25%, best 0.38%), we adopt α s = 0.5 as the fixed default. Figure 7 shows the complete α s sweep curve.
3.5.5. Layer protection ablation Table 14: Layer protection ablation (10 reps) Method
n
L2 (%)
Param (%)
α (%)
Avg. selective decay
Wilcoxon
Depth-aware (βh = 0.55) No protection (βh = 0)
10 10
0.074 ± 0.015 0.092 ± 0.023
3.782 ± 3.068 7.478 ± 2.869
11.312 ± 9.205 22.395 ± 8.592
0.044 ± 0.011 0.101 ± 0.040
— p = 0.002
TGSR-PINN controls the selective soft decay intensity at different depths through the layer protection coefficient βh . Intuitively, shallow layers are more likely to contain input encoding and low-order spatial structures and thus receive stronger protection; deeper layers are more likely to carry source-taskrelated local patterns and therefore allow more thorough correction. The overall intensity matching control first computes layer protection coefficients under the default βh = 0.55, then takes a neuron-count-weighted average to obtain a uniform protection coefficient applied to all hidden layers with the same weak-adaptation signal scaling intensity. Table 14 shows that after removing layer protection, the relative L2 field error increases from 0.074% to 0.092%, the average parameter error from 3.782% to 7.478%, and the α error from 11.312% to 22.395%. Paired statistics further show that the noprotection version produces higher average parameter error on all 10/10 paired repetitions, with a median paired increase of +3.437 percentage points and a two-sided Wilcoxon p = 0.002. This result shows that under this high-Péclet diagnostic protocol, completely removing shallow protection significantly in-
Figure 7: α s parameter sweep curve. The α s ∈ [0.3, 0.8] range forms a lowerror plateau, with higher errors at both ends.
3.5.4. GMM and rank fallback ablation TGSR-PINN employs a dual-path design with GMM distribution diagnosis and rank fallback for weak-adaptation signal 13
creases selective soft decay intervention intensity and degrades parameter recovery. An additional overall-intensity-matched diagnostic is used only as supplementary mechanism evidence. It suggests that controlling the total intervention intensity is an important stabilizing factor, whereas the extra contribution of depth-wise allocation is more modest and task-dependent. The applicability boundary of layer protection is taskdependent. Under the high-Péclet diagnostic setting, shallow protection as an empirical stability constraint helps avoid additional perturbation from excessive selective soft decay. The supplementary overall-intensity-matched diagnostic indicates that total intervention intensity is a major stabilizing factor, while the extra effect of depth-wise allocation is modest and not used here as primary positive evidence. For scenarios with stronger source–target differences or severely misaligned target parameter initial values, layer protection strength can be further adapted in a task-specific manner.
performs significantly better than linear, sigmoid, and hardthreshold mappings. In the current high-Péclet diagnostic setting, rank fallback provides the main stable route for weakadaptation signal estimation, while the GMM branch remains a conditional distributional enhancement when clear low-scoring neuron clusters are present. Tables 7, 9, 10, and 11 primarily serve as early mechanism observations; Tables 8, 14, and 15 provide the main 10-repetition paired mechanism evidence.
4. Discussion
4.1. Joint Analysis of Field Error and Parameter Error 3.5.6. Selective soft decay mapping form ablation Table 15: Selective soft decay mapping form ablation (10 reps, mean±std) Mapping
n
L2 (%)
Param (%)
α (%)
Wilcoxon
Linear Piecewise cubic (default) Sigmoid Hard threshold
10 10 10 10
0.095 ± 0.017 0.074 ± 0.015 0.092 ± 0.021 0.093 ± 0.020
7.051 ± 3.795 3.782 ± 3.068 7.361 ± 3.358 6.430 ± 3.283
21.106 ± 11.380 11.312 ± 9.205 22.040 ± 10.077 19.249 ± 9.852
p = 0.002 — p = 0.004 p = 0.027
In PINN inverse problems, physical field errors and physical parameter errors are not always synchronized. The network may compensate for parameter bias by adjusting the field representation, achieving low field prediction errors while still failing to recover the correct target physical parameters. Therefore, this paper uses both relative L2 field error and parameter error as the basis for evaluating inverse transfer effectiveness, rather than judging transfer success solely by field reconstruction accuracy.
Eq. (19) uses a piecewise cubic function to map weakadaptation signals to continuous scaling coefficients. Table 15 shows that under this 10-repetition paired setting, the default piecewise cubic mapping achieves the lowest field error, average parameter error, and α error. Relative to the default mapping, linear, sigmoid, and hard threshold mappings increase average parameter error by 3.269, 3.579, and 2.648 percentage points respectively, and α error by 9.794, 10.728, and 7.937 percentage points respectively, with two-sided Wilcoxon p = 0.002, 0.004, and 0.027. The relatively small field error differences contrasted with pronounced parameter error differences indicate that the selective soft decay mapping form primarily affects subsequent parameter recovery rather than merely changing field reconstruction accuracy. Synthesizing the ablation results in Section 3.5, TGSRPINN’s weak-adaptation signal estimation and correction pipeline can be summarized as: target short adaptation provides the target-side diagnostic state → neuron target scoring through Taylor sensitivity and pre-activation variance (α s = 0.5) → weak-adaptation signal estimation through GMM and rank fallback → low-scoring neuron identification → layer-protected selective soft decay (βh = 0.55) with a bounded decay mapping. These results support the following empirical interpretation: neuron target scoring provides target-side evidence, and the paired counterfactuals show that random soft decay and score shuffling are significantly worse than TGSR-PINN at n = 10; layer protection reduces the risk of shallow-layer overcorrection, with no-protection variants significantly worse on all paired repetitions; and the default piecewise cubic mapping
From the high-Péclet number 2D advection–diffusion and Allen–Cahn → Burgers cross-PDE-family transfer results, TGSR-PINN’s primary value lies in improving target parameter recovery when field accuracy is comparable. In the highPéclet task, the average parameter error gap between TGSRPINN and Lightweight FT is limited, and the average α error is still approximately 13.25%; thus the advantage in this task manifests as a comprehensive performance across physical field error, average parameter error, best α recovery potential, and repetition-level diagnostics. At the same time, TGSR-PINN’s parameter error exhibits a relatively large standard deviation in the high-Péclet task, indicating sensitivity to random initialization, observation sampling, and optimization paths; this paper therefore reports mean, standard deviation, best repetition, and parameter-level errors simultaneously to more fully characterize parameter recovery stability. The Allen–Cahn → Burgers task further indicates that when source and target dominant physical mechanisms differ but field errors are at the same magnitude, neuron target scoring and selective soft decay can still improve target physical parameter recovery. The 5%-noise reaction–diffusion experiment provides supplementary evidence in a noisy dual-parameter scenario under milder source–target differences, but its smaller performance gap and larger parameter-error variance should not be overinterpreted as the main robustness evidence. 14
From an optimization dynamics perspective, PINN inverse problems may exhibit compensatory states where “field prediction is acceptable but parameters are wrong.” When network representations and physical parameters are jointly optimized, certain transferred representations may make subsequent training more likely to enter such compensation regions. Selective soft decay modulates the pre-activation magnitude of lowscoring neurons, changing the initial state of the main training phase, thereby alleviating part of the parameter compensation caused by mismatched transferred representations. Figure 9 shows the parameter-error convergence curves during target training, whereas Figure 10 compares the parameter errors before target short adaptation, after selective soft decay, and at the final training stage. Together, these two figures indicate that selective soft decay mainly influences the subsequent parameter recovery trajectory rather than acting as a simple field-error reduction trick.
Figure 8: Aggregated scatter plot of field error vs. parameter error on the highPéclet task
Figure 8, based on the 7-repetition aggregated results from Table 3, illustrates the potential decoupling between relative L2 field error and average parameter error in the high-Péclet experiment. Each point represents a method’s cross-repetition mean, with horizontal and vertical error bars indicating field error and parameter error standard deviations respectively. The figure highlights that several baselines obtain field errors close to TGSR-PINN, but their parameter errors remain substantially higher. Therefore, for inverse problems where parameter identification is the core objective, reporting only field error is insufficient; parameter-level error and repetition-level stability must be simultaneously analyzed. 4.2. Analysis of Selective Soft Decay As described in Section 2.2, TGSR-PINN implements transfer representation correction through the pipeline of “target short adaptation → neuron target scoring → weak-adaptation signal estimation → layer protection → selective soft decay.” This section discusses why this pipeline may produce benefits and combines ablation results to illustrate the functional boundaries of different components. The mechanism ablation results on the high-Péclet task support the role of selective soft decay. Target-Adaptation Only shows that merely exposing the transferred model briefly to the target loss is insufficient to stably improve parameter recovery; Random Reset shows that strong random resetting of low-scoring neurons may destroy partially reusable source representations and introduce additional optimization perturbation. In contrast, TGSR-PINN employs selective soft decay, reducing the influence of low-scoring neurons while preserving network topology and recoverability. Further 10-repetition paired counterfactual ablation shows that Random Soft Decay and shuffling the score–neuron correspondence significantly increase parameter error. These results indicate that the full method’s benefit cannot be simply attributed to general weight perturbation but more likely arises from the combined effect of target-informed neuron selection, score–neuron correspondence, layer protection, and selective soft decay.
Figure 9: Parameter error convergence curves on the high-Péclet task
Figure 10: Parameter error changes before and after target short adaptation and selective soft decay
15
selective soft decay. Table 16 provides the time and memory overhead for each step. The overhead statistics are based on the same network width and sampling batch settings as the high-Péclet main experiment (PDE 10000, BC 2500, IC 2500, data 1000), on a Windows 11 workstation with AMD Ryzen 5 9600X CPU and NVIDIA GeForce RTX 5070 GPU (12 GB), using Python 3.13.7, PyTorch 2.11.0.dev20260127+cu128, and CUDA 12.8. The reported times cover only the training process and TGSR-PINN additional steps, excluding offline data generation, plotting, final dense-grid evaluation, and result aggregation; these post-processing steps are identical across all methods and are therefore not counted as additional method overhead. All methods use the same batch size settings and the same LBFGS implementation. Each LBFGS outer epoch uses max_iter=20 and history_size=100, with a fixed training batch per LBFGS stage to maintain stable closure evaluation. Training early stopping is triggered only by training loss improvement: 10 consecutive rounds without sufficient improvement or adjacent loss change below 10−8 , without using test field error or true physical parameter error. Since different repetitions may trigger different early stopping rounds, the times in Table 16 should be understood as empirical overhead from representative runs rather than hardware-independent complexity conclusions; complete reproduction materials retain perrepetition runtime and early stopping information. In the highPéclet 2D task, TGSR-PINN’s total training time is 266s (Full FT: 235s), with approximately 13% overhead. It is worth noting that the main training stage of TGSR-PINN is slightly faster than Full FT in this representative run, but this observation should not be interpreted as a general acceleration guarantee across tasks. The cross-PDE task’s overhead proportion is slightly higher (19%), mainly because 1D problems have shorter main training times, making the target short adaptation relatively larger. Therefore, TGSR-PINN’s additional overhead should be understood as the cost of trading limited training time for parameter recovery stability, not as a training acceleration strategy. GMM fitting performs two-component EM estimation on the 1D neuron target score distribution of 100 neurons per layer, with computational complexity independent of the problem’s spatial dimensionality (1D/2D/3D). Under the network scale used in this paper, GMM fitting and selective soft decay consume far less time than target short adaptation and neuron target scoring; therefore, TGSR-PINN’s main computational cost comes from target short adaptation, which can be further compressed by adjusting the adaptation proportion (Ead ). GMM fitting, as a low-overhead distributional diagnosis step following neuron target scoring, provides intra-layer distributional information for weak-adaptation signal estimation. In terms of applicability boundaries, TGSR-PINN is more suitable for inverse transfer scenarios where source and target share partially reusable representations, but direct transfer may introduce parameter recovery bias. The method’s effectiveness depends on the target short adaptation phase providing reasonably reliable target evidence. When the initial target physical parameters deviate severely from true values, or short adapta-
Figure 11: Layer-wise selective soft decay diagnosis for layer protection and overall intensity matching
The design motivation for layer protection is: shallow representations of the source model may contain more general input encoding and low-order spatial structures, while deeper representations are more likely to carry source-task-related local patterns. Therefore, when TGSR-PINN performs selective soft decay, it applies weaker intervention to shallow layers and allows stronger correction for deeper layers. Figure 11 further provides layer-wise diagnostic results for the layer protection ablation: Figure 11(a) compares the layer-wise average selective soft decay intensity under default depth-aware protection, overall intensity matching, and no protection; Figure 11(b) shows the corresponding layer protection coefficients. This result is consistent with Table 14, showing that removing layer protection significantly increases selective soft decay intensity and degrades parameter recovery. The overall intensity matching control is treated as supplementary context: it helps distinguish the effect of total intervention strength from the weaker and taskdependent effect of depth-dependent allocation. TGSR-PINN differs from traditional pruning methods in its objective. Traditional pruning typically serves model compression and inference acceleration, while TGSR-PINN targets representation adaptation in PINN inverse transfer. Selective soft decay does not delete neurons or randomly reset them, but rather reduces the influence of low-scoring neurons on target optimization through continuous decay, improving target-side parameter recovery while retaining reusable source knowledge. The weak-adaptation signal in this paper serves as the continuous basis for selective soft decay, characterizing the lowscoring neurons under the current target loss and target sample response. 4.3. Computational Overhead and Applicability Boundaries
Table 16: TGSR-PINN additional computational overhead breakdown Component Target short adaptation Neuron scoring GMM + selective soft decay Total
Time (s)
Memory (MB)
Notes
∼45 ∼3 ∼2 ∼50
∼234 ∼470 <1 ∼470
20% of main training Forward+backward 1D distribution 13–19% of Full FT
TGSR-PINN introduces three additional steps on top of Full Fine-Tuning: target short adaptation, neuron target scoring, and 16
tion has already entered an incorrect parameter compensation region, neuron target scores and weak-adaptation signals may be contaminated, leading to insufficient parameter updates or parameter locking. Layer protection, GMM/rank fallback estimation, and piecewise cubic selective soft decay have demonstrated empirical effectiveness as a combined pipeline under the current experimental settings; their task-adaptive generalization is one direction for future research. Furthermore, the experiments in this paper are all based on benchmark inverse problem settings with known true values, facilitating parameter error and repetition-level stability assessment; model error, observation bias, unknown boundary conditions, and measurement noise in real engineering data still require dedicated future validation. Adaptive selection of the bounds for selective soft decay (γm and γmin ) is also left as future work.
fallback ablation indicates that rank fallback is the main active path in the current high-Péclet diagnostic setting, while GMM provides conditional distributional enhancement when clear low-scoring neuron clusters exist; the layer protection ablation shows that removing shallow protection significantly degrades parameter recovery; and the selective soft decay mapping ablation suggests that gentle continuous mapping helps reduce perturbation from excessive intervention. Overall, TGSR-PINN transforms coarse-grained weight reuse in PINN inverse transfer into a target-evidence-driven neuron-level representation diagnosis and soft correction process. The framework’s main contribution lies in reassessing hidden representations using target-side evidence after transfer and reducing the influence of low-scoring neurons on target optimization through selective soft decay. Future work may address real engineering inverse problem validation, adaptive target adaptation step count, selective soft decay bound selection, parameter update sufficiency detection, dynamic loss weighting, explicit gating versus weight scaling comparison, multisource transfer, and integration with neural operator methods to enhance TGSR-PINN’s applicability and scalability in more complex engineering inverse problems.
5. Conclusion This paper proposes TGSR-PINN, a target-evidence-driven neuron-level representation adaptation method for PINN inverse transfer learning. The method does not transfer sourcetask physical parameters; instead, after target short adaptation, it computes neuron target scores using Taylor sensitivity and pre-activation variance, estimates weak-adaptation signals through a GMM and rank fallback dual-path mechanism, and applies selective soft decay to low-scoring neurons. Compared with full fine-tuning, frozen-layer transfer, or partiallayer transfer, TGSR-PINN focuses more on the empirical adaptation evidence of transferred hidden representations in target inverse problems and reduces the influence of low-scoring neurons on subsequent target optimization through gentle continuous reweighting. Experimental results show that TGSR-PINN presents empirical advantages in the examined inverse transfer scenarios with significant source–target differences and strong parameter coupling. In the high-Péclet number 2D advection–diffusion task, TGSR-PINN achieves the lowest average field error and average parameter error, and shows the best α parameter recovery potential; this result indicates that target-side representation correction can benefit parameter recovery in challenging advection-dominated inverse problems. In the Allen–Cahn → Burgers cross-PDE-family transfer, TGSR-PINN achieves lower ν parameter error while maintaining field error comparable to Full Fine-Tuning, indicating that target-side representation correction provides benefits for parameter recovery. The 5%-noise reaction–diffusion experiment serves as supplementary validation in a noisy dual-parameter scenario, where the advantage is smaller and should be interpreted as supporting rather than primary evidence. Mechanism ablation and counterfactual ablation show that target short adaptation alone or random resetting is insufficient to replace the full TGSR-PINN; Random Soft Decay and score shuffling are significantly worse than TGSR-PINN in the 10repetition paired counterfactual comparison. The scoring component sweep shows that the joint scoring of Taylor sensitivity and pre-activation variance forms a relatively stable low-error plateau in the intermediate weight range; the GMM and rank
Code Availability The source code, configuration files, and scripts for reproducing the experiments in this paper are publicly available at https://github.com/jooycelee/TGSR-Pinns. The repository includes the TGSR-PINN implementation, benchmark task configurations, and post-processing utilities used to generate the reported tables and figures. Acknowledgements This work is supported by the Foundation of Fujian Provincial Department of Education, China (Grant No. JAT251084 and JAT251087). The authors also thank the anonymous reviewers for their constructive comments that helped improve this paper. References [1] Raissi M, Perdikaris P, Karniadakis G E. Physicsinformed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019, 378: 686–707. [2] Karniadakis G E, Kevrekidis I G, Lu L, et al. Physicsinformed machine learning. Nature Reviews Physics, 2021, 3: 422–440. [3] Grossmann T G, Komorowska U J, Latz J, Schonlieb C B. Can physics-informed neural networks beat the finite element method? IMA Journal of Applied Mathematics, 2024, 89(1): 143–174. 17
[4] Cuomo S, Di Cola V S, Giampaolo F, et al. Scientific machine learning through physics-informed neural networks: Where we are and what’s next. Journal of Scientific Computing, 2022, 92(3): 88.
[17] Moseley B, Markham A, Nissen-Meyer T. Finite basis physics-informed neural networks (FBPINNs). Advances in Computational Mathematics, 2023, 49: 62. [18] Musgrave J, Huang S W. Fourier Domain Physics Informed Neural Network. arXiv:2409.19895, 2024.
[5] Guo J, Zhu H, Yang Y, Guo C. Advances in physicsinformed neural networks for solving complex partial differential equations and their engineering applications. Engineering Applications of Artificial Intelligence, 2025, 161: 112044.
[19] Son H, Cho S W, Hwang H J. Enhanced physics-informed neural networks with augmented Lagrangian relaxation method. Neurocomputing, 2023, 548: 126424. [20] Chen W, Howard A A, Stinis P. Self-adaptive weights based on balanced residual decay rate for PINNs and deep operator networks. Journal of Computational Physics, 2025, 542: 114226.
[6] Plankovskyy S, Tsegelnyk Y, Shyshko N, et al. Review of physics-informed neural networks: Challenges in loss function design and geometric integration. Mathematics, 2025, 13(20): 3289.
[21] Song Y, Wang H, Yang H, Taccari M L, Chen X. Lossattentional physics-informed neural networks. Journal of Computational Physics, 2024, 501: 112781.
[7] Baniya S, Maity D. A comprehensive review of theoretical concepts and advancements in physics-informed neural networks with applications in structural engineering. Artificial Intelligence Review, 2026, 59: 49.
[22] Pan S J, Yang Q. A survey on transfer learning. IEEE TKDE, 2010, 22(10): 1345–1359.
[8] Wang S, Teng Y, Perdikaris P. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 2021, 43(5): A3055–A3081.
[23] Lin S, Chen Y. Gradient-enhanced physics-informed neural networks based on transfer learning for inverse problems. Physica D, 2024, 459: 134023.
[9] Wang S, Yu X, Perdikaris P. When and why PINNs fail to train: A neural tangent kernel perspective. Journal of Computational Physics, 2022, 449: 110768.
[24] Zhou W, Xu Y F. Data-guided physics-informed neural networks for solving inverse problems in PDEs. arXiv:2407.10836, 2024.
[10] Shin Y, Darbon J, Karniadakis G E. On the convergence of physics informed neural networks for linear secondorder elliptic and parabolic type PDEs. Communications in Computational Physics, 2020, 28(5): 2042–2074.
[25] Kapoor T, Wang H, Nunez A, Dollevoet R. Transfer learning for improved generalizability in causal PINNs for beam simulations. Engineering Applications of Artificial Intelligence, 2024, 133: 108085. [26] Tang H, Liao Y, Yang H, Xie L. A transfer learningPINN (TL-PINN) for vortex-induced vibration. Ocean Engineering, 2022, 266: 113101.
[11] Krishnapriyan A S, Gholami A, Zhe S, Kirby R M, Mahoney M W. Characterizing possible failure modes in physics-informed neural networks. NeurIPS, 2021, 34: 26548–26560.
[27] Wang Y, Bai J, Eshaghi M S, et al. Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation. Int J Mechanical System Dynamics, 2025, 5(2): 212–235.
[12] Rathore P, Lei W, Frangella Z, et al. Challenges in training PINNs: A loss landscape perspective. ICML, 2024, 235: 42159–42191.
[28] Liu Y, Liu W, Yan X, Guo S, Zhang C A. Adaptive transfer learning for PINN. Journal of Computational Physics, 2023, 490: 112291.
[13] Yu J, Lu L, Meng X, et al. Gradient-enhanced physicsinformed neural networks for forward and inverse PDE problems. CMAME, 2022, 393: 114823.
[29] Yosinski J, Clune J, Bengio Y, Lipson H. How transferable are features in deep neural networks? NeurIPS, 2014, 27: 3320–3328.
[14] McClenny L, Braga-Neto U. Self-adaptive physicsinformed neural networks. Journal of Computational Physics, 2023, 474: 111722.
[30] Lee S, Popovics J S. Applications of physics-informed neural networks for property characterization of complex materials. RILEM Technical Letters, 2023, 7: 178–188.
[15] Xiang Z, Peng W, Liu X, Yao W. Self-adaptive loss balanced physics-informed neural networks. Neurocomputing, 2022, 496: 11–34.
[31] Shukla K, Di Leoni P C, Blackshire J, Sparkman D, Karniadakis G E. Physics-informed neural network for ultrasound nondestructive quantification of surface breaking cracks. Journal of Nondestructive Evaluation, 2020, 39(3): 61.
[16] Jagtap A D, Kharazmi E, Karniadakis G E. Extended physics-informed neural networks (XPINNs). Communications in Computational Physics, 2020, 28(5): 2002– 2041. 18
[32] Molchanov P, Mallya A, Tyree S, Frosio I, Kautz J. Importance Estimation for Neural Network Pruning. CVPR, 2019: 11264–11272. [33] Han S, Pool J, Tran J, Dally W J. Learning both weights and connections for efficient neural networks. NeurIPS, 2015, 28: 1135–1143. [34] Han S, Mao H, Dally W J. Deep Compression. ICLR, 2016. [35] Chen Y, Chen Y, Guo W, Zhong X. Unlearning noise in PINNs: A selective pruning framework for PDE inverse problems. arXiv:2602.19967, 2026. [36] Frankle J, Carbin M. The Lottery Ticket Hypothesis. ICLR, 2019. [37] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. AISTATS, 2010, PMLR 9: 249–256. [38] Ioffe S, Szegedy C. Batch Normalization. ICML, 2015, PMLR 37: 448–456. [39] Ba J L, Kiros J R, Hinton G E. Layer Normalization. arXiv:1607.06450, 2016. [40] Dempster A P, Laird N M, Rubin D B. Maximum likelihood from incomplete data via the EM algorithm. JRSS B, 1977, 39(1): 1–38. [41] Bhattacharyya A. On a measure of divergence between two statistical populations. Bulletin of the Calcutta Mathematical Society, 1943, 35: 99–109. [42] Schwarz G. Estimating the dimension of a model. The Annals of Statistics, 1978, 6(2): 461–464. [43] He Y, Kang G, Dong X, Fu Y, Yang Y. Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks. IJCAI, 2018: 2234–2240. [44] Kusupati A, Ramanujan V, Somani R, et al. Soft Threshold Weight Reparameterization for Learnable Sparsity. ICML, 2020, PMLR 119: 5544–5555. [45] Sanh V, Wolf T, Rush A M. Movement Pruning: Adaptive Sparsity by Fine-Tuning. NeurIPS, 2020, 33: 20378– 20389.
19