Recovering Governing Equations from Solution Data: Identifiability Bounds for Linear and Nonlinear ODEs

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Recovering Governing Equations from Solution Data: Identifiability Bounds for Linear and Nonlinear ODEs

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

machine learning, deep learning, neural networks

arXiv:2606.27285v1 [cs.LG] 25 Jun 2026

Recovering Governing Equations from Solution Data: Identifiability Bounds for Linear and Nonlinear ODEs Yang Pan [email protected]

Helmut Bölcskei [email protected]

June 26, 2026 Abstract Learning governing equations from observed solution data is a fundamental challenge in scientific machine learning [2, 10, 15, 25, 22], yet the theoretical conditions under which a ground-truth ODE can be uniquely and stably identified from multiple solution observations remain largely undeveloped, and no quantitative analysis of the sample complexity of such learning tasks exists in the literature. To address this gap, we introduce the Hausdorff distance on solution sets as the natural metric for comparing differential equations, since it captures the worst-case separation between two equations over all admissible initial conditions and thus encodes the minimax structure of the identification problem. We establish identifiability bounds for governing ODEs across a wide class of structure equations — ranging from linear ODEs to nonlinear classes with Lipschitz (Hölder)-continuous vector fields — characterizing precisely when two distinct equations can be distinguished from solution data. Using this metric, we derive metric entropy estimates for the relevant ODE classes and analyze sample complexity bounds, quantifying how many solution observations are needed to reliably recover the governing equation.

Introduction

Learning physics (PDEs) and discovering governing equations from solution data is a topic of significant interest [2, 10, 15, 25, 22]. The current literature focuses mostly on algorithms to achieve this goal [2, 25, 15]. Uniqueness issues in finding the equations are discussed in [27, 6, 28] to a limited extent. Specifically, [2, 25, 6] exploit regularization minimization to search for the structure equation with “minimal energy” that fits the solution data, with existence and uniqueness analyzed in [6]. Equivalent and practical conditions for one function (solution) to uniquely determine its underlying differential equation are derived in [27, 28] for linear/polynomial/algebraic PDEs. However, theoretical foundations–under what conditions the ground-truth governing equation can be uniquely and stably identified from multiple solution observations–remain undeveloped. Moreover, a quantitative analysis on the complexity in learning equations, e.g., sample complexity 1

(the amount of solution observations), does not exist in the literature. A theoretical foundation is thus still missing. This paper intends to fill the gaps mentioned above for a wide class of ODEs as a starting point for more work to be done in the future for general PDEs. Specifically, we manage to find out when uniqueness is guaranteed in discovering governing equations and provide a quantitative analysis of complexity in such learning tasks. To make the identification problem precise, we measure the distance between two ODEs by the Hausdorff distance of their solution sets over a prescribed set of initial conditions. This is the natural choice: it captures the worst-case separation between solution trajectories, reflecting the minimax structure of the identification problem. Based on the prescribed distance measure, we set out to consider specific classes of ODEs–linear, Lipschitz, Hölder, and polynomial ODEs, establish identification bounds by proving upper and lower bounds on Hausdorff distance between ODEs by the distance between the underlying structure equations, and then use identification bounds to derive complexity results in learning equations based on sample complexity (i.e., how many samples we need to learn the equation) and metric entropy [32] (how “rich” the class of ODEs is in terms of the Hausdorff distance). We summarize in Table 1 representative examples of lower and upper bounds on the Hausdorff distance between ODEs for several ODE classes considered in this work. ODE class

Hausdorff distance between

Lower bound

Linear ODEs

x9 “ Ax

´ & x9 “ Ãx

¯ A ´ Ã

ˆ Lipschitz ODEs

x9 “ f pxq & x9 “ f˜pxq

Upper bound

f ´ f˜

´ O

2 2

´ O

2 L8

¯ A ´ Ã f ´ f˜

¯ 2 L8

Table 1: Lower and upper bounds on Hausdorff distance within linear ODE class or Lipschitz ODE class. Organization of the paper. The paper is organized as follows. Section 2 reviews related work in the literature. In Section 3, we formalize the setup of learning structure equations of ODEs and introduce Hausdorff distance for ODEs. Section 4 is devoted to identification results for a wide class of ODEs based on Hausdorff distance. In Section 5, we analyze the complexity of learning ODEs in terms of sample complexity and metric entropy.

1.1

Notation

N0 and N` denote the set of natural numbers including and excluding 0, respectively. Rdˆd stands for the set of d by d matrices with real entries. The transpose of the matrix Abis AT . The N ˆ N identity matrix is IN . For the vector x P Rd , we let řd |xi |2 . For the matrix A P Rdˆd , define ∥A∥2 “ sup∥x∥2 “1 ∥Ax∥2 and ∥x∥2 :“ bři“1 ř d d 2 ∥A∥F “ i“1 j“1 |Aij | . Additionally, denote σ1 pAq ě σ2 pAq ě ¨ ¨ ¨ ě σn pAq as the singular values of A. Moreover, λ1 pAq, λ2 pAq, . . . , λn pAq are the eigenvalues 2

of A ordered such that |λ1 pAq| ě |λ2 pAq| ě ¨ ¨ ¨ ě |λn pAq|. ρ pAq :“ |λ1 pAq| is the spectral radius of A. For a nonsingular matrix A, the condition number induced by ∥¨∥ is κ∥¨∥ pAq :“ ∥A∥ ¨ ∥A´1 ∥. logp¨q refers to the logarithm to base 2, logpnq “ log ˝ ¨ ¨ ¨ ˝ log is the n-fold iterated logarithm, and logτ p¨q “ plogp¨qqτ , for τ P R. For a set X Ă R, we ř define C k pXq “ tf : X Ñ R | f has k-th continuous derivativesu and ∥f ∥C k pXq “ ki“0 supxPX |f piq pxq|, where f piq denotes the i-th derivative of f . When k “ 0, we use ∥f ∥L8 pXq instead of ∥f ∥C 0 pXq . For a function f : Rd Ñ R, denote supppf q “ tx P Rd | f pxq ‰ 0u as the support set of f . Let f pϵq and gpϵq, in both cases for ϵ ą 0, be strictly positive for all small enough values of ϵ. We use pϵq pϵq “ 0 and we express lim supϵÑ0 fgpϵq ă8 f pϵq “ o pgpϵqq to indicate that limϵÑ0 fgpϵq by f pϵq “ O pgpϵqq. Moreover, we write f pϵq À gpϵq when f pϵq ď O pgpϵqq, and we write f pϵq — gpϵq when both f pϵq “ O pgpϵqq and gpϵq “ O pf pϵqq. Constants are always understood to be in R unless explicitly stated otherwise.

Related literature

Across science and engineering, “identifying a system” can be viewed as the task of turning observations into a usable representation of how a phenomenon behaves. A common way to frame this topic is the white/grey/black-box spectrum [31]: whitebox models aim to express the internal mechanisms explicitly, grey-box models assume partial mechanistic structure but leave key components or parameters to be learned from data, and black-box models instead learn an input–output relationship that is aimed for prediction or control. A wide range of research topics and literature concerns the problem of “identifying a system”, which can be broadly categorized according to the white-/grey/black-box spectrum introduced above: • The white-box modeling includes – Sparse equation discovery methods, such as SINDy [2, 25], which seek to recover explicit governing equations directly from data by constructing a candidate feature library and promoting sparsity in the identified dynamics. – Symbolic regression approaches [33], including equation-learning systems such as EQL [16, 26], which search over spaces of analytic expressions to produce closed-form, human-interpretable models. • The grey-box modeling includes – Inverse problems based on physics-informed neural networks (PINNs) [21, 4], which incorporate known physical laws—typically expressed as differential-equation constraints or residuals—into the learning objective while estimating unknown functions or parameters from data. – Classical system identification [13] in its structured or parametric formulations (often referred to as grey-box or semi-physical identification), where the model structure is derived from domain knowledge and only a subset of parameters is learned from data. 3

• The black-box modeling includes – Classical system identification [13] in its black-box form, which aims at predictions through flexible input–output models without enforcing mechanistic interpretability. – Operator learning [11, 12, 23], which aims to learn mappings between function spaces (e.g., from input coefficients, forcing terms, or boundary conditions to solution fields) in PDE settings. Although white-, grey-, and black-box models differ in how much structure is prescribed a priori, they share a common objective: to identify system behavior from measurements. In this work, we operate in a hybrid viewpoint: we assume an ODE form x9 “ f pxq, which is a “white-box” at the level of state-space ODE structure, while the choice of hypothesis class for the unknown vector field f could be either a white-, grey-, or black-box model. Moreover, rather than designing a particular white-, grey-, or black-box parametrization for f , our focus is on the underlying identification problem itself: we analyze the uniqueness and stability of the identification process, namely, under what conditions f can be uniquely recovered from solution data, and how sensitive this recovery is to perturbations in the data. We address these questions by developing a theoretical framework that characterizes identifiability and stability independently of any specific parameterization of f . Based on this framework, we further analyze the complexity in the identification problem and motivate a loss function that could be used in identifying governing differential equations based on our frameworks. Depending on the structural assumptions imposed on f , this philosophy naturally applies to white-box, grey-box, or black-box modeling approaches. We hasten to add that questions of uniqueness and stability in system identification have been studied extensively in several mature theoretical fields, most notably in control theory and inverse problems for ODEs and PDEs. In control theory, identifiability is typically defined relative to a prespecified model structure [5, 14]: one assumes that the dynamics (often an input–output or a state-space model) belong to a known parametric family—often linear time-invariant systems, or nonlinear systems with fixed functional form—and asks whether the unknown parameters can be uniquely determined from input–output or state–trajectory data under suitable conditions. In this setting, identifiability is not formulated as the recovery of an arbitrary vector field f from solutions of x9 “ f pxq, but rather as parameter estimation within an assumed class of dynamics. Likewise, the inverse problems for ODEs and PDEs aim to identify the unknown coefficients, source terms, boundary or initial conditions, or other finite- or infinitedimensional parameters appearing in a known governing differential equation [8, 30, 17] and rigorous identifiability results in inverse problems almost always assume that the form of the differential operator is known a priori. 4

However, to the best of our knowledge, there is relatively little theory addressing the identifiability and stability of recovering a general vector field f directly from solution trajectories without assuming a fixed parameterization or model class. Our work is intended to complement these existing theories by studying identifiability at this more intrinsic, representation-independent level.

Problem Setup

In the paper, we consider the problem of identifying the structure f of an ODE ` ˘ xpmq “ f xpm´1q , . . . , xp1q , x (1) given access to its solution data xptq or a set of solutions txptq | x satisfies (1)u. The question is, if the solution data is given, is it possible to uniquely and stably identify f ? Heuristically, if the solution data satisfy different ODEs in nature, then we cannot find an algorithm that recovers the unique ground-truth equation without further restrictions. To formalize the matter, we first introduce the classes of differential equations considered. Then, we define the distance between different differential equations based on the Hausdorff distance between their solution sets, a key notion that lays the foundation for the analysis of the uniqueness and complexity of equation learning for the rest of this paper.

3.1

Classes of Ordinary differential equations

In this paper, we focus on autonomous ODEs of the form ` ˘ xpmq “ f xpm´1q , . . . , xp1q , x .

(2)

To this end, we consider the class of autonomous ODEs in the form of (2) characterized by the class of functions. Definition `3.1. Given d, m P ˘N` , let F be a class of functions f : Rmd Ñ Rd , and let xpmq “ f xpm´1q , . . . , xp1q , x be a differential equation of order m and dimension d. We define the corresponding class of ODEs, denoted as Am pFq, to be the set of all such equations where f P F, that is ␣ ` ˘ ( Am pFq :“ xpmq “ f xpm´1q , . . . , xp1q , x | f P F . (3) When m “ 1, we simply write A pFq. For notational simplicity, we write Of P A pFq as an ODE of the form (2) with structure f P F, unless otherwise specified. There are different types of ODEs depending on the structure f . In this paper, we focus on the following classes of ODEs. Definition 3.2. Given d, m P N` , we consider the class of m-th order, d-dimensional

˘˘ ` ` 1. linear ODEs: Am LinK Rmd , Rd , for K ą 0, where ˜ ¸ ` md d ˘ LinK R , R :“ tf : Rmd Ñ Rd |f pxq “ Ax “ A x m´1 . . . A1 A0 Ai P Rdˆd , ∥Ai ∥2 ď K, i “ 0, 1, . . . , m ´ 1u (4) ` ` md d ˘˘ Specifically, we write OA P Am LinK R , R as an ODE of the form (2) with structure f P F being f pxq “ Ax. ` ` ˘˘ 2. Lipschitz ODEs: Am LipL,K Rmd , Rd , for L, K ą 0, where ˘ ` LipL,K Rmd , Rd :“ tf : Rmd Ñ Rd | Lippf q :“

∥f pxq ´ f pyq∥2 ď L, ∥x ´ y∥2 x‰yPRmd sup

∥f p0q∥2 ď Ku. (5) Moreover, we consider the class of first order, 1-dimensional ¯ ´ k,α , for k P N0 , α P p0, 1s, L, K ą 0, where 1. Hölder ODEs: A CL,K k,α CL,K :“ tf : R Ñ R | f P C k pRq,

2. polynomial ODEs: A pPq,K q where Pq,K :“ tp : R Ñ R | ppxq “ xq `aq´1 xq´1 `¨ ¨ ¨`a0 , |ai | ď K for i “ 0, 1, . . . , q´1u. (7)

3.2

Hausdorff distance for differential equation identification

Now, given prior knowledge about the type of ordinary differential equations (ODEs) N under consideration, and assuming access to solution data tpti , xk pti qqM i“1 uk“1 , can we uniquely and stably identify the governing equation (2)? If identifiability is achievable, what is the sample complexity, i.e., how many data points M, N are required to learn f within a prescribed accuracy ϵ? The first question has been discussed in [25, 27] in the PDE setting, where nonuniqueness issues are found. Specifically, the KdV evolution and the one-way wave equation share a set of solutions. Example 3.1. Consider the Korteweg–De Vries (KdV) equation wt ` 6wwx ` wxxx “ 0. ? It is solved by wpt, xq “ c{2 sech2 p c{2px ´ ct ´ aqq. However, it also solves the one-way wave equation wt ` cwx “ 0. 6

This means that inferring the structure of a PDE from one solution or a set of solutions is potentially an ill-posed problem. The same issue lies in identifying the structure equation of an ODE, as can be seen in the following examples. Example 3.2 (Linear ODE). Let ¨

˚1 0‹ ˚ ‹ ‹, A“˚ ˚ ‹ ˝ ‚ 1 1 ¨ ˛

(8)

˚1 0‹ ˚ ‹ ‹. B“˚ ˚ ‹ ˝ ‚ 5 1 ˜

For x0 “ c 0 1 , where c P R, we have that the solutions of x9 “ Ax and x9 “ Bx with xp0q “ x0 satisfy sup tPr0,T s

eAt x0 ´ eBt x0 2 “ sup

tPr0,T s

et x0 ´ et x0 2 ” 0.

(9)

On the other hand, we have ∥A ´ B∥2 “ 4.

(10)

Example 3.3. Consider Lipschitz functions f, f˜ depicted in Figure 1.

Figure 1: Lipschitz functions f and f˜ We have f ´ f˜

L8 pr´D,Dsq

“ D{6. However, for all |x0 | ď D{3, we have x “ x̃,

where x and x̃ satisfy the ODEs x9 “ f pxq, x9̃ “ f px̃q,

xp0q “ x0 , x̃p0q “ x0 . 7

(11)

The above examples show that identifying the governing equation from solution data in a unique and stable manner is generally ill-posed, even for linear ODEs. The essence of this issue lies in the fact that two differential equations could share a set of solutions. This leads to the question that, if two differential equations share exactly the same solution sets, or their “entire” solution sets are close in terms of some specified distance measure, what is the implication for the structure equation? To address these questions, it is helpful to start with linear autonomous ODEs and ask when the structure matrix A in x9 “ Ax can be identified from solution trajectories. Assume, for simplicity, that A is diagonalizable. Then A is fully determined by its eigenvalues and eigenvectors. Concretely, suppose ˜ ¸ A “ V ΛV ´1 , V “

v1 v2 . . . vn , Λ “ diagpλ1 , λ2 , . . . , λn q.

For an initial condition xp0q “ x0 , the corresponding solution is xptq “ V eΛt V ´1 x0 . From the perspective of identification, recovering A reduces to recovering V and Λ. If V were known, then each eigenvalue λi could be revealed by initializing the system along the i-th eigenvector direction. Indeed, choosing x0 “ vi “ V ei yields xptq “ V eΛt ei “ eλi t vi . So the trajectory is a single exponential mode, and the mode λi can be read off directly. Therefore, if one can “excite” (i.e., initialize) the system at tv1 , v2 , . . . , vn u, the matrix A is identified. This observation also supports a natural uniqueness intuition: the solution set with initial conditions tv1 , v2 , . . . , vn u spans the full solution set of the linear ODE, and so it is reasonable to conjecture that the entire solution set strongly constrains the governing equation of the ODE. A concrete example is the spring (second-order linear) equation x : ` 2ζωn x9 ` ωn2 x “ 0, where ωn is the undamped natural frequency and ζ is the damping ratio. Clearly, the solution to the spring equation can be written as superpositions of complex exponentials, i.e., ¯ ´ ? ¯ ´ ? ´ωn t ζ´ ζ 2 ´1 ´ωn t ζ` ζ 2 ´1 ` Be . x “ Ae 9 T , one can rewrite the dynamics as a Equivalently, by introducing the state z “ px, xq two-dimensional linear ODE z9 “ Az. The two exponential modes above are precisely the eigenvalues of that matrix A, so estimating those exponentials from trajectory 8

data identifies ωn and ζ. In practice, one way to estimate such exponentials from sampled trajectories is to use ESPRIT [24]. Nevertheless, this procedure is fundamentally structure-dependent: it relies on knowing a priori that the system is a linear, autonomous ODE of a fixed dimension, for which the solution decomposes into exponential modes. For general nonlinear ODEs, one typically lacks such a linear modal decomposition, so the questions are: how to characterize and compare the solution sets induced by different (nonlinear) ODEs, and what implication of this comparison has on the governing differential equations of the corresponding ODEs. To this end, we first formally define the notion of solution set (behavior) for ODEs. Definition 3.3 (Solution set (behavior)). Given d, m P N` , let F be a class of functions f : Rmd Ñ Rd , and let Am pFq be the corresponding class of ODEs. Fix T ą 0. For f P F and the corresponding ODE Of P Am pFq, we define its behavior as ` ˘ BpOf q “ tx P pC m pr0, T sqqd | xpmq “ f xpm´1q , . . . , xp1q , x u. Moreover, for the rest of this paper, we consider initial value problems (IVPs) for ODEs and define the behavior with respect to a set of initial conditions I Ă Rmd according to ` ˘ BI pOf q “ tx P pC m pr0, T sqqd |xpmq “ f xpm´1q , . . . , xp1q , x , ´ ` ˘T ` ˘T ¯ T pxp0qqT , xp1q p0q , . . . , xpm´1q p0q P Iu. The following presents an example of behavior in terms of linear ODEs. Example ODEs). Consider A P Rdˆd and a first-order linear ODE ` 3.4` (Linear ˘˘ OA P A LinK Rd , Rd . Then, its behavior with respect to initial conditions I is BI pOA q “ tx P pCpr0, T sqqd | x9 “ Ax, xp0q P Iu “ tx P pCpr0, T sqqd | xptq “ eAt x0 , x0 P Iu.

(12)

Now, we define the Hausdorff distance for sets and the Hausdorff distance for ODEs based on their behavior as follows. Definition 3.4 (Hausdorff distance between sets). Given a metric space pX, ∥¨∥X q and the sets U, V Ă X, we define the Hausdorff distance between U and V as " * hd pU, V ; ∥¨∥X q “ max sup inf ∥u ´ v∥X , sup inf ∥u ´ v∥X . (13) uPU vPV

vPV uPU

Definition 3.5 (Hausdorff distance between ODEs). Given d, m P N` , let F be a class of functions f : Rmd Ñ Rd , and let Am pFq be the corresponding class of ODEs. Fix T ą 0. For every f1 , f2 P F and their corresponding differential equations Of1 , Of2 P Am pFq, we define the solution variation with respect to initial conditions I as ` ˘ svI Ofi , Ofj ; ∥¨∥C m´1 :“ sup inf ∥∥x ´ x̃∥2 ∥C m´1 pr0,T sq for ti, ju “ t1, 2u. x̃PBI pOfj q xPBI pOfi q

(14) 9

The Hausdorff distance between these two equations with respect to initial conditions I is hdI pOf1 , Of2 ; ∥¨∥C m´1 q :“ maxtsvI pOf1 , Of2 ; ∥¨∥C m´1 q , svI pOf2 , Of1 ; ∥¨∥C m´1 qu. (15) One can think of this Hausdorff distance between ODEs as the Hausdorff distance between their behaviors (solution sets), i.e., hdI pOf1 , Of2 ; ∥¨∥C m´1 q “ hd pBI pOf1 q, BI pOf2 q; ∥¨∥C m´1 q .

(16)

For ease of notation, we abuse the notation of Hausdorff distance for ODEs and Hausdorff distance for sets. In the rest of this paper, we set T ą 0 to be a universal constant characterizing the horizon for ODEs. For simplicity, domain r0, T s and vector norm ∥¨∥2 is omitted in the notation of Hausdorff distance and write hd p¨, ¯ ¨; ∥¨∥L8 q or ´ ¯ we simply ´ hd p¨, ¨; ∥¨∥C k q instead of hd ¨, ¨; ∥∥¨∥2 ∥L8 pr0,T sq or hd ¨, ¨; ∥∥¨∥2 ∥C k pr0,T sq .

Identification results for classes of ODEs

In this section, we examine the relationship between the Hausdorff distance in ODEs and distances in structure equations. Practically speaking, if small Hausdorff distances in ODEs imply small distances in structure equations, i.e., ∥f1 ´ f2 ∥ ď δphd pOf1 , Of2 qq where δp0q “ 0 and lim` δpaq “ 0,

(17)

aÑ0

we can develop an algorithm that minimizes the Hausdorff distances between the solution sets of the model ODE Of2 and the ground-truth ODE Of1 , which then ensures the possibility of a unique and stable identification of the structure f1 .

4.1

First-order linear ODEs

` ` ˘˘ We first consider the class of 1st order, d-dimensional linear ODEs A LinK Rd , Rd as per Definition 3.2. Note that the solutions of linear ODEs are generally stable with respect to the perturbations in structure matrices. This leads to the following upper bound on the Hausdorff distance between linear ODEs. Lemma 4.1 (Upper bound). Fix an arbitrary D ą 0 and consider ` ` BD :“ ˘˘ tx P Rd | ∥x∥2 ď Du. For every two linear ODEs OA , OÃ P A LinK Rd , Rd with corresponding A, Ã P Rdˆd , we have that hdBD pOA , OÃ ; ∥¨∥L8 q ď DT eKT A ´ Ã

(18)

Proof. By applying the matrix exponent inequality from Lemma B.7, we have that eÃt x̃0 ´ eAt x̃0

ď A ´ Ã

DT eKT 2

(19)

for all t P r0, T s and all x̃0 such that ∥x̃0 ∥ ď D. This implies that svBD pOA , OÃ ; ∥¨∥L8 q “ “

sup

inf

x̃PBBD pOÃ q xPBBD pOA q

sup

inf

∥x̃0 ∥ ďD ∥x0 ∥2 ďD

∥∥x ´ x̃∥2 ∥L8 pr0,T sq

eA¨ x0 ´ eÃ¨ x̃0

ď sup

A¨

(20)

Ã¨

e x̃0 ´ e x̃0

∥x̃0 ∥ďD

ď DT eKT A ´ Ã

2 L8 pr0,T sq

. 2

By symmetry, this gives the desired bound for hdBD pOA , OÃ ; ∥¨∥L8 q. Next, we obtain a lower bound on the Hausdorff distance between linear ODEs with respect to the distance between structure matrices, which, according to the beginning of Section 4, implies identification results. Lemma 4.2 (Lower bound). Fix an arbitrary D ą 0 and consider ˘˘ tx P ` ` BD :“ Rd | ∥x∥2 ď Du. For every two linear ODEs OA , OÃ P A LinK Rd , Rd with corresponding A, Ã P Rdˆd , we have that ´ ¯ (˜ ␣ ¸ 12 1 1 1 2 ´ e 2 D min T, 2K e´5 mintT, 2K uK ? A ´ Ã hdBD pOA , OÃ ; ∥¨∥L8 q ě 1 2 2 1 ` e´5 mintT, 2K uK (21) Proof. For arbitrarily fixed t0 P p0, T s and x0 , x̃0 P BD , set B :“ At0 , B̃ :“ Ãt0 , y :“ x0 ´ x̃0 and b :“ peB ´ eB̃ qx̃0 . Then, ∥B∥2 “ B T 2 ď t0 K. Note that for arbitrarily fixed x̃0 P BD , we have ) ! inf eA¨ x0 ´ eÃ¨ x̃0 ě inf max ∥x0 ´ x̃0 ∥2 , eAt0 x0 ´ eÃt0 x̃0 ∥x0 ∥ďD

2 L8

∥x0 ∥ďD

˙ 21

2 1 ě ? inf ∥x0 ´ x̃0 ∥22 ` eB x0 ´ eB̃ x̃0 2 2 x0 PRd 1 ´ ¯ 1 2 2 “ ? inf ∥y∥22 ` eB y ` b 2 2 yPRd ´ ¯ 12 1 T BT B T B T ? “ inf y pI ` e e qy ` 2b e y ` b b 2 yPRd ¯ ¯ 12 1 ´ T´ B B T B ´1 B T “ ? b I ´ e pI ` e e q e b , 2

(22)

where the last equality holds since we are minimizing a quadratic form and I `eB eB is a symmetric positive definite matrix. Note that ¯´1 ´ T T T I ´ eB pI ` eB eB q´1 eB “ I ´ I ` e´B e´B T

is a symmetric matrix and thus only has real eigenvalues. Moreover, e´B e´B is a symmetric positive definite matrix and by Lemma B.8 (noting that AT 2 “ ∥A∥2 ď K and replacing A with ´B), T

λn pe´B e´B q ě e´5t0 K . 11

(23)

Now, note that ˆ ´ ¯´1 ˙ ´ ´ ¯¯´1 T ´B T ´B λn I ´ I ` e e “ 1´ 1 ` λi e´B e´B

for some i P t1, 2, . . . , nu,

which means ˆ λn

´ ¯´1 ˙ ´ ´ ¯¯´1 ´B T ´B ´B T ´B I ´ I `e e ě 1 ´ 1 ` λn e e (23)

(24)

` ˘´1 ě 1 ´ 1 ` e´5t0 K

“

e´5t0 K . 1 ` e´5t0 K

Thus, we have svBD pOA , OÃ ; ∥¨∥L8 q “ “

sup

inf

x̃PBBD pOÃ q xPBBD pOA q

sup

∥∥x ´ x̃∥2 ∥L8 pr0,T sq

eA¨ x0 ´ eÃ¨ x̃0

inf

∥x̃0 ∥ ďD ∥x0 ∥2 ďD 2

2 L8 pr0,T sq

´ ´ ¯ ¯ 12 1 T T bT I ´ eB pI ` eB eB q´1 eB b ě ? sup 2 ∥x̃0 ∥2 ďD ˆ ´5t0 K ˙ 12 (24) 1 e ě ? sup peAt0 ´ eÃt0 qx̃0 ´5t0 K 1 ` e 2 2 ∥x̃0 ∥2 ďD ˆ ´5t0 K ˙ 12 e D “? sup peAt0 ´ eÃt0 qx̃0 ´5t0 K 1 ` e 2 2 ∥x̃0 ∥2 ď1 ˆ ´5t0 K ˙ 12 e D eAt0 ´ eÃt0 . “? 2 2 1 ` e´5t0 K (22)

Next, we set out to find a t0 such that eAt0 ´ eÃt0

(25)

can be lower-bounded by a 2

positive scaling of A ´ Ã . Note that 2

At0

´e

Ãt0

8 ¯ ÿ 1 ´ “ pAt0 qk ´ pÃt0 qk k! 2 k“0

ě A ´ Ã Lemma B.6

t0 ´

“ A ´ Ã

8 ÿ k k“2

t0 2 ´ e

tk0

A ´ Ã

t0 ´ A ´ Ã `

t0 ´

Ak ´ Ãk

k“2

A ´ Ã

ě A ´ Ã

8 ÿ

˘ Kt0

8 ÿ 1 k“1

! )k´1 (26) max ∥A∥2 , Ã tk0

K k tk0

( ␣ 1 , then If we choose t0 “ min T, 2K e

At0

´e

Ãt0

" * ¯ 1 1 ´ 2 ě min T, 2´e A ´ Ã . 2K 2 2

(27)

Combining (25) and (27), we have ´ ¯ (˜ ␣ ¸ 21 1 1 1 2 ´ e 2 D min T, 2K e´5 mintT, 2K uK ? svBD pOA , OÃ ; ∥¨∥L8 q ě A ´ Ã . 1 2 2 1 ` e´5 mintT, 2K uK By symmetry, this gives the desired bound for hdBD pOA , OÃ ; ∥¨∥L8 q. In summary, we get the following bounds for Hausdorff distance between linear ODEs. Theorem 4.1. Fix an arbitrary D ą 0 `and consider BD :“ tx P Rd | ∥x∥2 ď Du. ` d d ˘˘ For every two linear ODEs OA , OÃ P A LinK R , R with corresponding A, Ã P dˆd R , we have that c A ´ Ã ď hdBD pOA , OÃ ; ∥¨∥L8 q ď C A ´ Ã , 2 looooooomooooooon2 looooooomooooooon

(28)

Structural stability

Identification stability

where c, C ą 0 are constants that only depend on T, K, D. The lower bound in Theorem 4.1 implies that the closeness in solution sets in terms of Hausdorff distance guarantees closeness in the corresponding structure equation, which, in turn, means that we can identify the governing equation of an ODE uniquely and stably based on the full knowledge of its solution set. Note that there are generally no conditions on the size of the ball BD , this means even a small perturbation region in initial conditions is enough for us to identify the linear ODE based on the information of the solution set. Nevertheless, a small ball in Rd is still a set with infinite cardinality. The question is, what is the least amount of solution data we need to identify the structure? Recall that in Section 3.2 an example was given in which the structure matrix of a linear ODE is diagonalizable. In that setting, having initial conditions that span the state space is sufficient to identify the system. This motivates the assumption that, for linear ODEs with general structure matrices, “completeness” of the initialcondition set (in the sense of spanning the state space) should likewise be sufficient for identification. To allow additional flexibility, it is convenient to work not only with bases but also with spanning, possibly redundant generating sets —namely, a frame—and, without disrupting the flow of the present paper, refer the reader to a short overview of basic frame theory in [1]. The following lemmata formalize and justify the above intuition. Lemma 4.3. Suppose G “ tg1 , g2 , . . . , gN u is a frame for Rd and tg̃1 , g̃2 , . . . , g̃N u is its corresponding canonical dual frame, such that there exist F1 , F2 with 0 ă F1 ď F2 , F1 ∥x∥22 ď

N ÿ

|xx, gk y|2 ď F2 ∥x∥22 ,

k“1 N ÿ

1 1 |xx, g̃k y|2 ď ∥x∥22 ď ∥x∥22 . F2 F1 k“1 13

˘˘ ` ` For every two linear ODEs OA , OÃ P A LinK Rd , Rd with corresponding A, Ã P Rdˆd , we have that ´ ¯? ␣ (˜ ¸ 12 1 1 1 2 ´ e2 F1 min T, 2K e´5 mintT, 2K uK ? hdG pOA , OÃ ; ∥¨∥L8 q ě A ´ Ã . 1 2 2N 1 ` e´5 mintT, 2K uK (29) Proof. The proof follows similar lines as that of Lemma 4.2. We defer it to Appendix D.1. From Lemma 4.3, it’s enough to identify the structure matrix of a linear ODE if the set of initial values is complete in the sense that it contains a basis of Rd . The following lemma shows that a basis is, in turn, necessary in identifying structure matrices for linear ODEs. Lemma 4.4. Given G “ tg1 , g2 . . . , gN u such that spantg1 , g2 , . . . , gN u ‰ Rd , there exist A, B P` Rdˆd ,` such that ˘˘ ∥A ´ B∥2 ą 0 and the corresponding linear ODEs d d satisfy hdG pOA , OB ; ∥¨∥L8 q “ 0. OA , OB P A LinK R , R Proof. Since the solutions of linear ODEs with constant coefficient in linear with respect to initial values, we can assume, without loss of generality, that N ă d and g1 , g2 . . . , gN is linearly independent. Now, we extend g1 , g2 . . . , gN to g1 , g2 , . . . , gN , gN `1 , . . . , gd such that spantg1 , g2 , . . . , gd u “ Rd . Moreover, define ˜ ¸ T “

g1 g2 . . . gd , ¨ ˛

˚IN ˚ Λ“˚ ˚ ˝

2Id´N

‹ ‹ ‹. ‹ ‚

Now, consider A “ T ΛT ´1 and B “ Id . We have Agi “ Bgi “ gi ,

for all i ď N,

which implies that hdG pOA , OB ; ∥¨∥L8 q “ 0. In the meanwhile, we have ∥A ´ B∥2

Lemma B.1

1 ? ∥A ´ B∥F d

1 |trpA ´ Bq| d ˇ 1ˇ “ ˇtrpT pΛ ´ Id qT ´1 qˇ d 1 “ |trpΛ ´ Id q| d d´N “ ą 0, d ě

which concludes the proof. 14

4.2

First-order nonlinear ODEs: Lipschitz ODEs

In this section, we consider the most regular nonlinear ODEs—Lipschitz ODEs. In particular, ` ` wed first ˘˘ consider the class of 1st order, d-dimensional Lipschitz ODEs d A LipL,K R , R . Since the solutions of Lipschitz ODEs are generally stable with respect to the perturbations in structure equations, we immediately have the following upper bound on the Hausdorff distance between Lipschitz ODEs. Lemma 4.5 (Upper bound). Fix an arbitrary D ą 0 and a compact set Q Ă p :“ tx P Rd | ∥x∥ ď pKT ` DqeLT u. For BD :“ tx P Rd | ∥x∥2 ď Du. Let Q ` ` ˘˘ 2 every two Lipschitz ODEs Of , Of˜ P A LipL,K Rd , Rd with corresponding f, f˜ P ` ˘ LipL,K Rd , Rd , we have that ˘ ` hdQ Of , Of˜; ∥¨∥L8 ď T eLT

f ´ f˜

(30)

p 2 L8 pQq

Proof. Let x̃ P BQ pOf˜q be a solution to Of˜. We then have żt f˜px̃psqqds,

x̃ptq “ x0 `

(31)

where x0 P Q. Taking the norm, noting that Lippf q ď L, ∥f p0q∥2 ď K, ∥x0 ∥2 ď D we have żt ∥x̃ptq∥2 ď L

∥x̃psq∥2 ds ` Kt ` D.

(32)

Appling Grönwall’s inequality, we obtain ∥x̃ptq∥2 ď pKt ` DqeLt ď pKT ` DqeLT

(33)

Now, consider the solution x to Of with xp0q “ x̃p0q P Q and define y :“ x ´ x̃. Then, y satisfy the ODE 9 “ f pyptq ` x̃ptqq ´ f˜px̃ptqq yptq

(34)

with yp0q “ 0. Taking the integral and bounding the norm, we have żt ∥yptq∥2 ď f pypsq ` x̃psqq ´ f˜px̃psqq ds 2 0 żt ď ∥f pypsq ` x̃psqq ´ f px̃psqq∥2 ds 0 żt ` f px̃psqq ´ f˜px̃psqq ds 2 ż 0t ď L ∥ypsq∥2 ds ` f ´ f˜ t.

(35)

p 2 L8 pQq

Applying Grönwall’s lemma and take supreme over t, we obtain ∥∥y∥2 ∥L8 pr0,T sq ď T eLT 15

f ´ f˜

. p 2 L8 pQq

(36)

This implies that inf xPBQ pOf q

∥∥x ´ x̃∥2 ∥L8 pr0,T sq ď ∥∥y∥2 ∥L8 pr0,T sq ď T eLT

f ´ f˜

(37)

p 2 L8 pQq

Since this holds for arbitrarily chosen x̃ P BQ pOf˜q, we have ˘ ` svQ Of , Of˜; ∥¨∥L8 ď T eLT

f ´ f˜

(38)

p 2 L8 pQq

and thus, by symmetry, the desired result. Next, we obtain a lower bound on the Hausdorff distance between Lipschitz ODEs, which will then be used to obtain identification results. Lemma 4.6. Fix an arbitrary D ą 0 and a compact set `Q Ă BD˘˘:“ tx P Rd | ∥x∥2 ď ` Du. For every two Lipschitz ODEs Of , Of˜ P A LipL,K Rd , Rd with corresponding ˘ ` f, f˜ P LipL,K Rd , Rd , we have $ , ˜ ’ / f ´ f & . f ´ f˜ ` ˘ 2 L8 pQq 2 L8 pQq ? ? hdQ Of , Of˜; ∥¨∥L8 ě min ,T . LT 2 LT ’ - 2 dp2 ` LT q % 2 dL p2DLT e ` KLT e ` 2Kq / (39) Proof. First, we pick x0 P Q such that f px0 q ´ f˜px0 q

“

f ´ f˜

2 L8 pQq

thanks to the continuity of f, f˜ and compactness of Q. Now, pick x̃ P BQ pOf˜q such that x̃p0q “ x0 . We then have żt (40) x̃ptq “ x0 ` f˜px̃psqqds. 0

Taking the norm, noting that Lippf q ď L, ∥f p0q∥2 ď K, we have żt ∥x̃ptq ´ x0 ∥2 ď L ∥x̃psq∥2 ds ` Kt.

(41)

Defining żt yptq “ e

´Lt

∥x̃psq∥2 ds,

(42)

we have that yp0q “ 0 and ˆ ˙ żt ´Lt 9 “e yptq ∥x̃ptq∥2 ´ L ∥x̃psq∥2 ds ď e´Lt p∥x0 ∥2 ` Ktq .

(43)

Integrating from 0 to t, we obtain ` ˘ K 1 ´ e´Lt p1 ` Ltq 1 ´ e´Lt yptq ď ∥x0 ∥2 ` , L L2 16

(44)

which, together with (41) and (42), implies that ˜ ` ˘¸ ´Lt ´Lt K 1 ´ e p1 ` Ltq 1 ´ e ∥x̃ptq ´ x0 ∥2 ď LeLt ∥x0 ∥2 ` ` Kt L L2 K ď peLt ´ 1q ∥x0 ∥2 ` peLt ´ 1 ´ Ltq ` Kt ˙ L ˆ 2 LT KLT e ` K t. ď DLT eLT ` 2

(45)

Setting $ ’ & t0 :“ min

f ´ f˜

, / .

L pQq ? ,T , LT ’ % 2 dL p2DLT e ` KLT 2 eLT ` 2Kq / -

(46)

we obtain ∥x̃ptq ´ x0 ∥2 ď

f ´ f˜ 8 2 ? L pQq , 4 dL

for t P r0, t0 s.

Define h :“ f ´ f˜. Noting that x0 is picked such that ∥hpx0 q∥2 “

(47) f ´ f˜

, 2 L8 pQq

there must exits j P t1, 2, . . . , du satisfying

|phpx0 qqj | ě

f ´ f˜ 8 2 ? L pQq . d

(48)

Since Lipphq ď 2L, we have |phpx̃ptqqqj ´ phpx0 qqj | ď ∥hpx̃ptqq ´ hpx0 q∥2 ď 2L ∥x̃ptq ´ x0 ∥2 , and thus

|phpx̃ptqqqj | ě |phpx0 qqj | ´ 2L ∥x̃ptq ´ x0 ∥2

f ´ f˜ f ´ f˜ 8 2 L8 pQq 2 ? ? L pQq ě ´ 2L d 4 dL f ´ f˜ 8 2 ? L pQq , for t P r0, t0 s. “ 2 d Now, for every x P BQ pOf q, define y “ x ´ x̃. Then, y satisfy the ODE yptq 9 “ f pyptq ` x̃ptqq ´ f˜px̃ptqq.

(49)

(50)

Integrating from 0 to t, we have żt żt´ ¯ yptq “ yp0q ` pf pypsq ` x̃psqq ´ f px̃psqqq ds ` f px̃psqq ´ f˜px̃psqq ds. (51) 0

Taking the absolute value, applying triangle inequality, and noticing Lippf q ď L gives żt´ żt ¯ ˜ ∥∥y∥2 ∥L8 ě ∥yptq∥2 ě f px̃psqq ´ f px̃psqq ds ´ ∥yp0q∥2 ´ L ∥ypsq∥2 ds 0 0 2 żt´ ¯ f px̃psqq ´ f˜px̃psqq ds ´ p1 ` LT q ∥∥y∥2 ∥L8 , ě 0

(52) which in turn gives 1 ∥∥y∥2 ∥L8 ě 2 ` LT

żt hpx̃psqqds 0

for all t P r0, T s.

(53)

By (49), since ph ˝ x̃qj is continuous, phpx̃ptqqqj must remain positive or negative for t P r0, t0 s. In either case, we have ż t0 (53),t“t0 1 ∥∥x ´ x̃∥2 ∥L8 “ ∥∥y∥2 ∥L8 ě hpx̃psqqds 2 ` LT 0 2 ˇˆ ż ˙ ˇˇ ˇ t0 1 ˇ ˇ hpx̃psqqds ˇ ě ˇ ˇ 2 ` LT ˇ 0 j f ´ f˜ 8 t0 2 ? L pQq ě 2 ` LT 2 d $ , ’ / f ´ f˜ & . 2 L8 pQq ? ě min ,T ’ % 2 dL p2DLT eLT ` KLT 2 eLT ` 2Kq / -

(49)

f ´ f˜ 2 L8 pQq ? . 2 dp2 ` LT q

Since this holds for arbitrary x P BQ pOf q, we obtain $ , ’ / f ´ f˜ & . 2 L8 pQq ? inf ∥∥x ´ x̃∥2 ∥L8 ě min ,T ’ xPBQ pOf q % 2 dL p2DLT eLT ` KLT 2 eLT ` 2Kq / -

f ´ f˜ 2 L8 pQq ? . 2 dp2 ` LT q

Taking the supremum over BQ pOf˜q and by symmetry, we arrive at the desired result. In summary, we get the following bounds on the Hausdorff distance between Lipschitz ODEs. Theorem 4.2. Fix an arbitrary D ą 0 and a compact set Q Ă BD :“ tx P Rd | p :“ tx P Rd | ∥x∥ ď pKT ` DqeLT u. For every two Lipschitz ∥x∥2 ď Du. Let Q ` ` ˘˘ 2 ` ˘ ODEs Of , Of˜ P A LipL,K Rd , Rd with corresponding f, f˜ P LipL,K Rd , Rd , we have that " * 2 ` ˘ ˜ ˜ min C1 f ´ f , C2 f ´ f ď hdQ Of , Of˜; ∥¨∥L8 2 L8 pQq 2 L8 pQq looooooooooooooooooooooooooooooooomooooooooooooooooooooooooooooooooon Identification stability

(54) ď C3 f ´ f˜ , p 2 L8 pQq looooooooooooomooooooooooooon Structural stability

where C1 , C2 , C3 ą 0 are constants that only depend on T, L, D, K, d. The lower bound in Theorem 4.2 shows that if we are given information on the solution data with initial values in Q, we are able to uniquely and stably identify the structure equation f within Q. Now, a natural question is, can we identify f outside Q if we are only given solution information with initial values in Q? The following lemma gives a negative answer when d “ 1. Lemma 4.7. Let d “ 1. Given a compact set Q Ă R, for arbitrarily fixed 0 ă δ ă 2K, there exist f, f˜ P LipL,K pR, Rq, such that " * |f pxq ´ f˜pxq| “ δ{2 for all x P z | inf |z ´ w| ą δ “: V pQ, δq, wPQ

` ˘ while the corresponding Of , Of˜ P A LipL,K pR, Rq satisfy ˘ ` hdQ Of , Of˜; ∥¨∥L8 “ 0. Proof. Fix f ” 0. Define the hat function $ ’ ’ ’ ’ 1 ´ Lx x P r0, 1{Ls ’ ’ ’ ’ ’ & Hpxq “ 1 ` Lx x P r´1{L, 0s ’ ’ ’ ’ ’ ’ ’ ’ ’ % 0 else and consider the translated and dilated version ˆ ˙ δ 2 Hy pxq “ H px ´ yq . 2 δ

(55)

(56)

Then, define f˜pxq “

sup Hy pxq.

(57)

yPV pQ,δq

It is straightforward that ˇ ˇ ˇ ˇ δ ˇ ˇ ˜ |f p0q| “ ˇ sup Hy p0qˇ ď ď K, ˇyPV pQ,δq ˇ 2 Lippf˜q ď

(58)

sup LippHy q ď L. yPV pQ,δq

Moreover, we see that for all x P tz | inf wPQ |z ´ w| ď δ{2u, f˜pxq “ f pxq “ 0, which implies for all x P BQ pOf q, x̃ P BQ pOf˜q, we have x “ x̃ ” 0 (due to uniqueness of solutions to Lipschitz ODEs, [35, Theorem 6.I, Theorem 10.VII]), and thus ˘ ` hdQ Of , Of˜; ∥¨∥L8 “ 0. 19

In the meanwhile, for all x P V pQ, δq, we have ˇ ˇ ˇ ˇ ˇ ˇ |f˜pxq ´ f pxq| “ ˇ sup Hy pxqˇ ˇyPV pQ,δq ˇ

(59)

δ “ |Hx pxq| “ . 2

4.3

Higher-order linear and Lipschitz ODEs

In this section, we generalize the results from Section 4.1 and 4.2 to higher-order linear and Lipschitz ODEs. This generalization is straightforward via state-space reduction, i.e., if we are given a higher-order ODE of the form ` ˘ xpmq “ f xpm´1q , . . . , xp1q , x , xptq P Rd , (60) ¸T

˜ we can apply state transformation y “

such that y

xT pxp1q qT . . . pxpm´1q qT

satisfy the first-order ODE ¸T

˜ y9 “ F pyq ,

y“

P Rmd

T y1T y2T . . . ym

where F pyq “

(61)

¸T

˜ T pf pym , ym´1 , . . . , y1 qqT y2T y3T . . . ym

First, we generalize the results from Theorem 4.1 to higher-order linear ODEs. md Theorem 4.3. Fix an arbitrary D ą 0 and consider BD :“ ∥y∥2 ď ` ty P` Rmd |d ˘˘ Du. For every two m-th order linear ODEs OA , OÃ P Am LinK R , R with corresponding ˜ ¸

A“ Ã “

mdˆd

Am´1 . . . A1 A0 P R ˜ ¸ Ãm´1 . . . Ã1 Ã0

, (62)

P Rmdˆd ,

we have that m´1 m´1 ÿ ÿ c Ai ´ Ãi ď hdBD pOA , OÃ ; ∥¨∥C m´1 q ď C Ai ´ Ãi , 2 2 i“0 i“0 looooooooooomooooooooooon looooooooooomooooooooooon Identification stability

Structural stability

where c, C ą 0 are constants that only depend on T, K, D, m, d. Proof. See Appendix D.2. 20

(63)

In a similar spirit, we can generalize the results from Theorem 4.2 to higher-order Lipschitz ODEs. Theorem 4.4. Fix an arbitrary D ą 0 and a compact set Q Ă BD :“ tx P Rmd | ? 2 `1T d L p :“ tx P R | ∥x∥ ď pKT ` Dqe ∥y∥2 ď Du. Let Q u. For every two m-th ˘˘ `2 ` order Lipschitz ODEs Of , Of˜ P Am LipL,K Rmd , Rd with corresponding f, f˜ P ` ˘ LipL,K Rmd , Rd , we have that * " 2 ˘ ` ˜ ˜ ď hdQ Of , Of˜; ∥¨∥C m´1 , C2 f ´ f min C1 f ´ f 2 L8 pQq 2 L8 pQq looooooooooooooooooooooooooooooooomooooooooooooooooooooooooooooooooon Identification stability

(64) ď C3 f ´ f˜ , p 2 L8 pQq looooooooooooomooooooooooooon Structural stability

where C1 , C2 , C3 ą 0 are constants that only depend on T, L, D, K, m, d. Proof. See Appendix D.3. One should note that C m´1 norm instead of L8 in Hausdorff distance for m-th order ODEs is necessary for the above results to hold, as shown by the following example. Example 4.1. Consider the ODE (0 ă ϵ ă 1) Oϵ : xp3q `

1 x“0 ϵ2

with intial conditions xp0q “ x0 , xp1q p0q “ x1 , xp2q p0q “ x2 , px0 , x1 , x2 qT P r0, 1s3 . Solving this ODE, we obtain ˆ ˙ ˆ ˙ t t 2 ´ ϵ x2 cos . xϵ ptq “ px0 ` ϵx2 q ` ϵx1 sin ϵ ϵ It is straight forward that ` ˘ hdr0,1s3 Oϵ , O2ϵ ; ∥¨∥L8 ď 5ϵ Ñ 0,

ϵ Ñ 0.

On the other hand, the distance in coefficients diverges to infinity ˇ ˇ ˇ1 ˇ 1 ˇ ´ ˇ “ 1 Ñ 8, ϵ Ñ 0. ˇ ϵ 2ϵ ˇ 2ϵ

4.4

First-order nonlinear ODEs: Hölder ODEs with applications to polynomial ODEs

In ´ this section, we extend the results to a wider class of ODEs–Hölder ODEs ¯ k,α (A CL,K as per Definition 3.1) for dimension d “ 1 and order m “ 1. Common examples of Hölder ODEs include ODEs with polynomial structures or sublinear structures. These ODEs exhibit behavior that is different from that of Lipschitz ODEs, with some not having a global solution, and some having multiple solutions for one initial value. 21

Example 4.2 (Finite escaping time). Consider the ODE x9 “ x2 with xp0q “ x0 ą 0. The solution is xptq “

x0 . 1 ´ x0 t

and it is only defined for t P r0, 1{x0 q Example 4.3 (Multiple solutions). Consider the ODE 1

x9 “ 2|x| 2 with xp0q “ 0. For every t0 ě 0, the function $ ’ ’ ’ ’ 0 t ď t0 & xptq “ ’ ’ ’ ’ % pt ´ t0 q2 t ą t0 is a solution. Despite these irregularities, the Hausdorff distance is still well-defined because it does not rely on the uniqueness or global existence of solutions with respect to initial value problems. Moreover, the identification results in the previous sections carry over to general Hölder ODEs. To this end, we first distinguish between three types of Hölder ODEs based on Definition 3.1: • sublinear Hölder ODEs (Example 4.3): k “ 0, α P p0, 1q; • Lipschitz ODEs: k “ 0, α “ 1; • superlinear Hölder ODEs (Example 4.2): k P N` , α P p0, 1s. Since the second type has been studied in the previous sections, we focus on sublinear (Section 4.4.1) and superlinear Hölder ODEs in the following sections. Moreover, as we are more interested in identifying structures based on solution data, we exclusively study the lower bound for Hausdorff distances. 4.4.1

Sublinear Hölder ODEs

We first study sublinear Hölder ODEs. In this case, we have the following identification results by slightly adjusting the proof of Lemma 4.6. Lemma 4.8. Assume α P p0, 1q. Fix an arbitrary D ą 0 and a compact set Ă˘ ` Q 0,α BD :“ tx P R | |x| ď Du. For every two sublinear Hölder ODEs Of , Of˜ P A CL,K 0,α with corresponding f, f˜ P CL,K , we have that ` ˘ hdQ Of , Of˜; ∥¨∥L8 * " α`1 α`1 1 (65) α α α2 ˜ ˜ ˜ ˜ , f ´f , f ´f , ě C min f ´ f , f ´f L8 pQq

L8 pQq

where C ą 0 is a constant depending only on α, L, K, T, D. 22

L8 pQq

Proof. See Appendix D.4. One should note that for sublinear ODEs, solutions are generally non-unique for initial value problems (Example 4.3). However, the Hausdorff distance is welldefined as it depends on the solution set instead of a single solution with respect to a fixed initial value. Moreover, one can see that the uniqueness of solutions is not required in the proof in Appendix D.4. 4.4.2

Superlinear Hölder ODEs

Next, we study superlinear Hölder ODEs. In this scenario, solutions could explode within a finite time (Example 4.2). Thus, to study the class of superlinear Hölder ODEs, the horizon T should be suitably chosen such that the solutions of each ODE in this class is at least well-defined on r0, T s. In the following lemma, we provide conditions on T and prove the identification results by noting that the superlinear Hölder ODEs we consider are nothing but Lipschitz ODEs for t P r0, T s. Lemma 4.9. For D, L, K ą 0, k P N` , α P p0, 1s, let ˆ ˙ 1 ´1 ´1 , T ă ppK ` Lqeq ϕD,k,α k`α´1 where ϕD,k,α prq “ rpr ` Dqk`α´1 , for r P R` . Now, given a compact set´ Q Ă ¯BD :“ k,α tx P R | |x| ď Du, for every two superlinear Hölder ODEs Of , Of˜ P A CL,K corresponding f, f˜ P C k,α , we have that

with

L,K

" `

hdQ Of , Of˜; ∥¨∥L8 ě min C1

f ´ f˜

2 2 L8 pQq

, C2

f ´ f˜

(66)

2 L8 pQq

where C1 , C2 ą 0 are constants depending only on k, α, L, K, T, D. Proof. See Appendix D.5 4.4.3

Polynomial ODEs

¯ ´ q´1,1 ř As per Definition 3.1, it is straightforward that A pPq,K q Ă A Cq!,K q´1 i! . Thus i“0

results in Lemma 4.9 can be applied to A pPq,K q. Moreover, due to the properties of polynomials, we obtain the following results. Corollary 4.1. Let I1 Ă BD :“ tx P R | |x| ď Du be a compact set with non-empty interior and let I2 “ tx1 , x2 , . . . , xq u P R be a set of q distinct real numbers. Then, for every two polynomial ODEs Op , Op̃ P A pPq,K q with corresponding p, p̃ P Pq,K , the following holds: (i). hdI1 pOp , Op̃ ; ∥¨∥L8 q “ 0 implies p “ p̃. (ii). hdI2 pOp , Op̃ ; ∥¨∥L8 q “ 0 implies p “ p̃. 23

4.5

Numerics showing bounds

We empirically examine the validity of lower and upper bounds on Hausdorff distance between ODEs, with a focus on first-order Linear ODEs and Lipschitz ODEs (Theorem 4.1 & Theorem 4.2). Specifically, we consider ODEs of dimension d “ 2 on a fixed time horizon t P r0, 0.1s and consider initial value set B1 “ tx P R2 | ∥x∥2 ď 1u. For numerical purpose, we discretize the time horizon and initial value set into T “ tt0 , t1 , . . . , t20 | where ti “ 0.005iu, ˜ ¸T x1 “ txi,j “ B | ri “ 0.25i, θ “ πj{5, i, j P t0, 1, . . . , 4u2 u. ri cospθj q ri sinpθj q x1 , trajectories are computed on T using For each ODE and each initial value x0 P B a fixed explicit integrator (RK4). 4.5.1

Linear ODEs

For numerical experiments on linear ODEs, we construct an ensemble of systems by first sampling 30 “base” matrices tÃ1 , Ã2 , . . . , Ã30 u where Ãi are independent random matrices, and then for each “base” matrix Ãk , k P t1, 2, . . . , 30u, we generate 20 perturbed variants of the form Ãk ` 0.02Bkl , where Bkl , l P t1, 2, . . . , 20u independent random matrices. This yields a collection of 600 matrices, which we denote by tA1 , A2 , . . . , A600 u. Figure ´ 2 reports the pairwise ¯ Hausdorff distance between linear ODEs OAi and OAj : hdBx1 OAi , OAj ; ∥∥¨∥2 ∥L8 pT q versus the pairwise structure distances between matrices: ∥Ai ´ Aj ∥2 . Each point thus represents one pair of linear ODEs, with the x-axis measuring the size of the perturbation in matrix space and the y-axis the induced discrepancy between solution sets–Hausdorff distance between two linear ODEs. The yellow and blue lines are the predicted lower and upper bounds in line with Theorem 4.1.

Figure 2: Red points represent the randomly generated samples. Yellow and blue lines represent the lower bound and upper bound on the Hausdorff distance between linear ODEs with respect to structure distances between matrices based on Theorem 4.1. 4.5.2

Lipschitz ODEs

For the numerical experiments on Lipschitz ODEs, we generate random Lipschitz vector fields using a two-layer ReLU network parametrization f pxq “ A2 ReLUpA1 x ` b1 q ` b2 . The construction proceeds as follows. First, we sample 30 base parameter tuples k 5ˆ2 tÃk1 , Ãk2 , b̃k1 , b̃k2 u30 , Ãk2 P R2ˆ5 , b̃k1 P R5 , and b̃k2 P R2 drawn indepenk“1 , with Ã1 P R dently at random. For each base index k, we then generate 20 perturbed parameter sets indexed by l P t1, . . . , 20u via k,l k Ak,l 1 “ Ã1 ` B1 ,

B1k,l P R5ˆ2 ,

k,l k Ak,l 2 “ Ã2 ` B2 ,

B2k,l P R2ˆ5 ,

k,l k bk,l 1 “ b̃1 ` c1 ,

5 ck,l 1 P R ,

k,l k bk,l 2 “ b̃2 ` c2 ,

2 ck,l 2 P R ,

k,l where B1k,l , B2k,l , ck,l 1 , c2 are independent random perturbations. This yields an ensemble of 600 Lipschitz vector fields tf1 , . . . , f600 u, where each fi corresponds to k,l k,l k,l some quadruple pAi1 , Ai2 , bi1 , bi2 q :“ pAk,l 1 , A2 , b1 , b2 q and is written as

fi pxq “ Ai2 ReLUpAi1 x ` bi1 q ` bi2 . 25

The global Lipschitz constant L and the affine growth constant K in Theorem 4.2 are determined from this ensemble as ␣ i ␣ i ( ( L “ max A2 2 Ai1 2 , K “ max A2 2 bi1 2 ` bi2 2 . i“1,...,600

i“1,...,600

Figure 3 reports, for all indices i, j P t1, . . . , 600u, ´ the pairwise Hausdorff ¯ distance between Lipschitz ODEs Ofi and Ofj : hdBx1 Ofi , Ofj ; ∥∥¨∥2 ∥L8 pT q (on a logrithmic scale) versus the pairwise structure distances between Lipschitz functions: ∥fi ´ fj ∥2 L8 . Each point in the figure thus represents one pair of Lipschitz ODEs, with the horizontal axis encoding the size of the perturbation in function space and the vertical axis the induced discrepancy between their solution sets– Hausdorff distance between two Lipschitz ODEs. The yellow and blue lines are the predicted lower and upper bounds in line with Theorem 4.2.

Figure 3: Red points represent the randomly generated samples. Yellow and blue lines represent the lower bound and upper bound on the Hausdorff distance between Lipschitz ODEs with respect to structure distances between Lipschitz functions based on Theorem 4.2. It is worth noting that our numerical experiments suggest that the identification lower bound derived for Lipschitz ODEs is not tight: the observed separation rates indicate room for improvement at the level of the exponents. Establishing sharp bounds for this class is left as an open problem for future work.

Complexity in learning governing equations of ODEs

In this section, we analyze the complexity in learning of structure equations for linear and Lipschitz ODEs based on Hausdorff-distance-based identification results from previous sections. Specifically, we study • how “rich” the class of ODEs is in terms of Hausdorff distance based on metric entropy [32] (Section 5.1), N • and how many solution samples tpti , xk pti qqM i“1 uk“1 we need to learn the structure equation (Section 5.2).

To the best of our knowledge, this work is the first to study the metric entropy and sample complexity in learning ODEs. In what follows, we restrict the discussions to first-order ODEs, as we can see from Section 4.3 that higher-order ODEs are merely a generalization of first-order ODEs.

5.1

Information-theoretic complexity in learning governing equations of ODEs

In this section, we study the “richness of” classes of first-order linear ODEs based on metric entropy. To motivate the analysis in this section, we briefly recall several notions related to metric entropy. Originating in the work of Kolmogorov, metric entropy was introduced as a way to quantify the complexity of compact subsets of metric spaces via covering or packing numbers [32]. Closely related but conceptually distinct is Kolmogorov–Sinai (KS) entropy, developed in ergodic theory and dynamical systems to measure the rate at which complexity is generated along the temporal evolution of a single dynamical system [9, 29]. Although dynamical systems are closely related to ODEs, the perspective in our paper differs fundamentally from that of KS entropy. Rather than quantifying the growth of complexity along trajectories of a fixed system, we are interested in using metric entropy to analyze the complexity of an entire class of ODEs by viewing it as a subset of an appropriate metric space. In this spirit, several works have studied metric entropy for families of dynamical systems by representing each system as an input–output operator and analyzing the resulting operator class [7, 20, 36]. The most closely related work to the present setting is [37], which investigates the metric entropy of the solution class induced by a class of ODEs: solutions are aggregated into a set, whose covering complexity is then bounded in terms of the complexity of the underlying set of governing vector fields (or equations) [37]. But unlike [37], we are not mixing the solutions of different ODEs in a set. Instead, we consider the set of solution sets induced by the class of ODEs. Each element in the set is the entire solution set of a specific ODE. And the metric used in analyzing the complexity is the Hausdorff distance. This unique and innovative way allows us to obtain identification results, as shown in Section 4. In this section, 27

we are interested in how the prescribed perspective allows us to study the “richness of” classes of first-order linear ODEs based on metric entropy. To this end, we start by noting that the Hausdorff distance is indeed a welldefined metric for the classes of linear ODEs. Lemma 5.1. Fix D ą 0 and consider BD :“ tx P Rd | ∥x∥2 ď Du. The Hausdorff BD p¨, ¨; ∥¨∥L8 q is a well-defined metric for the class of linear ODEs ˘˘ ` distance ` d hd d A LinK R , R . Proof.` Basically, ˘˘need to prove that hdBD p¨, ¨; ∥¨∥L8 q satisfies the metric axioms ` d we d on A LinK R , R . The symmetry, triangle inequality and hdBD pO, O; ∥¨∥L8 q “ 0 immediately follow from Definition ˘ For linear ODEs, we know from Theorem ` d 3.5. d 4.1 that for every A1 , A2 P LinK R , R , hdBD pOA1 , OA2 ; ∥¨∥L8 q “ 0 ô A1 “ A2 ô OA1 “ OA2 , which proves positivity. Then, we have the following results regarding the metric entropy of the class of linear ODEs. Theorem 5.1. Fix D ą 0 and consider BD :“ tx P `Rd | ∥x∥˘2 ď Du. For all ϵ ą 0, the metric entropy of the class of linear ODEs LinK Rd , Rd satisfies ` ` ˘˘ (67) log N pϵ; A LinK Rd , Rd , hdBD p¨, ¨; ∥¨∥L8 qqq — d2 logpϵ´1 q. Proof. See Appendix D.6. This means that the metric entropy of linear ODEs is essentially determined by the metric entropy of structure matrices characterizing the ODEs Things become tricky for Lipschitz ODEs. According to Theorem 4.2, the Hausdorff distance of two Lipschitz ODEs with respect to a compact initial set Q being zero only implies structural equivalence restricted to Q. Conversely, structural equivalence restricted to Q is not sufficient for the Hausdorff distance to be zero. Therefore, for the purposes of analysis, we restrict the scope of this section to a special subclass of Lipschitz ODEs—namely, ODEs with compactly supported Lipschitz vector fields. This assumption is mainly technical, but it does not undermine the reasonableness of the perspective taken here. First, metric-entropy arguments for Lipschitz functions are typically stated for functions defined on a bounded domain. For ODEs, however, working with a vector field that is only locally defined is inconvenient: even if the dynamics are well-defined in a neighborhood, the trajectory may exit that neighborhood, and the solution map is then no longer globally meaningful over the time horizon of interest. Second, while true governing vector fields are rarely compactly supported, compact support can be imposed without changing the dynamics on the trajectories under consideration. Indeed, if the initial conditions are restricted to a bounded set, then solutions of a Lipschitz ODE remain bounded on any finite time interval. Consequently, all relevant trajectories lie in some (possibly large) bounded region of state space. One may therefore modify the vector field outside this region—e.g., 28

by multiplying it with a smooth cutoff that equals one on the trajectory-containing set—thereby obtaining a compactly supported vector field that agrees with the original dynamics along all trajectories considered in this paper. In the following, we introduce the class of Lipschitz ODEs with compactly supported vector fields. For simplicity, we restrict our scope to ODEs with state dimension of 1. Definition 5.1. Fix D ą 0 and consider BD :“ tx P R | |x| ď Du. Consider the class of Lipschitz ODEs with compact support FL,K,D :“ tf : R Ñ R | Lippf q ď L, |f p0q| ď K, supppf q P BD u Ă LipL,K pR, Rq . Next, we show that hdBD p¨, ¨; ∥¨∥L8 q is a well-define metric for A pFL,K,D q. Lemma 5.2. Fix D ą 0 and consider BD :“ tx P Rd | ∥x∥2 ď Du. The Hausdorff distance hdBD p¨, ¨; ∥¨∥L8 q is a well-defined metric for the class of Lipschitz ODEs A pFL,K,D q. Proof. We need to prove that hdBD p¨, ¨; ∥¨∥L8 q satisfies the metric axioms on A pFL,K,D q. The symmetry, triangle inequality and hdBD pO, O; ∥¨∥L8 q “ 0 immediately follow from Definition 3.5. Applying Theorem 4.2 for d “ 1 and noting that the supporting sets of functions in FL,K,D are susbets of BD , we obtain * " 2 ` ˘ ď hdBD Of , Of˜; ∥¨∥C m´1 , C2 f ´ f˜ min C1 f ´ f˜ L8 pRq

L8 pRq

ď C3 f ´ f˜

L8 pRq

This implies for every f, f˜ P FL,K,D , ` ˘ hdBD Of , Of˜; ∥¨∥L8 “ 0 ô f “ f˜ ô Of “ Of˜, which proves positivity. Finally, we have the following results regarding the metric entropy of the class of Lipschitz ODEs A pFL,K,D q. Theorem 5.2. For all ϵ ą 0, the metric entropy of the class of Lipschitz ODEs A pFL,K,D q satisfies logp2q N pϵ; A pFL,K,D q , hdBD p¨, ¨; ∥¨∥L8 qq — logpϵ´1 q.

(68)

Proof. See Appendix D.7. This implies that the metric entropy of Lipschitz ODEs has asymptotically the same growth rate as that of Lipschitz functions on the log scale.

5.2

Sample complexity in learning governing equations of ODEs

In this section, we analyze the sample complexity in learning linear and Lipschitz ODEs. We would like to mention that the study of sample complexity for learning dynamical systems—whether for learning input–output maps or for identifying (typically linear) state-space models—is a widely researched topic [18, 3]. By contrast, the corresponding sample-complexity analysis in identifying the structure equations of general (including nonlinear) ODEs remains comparatively less explored—which we shall discuss in the remainder of this section. N Specifically, we consider samples tpti , xk pti qqM i“1 uk“1 generated through experiments or numerical simulations. Furthermore, the measurement errors in sample generation—including random experimental noise, numerical errors, and truncation errors arising from simulations—are assumed to admit a deterministic uniform upper bound Emeas . We summarize the above ideas in Lemma 5.3 and show how to learn the structure equation within a prescribed hypothesis class in a general principle. Lemma 5.3. Given a function class F with norm (or semi-norm) ∥¨∥F , the corresponding class of ODEs A pFq as per Definition 3.2 and an initial value set I for ODEs in A pFq, we assume the following: (1) Identification guarantee: There exists a function ϕ : R` Ñ R` with limaÑ0` ϕpaq “ 0 and ϕp0q “ 0, such that for every two ODEs Of1 , Of2 P A pFq, with corresponding f1 , f2 P F, the following holds ∥f1 ´ f2 ∥F ď ϕ phdI pOf1 , Of2 ; ∥¨∥L8 qq .

(69)

(2) Finite-sampling and bounded measurement error: Given a function f P F, we can sample N solutions X :“ txk ptquN k“1 Ă F at time 0 “ t1 ď t2 ď N ¨ ¨ ¨ ď tM “ T , leading to a sampling set D pXq :“ tpti , x̂k,i qM i“1 uk“1 such that hd pX, BI pOf qq ď Esampling pN q,

(70)

where Esampling pN q is the sampling error depending on the number of solution samples N , and ∥x̂k,i ´ xk pti q∥2 ď Emeas ,

for all k “ 1, 2, . . . , N, and i “ 1, 2, . . . , M, (71)

where Emeas is the measurement error. Here, D pXq can be interpreted as the set of sampled solutions with respect to ODE Of on finite times resolution. (3) Numerical error: We can numerically construct (e.g., by interpolation) a M function set C pD pXqq :“ tx̂k uN k“1 Ă F, where x̂k ptq depends on pti , x̂k,i qi“1 , such that ∥∥x̂k ´ xk ∥2 ∥L8 pr0,T sq ď Enum pEmeas , M q, (72) where Enum is the numerical error depending on both the measurement errors Emeas and number of time steps M used in sampling. Here, C pD pXqq is the approximation of the set of sampled solutions with respect to ODE Of based on reconstructions from D pXq. 30

(4) Training error: We can find a function f˜ (typically by training a model) such that ˘ ` (73) hd BI pOf˜q, C pD pXqq ; ∥¨∥L8 ď Etrain . Then, we have f ´ f˜

ď ϕ pEtotal q ,

(74)

where Etotal “ Esampling pN q ` Enum pEmeas , M q ` Etrain .

(75)

Proof. Note that ` ˘ Definition 3.5 ` ˘ hdI Of , Of˜; ∥¨∥L8 “ hd BI pOf q, BI pOf˜q; ∥¨∥L8 triangle inequality

hd pBI pOf q, X; ∥¨∥L8 q ` hd pX, C pD pXqq ; ∥¨∥L8 q ` ˘ ` hd C pD pXqq , BI pOf˜q; ∥¨∥L8 #

ď max sup

inf

xPX x̂PCpDpXqq

∥∥x ´ x̂∥2 ∥L8 ,

sup

inf ∥∥x ´ x̂∥2 ∥L8

x̂PCpDpXqq xPX

` Esampling pN q ` Etrain ď sup ∥∥x̂k ´ xk ∥2 ∥L8 ` Esampling pN q ` Etrain kPt1,2,...,N u

ď Esampling pN q ` Enum pEmeas , M q ` Etrain . Applying (69) concludes the proof. The lemma has some practical meanings. Most importantly, it provides a principled basis for the design of a numerical learning objective for recovering the governing differential equation. Specifically, if • the ODE class A pFq is identifiable according to (1) in Lemma 5.3, • we can sample the solution set of the ODE Of P A pFq up to some sampling, numerical, and measurement error and collect them in C pD pXqq then it is natural to train a model f˜—potentially a neural-network model, or a sparse superposition over a prescribed library of candidate functions as in SINDy—by minimizing the loss function ` ˘ Loss :“ hd BQ pOf˜q, C pD pXqq ; ∥¨∥L8 , which is the Hausdorff distance between the solution set of our model and the sampled solution set. Lemma 5.3 guarantees that any model that achieves a small training loss necessarily yields a small error in the recovered dynamics—the learning error between f and f˜ can be bounded above by the sampling, numerical, measurement, and training error. Note that this idea works for any ODEs class that satisfies the identification guarantee in Lemma 5.3 Now, we apply Lemma 5.3 to ODE classes in Section 4, for which the identification guarantee actually holds—namely, linear and Lipschitz ODEs. Since linear ODEs are merely a special case of Lipschitz ODEs, we first present the result on the sample complexity in learning Lipschitz ODEs. 31

Lemma 5.4 (Sampling results for Lipschitz ODEs). Arbitrarily `fix Nx , `Nt P N˘˘ `, d d D ą 0 and consider Q :“ r´D,`Dsd . Given a target ODE O P A Lip R , R f L,K ˘ with corresponding f P LipL,K Rd , Rd , assume we can generate solutions samples from BQ pOf q according to " * D d X “ xi P BQ pOf q | xi p0q “ i, where i P t´Nx , ´Nx ` 1, . . . , Nx u , (76) Nx and for each of the solution xi P X, we can measure the solution uniformly in time according to " * T D pXq :“ ptτ , x̂i,τ q | tτ “ τ, x̂i,τ “ xi ptτ q ` wi,τ , xi P X, τ P t0, 1, . . . , Nt u , Nt (77) where wi,τ is the measurement error admitting uniform bound ∥wi,τ ∥2 ď Emeas . Then, we construct ␣ C pD pXqq :“ x̂i : r0, T s Ñ Rd | x̂i is a piecewise linear interpolation such that ˙ * ˆ T τ “ x̂i,τ for xi P X, τ P t0, 1, . . . , Nt u x̂i Nt ` (78) ˘ ˜ as the (training) solution data. Now, suppose we can find a function f P LipL,K Rd , Rd (typically by training a model) such that ˘ ` (79) hd BQ pOf˜q, C pD pXqq ; ∥¨∥L8 ď Etrain . Then, we have the learning bound f ´ f˜

2 L8 pQq

c´ " * ¯¯ ? ? ´ ? Etotal ď 2 dp2 ` LT q max 2 dL 2 dDLT eLT ` KLT 2 eLT ` 2K Etotal , , T (80) where ` ˘ ? K ` LpD ` KT qeLT T dDeLT Etotal “ Emeas ` Etrain ` ` . (81) Nx 4Nt Proof. We follow the procedure outlined in Lemma 5.3. (i). Determine the form of ϕ. From Lemma 4.6, we know that ? Q Ă B?dD :“ tx P Rd | ∥x∥2 ď dDu ` ` d d ˘˘ and thus for every two ODEs O , O P A Lip R , R , with correspondf f L,K 1 2 ` ˘ ing f1 , f2 P LipL,K Rd , Rd , the following holds # + ∥∥f1 ´ f2 ∥2 ∥L8 pQq ? ` ? ˘, T hdQ pOf1 , Of2 ; ∥¨∥L8 q ě min 2 dL 2 dDLT eLT ` KLT 2 eLT ` 2K ¨

∥∥f1 ´ f2 ∥2 ∥L8 pQq ? , 2 dp2 ` LT q 32

which implies that ∥∥f1 ´ f2 ∥2 ∥L8 pQq ď ϕ phdI pOf1 , Of2 ; ∥¨∥L8 qq ,

(82)

with ? ϕpδq :“ 2 dp2`LT q max

δ , T

c´ ¯¯ * ? ´ ? LT 2 LT 2 dL 2 dDLT e ` KLT e ` 2K δ . (83)

(ii). Determine the sampling error Esampling . For arbitrarily x P BQ pOf q, we?can find i P t´Nx , ´Nx ` 1, . . . , Nx ud , such that xi P X, ∥xi p0q ´ xp0q∥2 ď NdD . x Since xi ´ x satisfy żt ∥xi ptq ´ xptq∥2 “ xi p0q ´ xp0q ` pf pxpsqq ´ f pypsqqdsq 0 2 żt ď ∥xi p0q ´ xp0q∥2 ` ∥f pxpsqq ´ f pypsqq∥2 ds 0 żt ď ∥xi p0q ´ xp0q∥2 ` L ∥xpsq ´ ypsq∥2 ds, 0

by Grönwall’s inequality, we have ? dDeLt ∥xi ptq ´ xptq∥2 ď ∥xi p0q ´ xp0q∥2 eLt ď . Nx Taking supreme over t, we have ? dDeLT . inf ∥∥y ´ x∥2 ∥L8 pr0,T sq ď ∥∥xi ´ x∥2 ∥L8 pr0,T sq ď yPX Nx

(84)

Noting that the above inequality holds for arbitrary x P BQ pOf q, we obtain ? dDeLT sup inf ∥∥y ´ x∥2 ∥L8 pr0,T sq ď . (85) Nx xPBQ pOf q yPX Conversely, for arbitrary xi P X, thanks to the fact that xi P BQ pOf q, we obtain sup inf ∥∥y ´ x∥2 ∥L8 pr0,T sq “ 0. (86) yPX xPBQ pOf q

Combing (85) and (86) gives ? dDeLT hd pX, BI pOf qq ď “: Esampling . Nx (iii). Determine the numerical error Enum . For every xi P X, we have żt ∥xi ptq∥2 “ xi p0q ` f pxi psqdsq s“0 2 żt ď D ` pK ` L ∥xi psq∥2 qds 0 żt ď D ` KT ` L ∥xi psq∥2 ds 0

(87)

which, by Grönwall’s inequality, implies that ∥xi ptq∥2 ď pD ` KT qeLt , and thus ∥∥xi ∥2 ∥L8 pr0,T sq ď pD ` KT qeLT . Then we can bound the Lipschitz constants of xi according to Lippxi q ď ∥∥x9i ∥2 ∥L8 pr0,T sq “ ∥∥f pxi q∥2 ∥L8 pr0,T sq ď K ` L ∥∥xi ∥2 ∥L8 pr0,T sq ď K ` LpD ` KT qeLT .

(88)

This gives that for x̂i P C pD pXqq, every τ P t0, 1, . . . , Nt u, and every ȷ „ T T pτ ´ 1q, τ “: rtτ ´1 , tτ s , tP Nt Nt h:“ NT

tτ ´ t t ´ tτ ´1 px̂i ptτ q ´ xptqq ` px̂i ptτ ´1 q ´ xptqq h h 2 tτ ´ t t ´ tτ ´1 pEmeas ` Lippxi qptτ ´ tqq ` pEmeas ` Lippxi qpt ´ tτ ´1 qq ď h h (88) ` ˘ ptτ ´ tqpt ´ tτ ´1 q ď Emeas ` K ` LpD ` KT qeLT h ` ˘ h ď Emeas ` K ` LpD ` KT qeLT ` ˘4 K ` LpD ` KT qeLT T “ Emeas ` “: Enum . 4Nt (89)

∥x̂i ptq ´ xi ptq∥2 “ t

Finally, combining (82),(83),(87),(89) and applying Lemma 5.3 conclude the proof. Remark 5.1. According to Lemma 5.4, when measurement error and training error are sufficiently ` in˘ order to achieve the learning error O pϵq, we need at least ` ˘ small, N “ O Nxd “ O ´ϵ2d number of samples and the sampling rate when measuring ` ´1 ˘ solutions should be O Nt “ O pϵ2 q. Since linear ODEs are a special case of Lipschitz ODEs, we immediately obtain the following sampling results for linear ODEs by adjusting the proof of Lemma 5.4. Lemma 5.5 (Sampling results for linear ODEs). Arbitrarily fix d, Nx , Nt P N` , D ą 0. Consider I :“ tg1 , g2 , . . . , gd u Ă Rd such that spantg1 , g2 , . . . , gd u “ Rd , tg̃1 , g̃2 , . . . , g̃d u is its corresponding canonical dual frame and there exists 0 ă F1 ď F2 , N ÿ 2 F1 ∥x∥2 ď |xx, gk y|2 ď F2 ∥x∥22 , k“1 N ÿ

1 1 |xx, g̃k y|2 ď ∥x∥22 ď ∥x∥22 . F2 F1 k“1 34

Further assume supjPt1,2,...,du ∥gj ∥2 ď D. ` ` ˘˘ Now, given a target ODE OA P A LinK Rd , Rd with corresponding A P Rdˆd , assume we can generate solutions samples from BI pOA q according to X “ txj P BI pOA q | xj p0q “ gj , where j P t1, 2, . . . , duu ,

(90)

and for each of the solution xj P X, we can measure the solution uniformly in time according to * " T τ, x̂j,τ “ xj ptτ q ` wj,τ , xj P X, τ P t0, 1, . . . , Nt u , D pXq :“ ptτ , x̂j,τ q | tτ “ Nt (91) where wj,τ is the measurement error admitting uniform bound ∥wj,τ ∥2 ď Emeas . Then, we construct ␣ C pD pXqq :“ x̂j : r0, T s Ñ Rd | x̂j is a piecewise linear interpolation such that ˆ ˙ * T x̂j τ “ x̂j,τ for xj P X, τ P t0, 1, . . . , Nt u Nt (92) as the (training) solution data. Now, suppose we can find matrix Ã P Rdˆd such that Ã ď K and 2

hd pBI pOÃ q, C pD pXqq ; ∥¨∥L8 q ď Etrain .

(93)

Then, we have the learning bound ˜ ¸´ 21 ? 1 2d e´5 mintT, 2K uK ¯? Etotal , A ´ Ã ď ´ ␣ ( 1 1 1 2 1 ` e´5 mintT, 2K uK 2 ´ e2 F1 min T, 2K where Etotal “ Emeas ` Etrain `

KDeKT T . 4Nt

(94)

(95)

Proof. Note that the function `f pxq “˘Ax is an Lipschitz function with Lippf q ď K and f p0q “ 0, i.e., f P LipK,0 Rd , Rd . We thus follow the proof of Lemma 5.4 to determine the numerical error Enum “ Emeas `

KDeKT T . 4Nt

Since the sampling procedure is precise, i.e., X “ BI pOA q, except for the measurement error already considered in the numerical error, we have Esampling “ 0. Finally, we apply Lemma 4.3 (replacing the the function class F with the class of matrices does not affect the result) to determine ˜ ¸´ 21 ? 1 ´5 mintT, 2K K u e 2d ¯? ϕpaq “ ´ a. ␣ ( 1 1 1 1 ` e´5 mintT, 2K uK 2 ´ e2 F1 min T, 2K

Remark 5.2. According to Lemma 5.5, when measurement error and training error are sufficiently small, in order to achieve the learning error O pϵq, we need at least d `number ˘ of samples and the sampling rate when measuring solutions should be ´1 O Nt “ O pϵq.

Appendices A

Metric entropy

Definition A.1 (Covering number[32]). Let pX , ρq be a metric space and C Ă X compact. The set tx1 , x2 , . . . , xN u Ă C (respectively tx1 , x2 , . . . , xN u Ă X ) is an ϵ-covering (respectively ϵ-net) for pC, ρq if, for each x P C, there exists an i P t1, 2, . . . , N u so that ρpx, xi q ď ϵ. The ϵ-covering number N pϵ; C, ρq (respectively the exterior ϵ-covering number N ext pϵ; C, ρq) is the cardinality of a smallest ϵ-covering (respectively smallest ϵ-net) for pC, ρq. Definition A.2 (Packing number[32]). Let pX , ρq be a metric space and C Ă X compact. An ϵ-packing for pC, ρq is a set tx1 , x2 , . . . , xN u Ă C such that ρpxi , xj q ą ϵ, for all distinct i, j. The ϵ-packing number M pϵ; C, ρq is the cardinality of a largest ϵ-packing for pC, ρq. Lemma A.1 ([32], Theorem IV). Let pX , ρq be a metric space and C Ă X compact. For all ϵ ą 0, we have M p2ϵ; C, ρq ď N ext pϵ; C, ρq ď N pϵ; C, ρq ď M pϵ; C, ρq.

(96)

Lemma A.2 ([32], p. 93). Let pX , ρX q and pY, ρY q be metric spaces and consider the compact sets CX Ă X and CY Ă Y. Assume that there exists an isometric isomorphism f : CX Ñ CY , i.e., f is bijective and for every pair a, b P CX , one has ρY pf paq, f pbqq “ ρX pa, bq. Then, N pϵ; CX , ρX q “ N pϵ; CY , ρY q

and

M pϵ; CX , ρX q “ M pϵ; CY , ρY q.

(97)

Lemma A.3. Let pX , ρX q and pY, ρY q be metric spaces and consider the compact set CX Ă X and CY Ă Y. Assume that there exists an isometry f : CX Ñ CY , i.e., for every pair a, b P CX , one has ρY pf paq, f pbqq “ ρX pa, bq. Then, M pϵ; CX , ρX q ď M pϵ; CY , ρY q.

(98)

An immediate result from Lemma A.3 is the following. Corollary A.1. Let pX , ρX q be a metric space and consider the compact sets C1 Ă C2 Ă X . Then, M pϵ; C1 , ρX q ď M pϵ; C2 , ρX q. (99) Proof. This follows from setting pX , ρX q “ pY, ρY q, CX “ C1 , CY “ C2 , f to be an identity mapping, and finally applying Lemma A.3. Lemma A.4. Let pX , ρX q and pY, ρY q be metric spaces and consider the compact sets CX Ă X and CY Ă Y. Assume that there exist a surjective f : CX Ñ CY and constants C1 ě C2 ą 0 such that for all a, b P CX , C2 ρX pa, bq ď ρY pf paq, f pbqq ď C1 ρX pa, bq,

(100)

N pϵ{C2 ; CX , ρX q ď N pϵ; CY , ρY q ď N pϵ{C1 ; CX , ρX q.

(101)

then

An immediate result from Lemma A.4 is the following. Corollary A.2. Let pX , ρ1 q be a metric space and consider a compact set CX Ă X . Assume that for another metric ρ2 on X , there exist constants C1 ě C2 ą 0 such that for all a, b P CX , C2 ρ1 pa, bq ď ρ2 pa, bq ď C1 ρ1 pa, bq,

(102)

then CX is also compact under ρ2 . Moreover, we have N pϵ{C2 ; CX , ρ1 q ď N pϵ; CX , ρ2 q ď N pϵ{C1 ; CX , ρ2 q.

(103)

Proof. This follows by setting pX , ρX q “ pX , ρ1 q, pY, ρY q “ pX , ρ2 q, CY “ CX , f to be an identity mapping, and finally applying Lemma A.4 Lemma A.5. Let d P N` and C ą 0. Consider a ball in Rd BCd :“ tx P Rd | ∥x∥2 ď Cu. Then, we have ` ˘ log N pϵ; BCd , ∥¨∥2 q — d log ϵ´1 .

(104)

Proof. It follows by adjusting the proof of [34, Lemma 5.7, Example 5.8] from unit balls to balls with radius C.

Matrix analysis

Lemma B.1. Let A P Rnˆn . We have ? n ∥A∥2 .

(105)

σ1 pAq . σn pAq

(106)

|λ1 pAq| . |λn pAq|

(107)

∥A∥2 ď ∥A∥F ď Lemma B.2. Let A P Rnˆn . We have κ∥¨∥2 pAq “ Moreover, if A is a normal matrix, then κ∥¨∥2 pAq “

Lemma B.3. Let A P Rnˆn . For all consistent matrix norms ∥¨∥, we have ∥A∥ ě ρ pAq .

(108)

Moreover, if A is a symmetric matrix, then ∥A∥2 “ ρ pAq .

(109)

Lemma B.4 (Weyl’s inequality). Let A P Rnˆn , then we have |λ1 pAqλ2 pAq . . . λk pAq| ď σ1 pAqσ2 pAq . . . σk pAq for all k “ 1, 2, . . . , n. 38

(110)

Lemma B.5. For all consistent matrix norm ∥¨∥ and all A P Rnˆn , we have eA ď e∥A∥ .

(111)

Lemma B.6. For all consistent matrix norm ∥¨∥ and all A, B P Rnˆn , we have B k ´ Ak ď k ∥B ´ A∥ maxt∥A∥ , ∥B∥uk´1 .

(112)

Proof. Note that k´1 ÿ

B k ´ Ak “

B l pB ´ AqAk´1´l ď

l“0 k´1 ÿ

k´1 ÿ

B l pB ´ AqAk´1´l

l“0

∥B∥l ∥B ´ A∥ ∥B∥k´1´l ď k ∥B ´ A∥ maxt∥A∥ , ∥B∥uk´1 .

l“0

Lemma B.7. For all consistent matrix norm ∥¨∥ and all A, B P Rnˆn , we have eA ´ eB ď ∥A ´ B∥ emaxt∥A∥,∥B∥u .

(113)

Proof. Applying Lemma B.6, we obtain eA ´ eB ď

8 ÿ 1 k“0

B k ´ Ak ď

“ ∥A ´ B∥ e

8 ÿ

k“0 maxt∥A∥,∥B∥u

1 ∥B ´ A∥ maxt∥A∥ , ∥B∥uk´1 k!

Lemma B.8. Let A P Rnˆn . Assume ∥¨∥ is a consistent matrix norm and maxt∥A∥ , AT u ď T K. Then we have eA eA is a symmetric positive definite matrix with T

λn peA eA q ě e´5K .

(114)

Proof. Since eA eA is a symmetric positive definite matrix, we have λi peA eA q P R, T λi peA eA q ą 0 for all i P t1, 2, . . . , nu, and T

´ T ¯ λ1 peA eA q Lemma B.2 “ κ∥¨∥2 eA eA T A A λn pe e q T

“ eA eA ď e

Lemma B.3

ď Lemma B.5

e´A e´A

A 2

´A

e´A

e∥A ∥ e∥A∥ e∥´A∥ e∥´A ∥ T

ď e4K . 39

(115)

Note that | Repλi pAqq| ď |λi pAq| ď ρpAq ď ∥A∥ ď K

for all i P t1, 2, . . . , nu.

Since λ1 peA q “ eλi pAq for some i P t1, 2, . . . , nu, we have |λ1 peA q| “ |eλi pAq | “ eRepλi pAqq ě e´K . Applying Lemma B.4, we obtain T

λ1 peA eA q “ σ1 peA q ě |λ1 peA q| ě e´K .

(116)

Finally, combining (115) and (116) gives ˜ λn pe

eA q “ λ1 pe

eA q

λ1 peA eA q λn peAT eA q

¸´1 ě e´5K .

(117)

Lemma B.9. Suppose tg1 , g2 , . . . , gN u is a frame for Rn and tg̃1 , g̃2 , . . . , g̃N u is its corresponding canonical dual frame, such that there exists 0 ă F1 ď F2 , N ÿ

F1 ∥x∥22 ď

|xx, gk y|2 ď F2 ∥x∥22 ,

k“1 N ÿ

1 1 |xx, g̃k y|2 ď ∥x∥22 ď ∥x∥22 . F2 F 1 k“1 Then, for every matrix A P Rnˆn , we have c sup ∥Agi ∥2 ě i

F1 ∥A∥2 . N

(118)

Proof. For every x P Rn , we have the following frame decomposition N ÿ

x“

xx, g̃k ygk ,

(119)

k“1

and thus we have ∥Ax∥2 “ A

N ÿ

xx, g̃k ygk

k“1

2 N ÿ

ď sup ∥Agi ∥2 i

|xx, g̃k y|

k“1

g fN ? fÿ ď sup ∥Ag ∥ N e |xx, g̃ y|2 i

c ď

i 2

k“1

N ∥x∥2 sup ∥Agi ∥2 . F1 i 40

(120)

This implies that c sup ∥Agi ∥2 ě i

F1 ∥Ax∥2 , N ∥x∥2

and thus

c sup ∥Agi ∥2 ě i

for all x P Rn zt0u

(121)

F1 ∥A∥2 N

(122)

Ordinary differential equations

Lemma C.1 (Gronwall–Bellman–Pachpatte inequality [19]). Let uptq, aptq, a1 ptq P CpR` , R` q,kpt, σq, BtB kpt, σq P CpD, R` q where D “ tpt, σq P R2` : 0 ď σ ď t ă 8u. Let g P CpR` , R` q be a non-decreasing function, gpuq ą 0 on p0, 8q. If żt uptq ď aptq ` kpt, σqgpupσqqdσ, (123) 0

for t P R` , then for 0 ď t ď t1 , „ ȷ żt żt ´1 uptq ď aptq ` kpt, σqgpupσqqdσ ď G Gpaptqq ` Apsqds , 0

(124)

where

żt

B Aptq “ kpt, tq ` kpt, σqdσ, 0 Bt żr ds , r ą 0, Gprq “ r0 gpsq r0 is arbitrary and G´1 is the inverse of G and t1 P R` is chosen so that żt Gpaptqq ` Apsqds P DompG´1 q, 0

for all 0 ď t ď t1 .

Proofs

D.1

Proof of Lemma 4.3

For arbitrarily fixed t0 P p0, T s and x0 , x̃0 P BD , set B :“ At0 , B̃ :“ Ãt0 , y :“ x0 ´ x̃0 and b :“ peB ´ eB̃ qx̃0 . Then, ∥B∥2 “ B T 2 ď t0 K. Note that for arbitrarily fixed

x̃0 P G, we have inf

x0 PG

eA¨ x0 ´ eÃ¨ x̃0

2 L8

) ! ě inf max ∥x0 ´ x̃0 ∥2 , eAt0 x0 ´ eÃt0 x̃0 x0 PG

˙ 12

2 1 ě ? inf ∥x0 ´ x̃0 ∥22 ` eB x0 ´ eB̃ x̃0 2 2 x0 PRd 1 ¯ ´ 1 2 2 “ ? inf ∥y∥22 ` eB y ` b 2 2 yPRd ´ ¯ 12 1 T “ ? inf y T pI ` eB eB qy ` 2bT eB y ` bT b 2 yPRd ¯ ¯ 12 1 ´ T´ B B T B ´1 B T ? “ b I ´ e pI ` e e q e b , 2

(125)

where the last equality holds since we are minimizing a quadratic form and I `eB eB is a symmetric positive definite matrix. Note that ´ ¯´1 B B T B ´1 B T ´B T ´B I ´ e pI ` e e q e “ I ´ I ` e e T

is a symmetric matrix and thus only has real eigenvalues. Moreover, e´B e´B is a symmetric positive definite matrix and by Lemma B.8 (replacing A with ´B), T

λn pe´B e´B q ě e´5t0 K .

(126)

Now, note that ˆ ´ ¯´1 ˙ ´ ´ ¯¯´1 T ´B T ´B λn I ´ I ` e e “ 1´ 1 ` λi e´B e´B

for some i P t1, 2, . . . , nu,

which means ˆ λn

´ ¯´1 ˙ ´ ´ ¯¯´1 T ´B T ´B I ´ I `e e ě 1 ´ 1 ` λn e´B e´B (126)

` ˘´1 ě 1 ´ 1 ` e´5t0 K

“

e´5t0 K . 1 ` e´5t0 K

inf

∥∥x ´ x̃∥2 ∥L8 pr0,T sq

(127)

Thus, we have svG pOA , OÃ ; ∥¨∥L8 q “

sup

x̃PBG pOÃ q xPBG pOA q

“ sup inf

x̃0 PG x0 PG

eA¨ x0 ´ eÃ¨ x̃0

2 L8 pr0,T sq

´ ´ ¯ ¯ 12 (125) 1 T B B T B ´1 B T ? ě sup b I ´ e pI ` e e q e b 2 x̃0 PG ˆ ´5t0 K ˙ 21 (127) 1 e ě ? sup peAt0 ´ eÃt0 qgi 2 2 1 ` e´5t0 K i c ˆ ´5t0 K ˙ 21 Lemma B.9 e F1 ě eAt0 ´ eÃt0 . 2N 1 ` e´5t0 K 2 42

(128)

Next, we set out to find a t0 such that eAt0 ´ eÃt0

can be lower-bounded by a 2

scaling of A ´ Ã . Note that 2

eAt0 ´ eÃt0

“

8 ¯ ÿ 1 ´ pAt0 qk ´ pÃt0 qk k! k“0

ě A ´ Ã Lemma B.6

t0 ´

“ A ´ Ã

Ak ´ Ãk

k“2

A ´ Ã

ě A ´ Ã

8 ÿ

t0 ´

8 ÿ k k“2

8 ÿ k

k! k“2

tk0

A ´ Ã

! )k´1 max ∥A∥2 , Ã tk0 2

K k´1 tk0

˘ ` t0 2 ´ eKt0 (129)

␣ ( 1 If we choose t0 “ min T, 2K , then e

At0

´e

Ãt0

* " ¯ 1 1 ´ 2 2´e A ´ Ã . ě min T, 2K 2 2

(130)

Combining (128) and (130), we have ´ ¯? (˜ ␣ ¸ 21 1 1 1 2 K ´5 mintT, 2K 2´e F1 min T, 2K u e ? svG pOA , OÃ ; ∥¨∥L8 q ě A ´ Ã . 1 2 2N 1 ` e´5 mintT, 2K uK By symmetry, this gives the desired bound.

D.2

Proof of Theorem 4.3 ¸T

We apply state transformation y “ xT pxp1q qT . . . pxpm´1q qT . Then, x P ` ` md d ˘˘ BBD pOA q with OA P Am Lin R ,R of the form (60) if and only if y P K ` ` md ˘˘ md BBD pOF pAq q with OF pAq P A LinK R , R of the form (61), where ¨ ˛ ˚ 0 Id 0 ˚ ˚ ˚ ˚ ˚ 0 0 Id ˚ ˚ ˚ ˚ .. .. F pAq :“ ˚ ... . . ˚ ˚ ˚ ˚ ˚ 0 0 0 ˚ ˚ ˚ ˝ ´A0 ´A1 ´A2 43

...

0 .. .

...

. . . ´Am´1

‹ ‹ ‹ ‹ ‹ ‹ ‹ ‹ ‹ ‹ ‹. ‹ ‹ ‹ ‹ ‹ ‹ ‹ ‹ ‚

(131)

We can upper bound the 2-norm of F pAq according to m´1 ÿ

∥F pAq∥2 “

Ei,i`1 b Id ´

i“1 m´1 ÿ

m´1 ÿ

∥Ei,i`1 b Id ∥2 `

m´1 ÿ

i“1

“

Ed,j`1 b Aj

j“0

∥Ed,j`1 b Aj ∥2

(132)

j“0 m´1 ÿ

∥Ei,i`1 ∥2 ∥Id ∥2 `

i“1

∥Ed,j`1 ∥2 ∥Aj ∥2

j“0

ď mpK ` 1q. Note that ¸ 12

m´1 ÿ

∥∥y∥2 ∥L8 r0,T s “

i“0 m´1 ÿ

m´1 ÿ

2 xpiq 2

xpiq 2

i“0

L8 r0,T s

xpiq 2 L8 r0,T s “ ∥∥x∥2 ∥C m´1 r0,T s

(133)

i“0

¸ 21

˜ ďm

m´1 ÿ

2 xpiq 2

“ m ∥∥y∥2 ∥L8 r0,T s ,

i“0

L8 r0,T s

which implies that ´ ¯ ´ ¯ hdBD OF pAq , OF pÃq ; ∥¨∥L8 ď hdBD pOA , OÃ ; ∥¨∥C m´1 q ď m hdBD OF pAq , OF pÃq ; ∥¨∥L8 . (134) Thus, we can upper bound the Hausdorff distance according to ´ ¯ hdBD pOA , OÃ ; ∥¨∥C m´1 q ď m hdBD OF pAq , OF pÃq ; ∥¨∥L8 Lemma 4.1,(132)

mDT empK`1qT F pAq ´ F pÃq

“ mDT empK`1qT

m´1 ÿ

Ed,j`1 b pAj ´ Ãj q

j“0

ď mDT empK`1qT

m´1 ÿ

∥Ed,j`1 ∥2 pAj ´ Ãj q

j“0

“ mDT empK`1qT

m´1 ÿ j“0

(135)

pAj ´ Ãj q

. 2

Similarly, we can lower bound the Hausdorff distance according to ´ ¯ hdBD pOA , OÃ ; ∥¨∥C m´1 q ě hdBD OF pAq , OF pÃq ; ∥¨∥L8 )˜ ¯ ! ´ ¸ 12 1 1 1 2 D min T, 2 ´ e Lemma 4.2,(132) 2mpK`1q e´5 mintT, 2mpK`1q umpK`1q ? ě F pAq ´ F pÃq 1 mpK`1q ´5 mintT, 2mpK`1q u 2 2 1`e ) ´ ¯ ! 1 ˜ ¸2 1 1 1 2 D min T, 2mpK`1q Lemma B.1 2 ´ e e´5 mintT, 2mpK`1q umpK`1q ? ě F pAq ´ F pÃq 1 F 2md 1 ` e´5 mintT, 2mpK`1q umpK`1q ) ´ ¯ ! ˜ ¸ 12 1 1 1 m´1 2 ´ e 2 D min T, 2mpK`1q ÿ e´5 mintT, 2mpK`1q umpK`1q ? “ Ai ´ Ãi 1 F 2md 1 ` e´5 mintT, 2mpK`1q umpK`1q i“0 ¯ ! ) ´ ˜ ¸ 21 1 1 1 m´1 2 D min T, 2mpK`1q ÿ Lemma B.1 2 ´ e e´5 mintT, 2mpK`1q umpK`1q ? Ai ´ Ãi . ě 1 2 2md 1 ` e´5 mintT, 2mpK`1q umpK`1q i“0 (136) Combining (135) and (136) concludes the proof.

D.3

Proof of Theorem 4.4 ¸T

˜ We apply state transformation y “

xT pxp1q qT . . . pxpm´1q qT

. Then, x P

BQ pOf q with Of of the form (60) if and only if y P BQ pOF pf q q with OF pf q of the form (61), where ¨ ˛ ˚ ‹ y2 ˚ ‹ ˚ ‹ ˚ ‹ ˚ ‹ ˚ ‹ y 3 ˚ ‹ ˚ ‹ ˚ ‹ ˚ ‹ .. F pf qpyq :“ ˚ ‹. . ˚ ‹ ˚ ‹ ˚ ‹ ˚ ‹ ˚ ‹ y m ˚ ‹ ˚ ‹ ˚ ‹ ˝ ‚ f py1 , y2 , . . . , ym q Note that

(137)

b ? LippF pf qq ď pLippf qq2 ` 1 ď L2 ` 1 ∥F pf qp0q∥2 “ ∥f p0q∥2 ď K, F pf q ´ F pf˜q F pf q ´ F pf˜q

L8 pQq

p 2 L8 pQq

“

f ´ f˜

“

f ´ f˜

, 2

L8 pQq

, p 2 L8 pQq

(138)

Moreover, by the same spirit of (133), we can prove that ´ ¯ ´ ¯ ˘ ` hdBD OF pf q , OF pf˜q ; ∥¨∥L8 ď hdBD Of , Of˜; ∥¨∥C m´1 ď m hdBD OF pf q , OF pf˜q ; ∥¨∥L8 . (139) Thus, we can upper bound the Hausdorff distance according to ´ ¯ ˘ ` hdBD Of , Of˜; ∥¨∥C m´1 ď m hdBD OF pf q , OF pf˜q ; ∥¨∥L8 Lemma 4.5,(138)

mT e L `1T

(138)

F pf q ´ F pf˜q

f ´ f˜

“ mT e L `1T

p 2 L8 pQq

(140)

. p 2 L8 pQq

Similarly, we can lower bound the Hausdorff distance according to ´ ¯ ˘ ` hdBD Of , Of˜; ∥¨∥C m´1 ě hdBD OF pf q , OF pf˜q ; ∥¨∥L8 $ , ˜q ’ / F pf q ´ F p f & . F pf q ´ F pf˜q Lemma 4.6,(138) 2 L8 pQq 2 L8 pQq ´ ¯, T ? ě min ? ’ / 2 dp2 ` L̂T q % 2 dL̂ 2DL̂T eL̂T ` K L̂T 2 eL̂T ` 2K $ , ˜ ’ / f ´f . f ´ f˜ & (138) 2 L8 pQq 2 L8 pQq ¯, T ´ ? “ min , ? ’ / % 2 dL̂ 2DL̂T eL̂T ` K L̂T 2 eL̂T ` 2K - 2 dp2 ` L̂T q ? where L̂ “ L2 ` 1. Combining (140) and (141) concludes the proof.

D.4

(141)

Proof of Lemma 4.8

First, we pick x0 P Q such that |f px0 q ´ f˜px0 q| “ f ´ f˜

L8 pQq

thanks to the continuity of f, f˜ and compactness of Q. Now, pick x̃ P BQ pOf˜q such that x̃p0q “ x0 . We then have żt x̃ptq “ x0 ` f˜px̃psqqds. (142) 0 0,α pR, Rq, we have Taking the absolute value and noting that f˜ P CL,K

żt |x̃ptq ´ x0 | ď

żt L|x̃psq|α ds,

|f px̃psqq|ds ď Kt ` 0

which implies that żt L|x̃psq|α ds.

|x̃ptq| ď |x0 | ` Kt ` 0

(143)

By Lemma C.1, we have żt |x̃ptq| ď |x0 | ` Kt `

` ˘ 1 L|x̃psq|α ď p|x0 | ` Ktq1´α ` p1 ´ αqLt 1´α .

(144)

Using (144) in (143), we obtain ` ˘ 1 |x̃ptq ´ x0 | ď p|x0 | ` Ktq1´α ` p1 ´ αqLt 1´α ´ |x0 |.

(145)

Now, set ` ˘ 1 ϕptq “ p|x0 | ` Ktq1´α ` p1 ´ αqLt 1´α ´ |x0 |.

(146)

Then ϕp0q “ 0 and for all ξ P p0, T q, ˆ ˙ α ˘ 1´α 1 ` p1 ´ αqK 1´α p|x0 | ` Kξq ` p1 ´ αqLξ ` p1 ´ αqL 1´α p|x0 | ` Kξqα α ˆ ˙ 1´α p1 ´ αqLξ 1 1` pp1 ´ αqK ` p1 ´ αqLp|x0 | ` Kξqα q “ 1´α p|x0 | ` Kξq1´α ˆ ˙ α (147) p1 ´ αqLξ α 1´α 1 1` pp1 ´ αqK ` p1 ´ αqLp|x0 | ` Kξqα q ď 1´α 1´α K ˆ ˙ α p1 ´ αqLT α 1´α 1 pp1 ´ αqK ` p1 ´ αqLpD ` KT qα q 1` ď 1´α 1´α K “: cpα, L, K, T, Dq.

9 ϕpξq “

By Lagrange’s mean value theorem, we therefore have for all t P p0, T s, 0 ď ϕptq

DξPp0,tq

“

9 ϕpξqt ď cpα, L, K, T, Dqt,

which implies |x̃ptq ´ x0 | ď cpα, L, K, T, Dqt.

(148)

Now, set $ ’ ’ & t0 :“ min

, / / .

α f ´ f˜

L8 pQq

1 α

’ ’ % p4Lq cpα, L, K, T, Dq

we obtain f ´ f˜ |x̃ptq ´ x0 | ď

(149)

/ / -

1 α

L8 pQq 1

for t P r0, t0 s.

(150)

p4Lq α 0,α . We have Define h :“ f ´ f˜, then h P C2L,2K

f ´ f˜ |hpx̃ptqq| ě |hpx0 q| ´ 2L|x̃ptq ´ x0 |α ě

L8 pQq

for t P r0, t0 s.

(151)

Now, for arbitrarily fixed x P BQ pOf q, define y “ x ´ x̃. Then, y satisfy the ODE 9 “ f pyptq ` x̃ptqq ´ f˜px̃ptqq. yptq 47

(152)

Integrating from 0 to t, we have żt´ żt ¯ ˜ f px̃psqq ´ f px̃psqq ds. (153) yptq “ yp0q ` pf pypsq ` x̃psqq ´ f px̃psqqq ds ` 0

which in turn gives # ˇż ˇ ˆ ˇ˙ α1 + ˇż t t ˇ ˇ ˇ ˇ 1ˇ 1 ˇ ˇ, ˇ ∥y∥L8 pr0,T sq ě min hpx̃psqqds hpx̃psqqds , for all t P r0, T s. ˇ 2LT ˇ ˇ 4ˇ 0 0 (155) Since h ˝ x̃ is continuous, by (151), hpx̃ptqq must remain positive or negative for t P r0, t0 s. In either case, we have # ˇż ˇ ˆ ˇż t0 ˇ˙ α1 + ˇ ˇ ˇ 1 1 ˇˇ t0 ˇ hpx̃psqqdsˇˇ , hpx̃psqqdsˇˇ ∥y∥L8 pr0,T sq ě min ˇ ˇ 4 0 2LT 0 # ˆ ˙ α1 + (151) t t0 0 , f ´ f˜ f ´ f˜ ě min 8 4LT L8 pQq L8 pQq * " α`1 α`1 1 (149) α α α2 , ě C min f ´ f˜ , f ´ f˜ , f ´ f˜ , f ´ f˜ L8 pQq

L8 pQq

where C ą 0 is a constant depending only on α, L, K, T, D. Since this holds for arbitrary x P BQ pOf q, we obtain ∥x ´ x̃∥L8 pr0,T sq " α`1 α ě C min f ´ f˜ , f ´ f˜ inf

xPBQ pOf q

L8 pQq

1 α , f ´ f˜

L8 pQq

, f ´ f˜

α`1 α2

L8 pQq

* .

Taking the supremum over BQ pOf˜q and by symmetry, we arrive at the desired result.

D.5

Proof of Lemma 4.9

Before proving Lemma 4.9, we need two auxiliary lemmata. The first lemma shows that under´the condition for T in Lemma 4.9, solutions of each superlinear Hölder ¯ k,α ODE in A CL,K is indeed well-defined on r0, T s. Lemma D.1. For D, L, K ą 0, k P N` , α P p0, 1s, let ˙ ˆ 1 ´1 ´1 T ă ppK ` Lqeq ϕD,k,α , k`α´1 48

where ϕD,k,α prq “ rpr ` Dqk`α´1 , for r P R´` . Consider a compact set Q Ă BD :“ ¯

k,α k,α tx P R | |x| ď Du. Then, for each Of P A CL,K with f P CL,K , the solutions to Of with respect to initial values in Q are well defined on r0, T s. Moreover, for all x P BQ pOf q, we have

` ˘´ 1 |xptq| ď pD ` pK ` LqeT q 1 ´ pk ` α ´ 1qpK ` LqepD ` pK ` LqeT qk`α´1 k`α´1 , (156) for all t P r0, T s. ´ ¯ k,α Proof. For each Of P A CL,K and x P BQ pOf q with xp0q “ x0 P Q, we have żt xptq “ x0 `

f pxpsqqds, 0

By Lemma C.1, we have 1 ´ ¯´ k`α´1 |xptq| ď p|x0 | ` C̃tq 1 ´ pk ` α ´ 1qC̃tp|x0 | ` C̃tqk`α´1 1 ´ ¯´ k`α´1 ď pD ` C̃T q 1 ´ pk ` α ´ 1qC̃T pD ` C̃T qk`α´1 ă 8,

for 0 ď t ď T ă C̃ ´1 ϕ´1 D,k,α

1 k`α´1

, where ϕD,k,α prq “ rpr ` Dqk`α´1 .

next lemma shows that if the solutions to superlinear Hölder ODEs in ´The ¯ k,α A CL,K are uniformly bounded, then the superlinear Hölder ODEs are nothing but Lipschitz ODEs. Lemma D.2. Assume D, L, K ą 0, k P N` , α P p0, 1s, and B ą 0. For all f P Hpk, α, L, Kq, we have Lippf |r´B,Bs q ď maxtL, 1upB ` KqpB ` 1qk´1 . 49

(157)

for all |x| ď B,

which implies the desired result. Proof of Lemma 4.9: Lemma ´D.1 implies that the absolute values of solutions ¯ k,α to superlinear Hölder ODEs in A CL,K are universally upper-bounded by ˘´ 1 ` B :“ pD ` pK ` LqeT q 1 ´ pk ` α ´ 1qpK ` LqepD ` pK ` LqeT qk`α´1 k`α´1 . Then, Lemma D.2 implies that f acting on xptq is a Lipschitz function with Lippf |r´B,Bs q ď maxtL, 1upB ` KqpB ` 1qk´1 . Thus, we can apply Lemma 4.6 with d “ 1, L̃ “ maxtL, 1upB ` KqpB ` 1qk´1 to obtain the desired results.

D.6

Proof Theorem 5.1

To prove Theorem 5.1, we first need an auxiliary result on the metric entropy of bounded balls in the space of matrices. Lemma D.3. Define MKd,d :“ tA P Rdˆd | ∥A∥2 ď Ku.

(158)

log N pϵ; MKd,d , ∥¨∥2 q — d2 logpϵ´1 q.

(159)

Then, for all ϵ ą 0, 2

Proof. We define a mapping V : MKd,d Ñ Rd such that pV pAqqpi´1qd`j “ Ai,j for i, j P t1, 2, . . . , du. 50

(160)

Then, we have LemmaB.1

∥V pAq∥2 “ ∥A∥F

LemmaB.1 ?

d ∥A∥2 (161) ´ ¯ ´ ¯ and thus V is an isometric isomorphism between MKd,d , ∥¨∥F and V pMKd,d q, ∥¨∥2 . Now, for arbitrary C ą 0, define ∥A∥2

BCd :“ tx P Rd | ∥x∥2 ď Cu. By (161), we obtain 2

d d Ă V pMKd,d q Ă B? BK , dK

(162)

which implies ` ˘ Lemma A.5 d2 d2 log ϵ´1 — log M p2ϵ; BK , ∥¨∥2 q (162), Corollary A.1

ď Lemma A.1

log M p2ϵ; V pMKd,d q, ∥¨∥2 q

log N pϵ; V pMKd,d q, ∥¨∥2 q

Lemma A.2

log N pϵ; MKd,d , ∥¨∥F q

Lemma A.1

log M pϵ; V pMKd,d q, ∥¨∥2 q

“

(162), Corollary A.1

(163)

d , ∥¨∥2 q log M pϵ; B? dK ` ˘ Lemma A.5 2 — d log ϵ´1 .

Finally, combining by (161), (163), and applying Corollary A.2, we have ` ˘ ` ˘ log N pϵ; MKd,d , ∥¨∥F q — d2 log ϵ´1 ñ log N pϵ; MKd,d , ∥¨∥2 q — d2 log ϵ´1 .

Proof of Theorem 5.1: Consider the mapping ` ` ˘˘ G : MKd,d Ñ A LinK Rd , Rd . A Ñ OA By Theorem 4.1, there exist constants 0 ă c ď C depending only on T, K, D, such that for every A1 , A2 P MKd,d ,we have c ∥A1 ´ A2 ∥2 ď hdBD pGpA1 q, GpA2 q; ∥¨∥L8 q ď C ∥A1 ´ A2 ∥2 . Then, applying Lemma D.3 and A.4 gives the desired result.

D.7

Proof of Theorem 5.2

To prove Theorem 5.2, we need a few auxiliary results on the metric entropy of the class of Lipschitz functions. First, we introduce the following class of Lipschitz functions FL,C ra, bs :“ tf : ra, bs Ñ R | Lippf q ď L; |f pxq| ď C, @x P ra, bsu. 51

(164)

Lemma D.4 ([32, Equation (11), Chapter 7]). Arbitrarily fix L, C ą 0, b ą a. We have ` ˘ logp2q N pϵ; FL,C ra, bs, ∥¨∥L8 pra,bsq q — log ϵ´1 . (165) Based on Lemma D.4, we obtain the following results for the metric entropy of FL,K,D . Lemma D.5. Arbitrarily fix L, K, D ą 0. We have ` ˘ logp2q N pϵ; FL,K,D , ∥¨∥L8 pRq q — log ϵ´1 .

(166)

Proof. Consider the linear mapping ȷ D Ñ FL,K,D E : FL,mint LD ,K u 0, 2 2 f Ñ Epf q, „

where

$ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ ’ &

f p0q px ` Dq D

x P r´D, 0s

f pxq

“ ‰ x P 0, D2

Epf qpxq “

’ ’ “ ‰ ’ 2f p D q ’ ’ ´ D2 px ´ Dq x P D2 , D ’ ’ ’ ’ ’ ’ ’ ’ ’ % 0 else We have "

* LD |Epf qp0q| “ |f p0q| ď min , K ď K, 2 ˇ+ # # ␣ LD ( + ˇ ˇ ˇ ˇ f p0q ˇ ˇˇ 2f p D2 q ˇˇ min 2 2 ,K ˇ,ˇ LippEpfqq ď max Lippf q, ˇˇ ď L, ˇ ď max L, ˇ ˇ D ˇ D D supppEpf qq Ă r´D, Ds, ∥Epf q∥L8 pRq “ ∥f ∥L8 pr0, D sq , 2

´ ¯ “ ‰ and thus E is a well-defined isometry from FL,mint LD ,K u 0, D2 , ∥¨∥L8 pr0, D sq to 2 2 pFL,K,D , ∥¨∥L8 pRq q. Thus, we obtain ´ ¯ Lemma A.1 ´ ¯ logp2q N ϵ; FL,K,D , ∥¨∥L8 pRq ě logp2q M 2ϵ; FL,K,D , ∥¨∥L8 pRq ˆ „ ȷ ˙ Lemma A.3 D p2q ě log M 2ϵ; FL,mint LD ,K u 0, , ∥¨∥L8 pr0, D sq 2 2 2 ˆ „ ȷ ˙ Lemma A.1 D p2q ě log N 2ϵ; FL,mint LD ,K u 0, , ∥¨∥L8 pr0, D sq 2 2 2 ` ´1 ˘ Lemma D.4 — log ϵ . (167) 52

On the other hand, consider the mapping I : FL,K,D Ñ FL,K`LD r´D, Ds f Ñ Ipf q, where Ipf q “ f pxq

for x P r´D, Ds.

´ ¯ Then I is a well-defined isometry from pFL,K,D , ∥¨∥L8 pRq q to FL,K`LD r´D, Ds, ∥¨∥L8 pr´D,Dsq . Thus, ´ ¯ Lemma A.1 ´ ¯ p2q p2q log N ϵ; FL,K,D , ∥¨∥L8 pRq ď log M ϵ; FL,K,D , ∥¨∥L8 pRq ´ ¯ Lemma A.3 ď logp2q M ϵ; FL,K`LD r´D, Ds, ∥¨∥L8 pr´D,Dsq ´ϵ ¯ Lemma A.1 ď logp2q N ; FL,K`LD r´D, Ds, ∥¨∥L8 pr´D,Dsq 2 ` ´1 ˘ Lemma D.4 — log ϵ . (168) Combining (167) and (168) concludes the proof. Proof of Theorem 5.2: Appling Theorem 4.2 for d “ 1 and setting Q “ BD , we have that for every two Lipschitz ODEs Of , Of˜ P A pFL,K,D q, there exists C1 , C2 , C3 ą 0 depending only on T, L, D, K, such that " * 2 ˘ ` , C2 f ´ f˜ hdB Of , O ˜; ∥¨∥ 8 ě min C1 f ´ f˜ D

L8 pBD q

" 2 supppf q,supppf˜qĂBD “ min C1 f ´ f˜

L8 pRq

, C2 f ´ f˜

(169)

L8 pRq

and hdBD

d |∥x∥ ďpKT `DqeLT u p ˘ Q:“txPR 2 Of , Of˜; ∥¨∥L8 ď C3 f ´ f˜

p supppf q,supppf˜qĂBD ĂQ

“

C3 f ´ f˜

p L8 pQq

L8 pRq

(170)

(169) implies that f ´ f˜

L8 pRq

! ` ˘ b ` ˘) ď C4 max hdBD Of , Of˜; ∥¨∥L8 , hdBD Of , Of˜; ∥¨∥L8 , (171)

for a constant C4 ą 0. Now, consider an ϵ-covering tOf1 , Of2 , . . . OfN u of pA pFL,K,D q , hdBD p¨, ¨; ∥¨∥L8 qq with N “ N pϵ; A pFL,K,D q , hdBD p¨, ¨; ∥¨∥L8 qq and corresponding tf1 , f2 , . . . , fN u Ă FL,K,D . According to (171), this implies that for every f P FL,K,D , we can find i P t1, 2, . . . , N u, such that 1

∥f ´ fi ∥L8 pRq ď C4 maxtϵ, ϵ 2 u,

which in turn gives that tf1 , f2 , . . . , fN u is a pC4 maxtϵ, ϵ 2 uq-covering of pFL,K,D , ∥¨∥L8 pRq q and thus 1

logp2q N pϵ; A pFL,K,D q , hdBD p¨, ¨; ∥¨∥L8 qq ě logp2q N pC4 maxtϵ, ϵ 2 u; FL,K,D ∥¨∥L8 pRq q ` ˘ Lemma D.5 1 log ϵ´1 . Á 2 (172) On the other hand, consider an ϵ{C3 -covering tf1 , f2 . . . , fN u of pFL,K,D , ∥¨∥L8 pRq q with N “ N pϵ{C3 , FL,K,D , ∥¨∥L8 pRq q. By (170), the corresponding tOf1 , Of2 , . . . OfN u is an ϵ-covering of pA pFL,K,D q , hdBD p¨, ¨; ∥¨∥L8 qq, which gives logp2q N pϵ; A pFL,K,D q , hdBD p¨, ¨; ∥¨∥L8 qq ď logp2q N pϵ{C3 ; FL,K,D ∥¨∥L8 pRq q ` ˘ Lemma D.5 À log ϵ´1 . Combining (172) and (173) concludes the proof.

(173)

References [1] Helmut Bolcskei. “Lecture Notes Mathematics of Information”. In: (2020). [2] Steven L. Brunton, Joshua L. Proctor, and J. Nathan Kutz. “Discovering Governing Equations from Data: Sparse Identification of Nonlinear Dynamical Systems”. In: Proceedings of the National Academy of Sciences 113.15 (Apr. 2016), pp. 3932–3937. issn: 0027-8424, 1091-6490. doi: 10.1073/pnas.1517384113. arXiv: 1509.03580 [math]. [3] Marco C Campi and Erik Weyer. “Finite sample properties of system identification methods”. In: IEEE Transactions on Automatic Control 47.8 (2002), pp. 1329–1334. [4] Zhao Chen, Yang Liu, and Hao Sun. “Physics-informed learning of governing equations from scarce data”. In: Nature communications 12.1 (2021), p. 6136. [5] M Grewal and Keith Glover. “Identifiability of linear and nonlinear dynamical systems”. In: IEEE Transactions on automatic control 21.6 (2003), pp. 833– 837. [6] Martin Holler and Erion Morina. On Uniqueness in Structured Model Learning. Oct. 2024. arXiv: 2410.22009. [7] Clemens Hutter, Recep Gül, and Helmut Bölcskei. “Metric entropy limits on recurrent neural network learning of linear dynamical systems”. In: Applied and Computational Harmonic Analysis 59 (2022), pp. 198–223. [8] Victor Isakov. Inverse problems for partial differential equations. Springer, 2006. [9] AN Kolgomorov. “Entropy per Unit Time as a Metric Invariant of Automorphism”. In: Doklady of Russian Academy of Sciences: Moscow, Russia 124 (1959), pp. 754–755. [10] Nikola Kovachki et al. “Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs”. In: Journal of Machine Learning Research 24.89 (2023), pp. 1–97. issn: 1533-7928. [11] Nikola Kovachki et al. “Neural operator: Learning maps between function spaces with applications to pdes”. In: Journal of Machine Learning Research 24.89 (2023), pp. 1–97. [12] Zongyi Li et al. “Fourier neural operator for parametric partial differential equations”. In: arXiv preprint arXiv:2010.08895 (2020). [13] Lennart Ljung. “System identification”. In: Signal analysis and prediction. Springer, 1998, pp. 163–173. [14] Lennart Ljung and Torkel Glad. “On global identifiability for arbitrary model parametrizations”. In: automatica 30.2 (1994), pp. 265–276. [15] Zichao Long et al. “PDE-Net: Learning PDEs from Data”. In: Proceedings of the 35th International Conference on Machine Learning. PMLR, July 2018, pp. 3208–3216.

[16] Georg Martius and Christoph H Lampert. “Extrapolation and learning equations”. In: arXiv preprint arXiv:1610.02995 (2016). [17] Richard Nickl. Bayesian non-linear statistical inverse problems. EMS press Berlin, 2023. [18] Samet Oymak and Necmiye Ozay. “Non-asymptotic identification of lti systems from a single trajectory”. In: 2019 American control conference (ACC). IEEE. 2019, pp. 5655–5661. [19] Baburao G Pachpatte. Integral and finite difference inequalities and applications. Vol. 205. Elsevier, 2006. [20] Yang Pan, Clemens Hutter, and Helmut Bölcskei. “Metric-Entropy Limits on Nonlinear Dynamical System Learning”. In: arXiv preprint arXiv:2407.01250 (2024). [21] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations”. In: Journal of Computational physics 378 (2019), pp. 686–707. [22] Bogdan Raonic et al. “Convolutional Neural Operators for Robust and Accurate Learning of PDEs”. In: Advances in Neural Information Processing Systems 36 (Dec. 2023), pp. 77187–77200. [23] Bogdan Raonic et al. “Convolutional neural operators for robust and accurate learning of PDEs”. In: Advances in Neural Information Processing Systems 36 (2024). [24] Richard Roy and Thomas Kailath. “ESPRIT-estimation of signal parameters via rotational invariance techniques”. In: IEEE Transactions on acoustics, speech, and signal processing 37.7 (2002), pp. 984–995. [25] Samuel H. Rudy et al. “Data-Driven Discovery of Partial Differential Equations”. In: Science Advances 3.4 (Apr. 2017), e1602614. doi: 10.1126/sciadv. 1602614. [26] Subham Sahoo, Christoph Lampert, and Georg Martius. “Learning equations for extrapolation and control”. In: International Conference on Machine Learning. Pmlr. 2018, pp. 4442–4450. [27] Philipp Scholl et al. “The Uniqueness Problem of Physical Law Learning”. In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Rhodes Island, Greece: IEEE, June 2023, pp. 1– 5. isbn: 978-1-72816-327-7. doi: 10.1109/ICASSP49357.2023.10095017. [28] Philipp Scholl et al. Well-Definedness of Physical Law Learning: The Uniqueness Problem. Jan. 2023. arXiv: 2210.08342 [math-ph]. [29] Yakov G Sinai. “On the notion of entropy of a dynamical system”. In: Doklady of Russian Academy of Sciences. Vol. 124. 3. 1959, pp. 768–771. [30] Andrew M Stuart. “Inverse problems: a Bayesian perspective”. In: Acta numerica 19 (2010), pp. 451–559.

[31] System identification - Wikipedia — en.wikipedia.org. Aug. 2025. url: https: //en.wikipedia.org/wiki/System_identification. [32] V. M. Tikhomirov. “ε-Entropy and ε-Capacity of Sets In Functional Spaces”. In: Selected Works of A. N. Kolmogorov: Volume III: Information Theory and the Theory of Algorithms. Ed. by A. N. Shiryayev. Dordrecht: Springer Netherlands, 1993, pp. 86–170. isbn: 978-94-017-2973-4. doi: 10.1007/97894-017-2973-4_7. [33] Silviu-Marian Udrescu and Max Tegmark. “AI Feynman: A physics-inspired method for symbolic regression”. In: Science advances 6.16 (2020), eaay2631. [34] Martin J Wainwright. High-dimensional statistics: A non-asymptotic viewpoint. Vol. 48. Cambridge university press, 2019. [35] Wolfgang Walter. Ordinary Differential Equations. Vol. 182. Graduate Texts in Mathematics. New York, NY: Springer, 1998. doi: 10.1007/978-1-46120601-9. [36] G Zames. “On the metric complexity of causal linear systems: Estimates of εentropy and ε-dimension”. In: 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications. IEEE. 1977, pp. 807–810. [37] Ying Zhu and Mozhgan Mirzaei. “Classes of ODE solutions: smoothness, covering numbers, implications for noisy function fitting, and the curse of smoothness phenomenon”. In: arXiv preprint arXiv:2011.11371 (2020).

Record · ID 310809 · SHA-256 dc22633abb4a961f
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.