Verifiable Regularity Criterion for Conditional Expectation Operators and Conditional Mean Embeddings with Applications to Nonparametric Regression, Bayesian Inverse Problems, and Koopman Operators

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS AND CONDITIONAL MEAN EMBEDDINGS WITH APPLICATIONS TO NONPARAMETRIC REGRESSION, BAYESIAN INVERSE PROBLEMS, AND KOOPMAN OPERATORS

arXiv:2608.06155v1 [math.DS] 6 Aug 2026

MAXIMILIANO HERTEL1 , ILJA KLEBANOV2 , MANUEL SCHALLER3 , AND KARL WORTHMANN1 Abstract. Conditional expectation operators (CEOs) and their associated conditional mean embeddings (CMEs) play a central role across applied mathematics and machine learning, appearing in nonparametric regression, Bayesian inverse problems, and Koopman operator theory. A fundamental question is when a CEO maps a function space on Y into a prescribed function space on X , particularly a reproducing kernel Hilbert space (RKHS). We show that such mapping properties are characterized by the regularity of the Radon–Nikodym density of the conditional law, and establish a simple, verifiable sufficient condition under which the CEO is bounded and Hilbert–Schmidt. For RKHSs norm-equivalent to Sobolev spaces, this condition reduces to Sobolev regularity of the conditional density. The result yields a direct route to validate CME representations and error bounds for Galerkin-type and CME-based estimators. We verify the regularity condition in three settings: nonparametric regression, Bayesian inverse problems, and Koopman operator theory for stochastic dynamical systems. We show in each case that classical regularity results on the underlying probabilistic model imply the required mapping properties. The resulting framework offers a unified perspective on conditional expectation operators across probability, operator theory, kernel methods, and stochastic dynamics.

Keywords. Bayesian inverse problems, conditional expectation operators, conditional mean embeddings, covariance operators, Hilbert–Schmidt operators, kernel ridge regression, Radon– Nikodym derivatives, RKHS, stochastic differential equations, stochastic Koopman operators. Mathematics subject classications. 47B10, 37A30, 46E22, 45P05, 62G08, 62F15 1. Introduction Conditional expectation is among the most fundamental operations in probability theory, and understanding its regularity as a function of the conditioning variable is a recurring theme across applied mathematics. This regularity question can be phrased in terms of the conditional expectation operator (CEO; Mollenhauer and Koltai 2023) K : FY → FX ,

(Kg)(x) := E[g(Y ) | X = x],

1

Optimization-based Control Group, Institute of Mathematics, Technische Universität Ilmenau, Germany ({maximiliano.hertel, karl.worthmann}@tu-ilmenau.de). 2 Department of Mathematics and Computer Science, Freie Universität Berlin, Germany (klebanov@zedat.fu-berlin.de) 3 Faculty of Mathematics, Chemnitz University of Technology, Germany (manuel.schaller@math.tu-chemnitz.de). M. Hertel gratefully acknowledges funding by Carl-Zeiss-Stiftung, project KI-MSO-O: AI-supported analysis, modeling and synthesis (design) of organoids, P2024-02-019. I. Klebanov was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy (EXC-2046/1, project 390685689) of the Berlin Mathematics Research Center MATH+. K. Worthmann is grateful for the support of the German Research Foundation (DFG) within the research unit ALeSCo: Active Learning for Systems and Control, project number 535860958. The authors furthermore thank Mattes Mollenhauer for the insightful and productive discussions. 1

2

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

where X and Y are random variables taking values in spaces X and Y, and FX and FY are spaces of functions on X and Y, respectively. Further, one may seek to represent Kg in a reproducing kernel Hilbert space (RKHS) using a conditional mean embedding (CME; Klebanov et al. (2020); Song et al. (2009)). Both, the CEO and its lifted counterpart, the CME in the RKHS feature space, specialize to well-studied objects across a range of settings: • Nonparametric regression, where Kg recovers the regression function and, more generally, encodes the conditional law of Y given X = x (Caponnetto and De Vito, 2007; Györfi et al., 2002; Tsybakov, 2009); • Bayesian inverse problems, where K computes posterior expectations given observed data (Stuart, 2010); • stochastic dynamical systems, where Kt g(x) = E[g(Xt ) | X0 = x] defines the stochastic Koopman operator or Markov (transition) operator/semigroup (Brunton et al., 2022; Koopman, 1931; Meyn and Tweedie, 2009). A fundamental question across all of these settings concerns the regularity of the image Kg. In other words, for which spaces FX and FY , is the linear operator K well-defined and, possibly, bounded, compact or Hilbert–Schmidt? Similarly, such mapping properties play a crucial role in the RKHS setting: In operator-based CME theory, this representability condition is the key assumption under which the standard operator-based representation of conditional expectations and conditional mean embeddings is valid (Grünewälder et al., 2012; Klebanov et al., 2020). However, this representability assumption is well known to be restrictive and difficult to verify, especially in continuous settings. Closely related questions about the mapping properties of CEOs also arise in stochastic Koopman theory and in the analysis of approximation methods such as (kernel) extended dynamic mode decomposition (EDMD) (Hertel et al., 2025; Williams et al., 2015). The starting point of this work is the observation that these regularity questions can be studied via the Radon–Nikodym density of the conditional law. Assume that, for PX -almost every x ∈ X , the conditional distribution PY |X=x is absolutely continuous with respect to a reference measure νY on Y, and denote its density by
dPY |X=x (y). η(y) (x) := dνY Viewing K as an integral operator with integral kernel η yields the key observation underlying this work, which amounts to the following simple implication for any Hilbert space H of functions on X : η ∈ L2 (νY ; H)

=⇒

K : L2 (νY ) → H is Hilbert–Schmidt.

This yields a conceptually transparent route to RKHS representability: instead of verifying directly that E[g(Y ) | X = • ] ∈ H for all g, it suffices to establish regularity of the conditional density in the conditioning variable, ensuring in particular the applicability of the CME operator-based representation. Based on this observation, this paper makes four main contributions, summarized in Figure 1.1: • Regularity criterion for Conditional Expectation Operators (CEOs). We develop an abstract regularity condition for CEOs based entirely on the regularity of the conditional density (Section 3.1). • Regularity criterion for Conditional Mean Embeddings (CMEs). The operatorbased CME representation is usually available only under an a priori representability assumption on the CEO. We instead derive this assumption from density regularity of the conditional law, turning it into a verifiable condition for the CME covarianceoperator representation in the RKHS feature space (Section 3.2).

under source condition f ∈ H ℓ (X )

Section 4.2: Bayesian inverse problems X = F (Y ) + ε under growth condition (4.5)

Theorem 4.2

Theorem 4.5

Density regularity condition (Assumption 3.2):

Section 3.1: Conditional expectation operator K ∈ HS(L2 (νY ), H)

η(x, y) =

Theorem 3.7

dPY |X=x (y) ∈ L2 (νY ; H) dνY

X

e = PX KPY = ΦX G−1 K XY G−1 Φ⊤ K YY Y XX

Corollary 3.8

† K = CXX CXY ∈ HS(HkY , HkX )

if HkY ,→ L2 (νY ) and under Assumption 2.1

Section 3.3: One- and two-sided Galerkin proj. b = PX K K = ΦX G−1 Φ⊤ K

Theorem 4.11

Section 3.2: Conditional mean embedding † CXY )∗ ϕX µY |X= = (CXX •

error bounds for numerical approx.

Theorem 3.12

XX

under ellipticity condition (Assumption 4.9)

if HkY ,→ L2 (νY ) and under Assumption 2.1

Theorem 3.3

error bound for numerical approx.

Section 4.3: SDEs X = X0 , Y = Xt , dXt = b(Xt ) dt+σ(Xt ) dWt

Theorem 3.16

Section 3.3: Empirical CME formula
−1 K ≈ K̂ε = ΦX GεXX Φ⊤ Y
−1 ⊤ ∗ ε µY |X= ≈ K̂ε ◦ϕX = ΦY GXX ΦX ◦ϕX

IMPLICATIONS OF η ∈ L2 (νY ; H) (Section 3)

Section 4.1: Nonparametric regression Y = f (X) + ε

3

VERIFICATION OF η ∈ L2 (νY ; H) (Section 4)

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

•

Figure 1.1. Overview of the paper. The central density regularity condition η ∈

L2 (νY ; H) (Assumption 3.2) is verified in three application settings under suitable conditions (top, Section 4). It implies regularity of the CEO (Theorem 3.3) and validity of the conditional mean embedding representation (Corollary 3.8), which in turn yield error bounds for Galerkin-type and CME-based numerical approximations (Section 3.3).

• Error bounds for data-driven approximations. We establish error bounds for Galerkin-type (kernel EDMD) and CME-based estimators of the CEO (Section 3.3) with explicit learning rates depending, e.g., on fill distance or eigenvalue decay. • Verification of regularity criterion and consequences. We verify the density regularity condition in three notable settings: nonparametric regression, Bayesian inverse problems, and stochastic differential equations (Section 4). Outline. Section 2 introduces notation and background on RKHSs. Section 3 develops the operator-theoretic consequences of the density regularity condition: mapping properties of CEOs (Section 3.1), verifiable conditions for conditional mean embeddings (Section 3.2), and the resulting approximation error bounds (Section 3.3). Section 4 verifies the density regularity condition in nonparametric regression (Section 4.1), Bayesian inverse problems (Section 4.2), and stochastic differential equations (Section 4.3). Section 5 concludes the work. 2. Preliminaries and Notation Throughout this paper, X , Y denote standard Borel spaces and (Ω, Σ, P) refers to some underlying probability space. We will work with two random variables X : Ω → X , Y : Ω → Y

4

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

and denote the distributions of X, Y and (X, Y ) by PX , PY and PXY respectively. By standard results (Kallenberg, 2021, Thm. 8.5), there exists a regular conditional distribution PY |X , unique up to PX -almost everywhere equality. Equivalently, PY |X is a Markov kernel from X to Y, assigning to each x ∈ X a probability measure PY |X=x on Y. Throughout this paper, we fix one such version of the regular conditional distribution. Accordingly, for every measurable function g : Y → R for which the expectation exists, we write Z g(y) PY |X=x (dy). (2.1) K : FY → FX , (Kg)(x) := E[g(Y ) | X = x] := Y

Here, FX and FY denote spaces of functions on X and Y, respectively, and will be specified in each result separately, whenever relevant. Technically speaking, Kg denotes a measurable function which represents the corresponding conditional expectation, that is, (Kg)(X) = E[g(Y ) | σ(X)] almost surely, where σ(X) denotes the σ-algebra generated by X; such a representative exists as a function of X due to the Doob–Dynkin lemma (Kallenberg, 2021, Lemma 1.14). Consequently, identities involving conditional expectations are understood up to PX -almost everywhere equality, while the chosen representative allows us to regard them as pointwise-defined functions of x. We write N := {1, 2, . . . } and N0 := N ∪ {0}, and for a multi-index α = (α1 , . . . , αd ) ∈ Nd0 we write |α| := α1 + · · · + αd and ∂ α = ∂xα11 · · · ∂xαdd (also denoted Dα ); we write ∂xα when differentiating with respect to a variable x to distinguish it from other arguments. By ∥ • ∥2 we denote the Euclidean norm on Rd ; norms and inner products on other (Hilbert) spaces carry an explicit subscript indicating the space. For an open set D ⊆ Rd , we denote by λ the Lebesgue measure on D. For ℓ ≥ 0, we denote by H ℓ (Rd ) the L2 -based Sobolev space of order ℓ, equipped with the norm Z
ℓ 2 := ∥f ∥H ℓ (Rd ) 1 + ∥ω∥22 |fb(ω)|2 dω Rd

where fb is the Fourier transform of f . If D ⊊ Rd is bounded with Lipschitz boundary (in the following abbreviated by Lipschitz bounded) we define the corresponding space by restriction, that is,  ∥f ∥H ℓ (D) := inf ∥F ∥H ℓ (Rd ) | F ∈ H ℓ (Rd ), F |D = f . For integer ℓ ∈ N0 these definitions agree with the usual weak-derivative Sobolev space W ℓ,2 (Rd ), P 2 with norm ∥f ∥W ℓ,2 (Rd ) = |α|≤ℓ ∥∂ α f ∥2L2 (Rd ) and analogously for Lipschitz bounded D ⊊ Rd . For normed spaces E, F , we denote by L(E, F ) the space of bounded linear operators A : E → F , equipped with the operator norm ∥A∥L(E,F ) := sup ∥Au∥F . ∥u∥E ≤1

If E and F coincide as vector spaces their norms are equivalent, we write E ≃ F . By Cb (X ) we denote the space of continuous and bounded real-valued functions on X , equipped with the supremum norm ∥f ∥∞ := supx∈X |f (x)|. For an operator A : G → H between Hilbert spaces G, H, we denote by A∗ its Hilbert space adjoint and by A† its Moore–Penrose pseudoinverse. If A is compact with non-zero singular values (σi )i∈N , we denote its Hilbert– Schmidt and trace-class norms by X 1/2 ∞ ∞ X 2 ∈ [0, ∞], ∥A∥Tr(G,H) := |σi | ∈ [0, ∞], ∥A∥HS := ∥A∥HS(G,H) := |σi | i=1

i=1

which are finite if and only if A is Hilbert–Schmidt respectively trace-class; the associated spaces are denoted by HS(G, H) and Tr(G, H), respectively. By Id we denote the identity operator on a given space. Finally, for quantities a, b depending on a parameter (e.g. n ∈ N), we write

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

5

a ≲ b if there exists a constant C > 0, independent of the parameter, such that a ≤ Cb, and a ≍ b if a ≲ b and b ≲ a. 2.1. Reproducing Kernel Hilbert Spaces (RKHS). We briefly recall the basics and notation for reproducing kernel Hilbert spaces used throughout the paper; standard references are (Aronszajn, 1950; Berlinet and Thomas-Agnan, 2004; Saitoh and Sawano, 2016; Steinwart and Christmann, 2008; Wendland, 2005). Let kX : X × X → R be a symmetric and positive definite kernel on a set X . We denote by HkX the corresponding reproducing kernel Hilbert space and by ϕX (x) := kX (x, • ) ∈ HkX the canonical feature map. The reproducing property then reads for all h ∈ HkX and x ∈ X .

h(x) = ⟨h, ϕX (x)⟩HkX

(2.2)

When an RKHS is used together with a random variable, the following compatibility assumption ensures that RKHS elements can be regarded as square-integrable random variables and that the associated feature maps and covariance operators (cf. Section 3.2) are well-defined (Klebanov et al., 2020). Assumption 2.1 (Kernel–random variable pair). The pair (X, kX ), consisting of a random variable X : Ω → X and a kernel kX on X , satisfies the following: (i) kX is measurable, symmetric and positive definite, and HkX is separable. (ii) The feature map is square-integrable along X, that is,     E ∥ϕX (X)∥2Hk = E kX (X, X) < ∞. X

(iii) The canonical embedding into L2 (PX ) is injective: for every h ∈ HkX , h=0

PX -a.e.

=⇒

h=0

in HkX .

Assumption 2.1(ii) implies that every h ∈ HkX is square-integrable with respect to PX and that the canonical embedding ιkX ,X : HkX ,→ L2 (PX ),

h 7→ h,

is bounded. Indeed, by the reproducing property and the Cauchy–Schwarz inequality,     ∥h∥2L2 (PX ) = E |h(X)|2 ≤ ∥h∥2Hk E kX (X, X) . X

Assumption 2.1(iii) ensures that this embedding is injective, so that elements of HkX can be identified uniquely with their PX -almost everywhere equivalence classes in L2 (PX ). In particular, we may view HkX as a subspace of L2 (PX ) and write f ∈ HkX for functions f ∈ L2 (PX ) whenever there exists h ∈ HkX such that f = h PX -almost everywhere. The injectivity condition holds, for example, if kX is continuous, X is a topological space, and the topological support of PX equals X . We recall the restriction property of RKHSs. For D ⊆ X , let kX |D×D denote the restriction of kX to D × D. Then the corresponding RKHS HkX (D) := HkX |D×D can be identified with the restriction space (Wendland, 2005, Theorem 10.47)  HkX (D) = {h|D | h ∈ HkX }, ∥g∥Hk (D) = inf ∥h∥HkX | h ∈ HkX , h|D = g . X

(2.3)

We will further make use of the following standard reweighting construction, which modifies a given kernel by a Gaussian weight and thereby enlarges the corresponding RKHS by functions of controlled growth at infinity.

6

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

Lemma 2.2. Let X = Rd , Γ ∈ Rd×d symmetric and strictly positive definite, and k0 be a symmetric and positive definite kernel on Rd . Then, for τ > 0, the reweighted kernel
kτ (x, x′ ) := mτ (x)−1 k0 (x, x′ ) mτ (x′ )−1 , mτ (x) := exp − τ2 ∥x∥2Γ , (2.4) is also a symmetric and positive definite kernel on Rd with HkX = {f | mτ f ∈ Hk0 } and ∥f ∥HkX = ∥mτ f ∥Hk0 . Proof. This is the standard rescaling property of RKHSs, see e.g. (Saitoh and Sawano, 2016, Corollary 2.5). ■ 2.2. Sobolev Reproducing Kernel Hilbert Spaces. A particularly important class of kernels are those whose reproducing kernel Hilbert spaces are norm-equivalent to Sobolev spaces (Wendland, 2005). Two features make them central to this work. First, Sobolev spaces provide the canonical scale of quantitative smoothness in analysis, approximation theory, and statistical learning. Second, and most importantly for our purposes, the abstract density regularity condition η ∈ L2 (νY ; H) then becomes one of Sobolev regularity of the conditional density in the conditioning variable—a reformulation that makes it verifiable through classical smoothness estimates for the conditional law, available in a wide range of models. We verify this Sobolev regularity in Section 4, for nonparametric regression, Bayesian inverse problems, and stochastic differential equations governed by uniformly elliptic diffusion. Proposition 2.3 (Matérn and Wendland kernels yield Sobolev-type RKHSs). Let ρ > 0, d ∈ N and either X = Rd or X ⊆ Rd be a bounded Lipschitz domain. Further, let Kβ , β > 0, denote the modified Bessel function of the second kind and ϕd,j denote the Wendland function of order j ∈ N0 (Wendland, 2005, Definition 9.11), with the additional assumption d ≥ 3 if j = 0. Then the RKHSs corresponding to the Matérn and Wendland kernels β p  21−β p 2β ∥x − x′ ∥2 Kβ 2β ∥x − x′ ∥2 , (2.5) kβMat (x, x′ ) = Γ(β)
Wen (x, x′ ) = ϕd,j ρ−1 ∥x − x′ ∥2 , kd,j (2.6) satisfy d

HkMat (X ) ≃ H β+ 2 (X ), β

d+1

HkWen (X ) ≃ H j+ 2 (X ). d,j

Moreover, the Matérn kernel kβMat is ⌈2β − 1⌉-times continuously differentiable on Rd . Proof. First, let X = Rd . The characterization of the RKHS generated by a Matérn kernel kβMat , β > 0, can be found, for example, in Porcu et al. (2024). Moreover, Da Costa et al. (2026, Appendix B.1) show that the Matérn kernel is exactly ⌈2β − 1⌉-times differentiable for β ∈ /N Mat ⌈2β−1⌉ d d and (2β − 1)-times differentiable for β ∈ N, such that in any case kβ ∈ C (R × R ). For j ∈ N0 , with the additional assumption d ≥ 3 if j = 0, Wendland (2005, Theorem 10.35) Wen . yields the assertion on Wendland kernels kd,j For a bounded Lipschitz domain X ⊊ Rd , both characterizations follow from the restriction result by Wendland (2005, Theorem 10.47 and Corollary 10.48). ■ Proposition 2.4 (A sufficient criterion for Hk (X ) ,→ L2 (X )). Let k(x, x′ ) = ψ(x − x′ ) be a translation-invariant kernel on Rd , where ψ ∈ L1 (Rd ) is continuous and positive definite such that ψb ∈ L∞ (Rd ). Then the corresponding RKHS Hk is continuously embedded into L2 (Rd ). Moreover, for every Lebesgue-measurable set X ⊆ Rd , the corresponding (restricted) RKHS Hk (X ) is continuously embedded into L2 (X ). More precisely, for all f ∈ Hk (X ), b ∞ d ∥f ∥2 ∥f ∥2L2 (X ) ≤ (2π)d/2 ∥ψ∥ L (R ) Hk (X ) .

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

Proof. The proof is provided in Section A.

7

■

By Bochner’s theorem (Wendland, 2005, Theorem 6.6), ψb is the Lebesgue density of the spectral measure associated with the kernel whenever it exists, and the assumption ψb ∈ L∞ (Rd ) follows from ψ ∈ L1 (Rd ) by the Riemann–Lebesgue lemma. Remark 2.5. Proposition 2.4 applies to many well-established kernels such as Matérn and Wendland kernels, cf. Proposition 2.3, Gaussian and Laplace kernels, as well as Cauchy kernels with β > d/2, whose profiles are given by       ∥x∥22 ∥x∥2 ∥x∥22 −β Gauss Lap Cauchy ψ (x) = exp − , ψ (x) = exp − , ψ (x) = 1 + 2 . 2σ 2 σ σ 3. Implications of the Density Regularity Condition η ∈ L2 (νY ; H) In this section, we develop the consequences of the density regularity condition introduced in Assumption 3.2. The key observation is that the CEO (2.1) admits an integral representation whose integral kernel is precisely the Radon–Nikodym density of the conditional law, so that regularity of the density translates directly into mapping properties of K. Section 3.1 establishes these mapping properties, Section 3.2 derives from them the covariance-operator representation of conditional mean embeddings, and Section 3.3 provides error bounds for the resulting numerical approximations. 3.1. Verifiable Regularity Criterion for Conditional Expectation Operators (CEOs). For random variables X and Y taking values in spaces X and Y, we study conditional expectation operators (CEOs) K : FY → FX of the form (2.1). In the context of a stochastic dynamical system (Xt )t≥0 , taking X = X0 and Y = Xt recovers the associated Markov operator, commonly referred to as the (stochastic) Koopman operator. Our first aim is to clarify the mapping properties of K that are crucial in view of data-driven and finite-rank approximations: for which function spaces FX , FY is K a well-defined bounded linear operator, and under which assumptions is it Hilbert–Schmidt, hence compact? Compactness is the classical prerequisite for the convergence of Galerkin-type approximation schemes for integral operators (Kress, 2014, Chapter 13), and underlies the growing body of work on data-driven Koopman operator approximation via (kernel) extended dynamic mode decomposition (EDMD) (Hertel et al., 2025; Klus et al., 2016; Köhne et al., 2025; Williams et al., 2015). For this purpose, we rewrite K as a kernel integral operator Z dPY |X=x (y) (3.1) Kg = Tη g := g(y) η( • , y) νY (dy), where η(x, y) := dνY Y denotes the Radon–Nikodym derivative of the conditional distribution PY |X=x with respect to an arbitrary measure νY on Y (under the assumption of absolute continuity PY |X=x ≪ νY for PX -almost every x ∈ X ). As usual when working with Bochner spaces, we will often view η as a map η : Y → {functions over X } and write η(y) := η( • , y) : X → R. We begin with a general result for integral operators with arbitrary integral kernel η, and later discuss the consequences when η is the Radon–Nikodym derivative introduced above. While the arguments are fairly standard, we give a self-contained proof for the sake of completeness. Proposition 3.1. Let (Y, ΣY , νY ) be a measure space, let H be a separable Hilbert space and let η ∈ L2 (νY ; H). Then the integral operator Z Tη : L2 (νY ) → H, Tη g := g(y) η(y) νY (dy), Y

8

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

is well-defined and Hilbert–Schmidt. In particular, it is bounded, compact, and satisfies ∥Tη ∥L(L2 (νY ),H) ≤ ∥Tη ∥HS = ∥η∥L2 (νY ;H) . Moreover, its adjoint Tη∗ : H → L2 (νY ) is given by (Tη∗ h)(y) = ⟨η(y), h⟩H for all h ∈ H. Proof. For g ∈ L2 (νY ), the map y 7→ g(y)η(y) is Bochner integrable, since Z |g(y)| ∥η(y)∥H νY (dy) ≤ ∥g∥L2 (νY ) ∥η∥L2 (νY ;H) . Y

Hence Tη is well-defined and linear. Further, the operator Uη : H → L2 (νY ) given by (Uη h)(y) = ⟨η(y), h⟩H satisfies Uη h ∈ L2 (νY ), because Z 2 ∥Uη h∥L2 (νY ) = |⟨η(y), h⟩H |2 νY (dy) ≤ ∥h∥2H ∥η∥2L2 (νY ;H) . Y ∗ Moreover, Tη = Uη since, for every g ∈ L2 (νY ) and h ∈ H,

Z ⟨Tη g, h⟩H =

Z

g(y)η(y) νY (dy), h Y

= H

Y

g(y)⟨η(y), h⟩H νY (dy) = ⟨g, Uη h⟩L2 (νY ) .

Let (hm )m∈N be an orthonormal basis of H. By invariance of the Hilbert–Schmidt norm under taking adjoints, Parseval’s identity and Tonelli’s theorem, Z ∞ ∞ Z X X 2 2 ∗ 2 2 ∥Tη ∥HS = ∥Tη ∥HS = ∥Uη hm ∥L2 (νY ) = |⟨η(y), hm ⟩H | νY (dy) = ∥η(y)∥2H νY (dy) m=1

m=1 Y

Y

= ∥η∥2L2 (νY ;H) , proving the claim.

■

We now specialize Proposition 3.1 to the CEO, taking η to be the Radon–Nikodym derivative from (3.1). The condition η ∈ L2 (νY ; H) is then a regularity condition on the transition density in the conditioning variable; for H = H ℓ (X ), it amounts to Sobolev regularity of the sections x 7→ η(x, y), an assumption satisfied by many Markov processes with sufficiently regular transition densities, such as uniformly elliptic diffusions (revisited in Section 4.3). The theorem below shows how this H-regularity transfers to classical target spaces by composition with embeddings: for sufficiently regular X ⊆ Rd equipped with νX = λ, Sobolev embedding theory determines whether K remains Hilbert–Schmidt or trace class when regarded as an operator into L2 (νX ), and whether it becomes compact when regarded as an operator into Cb (X ). For ℓ > d/2, the same Sobolev spaces can also be realized as RKHSs through Matérn or Wendland kernels, cf. Proposition 2.3, thereby connecting the present regularity criterion with RKHS interpolation theory to obtain explicit error rates of Galerkin-type approximations in Section 3.3.1. The result follows from Proposition 3.1 and the standard stability properties of compact, Hilbert–Schmidt and trace class operators under composition (see, e.g., Conway, 2007; Reed and Simon, 1980). Assumption 3.2 (Density regularity condition). (X , ΣX ) and (Y, ΣY ) are standard Borel spaces, X : Ω → X and Y : Ω → Y are random variables. We fix a regular conditional distribution PY |X= of Y given X. Further, νY is a σ-finite measure on (Y, ΣY ) such that PY |X=x is absolutely continuous with respect to νY for PX -almost every x ∈ X , with jointly measurable dP Radon–Nikodym derivative η(x, y) := Ydν|X=x (y). Further, H is a separable Hilbert space of Y 2 real-valued functions on X such that η ∈ L (νY ; H), where we identify η(y) with the section η( • , y) for y ∈ Y, that is, we view η as Bochner-measurable map Y → H, y 7→ η(y) = η( • , y). •

Theorem 3.3. Let Assumption 3.2 hold and, whenever H = H ℓ (X ) with ℓ ≥ 0, let either X = Rd or X ⊊ Rd Lipschitz bounded. Then: (a) K ∈ HS(L2 (νY ), H) with ∥K∥L(L2 (νY ),H) ≤ ∥K∥HS(L2 (νY ),H) = ∥η∥L2 (νY ;H) .

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

9

(b) If H is continuously embedded in L2 (νX ) for some measure νX on X with embedding operator E : H ,→ L2 (νX ), then K ∈ HS(L2 (νY ), L2 (νX )) with ∥K∥HS(L2 (νY ),L2 (νX )) ≤ ∥E∥L(H,L2 (νX )) ∥η∥L2 (νY ;H) . In particular, this is the case for H = H ℓ (X ) with ℓ ≥ 0 and νX = λ. (c) If, in addition, E is Hilbert–Schmidt, then K ∈ Tr(L2 (νY ), L2 (νX )) with ∥K∥Tr(L2 (νY ),L2 (νX )) ≤ ∥E∥HS(H,L2 (νX )) ∥η∥L2 (νY ;H) . In particular, this is the case for Lipschitz bounded X ⊊ Rd and H = H ℓ (X ) with ℓ > d/2 and νX = λ. (d) If H = H ℓ (X ) with ℓ > d/2, then K ∈ L(L2 (νY ); Cb (X )) is compact. Proof. (a) follows from Proposition 3.1, since, for g ∈ L2 (νY ), the Radon–Nikodym identity for conditional expectations gives Kg = Tη g for PX -almost every x ∈ X . (b) holds because K ∈ HS(L2 (νY ), L2 (νX )) is Hilbert–Schmidt as the composition of the Hilbert–Schmidt operator K ∈ HS(L2 (νY ), H) and the bounded operator E ∈ L(H, L2 (νX )), with the norm bound following from the corresponding submultiplicativity property. This holds in particular for H = H ℓ (X ) with ℓ ≥ 0 since in this case E is bounded by definition. (c) holds since K ∈ Tr(L2 (νY ), L2 (νX )) is trace class as the composition of the Hilbert– Schmidt operators K ∈ HS(L2 (νY ), H) and E ∈ HS(H, L2 (νX )), again, with the corresponding submultiplicativity argument for the norms. This holds in particular for Lipschitz bounded X and H = H ℓ (X ) with ℓ > d/2, since in this case E is Hilbert-Schmidt (Maurin, 1961). (d) holds because K ∈ L(L2 (νY ), Cb (X )) is compact as the composition of the Hilbert– Schmidt operator K ∈ HS(L2 (νY ), H) and the bounded embedding H ℓ (X ) ,→ Cb (X ) (by the Sobolev embedding theorem; Adams and Fournier 2003, Theorem 4.12). ■ Remark 3.4. The integral operator from Proposition 3.1 remains well-defined and bounded as a map Tη : Lp (νY ) → H whenever η ∈ Lq (νY ; H), for conjugate exponents 1 ≤ p, q ≤ ∞ with p−1 + q −1 = 1: Hölder’s inequality yields Z |g(y)| ∥η(y)∥H νY (dy) ≤ ∥g∥Lp (νY ) ∥η∥Lq (νY ;H) . Y

In this sense, Proposition 3.1 corresponds to the Hilbert case p = q = 2. An interesting special case is p = ∞ (hence q = 1): choosing H ≃ H ℓ (X ) with ℓ > d/2 (under the domain assumptions of Theorem 3.3), the Sobolev embedding H ℓ (X ) ,→ Cb (X ) yields boundedness of Tη : Cb (Y) → Cb (X ). More generally, if differentiation under the integral sign is justified, the same Hölder argument applies pointwise to spatial derivatives of the kernel: for every multi-index α and x ∈ X , |∂xα (Kg)(x)| ≤ ∥g∥Lp (νY ) ∥(∂xα η)(x, • )∥Lq (νY ) . In particular, if supx∈X ∥(∂xα η)(x, • )∥Lq (νY ) < ∞ for every |α| ≤ ℓ, then the CEO maps Lp -observables into functions with uniformly bounded derivatives up to order ℓ, i.e., Kg ∈ W ℓ,∞ (X ) for every g ∈ Lp (νY ). Remark 3.5 (Spectral structure and Koopman approximation). The Hilbert–Schmidt conclusions above imply compactness and hence provide a basic spectral approximation principle. In particular, if K ∈ HS(L2 (νY ), L2 (νX )), then it admits a singular value expansion X X Kg = σn ⟨g, un ⟩L2 (νY ) vn , σn2 < ∞. n≥1

n≥1

with orthonormal systems (un )n∈N ⊂ L2 (νY ) and (vn )n∈N ⊂ L2 (νX ). Truncating this expansion yields finite-rank approximations of K, which is the functional-analytic mechanism behind many spectral and Galerkin-type Koopman approximation methods. If, in addition, X = Y

10

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

and the Markov process is reversible with respect to the invariant measure ν = νX = νY , then the corresponding Koopman operator is self-adjoint on L2 (ν), and the singular value expansion reduces to an eigenfunction expansion (Klus et al., 2018), which provides a classical spectral setting for transfer and Koopman operators. 3.2. Verifiable Regularity Criterion for Conditional Mean Embeddings (CMEs). In this section, we specialize the preceding regularity criterion to the operator-theoretic representation of conditional mean embeddings (CMEs), a central tool in the kernel methods literature for representing conditional distributions and a versatile tool across machine learning, including nonparametric approaches to regression, probabilistic inference, and reinforcement learning (Grünewälder et al., 2012; Muandet et al., 2017; Song et al., 2009). For kernels kX , kY on X , Y with RKHSs HkX , HkY and feature maps ϕX , ϕY (cf. Section 2), the object to be represented is the conditional kernel mean embedding (CME)1 µY |X=x = E[ϕY (Y ) | X = x],

x ∈ X.

(3.2)

The operator-theoretic identity underlying this representation—expressing µY |X=x , and more generally the CEO K, in terms of covariance and cross-covariance operators of the joint law of (X, Y )—can be traced back to Fukumizu et al. (2004), Song et al. (2009), Klebanov et al. (2020), Park and Muandet (2020) and has since been studied extensively, also for other CEOs such as stochastic Koopman operators (Klus et al., 2020b). These works typically take the representability condition, that the CEO maps between the prescribed RKHSs, i.e. (3.4) below, as a standing assumption, either imposed directly or encoded through source and embedding conditions (Fukumizu et al., 2013; Grünewälder et al., 2012; Klebanov et al., 2021; Li et al., 2024). This assumption is well known to be restrictive and hard to verify directly. The main point of this section is that the abstract regularity criterion from Theorem 3.3 instead derives it from regularity of the conditional density. In particular, when HkX is norm-equivalent to a Sobolev space, representability reduces to Sobolev regularity of the transition density in the conditioning variable. Throughout this subsection, we use the following assumption and notation: Assumption 3.6 (RKHS pair setting). (X , ΣX ) and (Y, ΣY ) are standard Borel spaces, X : Ω → X and Y : Ω → Y are random variables. We fix a regular conditional distribution PY |X= of Y given X. Further, kX : X × X → R and kY : Y × Y → R are kernels such that both pairs (X, kX ) and (Y, kY ) satisfy Assumption 2.1. •

For Hilbert-space valued random variables U ∈ L2 (Ω, Σ, P; G) and V ∈ L2 (Ω, Σ, P; H), we denote the uncentred covariance operator by C[U, V ] := E[U ⊗ V ] ∈ L(H, G), where the expectation is understood as a Bochner integral (Diestel and Uhl, 1977, Section II.2). Here, for g ∈ G and h ∈ H, the rank-one operator g ⊗ h : H → G is defined by (g ⊗ h)h′ := ⟨h, h′ ⟩H g for h′ ∈ H. Under Assumption 3.6, we denote the kernel mean embeddings and the uncentred kernel covariance and cross-covariance operators by µY := E[ϕY (Y )] ∈ HkY ,

CY Y := C[ϕY (Y ), ϕY (Y )],

CY X := C[ϕY (Y ), ϕX (X)],

µX := E[ϕX (X)] ∈ HkX ,

CXY := C[ϕX (X), ϕY (Y )],

CXX := C[ϕX (X), ϕX (X)],

as well as the conditional mean embedding (CME) µY |X=x by (3.2). Note that µY |X=x is well-defined for PX -almost every x ∈ X . The reproducing property (2.2) translates to kernel mean embeddings:   ⟨g, µY ⟩HkY = ⟨g, E[ϕY (Y )]⟩HkY = E ⟨g, ϕY (Y )⟩HkY = E[g(Y )], g ∈ HkY , (3.3) 1Strictly speaking, we define these objects on X

Y

:= {x ∈ X | E[∥ϕY (Y )∥2Hk | X = x] < ∞}, which has full

PX measure by Assumption 2.1(ii), and set them to zero on X \ XY .

Y

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

RKHS feature spaces HkX , HkY

original spaces X , Y

x ∈ X , y ∈ Y, X ∼ PX , Y ∼ PY , (X, Y ) ∼ PXY

11

embed ϕX , ϕY

ϕX (x), ϕY (y), ϕX (X), ϕY (Y ), µX , µY , CXX , CY Y , CXY , CY X conditional mean embedding

conditioning on X = x

(Y | X = x) ∼ PY |X=x (Kg)(x) = E[g(Y ) | X = x]

embed ϕY

† µY |X= = (CXX CXY )∗ ϕX , •

† CXY g Kg = ⟨g, µY |X= ⟩HkY = CXX •

Figure 3.1. Conditioning in the original spaces (left) and in the RKHS feature spaces (right), linked by the canonical feature maps ϕX , ϕY . After embedding, conditioning becomes a linear transformation of the feature vector ϕX (x). and similarly ⟨g, µY |X=x ⟩HkY = E[g(Y ) | X = x] PX -almost everywhere. We briefly recall the basic CME representation result following Klebanov et al. (2020). Figure 3.1 illustrates the two viewpoints on conditioning connected by this result: conditioning of X, Y in the original spaces X , Y (left column), and its counterpart after embedding into the RKHS feature spaces HkX , HkY , where it is captured by the covariance-operator representation (right column). The appeal of the latter description is that conditioning becomes a linear transformation of the embedded objects, which can be estimated from samples of PXY by linear-algebraic operations alone. The following theorem is a simplified version of Klebanov et al. (2020, Theorem 5.3). Theorem 3.7 (CME representation; Klebanov et al. 2020, Theorem 5.3). Under Assumption 3.6, if Kg := E[g(Y ) | X = • ] ∈ HkX for every g ∈ HkY , (3.4) † then the linear operator CXX CXY : HkY → HkX is bounded and, PX -almost everywhere, † µY |X= = (CXX CXY )∗ ϕX ,

(3.5)

•

† Kg = CXX CXY g,

for every g ∈ HkY .

(3.6)

Note that (3.6) follows from (3.5) by the reproducing properties (2.2) and (3.3). Conversely, (3.6) determines the conditional mean embedding through its action on all g ∈ HkY . The preceding theorem shows that the CME representation is available once the representability condition (3.4) has been verified. The following corollary gives precisely such a verification criterion in terms of the Radon–Nikodym derivative of the conditional law. Corollary 3.8 (CME representation under the density regularity condition). Under Assumptions 3.2 and 3.6 with H = HkX , if HkY ,→ L2 (νY ) continuously, then K ∈ HS(HkY , HkX ), the CME assumption (3.4) is satisfied for every g ∈ HkY , and the CME representations (3.5)–(3.6) hold. Proof. Since Assumption 3.2 holds with H = HkX , Theorem 3.3(a) yields K ∈ HS(L2 (νY ), HkX ). Hence, K ∈ HS(HkY , HkX ) as the composition of the continuous embedding HkY ,→ L2 (νY ) and a Hilbert–Schmidt operator; in particular, Kg ∈ HkX for every g ∈ HkY . Thus (3.4) holds, and the assertion follows from Theorem 3.7. ■ Remark 3.9 (Special cases of Corollary 3.8). The above corollary includes the following cases:

12

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

(i) The function dPY |X=x (y) dνY is the transition density of the Markov kernel PY |X=x with respect to νY . A particularly important case is when νY is the Lebesgue measure on Y ⊆ Rd . In this setting, the embedding HkY ,→ L2 (νY ) can be ensured by the spectral density condition in Proposition 2.4, which is satisfied for many common kernels. (ii) If X = Rd or X ⊆ Rd is a bounded Lipschitz domain and η ∈ L2 (νY ; H) for an RKHS H ≃ HkX with ℓ ≥ 1, then the CME representations hold. This applies in particular to the following kernels, cf. Proposition 2.3: (a) Matérn kernels kX = kβMat with 2β + d ≤ 2ℓ, cf. (2.5); Wen with 2j + d + 1 ≤ 2ℓ, cf. (2.6). (b) Wendland kernels kX = kd,j (iii) If νY = PY , then HkY ,→ L2 (νY ) is automatically satisfied under Assumption 2.1. From a Bayesian viewpoint, dPY |X=x (y) η(x, y) := dPY is the likelihood function Lx (y) up to an x-dependent normalizing constant Z(x) = E[Lx (Y )]; cf. Section 4.2. η(x, y) =

In Sections 4.1–4.3, we illustrate how to apply and leverage Corollary 3.8 in various applications, including Bayesian inverse problems and conditional mean embeddings for Koopman operator approximation. 3.3. Error Bounds for Conditional Expectation Operator Approximation. The abstract regularity results of Section 3.1 and the covariance-operator representation derived in Section 3.2 naturally give rise to finite-dimensional approximations of the CEO K defined in (2.1). On the one hand, once K is known to be a Hilbert–Schmidt operator, it can be approximated by Galerkin projections onto finite-dimensional subspaces of the underlying RKHSs. On the other hand, the covariance-operator representation (3.6) yields the empirical conditional mean embedding estimator that is widely used in literature on kernel methods (Grünewälder et al., 2012; Klus et al., 2020b). In this section, we derive approximation error bounds for such estimators and relate the resulting convergence rates to the regularity of the conditional density established in the previous subsections. In particular, we consider numerical approximation via three approaches: b = PX K, (i) the one-sided Galerkin projection K e (ii) the two-sided Galerkin projection K := PX KPY , also known as kernel EDMD (Williams et al., 2015), † (iii) an approximation K̂ε to the CME representation K = CXX CXY , cf. (3.6), where PX denotes the orthogonal projection onto span{ϕX (x1 ), . . . , ϕX (xnX )} ⊂ HkX for data points X = {x1 , . . . , xnX } ⊂ X ⊆ Rd (and PY accordingly). For this purpose, we introduce the following notation: X Notation 3.10. Let X = (xi )ni=1 be a collection of data points in X and let ϕX : X → HkX denote the feature map associated with a kernel kX on X . We define

ΦX = (ϕX (x1 ), . . . , ϕX (xnX )),

and

nX nX nX fX := Φ⊤ , X f := (⟨ϕX (xi ), f ⟩HkX )i=1 = (f (xi ))i=1 ∈ R

for f ∈ HkX , and the corresponding unregularized and regularized Gram matrices nX GXX = Φ⊤ X ΦX = (kX (xi , xj ))i,j=1 , n

GεXX = GXX + nX ε IdnX ,

Y where ε ≥ 0. Additionally, let Y = (yi )i=1 be a collection of data points in Y and let ϕY : Y → HkY denote the feature map associated with a kernel kY on Y, and define ΦY , gY and GY Y

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

Notation & symbols

13

(on X ; the corresponding objects on Y are defined analogously)

domain X

X feature matrix ΦX = (ϕX (xi ))ni=1

random variable X : Ω → X

nX evaluation vector fX = Φ⊤ X f = (f (xi ))i=1

X data X = (xi )ni=1

Gram matrix GXX = Φ⊤ X ΦX

kernel kX

conditional-evaluation matrix K XY = E[kY (yj , Y ) | X = xi ] i,j

RKHS HkX

bXX = 1 ΦX Φ⊤ , C bY X = 1 ΦY Φ⊤ empirical covariance operators C X X n n

feature map ϕX

fill distance hX = supx∈X minxi ∈X ∥x − xi ∥2 One-sided projection

Two-sided projection

CME regression

operator

K : L2 (νY ) → HkX

K : HkY → HkX

K : HkY → HkX

approximation ansatz

b = PX K (projecting K images onto trial space)

e = PX KPY (projection K in domain and range)

† K = CXX CXY (covariance representation)

required data

X and zm ∼ PY |X=xi

(i) iid

X, Y and zm ∼ PY |X=xi

K-estimator

\ ⊤ ΦX G−1 XX ΦX K

−1 ⊤ c ΦX G−1 XX K XY GY Y ΦY

deterministic error

range projection

domain and range projection

regularization

statistical error

Monte Carlo estimation of (Kg)X

Monte Carlo estimation of K XY

empirical estimation of CXX and CXY

density regularity

enters range projection error bounds

enters range projection error bounds

implies RKHS inclusion, eigenvalue decay of CXX

main result

Theorem 3.12: fill distance bound (range)

Theorem 3.12: fill distance bound (range and domain)

Theorem 3.16: learning rate via eigenvalue decay

(i) iid

i.i.d.

pairs (xi , yi ) ∼ PXY
−1 ⊤ ΦY ΦX GεXX

Table 3.1. Notation overview (top) and comparison of the three approximation strategies for CEO (bottom). Projection-based methods approximate the action of the operator on trial spaces built from the nodes X, Y ; CME methods approximate the operator itself via empirical covariance operators. The hat in \ ⊤ K denotes the Monte Carlo estimate of g 7→ (Kg) Φ X based on the samples X (i) zm . similarly. We denote M   X
(i) kY (yj , zm ) K XY := E[kY (yj , Y ) | X = xi ] i,j ≈ M −1 m=1

i,j

cXY , =: K (i)

where we used a Monte–Carlo approximation with independent samples zm ∼ PY |X=xi . In the case of independent pairs (xi , yi ) ∼ PXY , with nX = nY =: n, the empirical (cross-)covariance operators are defined by n n X X 1 1 bXX = 1 bY X = 1 ϕX (xi ) ⊗ ϕX (xi ) = ΦX Φ⊤ , C ϕY (yi ) ⊗ ϕX (xi ) = ΦY Φ⊤ C X X. n n n n i=1

i=1

Then we obtain the following formulas for the one-sided and two-sided projections of the CEO,

14

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

⊤ PX = ΦX G−1 XX ΦX ,

−1 ⊤ PX KPY = ΦX G−1 XX K XY GY Y ΦY ,

PX Kg = ΦX G−1 XX (Kg)X ,

−1 PX KPY g = ΦX G−1 XX K XY GY Y gY ,

cXY and, similarly, (Kg)X is approxwhere in practice the matrix K XY is approximated by K
P (i) imated by M −1 M m=1 g(zm ) i . Further, the empirical CEO and (regularized) CME read (for the final identities see, e.g., Klus et al. 2020b, Remark 2.9)

−1 ⊤ bXX + ε Id −1 C b ⊤ = Φ X Gε ΦY , K ≈ K̂ε := C (3.7) YX XX
−1 ⊤ µY |X= ≈ K̂ε∗ ◦ ϕX = ΦY GεXX (3.8) ΦX ◦ ϕ X . •

Table 3.1 summarizes the notation and different approximation philosophies. Remark 3.11 (Which approximation for which application?). The three schemes differ in the b and K e need condata they require (cf. Table 3.1): the projection-based approximants K (i) ditional samples zm ∼ PY |X=xi , whereas the CME estimator K̂ε only needs joint samples (xi , yi ) ∼ PXY . The former are thus natural for stochastic dynamical systems (Section 4.3), where conditional sampling amounts to simulating the process forward from xi , while the CME estimator is preferable in Bayesian inverse problems (Section 4.2), where conditional sampling means costly posterior sampling, and in nonparametric regression (Section 4.1), where data naturally arrive as i.i.d. pairs (xi , yi ) ∼ PXY . 3.3.1. Error Bounds for Galerkin-type Approximations. The regularization results of Section 3.1 imply that CEOs often map into smooth function spaces, such as Sobolev spaces and their RKHS realizations. In the following result, we exploit this regularity gain to derive approxb and K, e the latter of which is imation error bounds for the Galerkin-type approximations K also known as (kernel) extended dynamic mode decomposition (EDMD) (Williams et al., 2015) and has since become a standard tool for data-driven Koopman operator approximation, with quantitative error bounds established by, e.g., Köhne et al. (2025), Hertel et al. (2025), Philipp et al. (2025). For this, we briefly recall the class of kernels induced by radial basis functions (RBF), which are a special case of translation-invariant kernels characterized by a function ψ : R≥0 → R such that k(x, x′ ) = ψ(∥x − x′ ∥2 ) for all x, x′ . Theorem 3.12 (Convergence rates for projection-based CEO approximations). Let HkX and HkY be RKHSs over X and Y, respectively. Assume that Assumption 3.2 holds with H = HkX . Let F be a normed linear space of functions over X such that H ,→ F continuously, and assume nY X ⊂ X and Y = (yi )i=1 ⊂ Y be that HkY ,→ L2 (νY ) continuously. Let data sets X = (xi )ni=1 given. Then, using Notation 3.10, the following hold: b L(L2 (ν ),F ) ≤ ∥η∥L2 (ν ;H) ∥Id −PX ∥L(H,F ) , where ∥η∥L2 (ν ;H) < ∞. (a) ∥K − K∥ Y Y Y (b) There exist CX , CY > 0 such that e L(H ,F ) ≤ CY ∥η∥L2 (ν ;H ) ∥Id −PX ∥L(H ,F ) ∥K − K∥ k k Y k Y

X

X

+ CX ∥η∥L2 (νY ;Hk ) ∥Id −PY ∥L(Hk ,L2 (νY )) . X

Y

(c) Generic RBF error rates: Let X ⊂ Rd be a Lipschitz bounded domain and k

X be a radial basis function kernel satisfying Assumption 2.1(i) such that kX ∈ C 2mX (X × X ), mX ∈ N0 , with continuous derivatives on X ×X of order 2mX . Then there exists C > 0 such that b L(L2 (ν ),C (X )) ≤ C ∥η∥L2 (ν ;H) hmX . ∥K − K∥ (3.9) X Y Y b

If, additionally, Y ⊂ Rd is Lipschitz bounded, νY ≪ λ, and kY ∈ C 2mY (Y × Y) is a radial basis function kernel satisfying Assumption 2.1(i) with continuous derivatives on

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

15

Y × Y of order 2mY , mY ∈ N0 , then there exist CX , CY > 0 such that e L(H ,C (X )) ≤ CY ∥η∥L2 (ν ;H ) hmX + CX ∥η∥L2 (ν ;H ) hmY . ∥K − K∥ Y X b k Y Y k k Y

(3.10)

X

X

In particular, this applies to Matérn kernels k⋆ = kβMat , β > 0, for which m⋆ = ⌈β⌉ − 1, where ⋆ ∈ {X , Y}. Wen is a Wendland kernel for some ⋆ ∈ (d) Improved Wendland error rates: If k⋆ = kd,j {X , Y}, then, in the corresponding fill-distance terms in (3.9) or (3.10), the exponent may be replaced by m⋆ = j + 1/2. Proof. Under Assumption 3.2, the operator K is well-defined and bounded by Theorem 3.3(a), and satisfies ∥K∥L(L2 (νY ),H) ≤ ∥η∥L2 (νY ;H) < ∞. Factoring K − PX K = (I − PX )K together with submultiplicativity of operator norms then immediately implies (a). If additionally the embedding EY : HkY ,→ L2 (νY ) is continuous, the restricted operator K ∈ L(HkY , HkX ). We now decompose: e L(H ,F ) ≤ ∥(Id −PX )K∥L(H ,F ) + ∥PX K(Id −PY )∥L(H ,F ) ∥K − K∥ k k k Y

Y

Y

≤ ∥ Id −PX ∥L(Hk ,F ) ∥K∥L(L2 (νY ),Hk ) ∥EY ∥L(Hk ,L2 (νY )) X

X

Y

+ ∥PX ∥L(Hk ,F ) ∥K∥L(L2 (νY ),Hk ) ∥ Id −PY ∥L(Hk ,L2 (νY )) . X

X

Y

By contractivity of the orthogonal projection, ∥PX ∥L(Hk ,Hk ) ≤ 1 and continuity (hence, X X boundedness) of HkX ,→ F, there exists CX > 0 such that ∥PX ∥L(Hk ,F ) ≤ CX . Finally, X continuity of the embedding EY yields ∥EY ∥L(Hk ,L2 (νY )) ≤ CY , proving (b). Y

If X ⊂ Rd is a Lipschitz bounded domain (and therefore satisfies the interior cone condition; see Adams and Fournier (2003, Paragraph 4.11) and Köhne et al. (2025, Appendix A)), Wendland (2005, Theorem 11.13) and the subsequent discussion then imply the existence of C > 0 such that X ∥f − PX f ∥∞ ≤ C hm X ∥f ∥H

∀f ∈ H.

X Then ∥Id −PX ∥L(H,Cb (X ) = sup∥f ∥H =1 ∥f − PX f ∥∞ ≤ C hm X by definition, so (3.9) follows from (a) with F = Cb (X ). If additionally Y is Lipschitz bounded and νY ≪ λ, we have ∥f ∥L2 (νY ) ≤ νY (Y)1/2 ∥f ∥∞ ≤ e e > 0. Cλ(Y)1/2 ∥f ∥∞ , and therefore ∥Id −PY ∥L(Hk ,L2 (νY )) ≤ C∥Id −PY ∥L(Hk ,Cb (Y)) with C Y

Y

If HkY is induced by a radial basis function kernel kY ∈ C 2mY (Y × Y), then (3.10) follows analogously as (3.9). In particular, by Proposition 2.3, the Matérn kernel kβMat is ⌈2β − 1⌉-times continuously differentiable. Furthermore, we have ⌈2β − 1⌉ ≥ 2⌈β − 1⌉ = 2(⌈β⌉ − 1), as can be seen the following way. For β ∈ N, it holds that 2⌈β − 1⌉ = 2(β − 1) < 2β − 1 = ⌈2β − 1⌉. For β = l + δ with l ∈ N, δ ∈ (0, 1), we have 2⌈β − 1⌉ = 2l and 2β − 1 = 2l − 2δ + 1 ∈ (2l − 1, 2l + 1), so ⌈2β − 1⌉ ≥ 2l = 2⌈β − 1⌉. Hence, we have kβMat ∈ C 2m (Rd × Rd ) for m = ⌈β − 1⌉ = ⌈β⌉ − 1. Consequently, its derivatives of order 2m are also continuous on X × X and, in the case of Lipschitz bounded Y ⊂ Rd , on Y × Y. Thus, (3.9) and (3.10) hold with m⋆ = ⌈β⌉ − 1, concluding the proof of (c). Wen , j ∈ N, ⋆ ∈ {X , Y}, Wendland (2005, Theorem Finally, for Wendland RKHS k⋆ = kd,j j+1/2

11.17) yields the sharper projection error bound ∥Id −PX ∥L(Hk ,Cb (X )) ≤ C hX , which X together with (a) implies (3.9) with convergence rate m⋆ = j + 1/2. By the same argument as j+1/2 above, under the according assumptions this also implies ∥Id −PY ∥L(Hk ,L2 (νY )) ≤ C hY , Y such that (3.10) follows analogously as (b) with convergence rate m⋆ = j + 1/2, proving (d). ■

16

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

Remark 3.13 (Projection as interpolation). The projection error bounds in Theorem 3.12(c) rely on the reproducing kernel Hilbert space structure. Indeed, the orthogonal projection PX onto the span of kernel sections coincides with the kernel interpolant at the sampling points X (see, e.g., Köhne et al., 2025). Consequently, the estimates reduce to classical interpolation error bounds from scattered data approximation (Wendland, 2005). The use of kernel feature dictionaries in kernel EDMD approximations (Bold et al., 2025; Williams et al., 2015) thereby circumvents the “finite-data error” of EDMD in arbitrary Hilbert spaces, where the inner products in the solution of the regression problem have to be approximated empirically. Remark 3.14 (Optimal choice of regularization order). Note that the “range projection” error rate in (3.9) and (3.10) improves with increasing kernel regularity, for example in the case of Sobolev RKHS in Theorem 3.12(d). However, since in this case the upper bound ∥η∥L2 (νY ;H) typically grows2 with regularization order mX , the ‘optimal choice of smoothness’ to achieve the tightest error bound depends on the specific transition density η(x, y) and its partial derivatives ∂xα η(x, y) with respect to the conditioning variable x. Remark 3.15 (One-sided vs. two-sided projection schemes). Classical kernel EDMD (Bold et al., 2025; Philipp et al., 2024, 2025; Zhang and Zuazua, 2023) typically employs the two-sided e = PX KPY , usually under the mapping assumption KHk ⊂ Hk .3 In approximation K Y X b = PX K acts on the full input space L2 (νY ) contrast, the one-sided Galerkin approximation K and projects only the image of the operator. The bounds (3.9) and (3.10) reveal a fundamental trade-off. Since the one-sided approximation does not restrict the domain of K, it fully exploits the regularizing effect of conditional expectation and incurs only the image-space projection error. In the RBF setting, this error is governed by the fill distance in the (typically much smoother) RKHS HkX . The two-sided approximation, on the other hand, introduces an additional domain projection error through the embedding HkY ,→ L2 (νY ), so that the overall convergence is limited by the slower of the domain and image approximation rates. This additional approximation error is compensated by a substantial computational advantage: the action of K is represented by a single finite-dimensional propagation matrix K XY , which can be assembled once and subsequently applied to arbitrary observables. In contrast, the one-sided approximation requires the conditional expectation (Kg)(xi ) = E[g(Y ) | X = xi ] to be estimated separately for each observable g. 3.3.2. Error Bounds for CME-based Approximation. By the choice of data points X and Y , the b and K e impose deterministic, fill-distance-type error rates. In projection-based approximants K † CXY g, turn, approximations based on the CME representation Kg = E[g(Y ) | X = • ] = CXX cf. (3.6), impose the RKHS setting K : HkY → HkX , but approximate K directly from sampled data pairs (xi , yi ) rather than through explicit conditional sampling. This regression-type interpretation, and the resulting empirical estimator K̂ε , were first developed by Grünewälder et al. (2012) and have since been studied extensively as a nonparametric approximation problem for CEOs (Mollenhauer and Koltai, 2023). By Corollary 3.8, the CME representation need not simply be assumed but follows from verifiable conditions, so our abstract results also yield error bounds for CME estimators. Convergence rates for such CME-type regularized regression range from the foundational vectorvalued rates of Caponnetto and De Vito (2007), through the operator-free consistency rate of 2Generally, the growth of the operator norm with ℓ is expected to be faster than the fill distance rate can

compensate for. Not only does the number of derivative terms entering the Sobolev norm equal ℓ+d , but d higher-order derivatives with respect to x also might produce larger integrals over the bounded domain X . 3Often, the case X = Y with identical kernels k = k is considered, such that the RKHS H X Y kX is assumed (Hou et al., 2026; Klus et al., 2020b) or proven (Hertel et al., 2025; Köhne et al., 2025) to be invariant under the Koopman operator K.

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

17

Park and Muandet (2020) and the general-regularization-scheme rates of Mollenhauer et al. (2024), to the sharpest, source-condition-adaptive rates of Li et al. (2024), which also cover the misspecified case. The following result is a direct consequence of Li et al. (2024, Theorem 3) with α = β = 1 and γ = 0, combined with Corollary 3.8, which provides the well-specifiedness of the CME in our setting. Theorem 3.16 (Convergence rate for CME approximation). Let Assumptions 3.2 and 3.6 hold with H = HkX and HkY ,→ L2 (νY ), and assume that kX is bounded, i.e., supx∈X kX (x, x) < ∞. Let (λj )j∈N denote the non-increasing eigenvalues of CXX , and suppose there exist c > 0 and p ∈ (0, 1] such that λj ≤ c j −1/p for all j ∈ N. Then the regularized CME estimator bXX + εn Id)−1 C bXY based on n i.i.d. pairs (xi , yi ) ∼ PXY with εn ≍ n−1/(1+p) K̂εn := (C satisfies, for all τ ≥ log(5), 1 − 1+p

∥K̂εn − K∥2HS(Hk , L2 (PX )) ≤ τ 2 A n

(3.11)

Y

n -probability at least 1 − 5e−τ , for some constant A > 0 independent of τ and all with PXY sufficiently large n. In particular, the assumption λj ≤ c j −1/p always holds with p = 1 and rate n−1/2 . If moreover HkX ≃ H ℓ (X ) on a bounded Lipschitz domain X ⊂ Rd with ℓ > d/2 (e.g. for Matérn or Wendland kernels, cf. Proposition 2.3), then p = d/(2ℓ) < 1 and the rate improves to n−2ℓ/(2ℓ+d) .

Proof. In the notation of Li et al. (2024), we verify the assumptions of their Theorem 2 as follows. Corollary 3.8 yields K ∈ HS(HkY , HkX ), so K ∗ ∈ HS(HkX , HkY ), which is the wellspecified case of Li et al. (2024) with output space Y = HkY , covariate RKHS HX = HkX , and regression function F ∗ = K ∗ ◦ ϕX : X → HkY , giving (SRC) with β = 1. Boundedness of kX gives (EMB) at level α = 1 since [HkX ]1 ≃ HkX ,→ L∞ (PX ) with constant A = supx∈X kX (x, x)1/2 . The eigenvalue decay assumption is (EVD) with the given p. Since β + p = 1 + p > 1 = α for all p ∈ (0, 1], we are always in Case 2 of Li et al. (2024, Theorem 3) with γ = 0, yielding ∥K̂ε∗n − K ∗ ∥2HS(L2 (PX ), Hk ) ≤ τ 2 A n−1/(1+p) . Y

Since the Hilbert–Schmidt norm is invariant under taking adjoints, we obtain the stated bound. The regularized CME estimator K̂ε∗ coincides with the estimator of Li et al. (2024) as shown in Mollenhauer and Koltai (2023, Section 3); see also the proof of Li et al. (2024, Theorem 3). Since ∥CXX ∥Tr = E[kX (X, X)] < ∞ by Assumption 2.1, CXX is trace class on HkX , implying λj = O(j −1 ), i.e., (EVD) with p = 1 and rate n−1/2 . For HkX ≃ H ℓ (X ) on a bounded 2 2 Lipschitz domain X ⊂ Rd with ℓ > d/2, note that CXX = (E L (PX ) )∗ E L (PX ) : HkX → HkX (Li et al., 2024, Section 2), so the eigenvalues of CXX are the squared singular values of the embedding HkX ,→ L2 (PX ). By norm-equivalence HkX ≃ H ℓ (X ) and the classical decay 2 2 aj (E L (X ) ) ≍ j −ℓ/d of the approximation numbers of the Sobolev embedding E L (X ) : H ℓ (X ) ,→ L2 (X ) (Edmunds and Triebel, 1996), we obtain λj (CXX ) ≍ j −2ℓ/d , giving p = d/(2ℓ) and rate n−2ℓ/(2ℓ+d) . ■ Remark 3.17 (Eigenvalue decay on unbounded domains). On unbounded domains such as X = Rd , the Sobolev embedding H ℓ (Rd ) ,→ L2 (Rd ) is not compact, so the sharp rate p = d/(2ℓ) of Theorem 3.16 is not available in general, and p = 1 is the best guaranteed by trace class alone. For HkX ≃ H ℓ (Rd ) and PX with sufficiently fast-decaying tails, a sharper rate p < 1 may be recoverable via weighted Sobolev embedding theory (Edmunds and Triebel, 1996), but this falls outside the scope of the present work. Remark 3.18 (Strong norm bounds from extra smoothness). If the transition density satisfies η ∈ L2 (νY ; H s (X )) for some s > ℓ, then applying Li et al. (2024, Theorem 3) with γ = 1 yields

18

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

a bound in the stronger norm HS(HkY , HkX ) of the form s−ℓ − s+d/2

∥K̂εn − K∥2HS(Hk , Hk ) ≤ τ 2 A n Y

X

with high probability. In particular, stronger smoothness of the conditional law (s larger) gives a faster rate, reflecting the regularizing effect of the transition kernel. Remark 3.19 (CME convergence for trajectory data). Theorem 3.16 assumes i.i.d. data pairs (xi , yi ) ∼ PXY . In applications where data arise as a single trajectory of a stochastic process, the same rate is expected to hold P∞up to logarithmic corrections, provided the process is α-mixing with summable coefficients t=1 α(t) < ∞ and the kernel kX is bounded. This condition is satisfied, for instance, for geometrically ergodic Markov processes (with α(t) = O(e−ct )), including solutions to SDEs satisfying common assumptions (cf. Assumption 4.9 in Section 4.3). A rigorous proof would replace the Bernstein inequality for i.i.d. variables used in the variance bound of Li et al. (2024) with the analogous concentration inequality for α-mixing processes from (Mollenhauer et al., 2022, Theorem 21), and is left for future work. 4. Verification of the Density Regularity Condition η ∈ L2 (νY ; H) In the previous section, we established that the condition η ∈ L2 (νY ; H) implies a range of regularity, representation, and approximation results for conditional expectation operators. The purpose of the upcoming section is to show that Assumption 3.2 can be verified in several important problem classes. While the underlying models differ substantially, we exploit problem-specific regularity of the conditional density in every outlined application to establish the required Hilbert space regularity. The abstract results of Section 3 apply in each of these settings, once the standing assumptions have been verified. As the applications below address different communities with different interests, we restate only a selection of these results in each case. Remark 4.1 (CME rates for bounded vs. unbounded conditioning domains). Whenever kX is bounded, i.e., supx∈X kX (x, x) < ∞ (in particular, whenever X is bounded and kX continuous), the CME convergence rate of Theorem 3.16 applies directly, since it only requires this boundedness together with the eigenvalue decay of CXX . For unbounded X = Rd , we instead work with the exponentially reweighted kernel kτ from Lemma 2.2, whose reweighting compensates for the lack of spatial integrability but whose diagonal is necessarily unbounded. To the best of our knowledge, none of the existing convergence-rate results for CME-type or vector-valued ridge regression estimators (Caponnetto and De Vito, 2007; Li et al., 2024; Mollenhauer et al., 2024; Park and Muandet, 2020) cover kernels with unbounded diagonal; we also state only qualitative Hilbert–Schmidt and CME-representability results below whenever X is unbounded, and reserve quantitative rates for the case of bounded X . Establishing such a rate for the reweighted setting appears genuinely challenging and we leave this as an interesting direction for future work. 4.1. Nonparametric Regression (Forward Uncertainty). In statistical learning theory and nonparametric statistics, a canonical object of study is the regression model Y = f (X) + ε,

(4.1)

where X : Ω → X is the covariate, Y : Ω → R the response, f : X → R the unknown signal, and ε a centred noise variable independent of X. The classical goal is to estimate the regression function f ∗ (x) = E[Y | X = x] = f (x) from i.i.d. observations (xi , yi ) ∼ PXY . Beyond point estimation of f ∗ , the CME µY |X=x ∈ HkY provides a unified representation of PY |X=x from which E[g(Y ) | X = x] can be recovered for any g ∈ HkY via the reproducing property (3.3), for instance for (forward) uncertainty quantification (Gneiting et al., 2007).

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

19

The following theorem verifies the density regularity condition η ∈ L2 (νY ; HkX ) (Assumption 3.2) in the regression setting (4.1) under the source condition f ∈ H ℓ (X ) (Caponnetto and De Vito, 2007; Fischer and Steinwart, 2020), and the subsequent corollary establishes † the validity of the CME representation K = CXX CXY and the optimal estimation rate of Theorem 3.16. Theorem 4.2 (Sobolev regularity criterion for nonparametric regression). Let (Y, νY ) = (R, λ), X ⊆ Rd and consider the regression model (4.1), where f ∈ H ℓ (X ) for some ℓ ∈ N with ℓ > d/2 and ε is independent of X and admits a zero-mean Lebesgue density pε ∈ W ℓ,2 (R).4 Then the
dPY |X=x Radon–Nikodym derivative η(x, y) := dν (y) exists and satisfies η(x, y) = pε y − f (x) . Y Further, η ∈ L2 (νY ; H),

K ∈ HS(L2 (νY ), H)

with

∥K∥HS(L2 (νY ),H) = ∥η∥L2 (νY ;H) ,

(4.2)

if either of the following holds: (A) bounded domain, Sobolev RKHS: X ⊂ Rd is a bounded Lipschitz domain and H ≃ H ℓ (X ). In this case we additionally have K ∈ Tr(L2 (νY ), L2 (νX )) for νX = λ; (B) unbounded domain, reweighted Sobolev RKHS: X = Rd , k0 is a kernel on Rd with Hk0 ≃ H ℓ (Rd ), and H = HkX where kX = kτ is the Gaussian-reweighted kernel (2.4) for τ > 0 and symmetric, strictly positive definite Γ ∈ Rd×d .5 Proof. The proof is given in Section A.1.

■

Corollary 4.3 (CME representation for nonparametric regression). Let the assumptions of Theorem 4.2 hold and assume that the canonical embedding HkX ,→ L2 (PX ) is injective. In case (A), assume additionally that kX is measurable; in case (B), assume that k0 is continuous and that 
 E exp τ ∥X∥2Γ < ∞. (4.3) Then (X, kX ) satisfies Assumption 2.1. Further, if kY is a kernel on R such that (Y, kY ) also satisfies Assumption 2.1 and HkY ,→ L2 (νY ) continuously, then (a) K ∈ HS(HkY , HkX ), the CME assumption (3.4) is satisfied for every g ∈ HkY , and the CME representations (3.5)–(3.6) hold; (b) in case (A), the CME estimator K̂ε in (3.7) satisfies the estimation rate (3.11).6 Proof. The proof is given in Section A.1.

■

4.2. Bayesian Inverse Problems (Backward Uncertainty). The results of Corollary 3.8— and hence the CME assumption (3.4) and representations (3.5)–(3.6)—apply in particular to the standard setting of Bayesian inverse problems (Stuart, 2010), where X is a noisy measurement of F (Y ) corrupted by Gaussian noise, under rather mild tail conditions on F (Y ). Throughout this section, we use the following notation for a reweighted inner product on Rd and the corresponding norm, where Γ ∈ Rd×d denotes symmetric and positive definite matrix and x, y ∈ Rd : ⟨x, y⟩Γ := x⊤ Γ−1 y, ∥x∥2Γ := ⟨x, x⟩Γ = x⊤ Γ−1 x. Assumption 4.4 (Bayesian setting). X = Rd , Y is a standard Borel space, Y : Ω → Y a random variable, and X = F (Y ) + ε, ε ∼ N (0, Γ), 4This covers Gaussian noise ε ∼ N (0, σ 2 ), σ > 0, for every ℓ ∈ N. 5The reweighting cannot be omitted: since R |η(x, y)|2 dy = ∥p ∥2 2

ℓ

d

R

ε L2 (R) > 0 for every x ∈ R

invariance, one has η ∈ / L (νY ; H (R )) already for ℓ = 0, regardless of f . 6In case (B), Theorem 3.16 does not apply, since sup x∈Rd kτ (x, x) = ∞, cf. Remark 4.1.

d

by translation

20

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

where F : Y → Rd is measurable, Γ ∈ Rd×d is symmetric and strictly positive definite, and ε is dPY |X=x independent of Y . We denote by η(x, y) := dP (y) the Radon–Nikodym derivative of the Y posterior with respect to the prior, that is,  
exp ⟨x, F (y)⟩Γ − 12 ∥F (y)∥2Γ exp − 12 ∥x − F (y)∥2Γ
η(x, y) = R = , (4.4) Ξ(x) exp − 21 ∥x − F (ỹ)∥2Γ PY (dỹ) where Z Ξ(x) := Y

exp ⟨x, F (y)⟩Γ − 21 ∥F (y)∥2Γ PY (dy) > 0.

Recall from Lemma 2.2 the Gaussian-reweighted kernel kτ (x, x′ ) = mτ (x)−1 k0 (x, x′ ) mτ (x′ )−1
τ 2 with weight mτ (x) = exp − 2 ∥x∥Γ , cf. (2.4), whose RKHS consists of all functions f with mτ f ∈ Hk0 . Theorem 4.5 (Weighted Sobolev regularity criterion for Bayesian inverse problems). Let Assumption 4.4 hold. Let ℓ ∈ N0 and let k0 be a continuous, translation-invariant, symmetric and positive definite kernel on X = Rd such that Hk0 ≃ H ℓ (X ).7 Let τ > 0 and kX (x, x′ ) := kτ (x, x′ ) with the reweighted kernel kτ given by (2.4). Assume that Λ := log Ξ ∈ C ℓ (X ) and that its derivatives grow at most polynomially, that is, there exist CΛ > 0 and r ≥ 1 such that, for every multi-index α ∈ Nd0 with 1 ≤ |α| ≤ ℓ and all x ∈ X , |Dα Λ(x)| ≤ CΛ (1 + ∥x∥2 )r .

(4.5)

Then η(y) ∈ HkX for PY -almost every y ∈ Y. Moreover, η ∈ L2 (PY ; HkX ), and K ∈ HS(L2 (PY ), HkX ) with ∥K∥HS(L2 (PY ),Hk ) = ∥η∥L2 (PY ;Hk ) if X

τ >1

or

∃λ > 0 :

X

τ>


 1 and E exp λ∥F (Y )∥2Γ < ∞. 1+λ

Proof. The proof is given in Section A.2.

(4.6) ■

Theorem 4.5 is formulated in terms of the growth condition (4.5) on the log-partition function Z   Λ(x) = log Ξ(x), Ξ(x) = exp ⟨x, F (y)⟩Γ − 12 ∥F (y)∥2Γ PY (dy). Y

In the Bayesian setting of Assumption 4.4, the Radon–Nikodym derivative (4.4) shows that the posterior distribution PY | X=x is obtained from the prior PY by exponential tilting with potential ⟨x, a(y)⟩ − 21 ∥F (y)∥2Γ , where a(y) := Γ−1 F (y). The derivatives of Λ admit the probabilistic representations ∇Λ(x) = E[a(Y ) | X = x],

D2 Λ(x) = C(a(Y ) | X = x).

More generally, Dα Λ(x) equals the cumulant of order |α| of a(Y ) under the conditional law PY | X=x . Hence the growth condition (4.5) amounts to a uniform control of the cumulants of a(Y ) under the posterior laws PY | X=x . The following proposition provides several concrete sufficient conditions ensuring that this assumption holds. Their logical relations are illustrated in Figure 4.1. Proposition 4.6 (Concrete sufficient conditions for the log-partition growth condition (4.5)). The log-partition growth condition (4.5) in Theorem 4.5 is satisfied if any of the following hold: (a) F is bounded (or F is PY -essentially bounded). (b) Y is a Banach space, Y is Gaussian, and F (y) = Ay + b with b ∈ Rd and a bounded linear operator A : Y → Rd . 7By Proposition 2.3, Matérn and Wendland kernels satisfy this condition for certain choices of parameters.

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

(a) F bounded

(d) strong log-concavity of the pushforward law of a(Y ) = Γ−1 F (Y )

(b) affineGaussian case

bounded support of a(Y ) under the posterior laws

(c) uniform linear bound on conditional mean and centered moments of a(Y ) = Γ−1 F (Y )

log-partition growth condition (4.5)

21

Figure 4.1. Logical relations between the sufficient conditions in Proposition 4.6 ensuring the log-partition growth condition (4.5). (c) Let a(y) := Γ−1 F (y) and assume Ξ ∈ C ℓ (Rd ). For x ∈ Rd , define the conditional mean and conditional centered moments by   ma1 (x) := E[a(Y ) | X = x], maj (x) := E ∥a(Y ) − ma1 (x)∥j2 | X = x , j = 2, . . . , ℓ. Assume that there exists a constant c > 0, such that, for every x ∈ Rd ,
max ∥ma1 (x)∥2 , ma2 (x), . . . , maℓ (x) ≤ c (1 + ∥x∥2 ). (d) The pushforward law of a(Y ) = Γ−1 F (Y ) is strongly log-concave. Proof. The proof is given in Section A.2.

■

Corollary 4.7 (CME representation for Bayesian inverse problems). Let the assumptions of Theorem 4.5 hold and let τ > 0 and λ > 0 be such that8 
 1 λ <τ < and E exp λ∥F (Y )∥2Γ < ∞. (4.7) 1+λ 1 + 2λ Then the pair (X, kX ) satisfies Assumption 2.1. Further, if kY is a kernel on Y such that (Y, kY ) also satisfies Assumption 2.1, then K ∈ HS(HkY , HkX ), the CME assumption (3.4) is satisfied for every g ∈ HkY , and the CME representations (3.5)–(3.6) hold. Proof. The proof is given in Section A.2.

■

Remark 4.8 (Role of the τ -reweighting and the exponential-moment assumption in (4.6) and (4.7)). The kernel reweighting in the definition of kτ (cf. Lemma 2.2) can, in general, not be omitted. Indeed, even in the degenerate case where F ≡ c is constant, one has η( • , y) ≡ 1 for every y, so η( • , y) ∈ / L2 (Rd ) and hence η( • , y) cannot belong to an RKHS of Sobolev d type on R . The reweighting by mτ replaces the requirement η( • , y) ∈ H ℓ (Rd ) by the weaker condition mτ ( • )η( • , y) ∈ H ℓ (Rd ), ensuring η( • , y) ∈ HkX . By Theorem 4.5, this pointwise RKHS-membership is guaranteed under the stated assumptions for every τ > 0. If τ is chosen sufficiently large, for example τ > 1 as in (4.6), this even yields η ∈ L2 (PY ; HkX ). Note that for τ > 0 the kernel kτ is unbounded, hence Theorem 3.16 is not applicable, cf. Remark 4.1. On the other hand, increasing τ makes the diagonal of the reweighted kernel grow faster, and therefore makes it harder to satisfy the integrability condition E[kX (X, X)] < ∞ from Assumption 2.1. Thus, condition (4.7) balances the choice of τ so that both η ∈ L2 (PY ; HkX )

and

(X, kX ) satisfies Assumption 2.1 √

8Note that the two conditions on τ in (4.7) are jointly satisfiable only if λ > 1+

2

5

, the golden ratio.

22

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS


 hold simultaneously under the additional assumption E exp λ∥F (Y )∥2Γ < ∞ for some λ > √ 1+ 5 2 , which is a natural exponential-tail condition on the pushforward random variable F (Y )

and is satisfied, for example, in each of the following situations: (i) F is bounded (or F is PY -essentially bounded). (ii) Y is Gaussian in a Banach space Y, F (y) = Ay + b with a bounded linear operator A : Y → Rd , and the covariance matrix ΣF of the (Gaussian) random variable F (Y ) satisfies
1 √ , (4.8) λmax Γ−1/2 ΣF Γ−1/2 < 1+ 5 where λmax denotes the largest eigenvalue (that is, the covariance of F (Y ) is sufficiently small relative to Γ). (iii) more generally, F (Y ) is sub-Gaussian with parameter matrix ΣF satisfying (4.8). (iv) more generally still, F (Y ) has sufficiently light √tails so that ∥F (Y )∥2Γ admits an exponential moment of order strictly larger than 1+2 5 . 4.3. Koopman Operator of Stochastic Differential Equations. In this subsection, we verify the density regularity assumption ηt ∈ L2 (νY ; H) (Assumption 3.2) for uniformly elliptic stochastic differential equations. Throughout, we take Y = Rd and the reference measure νY = λ to be the d-dimensional Lebesgue measure. From the perspective of conditional expectation operators, stochastic differential equations provide a dynamical counterpart to the regression setting of Section 4.1. While regression studies a single conditional expectation x 7→ E[Y | X = x], stochastic dynamics give rise to an entire family of conditional expectation operators describing the evolution of observables over time, namely the Koopman semigroup (Kt )t≥0 . Stochastic differential equations therefore constitute one of the principal applications for Koopman analysis of stochastic dynamical systems (Črnjarić-Žic et al., 2020) with broad application areas such as molecular dynamics (Nateghi and Nüske, 2025), climate science (Santos Gutiérrez et al., 2021), and quantum systems (Klus et al., 2022), among many others. Kernel-based numerical methods such as kernel EDMD and related covariance operator representations have been widely studied for the approximation of Koopman operators (Klus et al., 2016, 2020a,b; Williams et al., 2015), while more recent works develop statistical learning approaches in RKHSs (Hou et al., 2026; Kostic et al., 2022) and approximation error bounds (Hertel et al., 2025; Philipp et al., 2025). The regularity theory developed in this work provides a unified analytical foundation for these results by relating the smoothing properties of stochastic Koopman operators directly to regularity of the underlying transition densities. Consequently, our key analytical ingredient is the existence of such smooth transition densities satisfying quantitative derivative estimates with respect to the initial (or backward ) variable x. These bounds allow us to establish the density regularity condition for suitable Sobolev-type Hilbert spaces H. For this, we consider the time-homogeneous Itô diffusion dXt = b(Xt ) dt + σ(Xt ) dWt ,

t > 0,

X0 = x ∈ X ⊆ Rd ,

(4.9)

where (Wt )t≥0 is a d-dimensional Brownian motion, b : Rd → Rd denotes the drift, and σ : Rd → Rd×d the diffusion coefficient. Under standard Lipschitz and growth assumptions, (4.9) admits a unique strong solution (Xt )t≥0 (Øksendal, 2003, Theorem 5.2.1). Its transition densities (ηt )t>0 induce the family of stochastic Koopman operators Z := Kt g(x) E[g(Xt ) | X0 = x] = g(y) ηt (x, y) νY (dy) (4.10) Y

for bounded, measurable g : Rd → R, which coincide precisely with the CEO (2.1) for X = X0 and Y = Xt .

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

23

The family (Kt )t≥0 forms a Markov semigroup associated with the diffusion (4.9). Moreover, the transition densities (ηt )t>0 correspond to the fundamental solution of the parabolic operator associated with the SDE. Under the ellipticity and smoothness assumptions below, classical parabolic regularity theory yields Gaussian-type bounds on the derivatives of ηt w.r.t. x. Integrating these bounds in the forward variable y implies the required Sobolev regularity of ηt and thereby verifies Assumption 3.2. For the remainder of this section, we fix the following assumptions on the coefficients of the SDE (4.9). Assumption 4.9 (Regular Itô diffusion with uniformly elliptic noise). We consider the SDE (4.9) and assume: (i) Uniform ellipticity: There exists κ > 0 such that κ ∥ξ∥22 ≤ ∥σ(x)⊤ ξ∥22 for all x, ξ ∈ Rd , or equivalently, a(x) = σ(x)σ(x)⊤ is uniformly positive definite. (ii) Coefficient regularity: For some ℓ ∈ N≥1 , the SDE coefficients satisfy b ∈ Cbℓ (Rd , Rd ) and σ ∈ Cbℓ (Rd , Rd×d ), where Cbℓ (S, T ) denotes the space of ℓ-times continuously differentiable functions f : S → T whose derivatives up to order ℓ are bounded. While not necessarily minimal (see Remark 4.13), Assumption 4.9 provides a convenient regime in which classical parabolic regularity theory yields the required density regularity condition ηt ∈ L2 (νY , H ℓ (X )). For this, we first gather the following properties, which are standard results in both the partial differential equations (PDE) and stochastic analysis literature (Friedman, 1964; Stroock and Varadhan, 2006). Proposition 4.10 (SDE density regularity and derivative bounds). Let Assumption 4.9 hold and fix T > 0. Then the SDE (4.9) admits a unique strong solution (Xt )t∈[0,T ] . Moreover, for every t ∈ (0, T ], the conditional law PXt |X0 =x is absolutely continuous w.r.t. νY = λ with transition density ηt satisfying (4.10) for all bounded measurable g : Rd → R. Further, for every multi-index |α| ≤ ℓ, the map x 7→ ηt (x, y) admits weak partial derivatives ∂xα ηt , and there exist constants Cα , κ > 0 independent of t, x, y such that   ∥x − y∥22 α −(|α|+d)/2 . |∂x ηt (x, y)| ≤ Cα t exp −κ t Proof. The proof is given in Section A.3.

■

We note that the classical results from Friedman (1964) were originally formulated for SDEs with time-dependent coefficients b = b(t, x) and σ = σ(t, x). Hence, the arguments in the proof of Proposition 4.10 extend to this case, see Section A. We restrict our presentation to time-homogeneous diffusions because the associated CEOs then form a Koopman semigroup (Kt )t≥0 satisfying Kt+s = Kt Ks , which is fundamental for prediction by iterative application of the learned operator approximation over one time interval. The Gaussian derivative bounds of Proposition 4.10 are sufficient to verify the density regularity condition ηt ∈ L2 (νY ; H) from Assumption 3.2. The remaining issue is the choice of the Hilbert space H on the conditioning domain X . As the following theorem shows, this naturally splits into two cases, cf. Remark 4.1: (A) If X ⊂ Rd is a bounded Lipschitz domain, one may take H ≃ H ℓ (X ). (B) If X = Rd , one instead works with a reweighted Sobolev-type RKHS H = Hkτ , where kτ (x, x
′ ) = mτ (x)−1 k0 (x, x′ ) mτ (x′ )−1 with Hk0 ≃ H ℓ (X ) and weight mτ (x) = exp − τ 2 2 ∥x∥Γ , cf. (2.4). Theorem 4.11 (Regularity of Koopman operators for uniformly elliptic SDEs). Let X ⊆ Rd with νX := λ, let (Y, νY ) = (Rd , λ), and let Assumption 4.9 hold with ℓ ∈ N≥1 . Let ηt denote the transition density and Kt the Koopman operator (4.10) associated with the SDE (4.9) at time t > 0. Then ηt ∈ L2 (νY ; H),

Kt ∈ HS(L2 (νY ), H)

with

∥Kt ∥HS(L2 (νY ),H) = ∥ηt ∥L2 (νY ;H) ,

24

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

if either of the following holds: (A) bounded domain, Sobolev RKHS: X ⊂ Rd is a bounded Lipschitz domain and H ≃ H ℓ (X ). In this case we additionally have Kt ∈ HS(L2 (νY ), L2 (νX ))

and

Kt ∈ Tr(L2 (νY ), L2 (νX )) if ℓ > d/2;

(B) unbounded domain, reweighted Sobolev RKHS: X = Rd , k0 is a kernel on Rd with Hk0 ≃ H ℓ (Rd ), and H = HkX where kX = kτ is the Gaussian-reweighted kernel (2.4) for τ > 0 and symmetric, strictly positive definite Γ ∈ Rd×d . Proof. The proof is given in Section A.3.

■

By Theorem 4.11, under sufficiently strong regularity assumptions on the coefficients of the SDE, the abstract results of Section 3 immediately yield Hilbert–Schmidt properties of the associated stochastic Koopman operators. The only distinction between the bounded and unbounded settings lies in the choice of the target Hilbert space: bounded domains naturally lead to Sobolev spaces, whereas on Rd exponential reweighting compensates for the lack of spatial integrability. In the former case, the results of Section 3.3 immediately become available. Corollary 4.12 (Error rates for Galerkin-type approximations of SDE Koopman operators). Let X ⊂ Rd be Lipschitz bounded, let Assumption 4.9 hold with N ∋ ℓ > d/2 and let H ≃ H ℓ (X ) be an RKHS induced by (cf. Proposition 2.3) (i) the Matérn kernel with parameter β = ℓ − d/2, where we assume ℓ > d/2 + 1 and set m = ⌈β⌉ − 1, or (ii) the Wendland kernel with parameter j = ℓ − (d + 1)/2, where we assume d to be odd and d ≥ 3 if j = 0, and set m = j + 1/2. Using Notation 3.10, we obtain for X = X0 , Y = Xt and a data set X = {xi }ni=1 , ∥Kt − PX Kt ∥L(L2 (νY ),Cb (X )) ≤ C ∥ηt ∥L2 (νY ;H) hm X. Proof. The proof is given in Section A.3.

(4.11) ■

Corollary 4.12 distinguishes between Matérn kernels, which induce H ≃ H ℓ (X ) for arbitrary smoothness levels ℓ > d/2, and Wendland kernels, which can only realize Sobolev spaces of integer and half-integer smoothness. In turn, whenever applicable, Wendland kernels yield a slightly sharper fill distance rate. The analysis in this subsection focuses on uniformly elliptic SDEs (cf. Assumption 4.9) with Lipschitz bounded X and Y = Rd , leading naturally to the one-sided Galerkin approximation considered in Corollary 4.12. This raises the question of how the CEO regularity theory developed in this work extends to broader classes of stochastic dynamics and to additional approximation schemes. Remark 4.13 (Extensions of CEO regularity and approximation results for SDEs). Several additional results from Sections 3.2 and 3.3 may admit analogous applications in the SDE setting. • Error rates for two-sided Galerkin-type approximations. The two-sided projection estimates from Theorem 3.12 are not directly applicable in our present SDE setting, since they require the state space Y to be a bounded Lipschitz domain to obtain a fill distance bound on the domain of the Koopman operator. For uniformly elliptic SDEs on Rd , however, the transition law typically has unbounded support. A possible remedy could be to extend the present analysis, in particular the transition density regularity analysis, to diffusions confined to bounded domains, e.g., by considering reflected diffusions or suitably modified coefficients. • CME representation and approximation bounds. For any RKHS HkY ,→ L2 (λ), in both cases of Theorem 4.11 the assumptions required for the covariance-operator representation of conditional mean embeddings (Corollary 3.8) are satisfied. Consequently, the corresponding CME-based approximation and convergence results from

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

25

Section 3.3.2 apply equally to stochastic Koopman operators associated with uniformly elliptic SDEs, provided the assumptions for the CME convergence rate (Theorem 3.16) are met. However, the latter requires a bounded kernel, which only holds in case (A) but fails in case (B) of Theorem 4.11, cf. Remark 4.1. A detailed investigation of the CME-based approximation results lies beyond the scope of the present work. Beyond the presented uniformly elliptic setting, relaxed or modified assumptions on the SDE coefficients might provide notable extensions to the CEO regularity and approximation results of this subsection. • Relaxing uniform ellipticity. Smooth transition densities can also be obtained under substantially weaker non-degeneracy conditions, most notably Hörmander’s bracket condition via Malliavin calculus (Hairer, 2011; Kusuoka and Stroock, 1985; Malliavin, 1978), or through differentiability properties of the associated stochastic flow (Kunita, 1990; Stroock, 1981). In the case of state space Y = Rd , the key question then becomes the availability of quantitative (e.g., Gaussian-type) bounds for spatial derivatives as in Proposition 4.10. • Relaxing regularity and boundedness of coefficients. Gaussian bounds for transition densities of uniform elliptic diffusions were originally proven for bounded measurable coefficients (Aronson, 1967) and have recently been extended to settings with linearly growing, unbounded drift by Menozzi et al. (2021), while still preserving on uniform ellipticity. Moreover, under suitable Hölder regularity assumptions on the coefficients, the latter work also derives Gaussian estimates for arbitrary spatial derivatives of the transition density. Extending the Sobolev regularity analysis developed here to this broader setting would substantially enlarge the class of stochastic Koopman operators covered by our framework. In particular, it would cover benchmark models such as the Ornstein–Uhlenbeck process, whose drift is linear and therefore falls outside Assumption 4.9, despite the process admitting an explicit Gaussian density satisfying the required derivative estimates, see Example 4.14. Example 4.14 (Ornstein–Uhlenbeck process: Explicit density under unbounded drift). On a Lipschitz bounded initial domain X ⊂ Rd and Y = Rd , we consider an Ornstein–Uhlenbeck process, i.e., the solution to the linear SDE dXt = AXt dt + B dWt ,

(4.12)

with constant matrices A, B ∈ Rd×d . Since the drift function b : Rd → Rd , b(x) = Ax is clearly unbounded, we cannot directly apply Theorem 4.11(A). However, the solution process is GaussRt ⊤ ian and satisfies (Xt | X0 = x) ∼ N (etA x, Qt ), where Qt = 0 esA BB ⊤ esA ds, see, e.g., (Vatiwutipong and Phewchean, 2019, Corollary 1). Assume that BB ⊤ > 0, such that Qt is positive definite (and the uniform ellipticity condition Assumption 4.9(i) is satisfied) and we obtain the
tA explicit transition density ηt (x, y) = (2π)−d/2 det(Qt )−1/2 expR − 21 (y − etA x)⊤ Q−1 t (y − e x) . Hence, the Koopman operator Kt g(x) = E[g(Xt ) | X0 = x] = Rd g(y) ηt (x, y) dy is well-defined and bounded. We now verify the Sobolev-type integrability directly as in the proof of Theorem 4.11. Since the exponent is quadratic in (x, y), every x-derivative yields a polynomial prefactor, i.e., ∂xα ηt (x, y) = fα (x, y) ηt (x, y), where fα is a polynomial of degree at most |α|. Consequently,

tA 2 |∂xα ηt (x, y)|2 ≤ Cα 1 + ∥x∥2|α| + ∥y∥2|α| ηt (x, y)2 ≤ C 1 + ∥x∥2|α| + ∥y∥2|α| e−c∥y−e x∥ with suitable constants Cα , C, c > 0, where we used the explicit Gaussian form of ηt . Hence, for fixed x and with the change of variables z = y − etA x, Z Z
tA 2 α 2 |∂x ηt (x, y)| dy ≤ C 1 + ∥x∥2|α| + ∥y∥2|α| e−c∥y−e x∥ dy Rd

Rd

26

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

Z =C Rd

2 e + ∥x∥2|α| ) 1 + ∥x∥2|α| + ∥z + etA x∥2|α| e−c∥z∥ dz ≤ C(1

e > 0, where we used finiteness of Gaussian moments. Finally, integrating over the with C R R bounded domain X yields X Rd |∂xα ηT (x, y)|2 dy dx < ∞ for all |α| ≤ ℓ. This proves ηt ∈ L2 (νY ; H ℓ (X )) for νY = λ and arbitrary ℓ ∈ N. In particular, the C ∞ -density means that the Koopman operator Kt associated with the SDE is ‘arbitrarily smoothing’. 5. Conclusions In this work, we traced the regularizing effect of conditional expectation operators (CEOs) back to the regularity of the associated conditional densities. Our main result establishes a simple, verifiable regularity criterion: if the conditional density belongs to a suitable Bochner space L2 (νY ; H), then the corresponding CEO defines a bounded, and in fact Hilbert–Schmidt, operator from L2 (νY ) into H. This provides a unified perspective on regularity, compactness, and smoothing properties of CEOs. Beyond its operator-theoretic interest, this criterion has two immediate consequences. On the one hand, it provides a verifiable sufficient condition for the representability assumptions underlying conditional mean embeddings. Rather than assuming a priori that conditional expectations take values in a prescribed RKHS, we show that this property follows from regularity of the conditional density, justifying operator-based CME representations through analytical properties of the underlying probabilistic model. On the other hand, the regularity criterion yields quantitative approximation results for CEOs. In particular, it provides error bounds for Galerkin-type and CME-based approximations whose convergence rates are determined by the smoothing properties of the underlying conditional density. Since stochastic Koopman operators are themselves CEOs, these results apply directly in the setting of stochastic dynamical systems. To demonstrate the practical verifiability of the proposed abstract regularity criterion, we established it in three representative applications. In nonparametric regression, smoothness assumptions on the regression function and noise model imply the required density regularity and thereby justify the operator-based CME representation. For Bayesian inverse problems, we derived sufficient conditions ensuring that posterior densities induce regularization into weighted Sobolev-type RKHSs. For uniformly elliptic stochastic differential equations, classical transition density regularity results and derivative bounds yield smoothing properties of the associated stochastic Koopman operators. Overall, the results show that operator-theoretic properties of CEOs can be reduced to verifiable regularity properties of the associated conditional densities. By replacing abstract RKHS representability assumptions with analytical conditions that can be checked in concrete probabilistic models, the proposed framework unifies conditional mean embeddings and stochastic Koopman theory across many application domains and provides a natural bridge between probability theory, operator theory, and numerical approximation. References R. A. Adams and J. J. F. Fournier. Sobolev spaces, volume 140 of Pure and Applied Mathematics (Amsterdam). Elsevier/Academic Press, Amsterdam, second edition, 2003. D. G. Aronson. Bounds for the fundamental solution of a parabolic equation. Bull. Amer. Math. Soc., 73:890–896, 1967. doi:10.1090/S0002-9904-1967-11830-5. N. Aronszajn. Theory of reproducing kernels. Trans. Amer. Math. Soc., 68:337–404, 1950. doi:10.2307/1990404. A. Berlinet and C. Thomas-Agnan. Reproducing kernel Hilbert spaces in probability and statistics. Kluwer Academic Publishers, Boston, MA, 2004. doi:10.1007/978-1-4419-9096-9.

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

27

L. Bold, F. M. Philipp, M. Schaller, and K. Worthmann. Kernel-based Koopman approximants for control: flexible sampling, error analysis, and stability. SIAM J. Control Optim., 63(6): 4044–4071, 2025. doi:10.1137/24M1715945. S. L. Brunton, M. Budišić, E. Kaiser, and J. N. Kutz. Modern Koopman theory for dynamical systems. SIAM Rev., 64(2):229–340, 2022. doi:10.1137/21M1401243. A. Caponnetto and E. De Vito. Optimal rates for the regularized least-squares algorithm. Found. Comput. Math., 7(3):331–368, 2007. doi:10.1007/s10208-006-0196-8. G. M. Constantine and T. H. Savits. A multivariate Faà di Bruno formula with applications. Trans. Amer. Math. Soc., 348(2):503–520, 1996. doi:10.1090/S0002-9947-96-01501-2. J. B. Conway. A Course in Functional Analysis, volume 96 of Graduate Texts in Mathematics. Springer-Verlag, New York, second edition, 2007. doi:10.1007/978-1-4757-4383-8. N. Črnjarić-Žic, S. Maćešic, and I. Mezić. Koopman operator spectrum for random dynamical systems. J. Nonlinear Sci., 30:2007–2056, 2020. doi:10.1007/s00332-019-09582-z. N. Da Costa, M. Pförtner, L. Da Costa, and P. Hennig. Sample path regularity of gaussian processes from the covariance kernel. Anal. Appl. (Singap.), pages 1–29, 2026. doi:10.1142/S0219530526500235. J. Diestel and J. J. Uhl. Vector Measures, volume 15 of Mathematical Surveys. American Mathematical Society, Providence, RI, 1977. doi:10.1090/surv/015. D. E. Edmunds and H. Triebel. Function spaces, entropy numbers, differential operators, volume 120 of Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, 1996. doi:10.1017/CBO9780511662201. S. Fischer and I. Steinwart. Sobolev norm learning rates for regularized least-squares algorithms. J. Mach. Learn. Res., 21(205):1–38, 2020. URL http://jmlr.org/papers/v21/ 19-734.html. A. Friedman. Partial differential equations of parabolic type. Prentice-Hall, Inc., Englewood Cliffs, NJ, 1964. K. Fukumizu, F. R. Bach, and M. I. Jordan. Dimensionality reduction for supervised learning with reproducing kernel Hilbert spaces. J. Mach. Learn. Res., 5(Jan):73–99, 2004. URL http://www.jmlr.org/papers/volume5/fukumizu04a/fukumizu04a.pdf. K. Fukumizu, L. Song, and A. Gretton. Kernel Bayes’ rule: Bayesian inference with positive definite kernels. J. Mach. Learn. Res., 14(118):3753–3783, 2013. URL https://jmlr.org/ papers/volume14/fukumizu13a/fukumizu13a.pdf. T. Gneiting, F. Balabdaoui, and A. E. Raftery. Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. Ser. B Stat. Methodol., 69(2):243–268, 2007. ISSN 1369-7412,1467-9868. doi:10.1111/j.1467-9868.2007.00587.x. S. Grünewälder, G. Lever, L. Baldassarre, S. Patterson, A. Gretton, and M. Pontil. Conditional mean embeddings as regressors. In Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML’12, page 1803–1810, Madison, WI, USA, 2012. Omnipress. URL https://icml.cc/2012/papers/898.pdf. L. Györfi, M. Kohler, A. Krzyżak, and H. Walk. A distribution-free theory of nonparametric regression. Springer Series in Statistics. Springer-Verlag, New York, 2002. doi:10.1007/b97848. M. Hairer. On Malliavin’s proof of Hörmander’s theorem. Bull. Sci. Math., 135(6-7):650–666, 2011. doi:10.1016/j.bulsci.2011.07.007. M. Hertel, F. M. Philipp, M. Schaller, and K. Worthmann. Koopman for stochastic dynamics: error bounds for kernel extended dynamic mode decomposition, 2025. arXiv:2512.20247, to appear in SIAM J. Appl. Dyn. Syst. B. Hou, S. Sanjari, N. Dahlin, A. Koppel, and S. Bose. Nonparametric sparse online learning of the Koopman operator, 2026. arXiv:2405.07432. O. Kallenberg. Foundations of modern probability, volume 99 of Probability Theory and Stochastic Modelling. Springer, Cham, 2021. doi:10.1007/978-3-030-61871-1. Third edition.

28

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

I. Klebanov, I. Schuster, and T. J. Sullivan. A rigorous theory of conditional mean embeddings. SIAM J. Math. Data Sci., 2(3):583–606, 2020. doi:10.1137/19M1305069. I. Klebanov, B. Sprungk, and T. J. Sullivan. The linear conditional expectation in Hilbert space. Bernoulli, 27(4):2267–2299, 2021. doi:10.3150/20-BEJ1308. S. Klus, P. Koltai, and C. Schütte. On the numerical approximation of the Perron-Frobenius and Koopman operator. J. Comput. Dyn., 3(1):51–79, 2016. doi:10.3934/jcd.2016003. S. Klus, F. Nüske, P. Koltai, H. Wu, I. Kevrekidis, C. Schütte, and F. Noé. Data-driven model reduction and transfer operator approximation. J. Nonlinear Sci., 28(3):985–1010, 2018. doi:10.1007/s00332-017-9437-7. S. Klus, F. Nüske, and B. Hamzi. Kernel-based approximation of the Koopman generator and Schrödinger operator. Entropy, 22(7):722, 2020a. doi:10.3390/e22070722. S. Klus, I. Schuster, and K. Muandet. Eigendecompositions of transfer operators in reproducing kernel Hilbert spaces. J. Nonlinear Sci., 30(1):283–315, 2020b. doi:10.1007/s00332-019-09574-z. S. Klus, F. Nüske, and S. Peitz. Koopman analysis of quantum systems. J. Phys. A, 55(31): 314002, 2022. doi:10.1088/1751-8121/ac7d22. F. Köhne, F. M. Philipp, M. Schaller, A. Schiela, and K. Worthmann. L∞ -error bounds for approximations of the Koopman operator by kernel extended dynamic mode decomposition. SIAM J. Appl. Dyn. Syst., 24(1):501–529, 2025. doi:10.1137/24M1650120. B. O. Koopman. Hamiltonian systems and transformations in Hilbert space. Proc. Natl. Acad. Sci. USA, 17(5):315–318, 1931. doi:10.1073/pnas.17.5.315. V. Kostic, P. Novelli, A. Maurer, C. Ciliberto, L. Rosasco, and M. Pontil. Learning dynamical systems via Koopman operator regression in reproducing kernel Hilbert spaces. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 4017–4031. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/2022/ file/196c4e02b7464c554f0f5646af5d502e-Paper-Conference.pdf. R. Kress. Linear integral equations, volume 82 of Applied Mathematical Sciences. Springer, New York, third edition, 2014. doi:10.1007/978-1-4614-9593-2. H. Kunita. Stochastic flows and stochastic differential equations, volume 24 of Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 1990. S. Kusuoka and D. Stroock. Applications of the Malliavin calculus. II. J. Fac. Sci. Univ. Tokyo Sect. IA Math., 32(1):1–76, 1985. URL https://api.semanticscholar.org/CorpusID: 118802825. Z. Li, D. Meunier, M. Mollenhauer, and A. Gretton. Towards optimal Sobolev norm rates for the vector-valued regularized least-squares algorithm. J. Mach. Learn. Res., 25(181):1–51, 2024. URL https://jmlr.org/papers/v25/23-1663.html. P. Malliavin. Stochastic calculus of variation and hypoelliptic operators. In Proceedings of the International Symposium on Stochastic Differential Equations (Res. Inst. Math. Sci., Kyoto Univ., Kyoto, 1976), Wiley-Intersci. Publ., pages 195–263. John Wiley & Sons, New York-Chichester-Brisbane, 1978. K. Maurin. Abbildungen vom Hilbert-Schmidtschen Typus und ihre Anwendungen. Math. Scand., 9:359–371, 1961. doi:10.7146/math.scand.a-10641. P. McCullagh. Tensor methods in statistics. Monographs on Statistics and Applied Probability. Chapman & Hall, London, 1987. doi:10.1201/9781351077118. S. Menozzi, A. Pesce, and X. Zhang. Density and gradient estimates for non degenerate Brownian SDEs with unbounded measurable drift. J. Differential Equations, 272:330–369, 2021. doi:10.1016/j.jde.2020.09.004. S. Meyn and R. L. Tweedie. Markov chains and stochastic stability. Cambridge University Press, Cambridge, second edition, 2009. doi:10.1017/CBO9780511626630.

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

29

M. Mollenhauer and P. Koltai. Nonparametric approximation of conditional expectation operators, 2023. arXiv:2012.12917. M. Mollenhauer, S. Klus, C. Schütte, and P. Koltai. Kernel autocovariance operators of stationary processes: estimation and convergence. J. Mach. Learn. Res., 23(327):1–34, 2022. URL http://jmlr.org/papers/v23/20-442.html. M. Mollenhauer, N. Mücke, and T. J. Sullivan. Learning linear operators: Infinite-dimensional regression as a well-behaved non-compact inverse problem, 2024. arXiv:2211.08875. K. Muandet, K. Fukumizu, B. Sriperumbudur, and B. Schölkopf. Kernel mean embedding of distributions: A review and beyond. Foundations and Trends in Machine Learning, 10(1–2): 1–141, 2017. doi:10.1561/2200000060. V. Nateghi and F. Nüske. Kinetically consistent coarse graining using kernel-based extended dynamic mode decomposition. Journal of Chemical Theory and Computation, 21(15):7236– 7248, 2025. doi:10.1021/acs.jctc.5c00479. B. Øksendal. Stochastic differential equations. Universitext. Springer-Verlag, Berlin, sixth edition, 2003. doi:10.1007/978-3-642-14394-6. J. Park and K. Muandet. A measure-theoretic approach to kernel conditional mean embeddings. In H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 21247–21259. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/f340f1b1f65b6df5b5e3f94d95b11daf-Paper.pdf. F. M. Philipp, M. Schaller, K. Worthmann, S. Peitz, and F. Nüske. Error bounds for kernelbased approximations of the Koopman operator. Appl. Comput. Harmon. Anal., 71:101657, 2024. doi:10.1016/j.acha.2024.101657. F. M. Philipp, M. Schaller, K. Worthmann, S. Peitz, and F. Nüske. Error analysis of kernel EDMD for prediction and control in the Koopman framework. J. Nonlinear Sci., 35(5): Paper No. 92, 36, 2025. doi:10.1007/s00332-025-10182-3. E. Porcu, M. Bevilacqua, R. Schaback, and C. J. Oates. The Matérn Model: A Journey Through Statistics, Numerical Analysis and Machine Learning. Statist. Sci., 39(3):469–492, 2024. doi:10.1214/24-STS923. M. Reed and B. Simon. Methods of Modern Mathematical Physics. I. Functional Analysis. Academic Press, New York, second edition, 1980. doi:10.1016/B978-0-12-585001-8.X5001-6. S. Saitoh and Y. Sawano. Theory of reproducing kernels and applications, volume 44 of Developments in Mathematics. Springer, Singapore, 2016. doi:10.1007/978-981-10-0530-5. M. Santos Gutiérrez, V. Lucarini, M. D. Chekroun, and M. Ghil. Reduced-order models for coupled dynamical systems: Data-driven methods and the Koopman operator. Chaos, 31 (5):053116, 2021. doi:10.1063/5.0039496. A. Saumard and J. A. Wellner. Log-concavity and strong log-concavity: a review. Stat. Surv., 8:45–114, 2014. doi:10.1214/14-SS107. L. Song, J. Huang, A. Smola, and K. Fukumizu. Hilbert space embeddings of conditional distributions with applications to dynamical systems. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 961–968, 2009. doi:10.1145/1553374.1553497. I. Steinwart and A. Christmann. Support vector machines. Information Science and Statistics. Springer, New York, 2008. doi:10.1007/978-0-387-77242-4. D. W. Stroock. The Malliavin calculus and its application to second order parabolic differential equations. I. Math. Systems Theory, 14(1):25–65, 1981. doi:10.1007/BF01752389. D. W. Stroock and S. R. S. Varadhan. Multidimensional diffusion processes. Classics in Mathematics. Springer-Verlag, Berlin, 1997 edition, 2006. doi:10.1007/3-540-28999-2. A. M. Stuart. Inverse problems: a Bayesian perspective. Acta Numer., 19:451–559, 2010. doi:10.1017/S0962492910000061.

30

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

M. E. Taylor. Partial differential equations III. Nonlinear equations, volume 117 of Applied Mathematical Sciences. Springer, New York, second edition, 2011. doi:10.1007/978-1-4419-7049-7. A. B. Tsybakov. Introduction to nonparametric estimation. Springer Series in Statistics. Springer, New York, 2009. doi:10.1007/b13794. P. Vatiwutipong and N. Phewchean. Alternative way to derive the distribution of the multivariate Ornstein–Uhlenbeck process. Adv. Difference Equ., pages Paper No. 276, 7, 2019. doi:10.1186/s13662-019-2214-1. H. Wendland. Scattered data approximation, volume 17 of Cambridge Monographs on Applied and Computational Mathematics. Cambridge University Press, Cambridge, 2005. doi:10.1017/CBO9780511617539. M. O. Williams, C. W. Rowley, and I. G. Kevrekidis. A kernel-based method for data-driven Koopman spectral analysis. J. Comput. Dyn., 2(2):247–265, 2015. doi:10.3934/jcd.2015005. C. Zhang and E. Zuazua. A quantitative analysis of Koopman operator methods for system identification and predictions. Comptes Rendus Mecanique, 351(S1):1–31, 2023. doi:10.5802/crmeca.138.

Appendix A. Proofs and Auxiliary Results In this section, we state the detailed proofs of our main results. Therein, we will make use of the following basic inequalities for arbitrary a, b, γ > 0: (a + b)2 = a2 + 2γ 1/2 aγ −1/2 b + b2 ≤ a2 + γa2 + γ −1 b2 + b2 = (1 + γ)a2 + (1 + γ −1 )b2 , (A.1) 2ab + b2 ≤ γa2 + (1 + γ −1 )b2 ,

(A.2)

where the second one follows directly from the first. Proof of Proposition 2.4. By Bochner’s theorem (Wendland, 2005, Theorem 6.6), ψb ≥ 0 is nonnegative. By (Wendland, 2005, Theorem 10.12), ( ) Z Z b(ω)|2 | f |fb(ω)|2 d 2 d 2 −d/2 Hk = f ∈ C(R ) ∩ L (R ) dω < ∞ , ∥f ∥Hk = (2π) dω. b b Rd ψ(ω) Rd ψ(ω) Hence, using Plancherel’s theorem and ψb ∈ L∞ (Rd ), ∥f ∥2L2 (Rd ) = ∥fb∥2L2 (Rd ) =

|fb(ω)|2 b ∞ d ∥f ∥2 . b ψ(ω) dω ≤ (2π)d/2 ∥ψ∥ L (R ) Hk b ψ(ω) Rd

Z

This proves the continuous embedding Hk ,→ L2 (Rd ). For the domain version, let g ∈ Hk (X ). By formula (2.3) for the restriction-space norm, for every ε > 0 there exists f ∈ Hk (Rd ) such that f |X = g and ∥f ∥Hk (Rd ) ≤ ∥g∥Hk (X ) + ε. Since ∥g∥L2 (X ) ≤ ∥f ∥L2 (Rd ) , the embedding on Rd yields
d/4 b 1/2 b 1/2 ∥g∥L2 (X ) ≤ ∥f ∥L2 (Rd ) ≤ (2π)d/4 ∥ψ∥ ∥f ∥ ∥ ψ∥ ∥g∥ + ε . d ) ≤ (2π) ∞ d ∞ d H (X ) H (R k k L (R ) L (R ) Letting ε → 0 proves the claim.

■

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

31

A.1. Proofs of Section 4.1. Proof of Theorem 4.2. Throughout, C > 0 denotes a constant depending only on d, ℓ, τ , Γ, and (in case (A)) on X , and may change from line
to line. Since ε is independent of X, the conditional density is given by η(x, y) = pε y − f (x) . Step 1: Derivative structure via Faà di Bruno. By the multivariate Faà di Bruno formula (Constantine and Savits, 1996, Theorem 2.1), for 1 ≤ |α| ≤ ℓ, ∂xα η(x, y) =

|α| X
(−1)s p(s) y − f (x) Gα,s (x), ε s=1

Gα,s (x) :=

X

cβ

β1 +···+βs =α |βj |≥1

s Y

∂ βj f (x), (A.3)

j=1

with combinatorial constants cβ ≥ 0, β = (β1 , . . . , βs ). (Here (A.3) is understood as an identity of weak derivatives on Q X , valid for f ∈ H ℓ (X ) by approximation with smooth functions; since ℓ > d/2, the products j ∂ βj f lie in L2 (X ), see (A.5) below.) Step 2: Integration in y. Fix x ∈ X . By translation invariance of the Lebesgue measure, Z
2 2 |p(s) y − f (x) |2 dy = ∥p(s) 0 ≤ s ≤ ℓ. ε ε ∥L2 (R) ≤ ∥pε ∥W ℓ,2 (R) , R

Hence, applying the Cauchy–Schwarz inequality to the finite sum (A.3), Z |α| X |∂xα η(x, y)|2 dy ≤ C ∥pε ∥2W ℓ,2 (R) G(x)2 , G(x) := max |Gα,s (x)| 1≤|α|≤ℓ

R

(A.4)

s=1

for 1 ≤ |α| ≤ ℓ, while for α = 0 the left-hand side equals ∥pε ∥2L2 (R) . P Step 3: Integration in x. For each tuple (β1 , . . . , βs ) in (A.4) we have j |βj | = |α| ≤ ℓ, so that the Gagliardo–Nirenberg–Moser product estimate (see Taylor, 2011, Ch. 13, §3) yields, using f ∈ H ℓ (X ) ,→ L∞ (X ) for ℓ > d/2, s Y

∂ βj f L2 (Rd )

j=1

∥f ∥H ℓ (Rd ) ≤ C∥f ∥sH ℓ (Rd ) , ≤ C∥f ∥s−1 L∞ (Rd )

(A.5)

ℓ and therefore ∥G∥L2 (X ) ≤ C 1 + ∥f ∥H ℓ (X ) is finite, where for bounded Lipschitz domains X we applied (A.5) to a Stein extension of f (Adams and Fournier, 2003, Theorem 5.24). Case (A). By Tonelli’s theorem and (A.4), XZ Z 2 ∥η∥L2 (νY ;H ℓ (X )) = |∂xα η(x, y)|2 dy dx ≤ λ(X )∥pε ∥2L2 (R) + C ∥pε ∥2W ℓ,2 (R) ∥G∥2L2 (X ) < ∞, |α|≤ℓ X

R

where the first summand is the α = 0 term. By norm-equivalence H ≃ H ℓ (X ), Assumption 3.2 holds, and the claim follows from Theorem 3.3(a) and (c). Case (B). By Lemma 2.2 and Hk0 ≃ H ℓ (Rd ),
∥η( • , y)∥Hkτ = ∥mτ η( • , y)∥Hk0 ≍ ∥mτ η( • , y)∥H ℓ (Rd ) , mτ (x) = exp − τ2 ∥x∥2Γ . Since every derivative of mτ is a polynomial times the same Gaussian weight, |∂ γ mτ (x)| ≤ C(1 + ∥x∥2 )|γ| mτ (x) for |γ| ≤ ℓ. Hence, by the Leibniz rule and (A.4), for every |α| ≤ ℓ, Z

2 |∂xα mτ η (x, y)|2 dy ≤ C∥pε ∥2W ℓ,2 (R) (1 + ∥x∥2 )2ℓ mτ (x)2 1 + G(x) . R

Since G ∈ L2 (Rd ) by (A.5) and wτ (x) := (1 + ∥x∥2 )2ℓ mτ (x)2 satisfies wτ ∈ L1 (Rd ) ∩ L∞ (Rd ), we conclude with Tonelli’s theorem that Z
2
2 2 ∥η∥L2 (νY ;Hk ) ≤ C ∥pε ∥W ℓ,2 (R) wτ (x) 1+G(x) dx ≤ C ∥pε ∥2W ℓ,2 (R) ∥wτ ∥L1 +∥wτ ∥L∞ ∥G∥2L2 τ

Rd

32

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

is finite. Assumption 3.2 thus holds with H = Hkτ , and the claim again follows from Theorem 3.3(a). ■ Proof of Corollary 4.3. We first verify that (X, kX ) satisfies Assumption 2.1; condition (iii) holds by assumption in both cases, so it remains to check (i) and (ii). Case (A). Since kX is measurable, symmetric, and positive definite as a kernel, and HkX ≃ ℓ H (X ) is separable, (i) holds. For (ii), note that since ℓ > d/2, the Sobolev embedding theorem gives H ℓ (X ) ,→ Cb (X ) continuously (Adams and Fournier, 2003, Theorem 4.12); composed with the norm-equivalence HkX ≃ H ℓ (X ), this yields a bounded embedding HkX ,→ Cb (X ) with some operator norm C > 0. Evaluating at the feature vector ϕX (x) = kX (x, • ) and using the reproducing property gives p kX (x, x) ≤ ∥ϕX (x)∥∞ ≤ C∥ϕX (x)∥HkX = C kX (x, x), hence E[kX (X, X)] ≤ supx∈X kX (x, x) ≤ C 2 < ∞ regardless of the law of X, proving (ii). Case (B). Since k0 is continuous and mτ (x) = exp(− τ2 ∥x∥2Γ ) > 0 is continuous, kτ is continuous, hence measurable. Symmetry and positive definiteness of kτ follow from Lemma 2.2, which also shows that the map f 7→ mτ f is an isometry from Hkτ onto Hk0 ≃ H ℓ (Rd ); since H ℓ (Rd ) is separable, so is Hkτ , proving (i). For (ii), translation invariance of k0 implies that c0 := k0 (x, x) is constant, so that, by (4.3),     
 E kX (X, X) = E mτ (X)−2 k0 (X, X) = c0 E exp τ ∥X∥2Γ < ∞. In both cases, Assumption 3.2 holds with H = HkX by Theorem 4.2, and HkY ,→ L2 (νY ) continuously, so Corollary 3.8 proves (a). In case (A), the boundedness supx∈X kX (x, x) < ∞ established above allows the application of Theorem 3.16, which yields (b). ■ A.2. Proofs of Section 4.2. Proof of Theorem 4.5. Throughout the proof, positive con′′ ′ are chosen as needed from the context without further , Cindex stants of the form Cindex , Cindex comment and may depend on the parameters indicated in the subscript. Step 1: Gaussian envelope for uy := mτ ( • )η( • , y). Since F (Y ) is finite P -almost surely, there exists R > 0 such that
πR := P ∥F (Y )∥Γ ≤ R > 0. Fix 0 < δ1 < τ . Restricting the integral defining Ξ(x) to the event {∥F (Y )∥Γ ≤ R} yields


Ξ(x) ≥ πR exp − R∥x∥Γ − 21 R2 ≥ πR exp − 21 δ1 ∥x∥2Γ + (1 + δ1−1 )R2 , where we used (A.2) with a = ∥x∥Γ , b = R and γ = δ1 . Consequently,  τ −δ  1 1 uy (x) := mτ (x)η(x, y) ≤ Cδ1 exp − ∥x∥2Γ + ⟨x, F (y)⟩Γ − ∥F (y)∥2Γ 2 2

2 τ −δ1 = Cδ1 exp aτ,δ1 ∥F (y)∥Γ exp − 2 ∥x − bτ,δ1 (y)∥2Γ ,

(A.6)

(y) 1+δ1 −τ := τF−δ with aτ,δ1 := 2(τ , where in the last step we completed the square. −δ1 ) and bτ,δ1 (y) 1

Step 2: Derivative bounds for uy . Write uy (x) = exp(hy (x)),

hy (x) = − τ2 ∥x∥2Γ + ⟨x, F (y)⟩Γ − 12 ∥F (y)∥2Γ − Λ(x).

Using (4.5) we obtain, for every multi-index α with 1 ≤ |α| ≤ ℓ,
r |Dα hy (x)| ≤ CΛ′ 1 + ∥x∥2 + ∥F (y)∥2 . By the multivariate Faà di Bruno formula, for every multi-index α with |α| ≤ ℓ,   Dα uy (x) = uy (x) Bα (Dβ hy (x))1≤|β|≤|α| ,

(A.7)

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

33

where Bα is a (multivariate) polynomial whose monomials have the form s Y

Dβj hy (x),

s ≤ |α|,

|β1 | + · · · + |βs | = |α|.

j=1

Since each factor Dβj hy (x) is bounded by CΛ′ (1+∥x∥2 +∥F (y)∥2 )r by (A.7), and each monomial contains at most |α| such factors, we obtain
r|α| |Dα uy (x)| ≤ CΛ′′ 1 + ∥x∥2 + ∥F (y)∥2 uy (x)

r|α|
2 1 ∥x − b (y)∥ . ≤ CΛ,δ1 1 + ∥x∥2 + ∥F (y)∥2 exp aτ,δ1 ∥F (y)∥2Γ exp − τ −δ τ,δ Γ 1 2 (A.8) Step 3: H ℓ -bound. Using ∥z + bτ,δ1 (y)∥2 ≤ ∥z∥2 + ∥bτ,δ1 (y)∥2 and ∥bτ,δ1 (y)∥2 ≲ ∥F (y)∥2 , we obtain, for every z ∈ Rd and δ2 > 0,
2r|α| 1 + ∥z + bτ,δ1 (y)∥2 + ∥F (y)∥2 ≤ CΛ,τ,δ1 (1 + ∥z∥2 )2r|α| (1 + ∥F (y)∥2 )2r|α|
≤ CΛ,τ,δ1 ,δ2 (1 + ∥z∥2 )2r|α| exp 2δ2 ∥F (y)∥2Γ , since polynomial factors in ∥F (y)∥Γ can be absorbed into exp(δ2 ∥F (y)∥2Γ ) up to a multiplicative constant (using equivalence of norms on Rd ). Squaring and integrating (A.8), and changing variables to z = x − bτ,δ1 (y), yields
′ ∥Dα uy ∥2L2 (Rd ) ≤ CΛ,τ,δ exp 2aτ,δ1 ∥F (y)∥2Γ × 1 Z
2r|α|
× 1 + ∥z + bτ,δ1 (y)∥2 + ∥F (y)∥2 exp − (τ − δ1 )∥z∥2Γ dz Rd Z

′ 2 ≤ CΛ,τ,δ1 ,δ2 exp 2(aτ,δ1 + δ2 )∥F (y)∥Γ (1 + ∥z∥2 )2r|α| exp − (τ − δ1 )∥z∥2Γ dz d
R ′′ 2 ≤ CΛ,τ,δ1 ,δ2 exp 2(aτ,δ1 + δ2 )∥F (y)∥Γ . Taking the square root and summing over all |α| ≤ ℓ gives
′ ∥uy ∥H ℓ (Rd ) ≤ Cℓ,τ,δ exp (aτ,δ1 + δ2 )∥F (y)∥2Γ . 1 ,δ2 Since uy = mτ ( • )η( • , y) and Hk0 ≃ H ℓ (Rd ), Lemma 2.2 implies η( • , y) ∈ HkX

and

∥η( • , y)∥HkX = ∥uy ∥Hk0 ≤ Cℓ,τ,δ1 ,δ2 exp (aτ,δ1 + δ2 )∥F (y)∥2Γ .

Step 4: η ∈ L2(PY ; HkX ) under (4.6). To prove η ∈ L2 (PY ; HkX ), we verify the finiteness of  E ∥η( • , Y )∥2Hk using the pointwise bound established in Step 3. X

choose δ1 , δ2 > 0 sufficiently small such that aτ,δ1 + δ2 < 0, and thereby If τ > 1, thenwe can 2 E ∥η( • , Y )∥2Hk ≤ Cℓ,τ,δ < ∞. 1 ,δ2 X 
 1 If τ > 1+λ for some λ > 0 with E exp λ∥F (Y )∥2Γ < ∞, then λ > 1−τ τ and we can choose δ1 , δ2 > 0 sufficiently small such that 2(aτ,δ1 + δ2 ) ≤ λ, yielding   
 2 E ∥η( • , Y )∥2Hk ≤ Cℓ,τ,δ E exp λ∥F (Y )∥2Γ < ∞. 1 ,δ2 X

In both cases, Theorem 3.3 with νY = PY implies that K ∈ HS(L2 (PY ), HkX ) with Hilbert– Schmidt norm ∥K∥HS(L2 (PY ),Hk ) = ∥η∥L2 (PY ;Hk ) . ■ X

X

Lemma A.1 (Cumulants bounded by centered moments). Let Z : Ω → Rd be a random vector such that E[∥Z∥j2 ] < ∞ for some integer j ≥ 2. For every multi-index α ∈ Nd0 with |α| = j, let   κα (Z) := Dα KZ (0), KZ (t) := log E e⟨t,Z⟩ ,

34

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

denote the corresponding mixed cumulant of order j. Then   there exists a constant cα > 0, depending only on α, such that |κα (Z)| ≤ cα E ∥Z − EZ∥j2 . Proof. Cumulants can be written as polynomials in the moments of Z; see (McCullagh, 1987, e = Z − EZ, and using that Section 2.3.4). Applying this identity to the centered vector Z cumulants of order j ≥ 2 are invariant under centering (see the affine transformation formula in (McCullagh, 1987, Section 2.4)), we obtain X Y κα (Z) = cα,π E[ZeβB ], π

B∈π

B∈π |βB | = j. By Jensen’s inequality,
i h i|βB |/j i j −1 PB∈π |βB | h i Y h Y h |β | j j e B ≤ e e e j . ≤ E ∥Z∥ E ∥ Z∥ = E ∥ Z∥ = E ∥ Z∥ 2 2 2 2 B∈π B∈π

where the sum contains finitely many terms and Y   e βB E Z B∈π

P

Summing over the finitely many terms yields |κα (Z)| ≤ cα E[∥Z − EZ∥j2 ] for some constant cα > 0 depending only on α.

■

Proof of Proposition 4.6. We verify that each of the conditions (a),(b),(c), (d) implies (4.5). (b) ⇒ (4.5). If Y is Gaussian and F (y) = Ay + b, then X = F (Y ) + ε is Gaussian on Rd with some mean mX and covariance matrix ΣX . Hence its density satisfies

pX (x) = c1 exp − 21 ∥x∥2Γ Ξ(x) = c2 exp − 12 ∥x − mX ∥2ΣX , for certain constants c1 , c2 > 0, where we used the definition of Ξ. Taking logarithms gives Λ(x) = 12 ∥x∥2Γ − 12 ∥x − mX ∥2ΣX + log cc21 . Thus Λ is a quadratic polynomial, and therefore satisfies (4.5). (c) ⇒ (4.5). Let Λ(x) = log Ξ(x). Since Ξ ∈ C ℓ (Rd ), the derivatives of Λ up to order ℓ are well-defined. By differentiating the log-partition function, ∥∇Λ(x)∥2 = ∥E[a(Y ) | X = x]∥2 = ∥ma1 (x)∥2 ≤ c (1 + ∥x∥2 ). Now let α ∈ Nd0 with 2 ≤ |α| = j ≤ ℓ. By definition of cumulants, Dα Λ(x) is the mixed cumulant of a(Y ) under the conditional law PY | X=x . Hence, by Lemma A.1 applied conditionally, h i |Dα Λ(x)| ≤ cα E ∥a(Y ) − ma1 (x)∥j2 | X = x = cα maj (x) ≤ cα c (1 + ∥x∥2 ). This shows (4.5). (a) ⇒ (c). If F is PY -essentially bounded, then a(Y ) = Γ−1 F (Y ) is also PY -essentially bounded. Hence there exists R > 0 such that ∥a(Y )∥2 ≤ R PY -a.s. Since PY | X=x ≪ PY for every x ∈ Rd , the same bound holds PY | X=x -almost surely. Therefore, for every x ∈ Rd , ∥ma1 (x)∥2 = ∥E[a(Y ) | X = x]∥2 ≤ R, and, for j = 2, . . . , ℓ,   maj (x) = E ∥a(Y ) − ma1 (x)∥j2 | X = x ≤ (2R)j . Thus, the bounds in (c) hold. (d) ⇒ (c). Let ρ denote the Lebesgue density of the pushforward law of a(Y ) = Γ−1 F (Y ). By assumption, ρ is strongly log-concave, so we may write ρ(z) ∝ e−V (z)

with

∇2 V (z) ⪰ cI

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

35

for some c > 0. For each x ∈ Rd , let Pxa denote the law of a(Y ) under the conditional distribution PY | X=x . By (4.4), the law Pxa has density    h i ρx (z) ∝ exp ⟨x, z⟩ − 12 z ⊤ Γz ρ(z) = exp − V (z) − ⟨x, z⟩ + 12 z ⊤ Γz . Hence Pxa is strongly log-concave with a lower Hessian bound that is uniform in x:   ∇2 V (z) − ⟨x, z⟩ + 12 z ⊤ Γz ⪰ λ∗ I, λ∗ := c + λmin (Γ) > 0. Let G, G′ ∼ N (0, λ−1 ∗ I) be independent. By Caffarelli’s contraction theorem (Saumard and Wellner, 2014, Theorem 9.7), for every x ∈ Rd , there exists a contraction Tx : Rd → Rd such that Tx (G) ∼ Pxa . Then, for j = 2, . . . , ℓ,       maj (x) = E ∥Tx (G) − E[Tx (G)]∥j2 ≤ E ∥Tx (G) − Tx (G′ )∥j2 ≤ E ∥G − G′ ∥j2 =: Cj , Thus the conditional centered moments are bounded uniformly in x. Further, since V is strongly convex (∇2 V ⪰ cI), the density ρ(z) ∝ e−V (z) has Gaussian tails. Hence the integrand in the definition of Ξ is dominated by an integrable function uniformly on compact sets in x. Differentiation under the integral sign is therefore justified to arbitrary order, yielding Ξ ∈ C ∞ (Rd ) and thus Λ ∈ C ∞ (Rd ). Since ma1 (x) = E[a(Y ) | X = x] = ∇Λ(x),

and

∥D2 Λ(x)∥ = ∥C[a(Y ) | X = x]∥ ≤ ma2 (x) ≤ C2

uniformly in x, ∥ma1 (x)∥2 ≤

Z 1 ∇Λ(0) +

D2 Λ(tx) x dt

0

≤ ∥∇Λ(0)∥2 + C2 ∥x∥2 ≤ C1 (1 + ∥x∥2 ) 2

for some C1 > 0 and all x ∈ Rd . Hence the bounds in (c) hold with C := maxj=1,...,ℓ Cj .

■

Proof of Corollary 4.7. We first verify that (X, kX ) satisfies Assumption 2.1. (i): Since k0 is continuous (Theorem 4.5) and mτ (x) = exp(− τ2 ∥x∥2Γ ) > 0 is continuous, kτ is continuous, hence measurable. Symmetry and positive definiteness of kτ follow from Lemma 2.2, which also shows that the map f 7→ mτ f is an isometry from Hkτ onto Hk0 ≃ H ℓ (Rd ); since H ℓ (Rd ) is separable, so is Hkτ . λ (ii): Since τ < 1+2λ , ζ := 2λ satisfies τ (1 + ζ) < λ

and

s := τ (1 + ζ −1 ) < 12 .

It follows from (A.1) that τ ∥X∥2Γ ≤ τ ∥F (Y )∥Γ + ∥ε∥Γ

2

≤ τ (1 + ζ)∥F (Y )∥2Γ + τ (1 + ζ −1 )∥ε∥2Γ ≤ λ∥F (Y )∥2Γ + s∥ε∥2Γ .

Since Y and ε are independent and c0 = k0 (x, x) is constant by translation-invariance of k0 ,     
 E kX (X, X) = E mτ (X)−2 k0 (X, X) = c0 E exp τ ∥X∥2Γ 
 
 ≤ c0 E exp λ∥F (Y )∥2Γ E exp s∥ε∥2Γ . The first expectation is finite by assumption, while the  second one  is finite since s < 1/2 and ∥ε∥2Γ ∼ χ2d with moment generating function E exp(s∥ε∥2Γ ) = (1 − 2s)−d/2 . Hence E[kX (X, X)] < ∞. (iii): Since X = F (Y ) + ε with ε ∼ N (0, Γ) and Γ strictly positive definite, the conditional distribution PX|Y =y = N (F (y), Γ) has a strictly positive density on Rd for every y. R Consequently, for every non-empty open set U ⊂ Rd , PX (U ) = PX|Y =y (U ) PY (dy) > 0, so supp(PX ) = Rd = X . Since kX = kτ is continuous (by (i)), this proves injectivity of the canonical embedding HkX ,→ L2 (PX ). Hence (X, kX ) satisfies Assumption 2.1. Consequently, if (4.7) holds and (Y, kY ) also satisfies Assumption 2.1 (implying HkY ,→ L2 (PY )), then Theorem 4.5 and Corollary 3.8 with ν = PY prove the remaining claims. ■

36

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

A.3. Proofs of Section 4.3. Throughout the proofs of this section, C and c denote positive constants that depend only on the quantities indicated in the subscript. Proof of Proposition 4.10. Under Assumption 4.9, ∥∇b∥∞ < ∞ and ∥∇σ∥∞ < ∞, so by the mean value theorem the coefficients b and σ are globally Lipschitz continuous. By (Øksendal, 2003, Theorem 5.2.1), the SDE (4.9) admits a unique strong solution. Let (Kt )t≥0 denote the associated Markov (Koopman) semigroup Kt f (x) := E[f (Xt ) | X0 = x]. Its infinitesimal generator A, defined on C02 (Rd ), is given by (Øksendal, 2003, Theorem 7.3.3) d

d

i=1

i,j=1

Kt g(x) − g(x) X 1 X = (σ(x)σ(x)⊤ )ij ∂xi ∂xj g(x), x ∈ Rd . bi (x)∂xi g(x) + t↓0 t 2

(Ag)(x) = lim

Under Assumption 4.9, A is a uniformly elliptic second-order differential operator with bounded coefficients and bounded derivatives up to order ℓ. Classical parabolic PDE theory (e.g. Friedman, R1964) now implies that A admits a fundamental solution p(t, x, y) which satisfies Kt f (x) = Rd f (y) p(t, x, y) dy and coincides with the transition density of the diffusion process, i.e., ηt (x, y) = p(t, x, y). Friedman (1964, Chapter 9, Theorem 7) then states that for every t > 0, p(t, • , y) admits weak derivatives ∂xα p(t, x, y) up to order |α| ≤ ℓ, satisfying the Gaussian bounds   ∥x − y∥22 α −(|α|+d)/2 |∂x p(t, x, y)| ≤ Cα t exp −κ , t with constants κ, Cα > 0.

■

Proof of Theorem 4.11. We first verify that the representation (4.10) holds with density ηt , before treating the two cases separately. Step 1: Koopman representation and derivative estimates. By Proposition 4.10, the R transition density ηt (x, y) exists and (4.10) holds, i.e., (Kt g)(x) = E[g(Xt ) | X0 = x] = multi-index |α| ≤ ℓ, Rd g(y)ηt (x, y) dy. Moreover, Proposition 4.10 shows that, for every
there exists κ > 0 such that |∂xα ηt (x, y)| ≤ Cα,t exp −κ(∥x − y∥22 )/t . Since these derivatives are in y, differentiation under the integral sign is justified, yielding ∂xα Kt g(x) = R integrable α Rd g(y) ∂x ηt (x, y) dy for every |α| ≤ ℓ. Thus, it remains to verify the required regularity of ηt ( • , y). Step 2: Case (A). Assume that X ⊂ Rd is Lipschitz bounded. Using the Sobolev norm representation, Fubini–Tonelli, and the derivative estimates above, we obtain   Z Z XZ Z X 2κ 2 α 2 2 ∥ηt ∥L2 (νY ;H ℓ (X )) = |∂x ηt (x, y)| dy dx ≤ Cα,t exp − ∥x − y∥2 dy dx. t X Rd X Rd |α|≤ℓ

|α|≤ℓ

2 For the inner integral we easily see that Rd exp − 2κ t ∥x − y∥2 dy = Cκ,t , independently of x. By boundedness of X , we obtain ∥ηt ∥2L2 (ν ;H ℓ (X )) ≤ Cℓ,t λ(X ) < ∞, which proves that R

Y

ηt ∈ L2 (νY ; H ℓ (X )). By Theorem 3.3(a), Kt ∈ HS(L2 (νY ), H ℓ (X )) with ∥Kt ∥HS(L2 (νY ),H ℓ (X )) = ∥ηt ∥L2 (νY ;H ℓ (X )) follows. Further, setting νX = λ, we immediately obtain Kt ∈ HS(L2 (νY ), L2 (νX )) by Theorem 3.3(b), and Kt ∈ Tr(L2 (νY ), L2 (νX )) for ℓ > d/2 by Theorem 3.3(c). Step 3: Case (B). Assume now that X = Rd . By Lemma 2.2 and the norm equivalence between Hk0 and H ℓ (Rd ), we have ∥f ∥Hkτ ≃ ∥mτ f ∥H ℓ (Rd ) , where mτ (x) = exp(− τ2 ∥x∥2Γ ). It therefore suffices to estimate ∥mτ ηt ( • , y)∥H ℓ (Rd ) . By the product rule, every derivative ∂xα (mτ ηt ( • , y)) is a finite sum of terms (∂ β mτ )(∂xα−β ηt ) with |β| ≤ |α| ≤ ℓ. Since every derivative of mτ is a polynomial multiplied by the same Gaussian
weight, there exist constants Cα,β such that |∂ β mτ (x)| ≤ Cα,β (1 + ∥x∥2 )ℓ exp − τ2 ∥x∥2Γ .

VERIFIABLE REGULARITY CRITERION FOR CONDITIONAL EXPECTATION OPERATORS

37

Further, since Γ is symmetric and positive definite, there exist constants 0 < λmin ≤ λmax τ τ 2 2 such that λmin ∥x∥22 ≤ ∥x∥2Γ ≤ λmax ∥x∥22 , hence e− 2 ∥x∥Γ ≤ e− 2 λmin ∥x∥2 . Combining this with the Gaussian estimate for ∂ α−β ηt yields  
2κ 2 α 2 N 2 |∂ (mτ ηt )(x, y)| ≤ Cα,t,τ (1 + ∥x∥2 ) exp −τ λmin ∥x∥2 exp − ∥x − y∥2 t for some integer N . Completing the square in the exponent gives −τ ∥x∥2Γ −

2 B 2κ ∥x − y∥22 = −(A + B) x − y − cτ,κ,t ∥y∥22 , t A+B

with positive constants A = τ λmin , B = 2κ/t, and cτ,κ,t > 0. Consequently, after
integrating over x for fixed y ∈ Rd , we obtain ∥mτ ηt ( • , y)∥2H ℓ (Rd ) ≤ Cℓ,t,τ exp −cτ,κ,t ∥y∥22 , where we used the finiteness of Gaussian moments for any N ∈ N. The expression on the right-hand side is clearly integrable over Y = Rd , hence ηt ∈ L2 (νY ; Hkτ ) follows. This means that Assumption 3.2 holds, such that Theorem 3.3(a) implies Kt ∈ HS(L2 (νY ), Hkτ ). ■ Proof of Corollary 4.12. Under Assumption 4.9, Theorem 4.11(A) yields ηt ∈ L2 (νY ; H ℓ (X )) with νY = λ. Thus Assumption 3.2 holds with H = H ℓ (X ). By Proposition 2.3, the Matérn kernel with parameter β = ℓ−d/2 > 0 induces the RKHS HkMat (X ) ≃ H ℓ (X ). Theorem 3.12(c) β then implies (4.11) with m = ⌈β⌉ − 1 = ⌈ℓ − d/2⌉ − 1. Wen with odd d and parameter j = Further, by Proposition 2.3 the Wendland kernel kd,j ℓ − (d + 1)/2 ∈ N0 induces the RKHS HkWen ≃ H ℓ (X ). Analogously, (4.11) then follows from d,j Theorem 3.12(d) with m = j + 1/2 = ℓ − d/2. ■

Related documents

• Hessian Schatten-Norm Regularization for Linear Inverse Problems#235364
• Symbolic Regression via Latent Iterative Refinement#229493
• Bayesian Symbolic Regression with Entropic Reinforcement Learning#441810
• Bayesian regression filter and the issue of priors#429219
• Oracle high-dimensional $M$-estimation using smooth reparameterization for sparsity#1014150