arXiv CSopen access

Optimal estimation for Functional Linear Regression with Noisy Discretized Data

· arxiv_cs

arXiv CS · Papers · License: Open Access

machine learning, deep learning, neural networks

Optimal estimation for Functional Linear Regression with Noisy Discretized Data Sixtine Sphabmixay Université Paris Cité, CNRS, MAP5, F-75006 Paris, France

arXiv:2609.08671v1 [stat.ML] 8 Sep 2026

September 9, 2026 Abstract In this paper, we consider the scalar-on-function linear regression model under a realistic sampling scheme in which the functional covariates are observed on a regular grid and contaminated by additive noise. We propose a two-step estimation procedure: first, the underlying curves are reconstructed from the discrete noisy observations using a Fourier-based projection method; second, the slope function is estimated by a penalized least-squares criterion over finite-dimensional trigonometric spaces, with data-driven selection of the model dimension. We establish oracle-type inequalities for the prediction error, both with respect to the reconstructed curves and to the true latent curves. Under regularity assumptions on the slope function and polynomial decay of the eigenvalues of the covariate, we derive convergence rates for the prediction error and show that our estimator attains the minimax rate when the number of grid points is sufficiently large. Finally, the proposed method is illustrated on simulated data and on a real meteorological dataset.

Introduction

Functional data analysis (FDA), as formalized and popularized by Ramsay and Silverman [14], is a statistical framework designed to analyze data that are naturally expressed as functions rather than finite-dimensional vectors, which are commonly considered in classical statistical methods. This methodology is particularly well suited to modern datasets, where observations evolve continuously over domains such as time or space. Moreover, advances in data collection and storage have also led to a substantial increase in the availability of functional data. Consequently, FDA has been widely applied across various disciplines, including medicine (Frøslie et al. [10]), biology (Dieng et al. [9]), and finance (Kokoszka et al. [13]). A central inferential tool in FDA is the functional linear regression model, which extends classical regression by relating scalar or functional responses to functional predictors. In this paper, we focus on the scalar-on-function case. More precisely, we assume that a functional covariate X taking values in a Hilbert space X and a real-valued response variable Y in R are linked through the model, Z 1

β(t)(X(t) − µ(t))dt + ϵ,

Y = µY +

(1)

where µY = E(Y ), µ(t) = E(X(t)) for all t ∈ [0, 1] and ϵ is a random variable which is centered and of finite variance σ 2 . This model was popularized in the FDA literature by Ramsay and Silverman [14] and has since been extensively developed, with numerous approaches proposed for estimating the coefficient function β. For instance, Cardot et al. [5] introduced an estimator based on functional principal component analysis. James et al. [12] proposed the FLiRTI method, designed to produce interpretable estimates of β that highlight regions where the relationship between Y and X is negligible. More recently, Grollemund et al. [11] developed a Bayesian approach, known as Bliss, which leverages the conditional distribution Y | X, β ∼ N (⟨β, X⟩X , σ 2 ). This approach also aims to identify the time intervals that have the greatest influence on the response. 1

Studies of the statistical properties of existing estimators have also been developed under the assumption that the functional data are observed continuously and without noise, as in Cardot et al. [6] who derived minimax rates and oracle-type inequalities. However, this idealized setting is rarely encountered in practice, since observations are typically available only on a finite grid, which may be regularly or irregularly spaced, deterministic or random. This more realistic setting has received comparatively less attention. To our knowledge, Cardot et al. [7] were among the first to study it. Although their approach is more practically relevant, it requires the grid size to be sufficiently large for their results to hold. In this paper, we aim to to extend the approach of Brunel and Roche [4] to the more realistic setting where observations are available on a regular grid and are corrupted by noise. As them, we assume that the covariates are centered, periodic, and second-order stationary, with X(0) = X(1). The sample (Xi )i=1,...,n consists of independent copies of X. Here we further assume that the functional covariates are observed on a common regular grid th = h/p for all h ∈ {0, . . . , p − 1}, and that the observations are contaminated by noise. Thus, rather than observing the full curves Xi , we observe Zi (th ) = Xi (th ) + ηi,h , (2) where the noise variables (ηi,h ) are centered, i.i.d., with variance τ 2 , and independent of the processes Xi . For technical simplicity, we assume throughout this paper that σ 2 , the variance of ϵ defined in (1) is known. In the idealized setting of continuously observed noiseless curves, Brunel and Roche [4] proposed an estimator of the slope function β based on a least-squares contrast over finite-dimensional spaces generated by trigonometric bases, together with a penalization procedure for selecting the optimal dimension of the basis. They proved an oracle inequality and obtained convergence rates when the eigenvalues of the covariance operator associated to the true curves decay in either a polynomial or exponential way, and that β belongs to a periodic Sobolev space, W per (k, L) := {f ∈ W2k (L), ∀j = 0, ..., k − 1, f (j) (0) = f (j) (1)}, with

(3)

W2k (L) := {f : [0, 1] → R, f (k−1) is absolutely continuous and ∥f (k) ∥L2 ≤ L},

for k a positive integer and L a positive real number. Here ∥ · ∥L2 is defined as Z 1 ∥f ∥L2 =

1/2 f (t) dt 2

for any square-integrable function f on [0, 1].

Our approach follows a similar philosophy. We estimate β by minimizing a least-squares criterion after a preliminary step that reconstructs the underlying curve X from the noisy discrete observations. This reconstruction is based on a Riemann-sum approximation and a projection onto a finite-dimensional space spanned by a truncated Fourier basis. We then introduce a penalization procedure to automatically select the optimal number of Fourier terms used in the estimation of β. This strategy differs from the work of Cardot et al. [7] where they proposed a smoothing splines estimator to estimate the unknown slope function. We derive two oracle-type inequalities for the prediction error, one with respect to the reconstructed curves and one with respect to the true curves. We then establish convergence rates for the prediction error with respect to the true curves when β belongs to W per (k, L) and the eigenvalues of the covariance operator decay polynomially. We recover the same rate as Brunel and Roche [4] when p is large enough but not necessarily infi2k+2a nite, namely that the prediction error decays as n− 2k+2a+1 , where a denotes the decay parameter of the covariance eigenvalues. In this regime, we also establish a matching lower bound, showing that observing the trajectories only on a sufficiently fine grid incurs no minimax loss compared to the continuous setting. When p is moderately large, the number of observations on the grid also plays 2k+2a 2a−1 a role, and the prediction error decays as n− 2k+2a+1 + p− 2a . When p is small the error decays 2a−1 as p− 2a . Finally, we apply our method on both simulated data and real-world meteorological observations.

Outline of the paper. Section 2 introduces the estimation procedure for β and the risk measure considered in this work. Section 3 establishes oracle-type inequalities. Section 4 derives the convergence rates over periodic Sobolev spaces. Section 5 presents the lower bound in the case where p is large. Section 6 reports numerical simulation results on artificial datasets and Section 7 on a true meteorological dataset. Notation: We denote by L2 the space of square-integrable functions on the interval [0, 1] equipped R1 1/2 with the inner product ⟨f, g⟩L2 = 0 f (t)g(t)dt and the associated norm ∥f ∥L2 = ⟨f, f ⟩L2 . For any set A, we denote by 1A the indicator function and for all non-negative integers i and j we define δij as the Kronecker delta. For any square matrix A we define the operator norm by ∥A∥op = sup∥a∥2 =1 ∥Aa∥2 , where for any vector u, ∥u∥2 denotes the Euclidean norm. In particular, we consider the space l2 consisting of all sequences c = (cl )l≥1 , such that ∥c∥22 < ∞. We also consider ⟨·, ·⟩2 the standard Euclidean inner product. Moreover for any a, b ∈ Z and m ∈ N\{0}, we write a ≡ b[m] to indicate that a and b are congruent modulo m.

Estimation procedure

2.1

Curve reconstruction

We consider an i.i.d. sample (Yi , Xi )i=1,...,n drawn from the distribution of (Y, X). Thus, for each i = 1, . . . , n, the pair (Yi , Xi ) satisfies Z 1 Yi = β(t)Xi (t)dt + ϵi , (4) 0

where the errors (ϵi )i=1,...,n are assumed to be i.i.d., centered, with finite variance σ 2 , and independent of the covariates (Xi )i=1,...,n . The first step of our methodology is to reconstruct the true unobserved curves Xi , from their noisy and discrete observations, Zi . To achieve this, we approximate each curve Xi by a finite series expansion in a suitable basis. Given our assumptions that the true curves are periodic, centered, second-order stationary and in L2 ([0, 1]), the Fourier basis provides a natural and efficient choice for this decomposition. This choice is further justified by the discussion in the next section (see Section 2.3). Let us first define Nn,p ∈ N\{0} and Mn,p = {1, . . . , Nn,p }. We denote Sm := Span{ϕ1 , ϕ2 , . . . , ϕ2m+1 },

(5)

for all m ∈ Mn,p with ϕ1 = 1,

ϕ2j (·) =

√

2 cos 2πj· ,

ϕ2j+1 (·) =

√

2 sin 2πj· .

We also define Dm the dimension of Sm (in particular, Dm = 2m + 1). As we only consider Dm elements, Sm represents a finite dimension space spanned by a truncated Fourier basis. Since the Zi ’s are only known at discrete points th , it’s impossible to compute ⟨Xi , ϕj ⟩L2 directly. Hence we define p−1 1X Zi (th )ϕj (th ). (6) x ei,j = p h=0

This quantity is a discrete approximation of the inner product ⟨Xi , ϕj ⟩L2 , based on noisy observations on a regular grid, and can be interpreted as a Riemann sum in the ideal continuous setting. This particular scheme and choice of grid create a convenient discrete orthogonality property for the chosen Fourier basis under the condition that DNn,p < p. This significantly simplifies the mathematical proofs required for the main results which are detailed in the appendix (Lemma B.1 and Lemma C.1). Then, we can define the reconstructed curves as follows : DNn,p

ei (t) := X

x ei,j ϕj (t),

for all i = 1, . . . , n

j=1

and for all t ∈ [0, 1].

(7)

ei is a random element of the finite-dimensional space SN , entirely determined by the Thus, X n,p discrete noisy sample Zi (t0 ), . . . , Zi (tp−1 ) . e a generic random element having the same distribution as any X ei . EquivWe also denote by X alently, if (X, η0 , . . . , ηp−1 ) is a generic copy of (Xi , ηi,0 , . . . , ηi,p−1 ), with Z(th ) = X(th ) + ηh , then DNn,p X e X(t) = x ej ϕj (t), t ∈ [0, 1], j=1

with

p−1

x ej =

1X Z(th )ϕj (th ). p h=0

e1 , . . . , X en are i.i.d. copies of X. e In particular, X

2.2

Estimating the slope function

In this section, we build an estimator of the true slope function β. Let m ∈ Mn,p . First of all, we define the following least-square criterion : n

γn (f ) :=

1X ei ⟩L2 )2 , (Yi − ⟨f, X n i=1

f ∈ Sm .

(8)

Because the trajectories Xi are unobserved, the ideal least-squares contrast (defined using Xi ei in (8)) cannot be directly minimized. We therefore use the least-square criterion rather than X ei )’s which are the only available curves. This induces a γn based on the reconstructed curves (X difference with the ideal model, which is quantified by the reconstruction error terms appearing in the oracle inequalities introduced in Section 3. We wish to minimize γn to get an estimate of β. More particularly, we define βbm as (9)

βbm ∈ argmin γn (f ). f ∈Sm

PDm Any f ∈ Sm can be rewritten as f = j=1 vj ϕj where v = (v1 , v2 , . . . , vDm ) is in RDm . Therefore, minimizing γn in Sm is equivalent to minimizing  2 Dm n X 1 X e i ⟩L2  F (v) := Yi − vj ⟨ϕj , X in RDm . (10) n i=1 j=1 Considering

Φm := and

1X x ei,k x ei,j n i=1 n

b :=

1X Yi x ei,k n i=1

! ,

(11)

1≤j,k≤Dm

! ,

(12)

1≤k≤Dm

we have the following result, ∇F (v) = −2b + 2Φm v,

(13)

where ∇F denotes the gradient of F . The computations that lead to this result are detailed in Lemma A.1. If Φm is invertible we find that the minimum of F , α e, is such that α e = (Φm )−1 b and P D m βbm = j=1 α e j ϕj . However, Φm is built from only n observations projected onto a Dm -basis. If the model dimension exceeds the sample size (Dm > n) or if some projected coordinates are linearly dependent in the data, Φm loses rank, giving at least one zero eigenvalue so its determinant vanishes and it cannot 4

be inverted. Even when Dm ≤ n, any direction that shows nearly zero empirical variance still makes the matrix singular or ill-conditioned. b(p) b(p) To deal with this issue, let us define the set Gm := {λ m ≥ sn } where λm is the smallest eigenvalue of Φm and 1 2 α > 0. (14) sn = α 1 − √ n ln n This form of sn is inspired by the one used in Brunel and Roche [4] introduced in Section 3.2. This condition guarantees positive definiteness and hence invertibility of Φm on Gm . Moreover it enables us to show that the probability of the event Gm not happening is small, for all m ∈ Mn,p . T Now, let G := m∈Mn,p Gm . On G we can compute the least squares estimate βbm of β in Sm for all m ∈ Mn,p . Hence on this set, we can define an integer m b ∈ arg min(γn (βbm ) + pen(m)), m∈Mn,p

where

pen(m) = θ(1 + δ)Dm σ 2 /n

for θ > 4 and δ > 0.

(15)

This penalty is crucial for model selection as it accounts for model complexity and thus helps to avoid overfitting. Moreover, we chose this particular penalty because its structure is specifically designed to control and bound certain terms that appear in the derivation of the oracle inequality (see Theorem 3.1). It is also inspired by the work of Brunel and Roche ([4], Equation (8)) and Brunel et al. ([3], Section 1.4). Finally, we set the estimate of the true slope function β as βbm on G, b e β= c 0 on G .

2.3

(16)

Risk measure

This section introduces the criterion used to evaluate the performance of the proposed estimator. We start by introducing the covariance operator of the true unobserved random curve X: Γf (·) = E (⟨f, X⟩L2 X(·)) ,

f ∈ L2 ([0, 1]).

(17)

Under our assumptions that the curves (Xi ) are periodic, centered and second-order stationary, the eigenfunctions of Γ coincide with the Fourier basis (as discussed in Section 2.1 of Comte and Johannes [8]). Denoting by (λj )j≥1 the associated eigenvalues, we have for each j ≥ 1, Γϕj = λj ϕj ,

⟨ϕj , ϕk ⟩L2 = δjk .

Let us define the kernel K associated with the operator Γ : K(s, t) = E (X(s)X(t)) ,

s, t ∈ [0, 1].

(18)

We also introduce the empirical version of this operator, that we denote Γn , and which is associated with the kernel n 1X Kn (s, t) = Xi (s)Xi (t), s, t ∈ [0, 1]. (19) n i=1 In particular for all f ∈ L2 ([0, 1]), n

Z 1

1X Γn f (·) = Kn (., t)f (t)dt = ⟨f, Xi ⟩L2 Xi (·). n i=1 0 e by We next define the covariance kernel of the reconstructed process X e t) = E X(s) e X(t) e K(s, , s, t ∈ [0, 1], 5

(20)

(21)

and its associated covariance operator Z 1 e (·) = e t)f (t)dt = E ⟨f, X⟩ e L2 X(·) e Γf K(·, ,

f ∈ L2 ([0, 1]).

(22)

ei )i=1,...,n , we also consider the empirical covariance kernel Given the reconstructed sample (X n X e n (s, t) = 1 ei (s)X ei (t), K X n i=1

s, t ∈ [0, 1],

(23)

e n , defined for any f ∈ L2 ([0, 1]) together with the corresponding empirical covariance operator Γ by Z 1 n X e e n (·, t)f (t)dt = 1 ei ⟩L2 X ei (·). Γn f (·) = K ⟨f, X (24) n i=1 0 These operators naturally induce the following bilinear forms (and associated seminorms) n

⟨f, g⟩Γn = ⟨Γn f, g⟩

1X = ⟨f, Xi ⟩L2 ⟨g, Xi ⟩L2 , n i=1

(25)

1X ei ⟩L2 ⟨g, X ei ⟩L2 , ⟨f, X n i=1 e g⟩L2 = E ⟨f, X⟩ e L2 ⟨g, X⟩ e L2 , ⟨f, g⟩Γe = ⟨Γf,

(26)

⟨f, g⟩Γ = ⟨Γf, g⟩L2 = E (⟨f, X⟩L2 ⟨g, X⟩L2 ) .

(28)

e n f, g⟩L2 = ⟨f, g⟩Γen = ⟨Γ

(27)

To quantify the performance of the estimator, we study the prediction error under the distribution of the true process. More specifically, we consider a new curve Xn+1 with a scalar response Yn+1 independent of the sample with (Xn+1 , Yn+1 satisfying (1). We wish to quantify the following error 2 2 e Xn+1 ⟩L2 − E (Yn+1 |Xn+1 ) X1 , · · · , Xn = E ⟨βe − β, Xn+1 ⟩L2 X1 , · · · , Xn E ⟨β, = ∥βe − β∥2Γ ,

(29) This criterion quantifies the average discrepancy between predictions based on βe and those based on β, when both are evaluated on the same unobserved curve. Remark 1. In the functional linear model, the quantity of interest is the prediction ⟨β, X⟩L2 , rather than the function β itself. For this reason, the natural risk is E ⟨βe − β, X⟩2L2 = E ∥βe − β∥2Γ , which measures the prediction error. The L2 -norm does not take into account the distribution of the covariate X, and in particular directions corresponding to small eigenvalues of the covariance operator Γ have little influence on the prediction, but may contribute significantly to the L2 -error.

Main results

In this section, we derive an upper bound of E(∥βe − β∥2Γe ) and E(∥βe − β∥2Γ ) in the oracle setting. Before stating our assumptions, we recall the notion of a sub-Gaussian random variable from Vershynin [16], used throughout this section. A random variable A is sub-Gaussian with variance factor v if for all t ∈ R t2 v E et(A−E(A)) ≤ e 2 . Throughout this section, we work under the following assumptions: 6

P • (H1) For all i ∈ {1, ..., n} and for all (cl )l≥1 ∈ l2 , l≥1 cl ⟨Xi , ϕl ⟩L2 is sub-Gaussian with P variance factor K12 l≥1 c2l λl , where K1 is a positive constant. • (H2) For all i = 1, ..., n and h = 0, ..., p − 1, ηi,h is sub-Gaussian with variance factor τ 2 . • (H3) We assume that the eigenvalues of Γ, namely the λj ’s decrease in a polynomial way : there exist two constants c′ > 0 and a > 1/2 such that for all j ≥ 1, λj ≤ c′ j −2a . Assumption (H1) is satisfied under the following set of sufficient conditions: Assume that, for all i ∈ {1, . . . , n} and all j ≥ 1, the Fourier coefficients ⟨Xi , ϕj ⟩L2 are subGaussian with variance factor λj . This assumption appears repeatedly in the works of Brunel and Roche [4] and Brunel et al. [3]. If in addition, for each i, the random variables ⟨Xi , ϕj ⟩L2 and ⟨Xi , ϕk ⟩L2 are independent whenever j ̸= k, then Assumption (H1) holds. We emphasize however, that these conditions may be restrictive in practice. In particular, the independence of the Fourier coefficients is a strong requirement that is generally satisfied only in specific settings, such as Gaussian processes represented in their Karhunen–Loève basis. Nevertheless, this independence assumption can be dispensed when Hypothesis (H3) holds for some a > 1, thereby providing an alternative set of sufficient conditions for Assumption (H1). Assumption (H2) is primarily introduced to derive concentration inequalities. It is a mild and reasonable condition in practice when the observational noise is light-tailed. Assumption (H3) imposes a polynomial decay on the eigenvalues of the covariance operator Γ. Such a condition is standard in functional data analysis (see, e.g., Brunel and Roche [4], Cardot and Johannes [6], and Comte and Johannes [8]) and reflects the regularity of the stochastic process X. In particular, it implies that most of the variability of X is captured by the first Fourier modes, whereas the contribution of higher-frequency components decreases progressively. Moreover, larger values of the parameter a correspond to smoother sample paths of X. For example, Brownian motion satisfies this assumption with a = 1, while integrated Brownian motion corresponds to a = 2, consistent with its smoother sample paths. e are diagonal operators on Span{ϕ1 , ϕ2 , . . . , ϕ2N +1 } with strictly Remark 2. Since both Γ and Γ n,p positive eigenvalues, the orthogonal projection of β onto Sm with respect to the scalar product ⟨·, ·⟩Γ coincides with its orthogonal projection with respect to ⟨·, ·⟩Γe . We denote this common projection by β (m) . The detailed proof of this result is established in Appendix E. First let us state the following result which will be useful to derive oracle inequalities : Proposition 3.1. Assume that there exists l > 4 such that υl := E(|ϵ|l ) < ∞ and that E ⟨β, X⟩4L2 < ∞. Moreover we suppose that assumptions (H1), (H2), (H3) are verified. If n DNn,p < min ,p , ln2 n 2 nα and α > 2a in the definition of sn (see in Equation (14)), then for all slope β ∈ L2 ([0, 1]) we have E(∥βe − β∥2Γe ) ≤ C0 min E(∥β (m) − β∥2Γe ) + pen(m) λDNn,p ≥

m∈Mn,p

e − X∥2 2 ) + ∥β∥2L2 E(∥X L 1/2 ∥β∥2L2 e 1 4 E∥X − X∥L2 + Cβ + + n

(30) 1 , np

where pen(m) is defined in (15), C0 is a positive constant which depends on K1 , θ, l, τl , δ, σ 2 . Moreover we have: Cβ = E(⟨β, X1 ⟩4L2 )1/2 + ∥β∥2L2 + 1. (31) 7

The following result provides an oracle inequality with respect to the norm ∥ · ∥Γe , thereby evaluating the prediction risk based on the reconstructed curves. Theorem 3.1. Suppose that the assumptions of Proposition 3.1 hold. Then for all slope β ∈ L2 ([0, 1]), 2 e E(∥β − β∥Γe ) ≤ C1 min E(∥β (m) − β∥2Γe ) + pen(m) m∈Mn,p

e − X∥2 2 ) + ∥β∥2L2 E(∥X L 1/2 ∥β∥2L2 e 1 1 E∥X − X∥4L2 + Cβ + , + n n np

(32)

where pen(m) is defined in (15), C1 is a positive constant which depends on θ, l, τl , δ, σ 2 and Cβ is defined in (31). The following result provides an oracle inequality with respect to the norm ∥ · ∥Γ , thereby evaluating the prediction risk based on the true curves. Theorem 3.2. Suppose that the assumptions of Theorem 3.1 are satisfied. Then for all slope β ∈ L2 ([0, 1]) we have, 2 e E(∥β − β∥Γ ) ≤ C2 min ∥β (m) − β∥2Γ + ∥β (m) − β∥2Γe + pen(m) m∈Mn,p

e − X∥2 2 ) + ∥β∥2L2 E(∥X L 1/2 2 ∥β∥L2 1 1 e − X∥4 2 + E∥X + , + C β L n n np

(33)

where pen(m) is defined in (15), C2 is a positive constant which depends on K1 , θ, l, τl , δ, σ 2 , c′ , a and Cβ is defined in (31). If we assume furthermore that β ∈ W per (k, L) we obtain the following oracle inequality : 2 e E(∥β − β∥Γ ) ≤ C3 min ∥β (m) − β∥2Γ + pen(m) m∈Mn,p

e − X∥2 2 ) + E(∥X L 1/2 1 1 1 1 e − X∥4 2 , + + + E∥X + L n p n np

(34)

where C3 is a positive constant which depends on K1 , θ, l, τl , δ, σ 2 , c′ , a. The sequence of Proposition 3.1 and Theorems 3.1–3.2 reflects a progressive refinement of the risk bounds. Proposition 3.1 establishes an oracle type inequality expressed in terms of the empirical covariance operator of the reconstructed data, which is the most directly observable quantity in our framework. Theorem 3.1 then replaces this empirical operator by the covariance operator of the reconstructed process, thereby removing the effect of sampling variability. Finally, Theorem 3.2 transfers the result to the covariance operator of the true underlying process by controlling the discrepancy between the reconstructed and true covariance structures. The bounds obtained in Theorems 3.1 and 3.2 are of oracle type. They show that the estimator βe performs, up to a multiplicative constant and additional remainder terms, nearly as well as the best estimator in the collection (Sm )m∈Mn,p . More precisely, the leading term in the bound of Theorem 3.1 is governed by the oracle criterion min ∥β (m) − β∥2Γe + pen(m) , m∈Mn,p

whereas Theorem 3.2 involves min

m∈Mn,p

∥β (m) − β∥2Γ + pen(m) . 8

e − X∥L2 quantify the error induced by reconstructing the curves The additional terms involving ∥X from discrete noisy observations and therefore measure the impact of the preliminary smoothing step on the final estimation procedure. The remaining terms, of order 1/n, 1/p, and 1/(np), arise from the proof of the oracle inequalities.

Convergence rates over Sobolev spaces

In this part, we derive convergence rates for the error E ∥βe − β∥2Γ . Theorem 4.1. Suppose that the assumptions of Theorem 3.2 are satisfied, with the exception that we consider a ≥ 1 instead of a > 1/2 in (H3). Then, for all β ∈ W per (k, L) : 2a then, • If p ≳ lnn2 n 2a+2k E ∥βe − β∥2Γ = O n− 2a+2k+1 . (35) 2a

• If n 2a+2k+1 ≲ p ≲

2a n then, ln2 n 2a+2k 2a−1 E ∥βe − β∥2Γ = O n− 2a+2k+1 + p− 2a .

(36)

2a−1 E ∥βe − β∥2Γ = O p− 2a .

(37)

• If p ≲ n 2a+2k+1 then,

The stronger assumption a ≥ 1 is required in Theorem 4.1 to control the contribution of the reconstruction error and to ensure that the tail sum of the eigenvalues decays sufficiently fast. Moreover, the three regimes described in Theorem 4.1 reveal a phase transition phenomenon that depends on the relative magnitudes of p and n. When p is large, the problem behaves essentially as if the curves were fully observed. In contrast, when p is small, the estimation error is dominated by the discretization error, and increasing the sample size n alone is not sufficient to improve the convergence rate. Furthermore, when the number of observation points is sufficiently large, namely 2a n p≳ , ln2 n the convergence rate obtained in Theorem 4.1 coincides with the optimal rate established in the literature for functional linear models with fully observed curves. In particular, we recover the rate derived by Brunel and Roche [4]. Finally, as shown in the proof of Theorem 4.1, the dependence of the rate on p is primarily driven by the observation noise rather than by the reconstruction procedure itself.

Lower Bound for the case p large

In this section we derive a lower bound for the case 1 of Theorem 4.1. Theorem 5.1. Let us assume that there exist two constants c′ > 0 and a > 1/2 such that for all j ≥ 1, j −2a /c′ ≤ λj ≤ c′ j −2a . We also make the further assumption that ϵi ∼ N (0, σ 2 )

∀i ∈ {1, . . . , n},

and that ηi,h ∼ N (0, τ 2 ) Then for p ≥

n ln2 n

∀i ∈ {1, . . . , n}, ∀h ∈ {0, . . . , p − 1}.

and for all β ∈ W per (k, L), 2a+2k E ∥βe − β∥2Γ ≥ C4 n− 2a+2k+1 ,

where C4 is a positive constant which depends on k, L, c′ , σ. 9

In Theorem 4.1, only the upper bound λj ≤ c′ j −2a is required. Indeed, the proof relies on controlling approximation and reconstruction errors, which involve tail sums of the eigenvalues and therefore only require an upper estimate on their decay. In contrast, the proof of Theorem 5.1 is based on a minimax lower-bound construction. To ensure that the candidate slope functions defined in (81) belong to W per (k, L) and remain sufficiently separated in the prediction norm ∥ · ∥Γ , it is necessary to control both λj and 1/λj . This requires the two-sided condition c1′ j −2a ≤ λj ≤ 2a+2k

c′ j −2a . Furthermore, Theorem 5.1 establishes that, when p is sufficiently large, the rate n− 2a+2k+1 constitutes a lower bound for the prediction risk over the class W per (k, L). Combined with the upper bound derived in Theorem 4.1, this shows that our estimator attains the minimax rate over this class.

Simulations

Across the next four paragraphs, we present a framework for evaluating our estimator on a simulated dataset. This framework consists of generating the true functional curves, computing the corresponding scalar outputs, creating noisy discrete observations, reconstructing the curves, choosing adequate values for θ and δ, estimating the slope function, and finally assessing the model’s performance. We study here three slope functions β1 (t) = t(t − 1),

(38)

t ∈ [0, 1], (1)

which is in W per (1, 1) and also studied in Brunel and Roche [4]. Since for all t ∈ [0, 1] β1 (t) = (1) (1) 2t − 1, therefore β1 (0) ̸= β1 (1) and β1 ∈ / W per (2, 1). (t − 0.70)2 (t − 0.30)2 − 2 exp − , t ∈ [0, 1], (39) β2 (t) = 3 exp − 2 · 0.072 2 · 0.082 which is defined as a linear combination of two Gaussian functions and β3 (t) = 4 sin(4πt) − sign(t − 0.3) − sign(0.72 − t),

t ∈ [0, 1],

(40)

which corresponds to the heavisine function.

Figure 1: Plot of β1 (left), β2 (middle), β3 (right)

6.1

Simulation setup and data generation

The simulation study begins with the generation of the true functional curves, denoted Xi , representing the underlying functional processes. These curves are constructed using a truncated Karhunen–Loève decomposition: X(t) =

2J+1 X

ξj ϕj (t),

t ∈ [0, 1].

(41)

j=1

Here, the sequence of independent, centered random variables {ξ1 , ..., ξ2J+1 } has variances Var(ξj ) = λj . We set J = 500 and consider the ξj ’s to be Gaussian.

To generate the true curves (Xi )i=1,...,n , we first construct a high-resolution grid G1 over the observation interval [0, 1], defined as: j j = 0, . . . , psim − 1 . G1 := psim − 1 This grid consists of psim = 10000 equally spaced points and provides a discrete approximation of the underlying continuous functions. Because it is not feasible to simulate the curves at every point in continuous time, this fine, regular grid serves as a practical compromise between approximation accuracy and computational cost. Next, we define a lower-resolution observation grid, G2 , consisting of p regularly spaced points where the noisy and discrete data are actually observed. This grid is defined as: h h = 0, . . . , p − 1 . G2 := th = p Because these observation points do not necessarily coincide with those of the high-resolution grid, we augment the latter to include G2 . The true curves are generated on this combined grid, G1 ∪G2 . Next, independent Gaussian noise terms ηi,h , with zero mean and variance τ 2 are added to the discretized observations. The value of τ 2 will vary and thus will be precised in each sections. This step aims to mimic measurement error in practical settings, yielding the observed sample (Zi )i=1,...,n , defined as: Zi (th ) = Xiobs (th ) + ηi,h , th ∈ G2 . (42) Once the functional predictors are generated, the corresponding scalar response Yi for each curve is simulated according to the functional linear model defined in (4). To compute the integral R1 term 0 Xi (t)β(t)dt, we employ a discrete numerical approximation evaluated over the grid G1 . In particular, Z 1 psim X−1 1 Xi (t)β(t)dt ≈ Xi (tj )β(tj ). psim − 1 j=0 0 Finally, the noise variables ϵi are generated independently from a zero-mean Gaussian distribution with variance σ 2 = 0.1.

6.2

Estimation of the slope function

Prior to estimating β, the functional predictors are reconstructed from the noisy and discretized observations. Following the procedure described in Section 2.1, we first compute the coefficients (e xi,j )i=1,...,n ; j=1,...,DNn,p as defined in Equation (6). These coefficients are then used to reconstruct ei according to Equation (7). This procedure yields a sample of reconstructed curves the curves X ei )i=1,...,n , which can be interpreted as smoothed versions of the original observations. The choice (X of the basis dimension DNn,p matters. In particular, DNn,p must be chosen so as to ensure the invertibility of the empirical covariance matrix with high probability, and therefore to be able to compute βe defined in (16). Theorem 3.1 shows that this condition is satisfied whenever n DNn,p < min , p . ln2 n Accordingly, in our simulations, we select the largest value of DNn,p satisfying this constraint in order to achieve the most accurate curve reconstruction possible. Based on the reconstructed curves, we proceed to estimate the slope function β. To this end, we adopt a penalized model selection approach to determine the appropriate model complexity, as described in Section 2. The performance of the resulting estimator is supported by an oracle inequality (Theorem 3.1), provided that the penalty parameters satisfy θ > 4 and δ > 0. In practice, these parameters must be calibrated; the corresponding selection procedure is detailed in Section 6.4.

Figure 2: Simulation of a functional curve: true, observed, and reconstructed. Here p = 50, psim = 10000, DNn,p = 11, τ 2 = 0.1 and λj = 1/j 4 for all j = 1, . . . , 2J + 1.

6.3

Prediction error

As discussed in the previous sections, we focus on the prediction error of the proposed estimator. More specifically, we aim to analyze the behavior of E ∥βe − β∥2Γ . P As ∥βe − β∥2Γ = j≥1 λj ⟨βe − β, ϕj ⟩2L2 , we approximate this quantity by using a finite number of elements in the Fourier basis, ∥βe − β∥2Γ ≈

2J+1 X

λj ⟨βe − β, ϕj ⟩2L2 ≈

j=1

2J+1 X

λj (e αj − βj )2 1j≤Dm + βj2 1j>Dm , c c

j=1

where the last equality is derived from the definition of βe given in Equation (16). Finally we estimate our prediction error using a Monte-Carlo method with nM C independent samples. In particular we derive nM C estimators of β that we denote as βe(l) for l = 1, ..., nM C . For each replication l ∈ {1, . . . , nMC }, we obtain an estimator βe(l) defined by Equation (16) and its definition PDm (l) c (l) α ej ϕj , where m b (l) denotes the model selected at iteration l. We is given on G by βe(l) = j=1 then compute an estimate of the prediction error: E ∥βe − β∥2Γ ≈ where ebl =

6.4

P2J+1 j=1

1 nM C

nX MC

ebl ,

l=1

(l) 2 λj (e αj − βj )2 1j≤Dm + β 1 . For our simulations we use nM C = 50. j>D j c (l) m c (l)

Calibration of θ and δ

The constant κ := θ(1 + δ), which appears in the penalty term defined by (15), must be calibrated. To determine a suitable value for it, we evaluate its predictive performance across a grid of candidate 12

values ranging from 1 to 20 (with a step size of 0.5), holding the sample size n = 1000 and the number of observation points p = 1000 fixed. Specifically, we conduct a Monte Carlo simulation study to assess the candidate values across the three distinct slope functions β1 , β2 , and β3 . For each slope function, the prediction error, which is approximated via the procedure detailed in Section 6.3, is averaged over 50 independent replications. We then calculate the overall mean error across all three scenarios to identify a single, universal κ that minimizes the aggregate prediction error. The results of this calibration process are visualized in Figure 3. As indicated by the vertical dashed line, the aggregate prediction error attains its minimum at κ = 3. However, given the constraints θ > 4 and δ > 0, the definition of the penalty parameter strictly requires κ > 4. Consequently, we select κ = 4.01, a value situated appropriately above this theoretical boundary but close to the lowest aggregate prediction error, for all subsequent simulations.

Figure 3: Average prediction error as a function of the penalty parameter κ ∈ [1, 20] across the three slope functions β1 , β2 , β3 .

6.5

Results

We now study the performance of our estimator when various slope functions are considered. 6.5.1

Study of β1

Now we consider β1 as the true slope function, which belongs to W per (1, 1) and λj = 1/j 4 for all j = 1, . . . , 2J + 1. In Figures 4a, 4b we analyze the impact of the sample size n by setting a fixed p = 7500. The variance of the noises τ 2 and σ 2 are both set to 0.1. The sample size, n, varies across the set {200, 400, 800, 1600, 3200, 6400}. The resulting log-log plot yields a slope of −0.92, which is quite close to the value that we can expect given the convergence rate established in Theorem 4.1. Indeed, here we have k = 1 and a = 2 hence we expect to have an error which decreases in n−(2k+2a)/(2a+2k+1) = n−6/7 . In Figures 4c, 4d we study the impact of the size of the grid p by setting a fixed n = 1000. The variance of the noise σ 2 = 0.1 but we use here a much higher variance for the noise of the observation τ 2 = 4. The number of observations, p, on the grid varies across the set {50, 100, 200, 300, 400, 500}. The resulting log-log plot yields a slope of −0.79 which matches the rate that we expect from Theorem 4.1. Indeed, here we have a = 2 hence we expect to have an error which decreases in p−(2a−1)/(2a) = p−3/4 . However, for smaller values of τ , the rate of decrease is significantly higher. This could be explained by the fact that, when τ is small, the influence of noise on the reconstruction error of the curves is reduced; instead, the error is primarily driven by the choice of reconstruction scheme. We also studied the case where the true curves are less smooth. In particular we consider now λj = 1/j 2 for all j = 1, . . . , 2J + 1. All the parameters and considered function remain unchanged. 13

a. Average prediction error vs. sample size n

b. Log-log plot of average prediction error vs. sample size n

c. Average prediction error vs. number of observation points p

d. Log-log plot of average prediction error vs. number of observation points p

Figure 4: Monte Carlo approximation of the mean squared Γ-norm error as a function of the sample size n and the grid density p. The solid red curves represent the empirical error averaged over the Monte Carlo replications. In panels a. and c. the blue dashed curves represent the 10th and 90th empirical deciles of the Monte Carlo errors. Panels b. and d. display the corresponding mean errors on a log-log scale; in these panels, the black dashed lines represent linear fits used to assess the empirical rate of decay of the error.

On Figure 5 we observe that the resulting log-log plot for the case p fixed - varying n yields a slope

a. Average prediction error vs. sample size n

b. Log-log plot of average prediction error vs. sample size n

c. Average prediction error vs. number of observation points p

d. Log-log Plot of average prediction error vs. number of observation points p

Figure 5: Monte Carlo approximation of the mean squared Γ-norm error as a function of the sample size n and the grid density p. The solid red curves represent the empirical error averaged over the Monte Carlo replications. In panels a. and c. the blue dashed curves represent the 10th and 90th empirical deciles of the Monte Carlo errors. Panels b. and d. display the corresponding mean errors on a log-log scale; in these panels, the black dashed lines represent linear fits used to assess the empirical rate of decay of the error. of −0.76, which is close to the value that we can expect given the convergence rate established in Theorem 4.1. Indeed, here we have k = 1 and a = 1 therefore we expect to have an error which decreases in n−(2k+2a)/(2a+2k+1) = n−0.8 . For the case n fixed - varying p the log-log plot gives an estimated slope of approximately −0.55, which indicates that the empirical decay of the prediction error is quite consistent with the expected p−(2a−1)/(2a) = p−1/2 behavior. 6.5.2

Study of β2

In this section, we consider the true slope function β2 . Because β2 does not perfectly satisfy the periodic boundary condition β2 (0) = β2 (1), this setup allows us to evaluate how our estimator perform even if the true slope function does not perfectly lie in the space W per . In this part we pick a = 2 and aim to compare our result with the first study of β1 . The same parameters are used for the simulations. The log-log plot in Figures 6b and 6d yield respectively slopes of −0.86 and −0.77, which are very close to the one that we got previously and which match our theoretical results. 6.5.3

Study of β3

In this section, we evaluate the estimator’s ability to handle change points by examining its behavior around the two discontinuities located at t1 = 0.30 and t2 = 0.72 of the true slope function β3 defined in Equation (40). Instead of using a threshold-based detection method, we adopt a visual approach to better understand the local behavior of our estimator. We plot the true slope function against 10 estimated curves obtained from independent samples. To illustrate the asymptotic 15

a. Average prediction error vs. sample size n

b. Log-log plot of average prediction error vs. sample size n

c. Average prediction error vs. number of observation points p

d. Log-log plot of average prediction error vs. number of observation points p

Figure 6: Monte Carlo approximation of the mean squared Γ-norm error as a function of the sample size n and the grid density p. The solid red curves represent the empirical error averaged over the Monte Carlo replications. In panels a. and c. the blue dashed curves represent the 10th and 90th empirical deciles of the Monte Carlo errors. Panels b. and d. display the corresponding mean errors on a log-log scale; in these panels, the black dashed lines represent linear fits used to assess the empirical rate of decay of the error.

behavior of the estimator, we compare two distinct settings: a moderate sample size (n = 800, p = 5000, DNn,p = 15) and a large sample size (n = 6400, p = 5000, DNn,p = 81). Figure

a. n = 800, p = 5000, and DNn,p = 15.

b. n = 6400, p = 5000, and DNn,p = 81.

Figure 7: Estimation of the slope function β3 for p = 5000 and different sample sizes n. 7a displays the results for the setting with n = 800. The estimated curves (in red) capture the global trend of β3 but completely smooth over the discontinuities. On the other hand, Figure 7b illustrates the results for n = 6400. With a larger sample size and a higher dimension for the basis (DNn,p = 81), the variance of the estimators is significantly reduced, yielding a good fit on the continuous segments of β3 . More importantly, around the jump location t2 , the estimators actively attempt to fit the discontinuities as it benefits from an increased flexibility provided by DNn,p = 81.

6.6

Effect of the unknown variance σ 2

Throughout this work we have assumed that σ 2 , the variance of ϵ defined in Equation (1), is known. In practice however, this is generally not the case and σ must therefore be estimated. To this end, for each model dimension m, we consider the empirical variance estimator 2 σ bm :=

n D E 2 1 X ei Yi − βbm , X . n i=1 L2

(43)

We then select the dimension m b plug by minimizing the following penalized criterion: 2 Dm σ bm m b plug ∈ arg min γn (βbm ) + θ(1 + δ) . n m∈Mn,p

(44)

Table 1: Approximate mean squared Γ-norm prediction errors for the slope function β1 with variance σ 2 known or unknown. The upper panel reports errors as n varies with p = 7500 fixed, while the lower panel reports errors as p varies with n = 1000 fixed. Slope β1 β1

σ2 known unknown

n = 200 1.42 × 10−4 1.27 × 10−4

Fixed p = 7500 n = 400 n = 800 6.90 × 10−5 3.52 × 10−5 8.56 × 10−5 3.73 × 10−5

n = 1600 1.73 × 10−5 1.83 × 10−5

n = 3200 1.09 × 10−5 8.68 × 10−6

n = 6400 6.55 × 10−6 6.11 × 10−6

Slope β1 β1

σ2 known unknown

p = 50 2.24 × 10−4 2.18 × 10−4

Fixed n = 1000 p = 100 p = 200 1.06 × 10−4 6.65 × 10−5 9.32 × 10−5 4.96 × 10−5

p = 300 5.09 × 10−5 4.72 × 10−5

p = 400 3.92 × 10−5 3.87 × 10−5

p = 500 3.94 × 10−5 3.44 × 10−5

Table 2: Approximate mean squared Γ-norm prediction errors for the irregular slope function β2 with variance σ 2 known or unknown. The upper panel reports errors as n varies with p = 7500 fixed, while the lower panel reports errors as p varies with n = 1000 fixed. Slope β2 β2

σ2 known unknown

n = 200 2.60 × 10−3 3.31 × 10−3

Fixed p = 7500 n = 400 n = 800 1.64 × 10−3 1.02 × 10−3 1.84 × 10−3 1.07 × 10−3

n = 1600 5.18 × 10−4 4.56 × 10−4

n = 3200 2.36 × 10−4 2.65 × 10−4

n = 6400 1.48 × 10−4 1.55 × 10−4

Slope β2 β2

σ2 known unknown

p = 50 5.93 × 10−3 5.63 × 10−3

Fixed n = 1000 p = 100 p = 200 2.66 × 10−3 1.66 × 10−3 2.81 × 10−3 1.51 × 10−3

p = 300 1.11 × 10−3 1.22 × 10−3

p = 400 1.01 × 10−3 1.07 × 10−3

p = 500 1.06 × 10−3 9.19 × 10−4

Tables 1 and 2 compare the mean squared Γ-norm prediction errors obtained when σ 2 is known and when it is estimated using (43). The comparison is performed for β1 (38) and β2 (39). As in the previous simulations, we consider two asymptotic regimes: first, n varies while p = 7500 is fixed; second, p varies while n = 1000 is fixed. The results show that replacing σ 2 by the 2 residual-based estimator σ bm has little impact on the performance of the estimator. Indeed, in the two tables the prediction errors obtained with known and unknown variance remain of the same order of magnitude and exhibit similar decreasing behavior as n or p increases. This suggests that the procedure remains reliable in the more realistic setting where the regression noise variance is unknown.

Application to meteorological data

We apply our estimation procedure to meteorological data provided by Météo-France, originally covering the period 1950–2024. For this application, we retain observations from the recent period 2019–2024 and average them over time to avoid making the estimated curve depend on the particular weather realization of a single year. After removing observations with missing values and excluding February 29 to obtain a common yearly calendar, the final dataset consists of n = 305 temperature curves, each corresponding to one location, and p = 365 daily measurements. The annual average precipitation recorded at each site is used as the scalar response variable, while the corresponding annual temperature profile is treated as a functional covariate. This yields a scalar-on-function regression framework, whose objective is to investigate how the shape of the 18

temperature curve is associated with annual rainfall across the selected French departments highlighted in Figure 8a. Figures 8b and 8c show examples of the functional data for four stations located in Finistère. The initial step involves partitioning the dataset into a training set (80% of observations) and a test set (20% of observations) to ensure an unbiased evaluation of the final model. We denote as trainIDX and testIDX the indexes of the data present in respectively the train and test set, as well as ntrain and ntest their dimensions. Both sets are subsequently centered to standardize the data. More particularly, the empirical means are computed exclusively from the training set. For the scalar response, we define Y train =

ntrain

Yi ,

i∈trainIDX

while, for each time point tj , the mean temperature curve is given by Z train (tj ) =

ntrain

Zi (tj ).

i∈trainIDX

Both the training and test observations are then centered using these training-set means. Thus, the centered training data are defined as Yetrain = Ytrain − Y train , and for all j = 0, . . . , p − 1 etrain (tj ) = Ztrain (tj ) − Z train (tj ). Z Similarly, the test data are centered using the same means computed from the training set: Yetest = Ytest − Y train , and for all j = 0, . . . , p − 1 Zetest (tj ) = Ztest (tj ) − Z train (tj ). We then reconstruct the curves with the same methodology as in Section 2.1. Afterward we estimate the variance σ 2 using only the elements of training set Zetrain and Yetrain , by using the estimator defined in Equation (43). The functional linear regression model is subsequently fitted on the training set, using the estimator of σ 2 to select the dimension m b plug (see (44)). After this step we obtain an estimator of the slope function of the form m b plug

e = β(t)

α ej ϕj (t),

j=1

and its predictive performance is evaluated on the held-out test set by calculating the root mean squared error (RMSE). Since the true curves are not available, we use the reconstructed ones to make prediction. More particularly, in order to do so, we reconstruct the curves from the noisy and discrete data of the test set (Zetest ). Thus we obtain DNn,p

etest (t) = X i

x ei,j ϕj (t),

for all i ∈ testIDX and for all t ∈ [0, 1],

j=1

and then, for each observation i in the test set we predict m b plug

Yepredi =

x ei,j α ej .

j=1

In Figure 9 we can see that the estimated slope function exhibits two extrema. One is negative during winter and early spring, with its lowest values around February–March, which indicates that higher temperatures during this period are associated with lower annual precipitation. The coefficient then plateau around 0 during the summer. Thus at this time the temperature curve 19

a. French departments included in the meteorological dataset.

b. Averaged annual temperature curves across four stations in Finistère.

c. Averaged annual temperature curves for the selected stations in Finistère.

Figure 8: Meteorological dataset map and the associated temperature curves in Finistère.

e Figure 9: Estimated functional coefficient β(t), describing the effect of the annual temperature profile on average annual precipitation.

a. Plot of the actual vs predicted values, the red dashed line is y = x.

b. Boxplot of the model’s residuals, showing the distribution of the prediction errors on the test set.

Figure 10: Performance assessment of the estimator on the test set.

does not impact the annual precipitation. The estimated slope function then increase quickly, reaching its maximum around October-November. This indicates a positive association between autumnal temperatures and annual precipitation. It then decreases during winter. Figure 10a shows a positive relationship between the observed and predicted centred values, which indicates that the model is able to reproduce the overall trend in annual precipitation. Figure 10b confirm this. In particular, it shows that the prediction errors are mostly centred around zero therefore we don’t have a systematic bias. However, the lower whisker is longer than the upper one, suggesting a slightly greater dispersion among negative residuals. Hence, some locations are more strongly overpredicted than underpredicted. The model achieves a RMSE of 0.58, which is quite low. However, this performance is only a small improvement over the baseline method where we use the average rainfall quantity with all the data from the training set to predict the average rainfall of the test set. This method yields a RMSE of 0.91. A primary reason for this small difference could be that the baseline method is already quite effective. Given that the target variable is the annual average rainfall, it is a stable and predictable value. The variations from year to year might be minimal, meaning that a simple prediction of the overall average is a already a strong starting point.

Gradient Calculation for the Least Squares Minimization

Lemma A.1. The function F which is such that  2 Dm n X 1 X ei ⟩L2  , vj ⟨ϕj , X Yi − F (v) := n i=1 j=1

for all v ∈ RDm ,

is differentiable and its gradient is ∇F (v) = −2b + 2Φm v, where Φm is defined by Equation (11) and b by Equation (12). Proof. The function F being convex and C 1 , finding a minimizer α e ∈ RDm of F is equivalent to Dm such that ∇F (e α) = 0. Let us compute ∇F . finding an α e in R First let us consider k ∈ {1, . . . , Dm }, then   Dm n X ∂F (v) 2X ei ⟩L2 Yi − ei ⟩L2  ⟨ϕk , X vj ⟨ϕj , X =− ∂vk n i=1 j=1   D N n,p n 2X  X Yi x ei,l ⟨ϕk , ϕl ⟩L2  =− n i=1 l=1    DNn,p DNn,p n Dm X X 2 XX + vj  x ei,l ⟨ϕk , ϕl ⟩L2   x ei,l′ ⟨ϕj , ϕl′ ⟩L2  n i=1 j=1 ′ l=1

=−

n X

l =1

Dm X

2 2 Yi x ei,k + x ei,k vj x ei,j . n i=1 n i=1 j=1

ei ’s by their definition The transition from the first to second line is obtained by replacing the X given in (7) and the one from the second to third by using the fact that the ϕj ’s are orthonormal. Hence, ∇F (v) = −2b + 2Φm v.

Proof of Section 3

B.1

Proof of Proposition 3.1

Let us start first by decomposing E(∥βe − β∥2Γe ) into two terms : n

E(∥βe − β∥2Γe ) = E(∥βe − β∥2Γe 1G ) + E(∥βe − β∥2Γe 1Gc ) n

2 2 = E(∥βbm b − β∥Γ e 1Gc ). e 1G ) + E(∥β∥Γ n

2 In a first hand we will focus on the left term E(∥βbm b − β∥Γ e 1G ). Let m ∈ Mn,p be fixed. Consider n

β (m) the orthogonal projection of β over Sm with respect to the scalar product ⟨·, ·⟩Γe . By definition of βbm , we have that γn (βbm ) ≤ γn (β (m) ). Moreover, the definition of m b implies that γn (βbm b ≤ γn (βbm ) + pen(m) ≤ γn (β (m) ) + pen(m). b ) + pen(m) Therefore, replacing the γn ’s by their definition given in (8) we have : n n 1X 1X 2 e ei ⟩L2 )2 + pen(m). 2 (Yi − ⟨βbm , X ⟩ ) + pen( m) b ≤ (Yi − ⟨β (m) , X i L b n i=1 n i=1

ei ⟩L2 + ⟨β, Xi − X ei ⟩L2 + ϵi the previous equation is equivalent to As for all i = 1, ..., n, Yi = ⟨β, X the following one : n

2 1 X e e β − βbm + pen(m) b b , Xi L2 + ϵi + β, Xi − Xi L2 n i=1 n

≤

2 1 X ei 2 + ϵi + β, Xi − X ei 2 + pen(m). β − β (m) , X L L n i=1

Simplifying the terms leads then to this inequality, 2 (m) 2 (m) (m) ∥β − βbm ∥Γe + 2νn (βbm ) + 2rn (βbm ) + pen(m) − pen(m). b b ∥Γ b −β b −β e ≤ ∥β − β n

where,

νn (f ) =

1X ei 2 , ϵi f, X L n i=1

rn (f ) =

1X ei 2 β, Xi − X ei 2 . f, X L L n i=1

Let us focus on the νn part. As νn is a linear process, we have that (m) (m) 2νn (βbm ) ≤ 2∥βbm ∥Γen b −β b −β

sup

νn (f ),

Γn f ∈Sm∨c m e

Γn 1 2 2 with Sm∨ b , ∥f ∥Γ en = 1}. Using the inequality 2xy ≤ θ x + θy for all x, y and for m b = {f ∈ Sm∨m all θ > 0, we get that e

1 (m) (m) 2 2νn (βbm ) ≤ ∥βbm ∥Γe + θ b −β b −β n θ

sup (νn (f ))2 . Γn f ∈Sm∨c m e

Let us focus now on the rn part. Using the Cauchy–Schwarz inequality for the scalar product in Rn we get that (m) rn (βbm )≤ b −β

n n 1/2 1 X 1 X 1/2 (m) e 2 e i ⟩2 2 ⟨βbm , Xi ⟩L2 ⟨β, Xi − X . b −β L n i=1 n i=1

Using now the Cauchy–Schwarz inequality for the scalar product ⟨·, ·⟩L2 , we get that n 1 X 1/2 (m) (m) e i ∥2 2 rn (βbm ) ≤ ∥βbm ∥Γen ∥β∥2L2 ∥Xi − X . b −β b −β L n i=1

Therefore, (m) 2rn (βbm )≤ b −β

n ∥β∥2L2 X 1 b (m) 2 ei ∥2 2 . + θ ∥β m − β ∥ ∥Xi − X b en L Γ θ n i=1

These two steps lead to the following inequality, 2 2 (m) (m) 2 ∥βbm − β∥2Γe + ∥βbm ∥Γe + θ b − β∥Γ b −β en ≤∥β n n θ +

sup (νn (f ))2 Γn f ∈Sm∨c m e

n θ∥β∥2L2 X ei ∥2 2 + pen(m) − pen(m). b ∥Xi − X L n i=1

(m) 2 2 (m) 2 Now, as ∥βbm ∥Γe ≤ 2∥βbm ∥Γe , we obtain for all θ > 4: b −β b − β∥Γ e + 2∥β − β n

4 2 1− ∥βbm − β∥ ≤ 1 + ∥β − β (m) ∥2Γe + θ sup (νn (f ))2 b en Γ n θ θ e f ∈S Γ m∨c m

θ∥β∥2L2 X ei ∥2 2 + pen(m) − pen(m), + ∥Xi − X b L n i=1 which leads to : θ+4 θ2 2 (m) 2 2 (ν (f )) E ∥β − β ∥ + E sup ≤ 1 E ∥βbm n b − β∥Γ en en G Γ θ−4 θ−4 en Γ f ∈Sm∨c m

∥β∥2L2

θ θ−4

n X ei ∥2 2 + ∥Xi − X E L i=1

θ E pen(m) − pen(m) b . θ−4

Now, for x > 0 and Z a random variable we have that Z = (Z − x)+ + x − (x − Z)+ ≤ (Z − x)+ + x. Therefore, sup (νn (f ))2 ≤ sup (νn (f ))2 − p(m, m) b + p(m, m), b Γn f ∈Sm∨c m

Γn f ∈Sm∨c m

1 b Obviously, for all x, y ≥ 0, max (x, y) ≤ x + y. with p(m, m) b = (1 + δ)σ 2 Dm∨m b /n = θ pen(m ∨ m). Thus pen(m) − pen(m) b + θp(m, m) b ≤ pen(m) − pen(m) b + pen(m) + pen(m) b = 2pen(m). Therefore,

θ+4 2 E ∥β − β (m) ∥2Γe E ∥βbm b − β∥Γ en 1G ≤ n θ−4 h X θ2 + E sup θ−4 ′ en Γ m ∈Mn,p

(νn (f ))2 − p(m, m′ )

f ∈Sm∨m′

n θ2 ∥β∥2L2 X ei ∥2 2 E ∥Xi − X L θ − 4 n i=1

2θ pen(m). θ−4

Using the Lemma 1 of Brunel et al. [3], we find that X m′ ∈Mn,p

E [

sup Γn f ∈Sm∨m ′ e

C(l, δ) σ2 . (νn (f ))2 − p(m, m′ )]+ ≤ n

i +

(The proof of this result is based on the Corollary 5.1 of Baraud [1]). To conclude this proof we need to control the term E(∥β∥2Γe 1Gc ). Applying Cauchy–Schwarz inequality for the scalar product n ⟨Z1 , Z2 ⟩ = E(Z1 Z2 ) we have that, i h c 1/2 . E(∥β∥2Γe 1Gc ) ≤ E(∥β∥4Γe )P(G ) n

Using the Lemma B.3 in the appendix leads to, c P(G ) ≤ DNn,p exp −

n . 2 2 4 ln n(K1 + 1)

This quantity is negligible for a sufficiently large n as we chose DNn,p < n/ ln2 n. Now, by definition of the norm ∥ · ∥Γen we have,  !2  n X 1 ei ⟩2 2  E ∥β∥4Γe = E  ⟨β, X L n n i=1   ! n X 2 1 X ei ⟩2 2 ⟨β, X ei ⟩4 2 + E  ei′ ⟩2 2  ⟨β, X ⟨β, X =E L L L n2 i=1 n2 ′ 1≤i<i ≤n 1 e1 ⟩4 2 + n − 1 ∥β∥4 . = E ⟨β, X e L Γ n n √ √ √ Using the fact that x + y ≤ x + y for all x, y ≥ 0 and that n is a positive integer we obtain E

where

∥β∥4Γe n

1/2

1/2 n−1 1 4 4 e E ⟨β, X1 ⟩L2 + ∥β∥Γe ≤ n n 1/2 1 e1 ⟩4 2 ≤ √ E ⟨β, X + ∥β∥2Γe L n eβ . ≤C

1/2 eβ = E ⟨β, X e1 ⟩4 2 C + ∥β∥2Γe + 1. L

Hence since DNn,p < lnn2 n we have for n large enough h

i 1/2 n c 1/2 eβ n ≤C E(∥β∥4Γe )P(G ) exp − n ln n 8 ln n(K12 + 1)2 eβ C . ≤ n

Lemma B.9 allows us to conclude the proof of this proposition. Lemma B.1. Let us define for all l, j ∈ N\{0} p−1

⟨ϕl , ϕj ⟩p :=

1X ϕl (th )ϕj (th ), p h=0

where we recall that th = hp and h = 0, . . . , p − 1. We have  1 if l = j = 1,     1 + 1 if l and j are even ,  {l≡j[2p]} {l≡−j[2p]}   1 − 1 if l and j are odd and strictly larger than 1 , {l≡−j+2[2p]} √{l≡j[2p]} ⟨ϕl , ϕj ⟩p = if j = 1 and l is even,   √21{l≡0[2p]}    21{j≡0[2p]} if l = 1 and j is even,   0 otherwise. Moreover, if l and j are strictly lower than p then ⟨ϕl , ϕj ⟩p = δlj . 25

(45)

Proof. We distinguish four cases. Case 1: j and l are even, meaning that there exist two positive integers a and b such that j = 2a and l = 2b. Using the fact that for all x ∈ R, cos(x) = (eix + e−ix )/2, we have that p−1 X

cos

2πah

h=0

cos

2πbh p

Now, let us define S(r) := Otherwise,

p−1

i2π(a−b)h i2π(a+b)h i2π(a+b)h 1 X i2π(a−b)h p p p p e . + e− +e + e− 4 h=0

Pp−1

h=0 e

i 2πrh p

for all integer r. If r ≡ 0[p] then S(r) =

S(r) =

1 − ei2πr i2πr

1−e p

Pp−1

h=0 1 = p.

= 0.

Moreover S(−r) = S(r) = S(r) as for all integer r, S(r) is a real number. Hence, p−1 X h=0

cos

2πah p

cos

2πbh p

i ph 1{b≡a[p]} + 1{b≡−a[p]} , 2

and ⟨ϕl , ϕj ⟩p = 1{l≡j[2p]} + 1{l≡−j[2p]} . Case 2: j and l are odd and strictly bigger than 1, meaning that there exist two positive integer a and b such that j = 2a + 1 and l = 2b + 1. Using the fact that for all x ∈ R, sin(x) = (eix − e−ix )/(2i) we have that p−1 X h=0

p−1 2πah 2πbh i2π(a−b)h i2π(a−b)h i2π(a+b)h i2π(a+b)h 1X p p p p sin sin =− [−e − e− +e + e− ]. p p 4 h=0

Therefore, p−1 X h=0

2πah 2πbh p sin sin = 1{b≡a[p]} − 1{b≡−a[p]} , p p 2

and ⟨ϕl , ϕj ⟩p = 1{l≡j[2p]} − 1{l≡−j+2[2p]} . Case 3: j is equal to 1. If l is also equal to 1 then ⟨ϕl , ϕj ⟩p = 1. If l is odd and strictly larger than 1, then there exists a positive integer b such that l = 2b + 1 and √ ⟨ϕl , ϕj ⟩p =

p−1 2 X 2πbh sin = 0. p p h=0

If l is even then there exists a positive integer b such that l = 2b and p−1 p−1 2πbh √2 X √ 2X cos = 1= 2 ⟨ϕl , ϕj ⟩p = p p p

√

h=0

provided that b ≡ 0[p], and ⟨ϕl , ϕj ⟩p = 0 otherwise. Consequently, we have √ ⟨ϕl , ϕj ⟩p = 2 whenever l ≡ 0[2p] and j = 1. By symmetry, the case l = 1 is treated analogously. Case 4: j is odd and strictly larger than 1, and l is even. In this situation there exist two positive integers a and b such that j = 2a + 1 and l = 2b. Then, p−1 X h=0

p−1 h 2πbh 1 X 2π(a + b)h 2π(a − b)h i 2πah sin cos = sin + sin . p p 2 p p h=0

For any integer r,

Pp−1

h=0 sin((2πrh)/p) = 0, hence p−1 X h=0

2πah 2πbh sin cos = 0, p p

and ⟨ϕl , ϕj ⟩p = 0. By symmetry of the roles of l and j we have the same result if l is odd and strictly bigger than 1, and j is even. Lemma B.2. Let us assume that DNn,p < p, then we have for all j, k ∈ {1, . . . , DNn,p } ej , E(e xi,j x ei,k ) = δjk λ where

ej = λj + λj + τ , λ p

(46)

and λj = 1{j even}

λj+2qp + λ2qp−j

q≥1

+ 1{j odd, j>1}

λj+2qp + λ2qp−j+2

q≥1

+ 21{j=1}

λ2qp .

q≥1

Proof. For all i ∈ {1, . . . , n} and j ∈ {1, . . . , DNn,p }, let x ei,j = xi,j + η i,j where p−1

1X xi,j = Xi (th )ϕj (th ) p h=0

p−1

1X and η i,j = ηi,h ϕj (th ). p

(47)

h=0

Then, as the ηi,h ’s are supposed to be i.i.d, independent of everything else, centered and of variance τ 2 we have for all j, k ∈ {1, . . . , DNn,p }, E(e xi,j x ei,k ) = E(xi,j xi,k ) + E(η i,j η i,k ). 2

Moreover E(η i,j η i,k ) = τp ⟨ϕj , ϕk ⟩p as the ηi,h ’s are supposed to be i.i.d, centered and of variance τ 2 . As j, k ≤ DNn,p < p the previous lemma leads to E(η i,j η i,k ) =

τ2 δjk . p

We can then write xi,j as xi,j = xi,j + ei,j

where

xi,j = ⟨Xi , ϕj ⟩L2

(48)

Therefore E(xi,j xi,k ) = E(xi,j xi,k ) + E(xi,j ei,k ) + E(xi,k ei,j ) + E(ei,k ei,j ). The first term can be rewritten as E(xi,j xi,k ) = λj δjk where the λj ’s are the eigenvalues of Γ. Indeed, for any j, k ≥ 1 and for any i ∈ {1, . . . , n} we have, using the fact that the Xi ’s are periodic and second order stationary, E(xi,j xi,k ) = E (⟨Xi , ϕj ⟩L2 ⟨Xi , ϕk ⟩L2 ) = ⟨E (⟨ϕj , Xi ⟩L2 Xi ) , ϕk ⟩L2 = ⟨Γϕj , ϕk ⟩L2 = ⟨λj ϕj , ϕk ⟩L2 = λj δjk . 27

(49)

Moreover for all i ∈ {1, . . . , n} and t ∈ [0, 1], Xi (t) = and using Fubini we get,

E xi,j ei,k = E xi,j

p−1 X 1 p

l≥1 xi,l ϕl (t), therefore Xi (th ) =

Xi (th )ϕk (th ) − xi,k

l≥1 xi,l ϕl (th )

h=0

 = E xi,j

xi,l

p−1 X 1 p

ϕl (th )ϕk (th ) − δlk 

h=0

l≥1



E xi,j xi,l

p−1 1 X p

l≥1

h=0

= λj ⟨ϕj , ϕk ⟩p − δjk

ϕl (th )ϕk (th ) − δlk

= 0, as j, k ≤ DNn,p < p. Similarly, we obtain E(xi,k ei,j ) = 0. To compute E(ei,j ei,k ), we first express ei,j for any j ∈ {1, . . . , DNn } as X ei,j = xi,l ⟨ϕl , ϕj ⟩p − δlj . l≥1

Using Lemma B.1, we have If j = 1,  √  2 if l ∈ (2pq)q≥1 , ⟨ϕl , ϕ1 ⟩p = 1 if l = j,  0 otherwise. If j > 1 and j is even, ⟨ϕl , ϕj ⟩p = If j > 1 and j is odd,

1 if l ∈ (j + 2qp)q≥0 or l ∈ (−j + 2qp)q≥1 , 0 otherwise.

  1 −1 ⟨ϕl , ϕj ⟩p =  0

if l ∈ (j + 2qp)q≥0 , if l ∈ (−j + 2 + 2qp)q≥0 , otherwise.

Therefore,  √ P 2 q≥1 xi,2qp if j = 1,   P DN if j = 2s with s ∈ {1, . . . , ⌊ 2n,p ⌋}, ei,j = q≥1 [xi,2s+2qp + xi,−2s+2qp ]  P DNn,p  ⌋}. q≥1 [xi,2s+1+2qp − xi,2qp+2−(2s+1) ] if j = 2s + 1 with s ∈ {1, . . . , ⌊ 2

(50)

Given Equation (49)), we notice that, when expanding E(ei,j ei,k ) into sums of terms of the form E(xi,t xi,s ) every term vanishes unless the indices coincide. In other words, E(ei,j ei,k ) = 0 as soon as the index sets used in ei,j and ei,k are disjoint. Let us first describe the sets of indices involved: • For j = 1: T1 = {2qp : q ≥ 1},

all even multiples of p.

• For j = 2s with 1 ≤ s ≤ ⌊(p − 1)/2⌋: T2s = {2s + 2qp, −2s + 2qp : q ≥ 1},

all even.

• For j = 2s + 1 with 1 ≤ s ≤ ⌊(p − 1)/2⌋: T2s+1 = {2s + 1 + 2qp, 2qp + 1 − 2s : q ≥ 1}, 28

all odd.

Trivial cases . From this, two trivial vanishing cases appear: when j and k don’t have the same parity and when j = 1 and k > 1 (or k = 1 and j > 1). As the role of j and k are symmetric, we can assume that j is odd and k is even. Then Tj contains only odd indices while Tk contains only even ones. Hence Tj ∩ Tk = ∅ and E(ei,j ei,k ) = 0. For the second case, let us consider j = 1 versus k > 1. If k is odd, this falls into the previous parity case. If k = 2s is even, we must check T1 ∩ T2s = ∅. Let us suppose that there exist q, q ′ ≥ 1 such that 2qp = 2s + 2q ′ p or 2qp = −2s + 2q ′ p. This would imply 2s ≡ 0[2p], or s ≡ 0[p]. But since 1 ≤ s ≤ ⌊ impossible. Therefore T1 ∩ T2m = ∅ and

DNn,p ⌋ ≤ ⌊(p − 1)/2⌋, this is 2

E(ei,1 ei,2s ) = 0. To conclude, j and k have different parity, or if one of them is 1 and the other is strictly larger than 1, then E(ei,j ei,k ) = 0. j and k are even . Then there exist a, b ∈ {1, . . . , ⌊ notation introduced above, the index sets are

DNn,p ⌋} with j = 2a and k = 2b. Using the 2

T2b = {2b + 2q ′ p, −2b + 2q ′ p : q ′ ≥ 1}.

T2a = {2a + 2qp, −2a + 2qp : q ≥ 1},

To determine whether E(ei,2a ei,2b ) can be nonzero we must look for coincidences between elements of T2a and T2b . Thus we consider equalities of the form (with q, q ′ ≥ 1): (I)

2a + 2qp = 2b + 2q ′ p, 2a + 2qp = −2b + 2q p,

(II)

′

(III) (IV)

′

− 2a + 2qp = 2b + 2q p, − 2a + 2qp = −2b + 2q ′ p. • Cases (I) and (IV) : From (I) we get 2(a − b) = 2p(q ′ − q)

⇐⇒

a − b = p(q ′ − q).

Since |a − b| ≤ ⌊ 2n,p ⌋ ≤ ⌊(p − 1)/2⌋ < p, the only multiple of p in that range is 0. Hence a − b = 0, or a = b. If a = b then (I) reduces to 2qp = 2q ′ p, or q = q ′ . The same reasoning applied to (IV) yields the identical conclusion: (IV) can occur only when a = b and q = q ′ . • Cases (II) and (III) : From (II) we obtain 2(a + b) = 2p(q ′ − q)

⇐⇒

a + b = p(q ′ − q).

But 2 ≤ a + b ≤ 2⌊ 2n,p ⌋ ≤ 2⌊(p − 1)/2⌋ ≤ p − 1, therefore 1 ≤ a + b ≤ p − 1 and therefore a + b cannot equal a nonzero multiple of p. Thus (II) is impossible. The same argument applied to (III), −2(a + b) = 2p(q ′ − q), shows (III) is impossible as well. Combining the four cases we conclude that if a ̸= b then no equality between elements of T2a and T2b can occur, therefore T2a ∩ T2b = ∅ and E(ei,2a ei,2b ) = 0

(a ̸= b).

If a = b the only coincidences are the trivial ones coming from matching identical indices 2a + 2qp with 2a + 2qp and −2a + 2qp with −2a + 2qp. Using Equation (49) we get X X E(e2i,2a ) = λ2a+2qp + λ−2a+2qp . q≥1

q≥1

Hence,

  j ̸= k, 0, X X E(ei,j ei,k ) = λ−j+2qp , j = k. λj+2qp +   q≥1

q≥1

j and k are odd and strictly bigger than 1. Then there exist a, b ∈ {1, . . . , ⌊ 2n,p ⌋} with j = 2a + 1 and k = 2b + 1. Recalling the notation introduced above, the index sets are T2a+1 = {2a + 1 + 2qp, 2qp + 1 − 2a : q ≥ 1}, T2b+1 = {2b + 1 + 2q ′ p, 2q ′ p + 1 − 2b : q ′ ≥ 1}. To determine whether E(ei,2a+1 ei,2b+1 ) can be nonzero we must look for coincidences between elements of T2a+1 and T2b+1 . There are four types of equalities to consider (with q, q ′ ≥ 1): 2a + 1 + 2qp = 2b + 1 + 2q ′ p, ′

2a + 1 + 2qp = 2q p + 1 − 2b,

(I) (II)

2qp + 1 − 2a = 2b + 1 + 2q p,

(III)

2qp + 1 − 2a = 2q ′ p + 1 − 2b.

(IV)

′

We treat each case separately. • Cases (I) and (IV) : From (I) we get 2(a − b) = 2p(q ′ − q)

⇐⇒

a − b = p(q ′ − q).

Since |a − b| ≤ ⌊ 2n,p ⌋ ≤ ⌊(p − 1)/2⌋ < p, the only multiple of p in that range is 0. Hence a − b = 0, or a = b. If a = b then (I) reduces to 2qp = 2q ′ p, hence q = q ′ . Thus (I) can occur only when a = b and q = q ′ . The same reasoning applied to (IV) yields the identical conclusion : (IV) can occur only when a = b and q = q ′ . • Cases (II) and (III) : From (II) we obtain 2(a + b) = 2p(q ′ − q)

⇐⇒

a + b = p(q ′ − q).

But 2 ≤ a + b ≤ 2⌊ 2n,p ⌋ ≤ 2⌊(p − 1)/2⌋ ≤ p − 1, thus 1 ≤ a + b ≤ p − 1 and therefore a + b cannot equal a nonzero multiple of p. Hence (II) is impossible for any a, b, q, q ′ . The same reasoning applied to (III) yields the identical conclusion. Combining the four cases we conclude that if a ̸= b then no equality between elements of T2a+1 and T2b+1 can occur, therefore T2a+1 ∩ T2b+1 = ∅ and E(ei,2a+1 ei,2b+1 ) = 0

(a ̸= b)

If a = b the only coincidences are the trivial ones coming from matching identical indices 2a + 1 + 2qp with itself and 2qp + 1 − 2a with itself. Using Equation (49) we therefore get the expression X X E(e2i,2a+1 ) = λ2a+1+2qp + λ2qp+1−2a q≥1

Hence,

q≥1

  j ̸= k, 0, X X E(ei,j ei,k ) = λj+2qp + λ2qp+2−j , j = k.   q≥1

q≥1

√ P j and k are equal to 1 . Let us recall that ei,1 = 2 q≥1 xi,2qp and the index set is T1 = {2qp : q ≥ 1}. To check possible coincidences between indices in the product e2i,1 we look for solutions q, q ′ ≥ 1 of 2qp = 2q ′ p ⇐⇒ q = q′ . 30

Hence we get E(e2i,1 ) = 2

X E x2i,2qp = 2 λ2qp .

q≥1

q≥1 2

Finally get that E(e xi,j x ei,k ) = δjk λj + λj + τp

ej and Var(e ej . = δjk λ xi,j ) = λ

Lemma B.3. Let us assume that λDNn,p ≥ n2α and α > 2a in the definition of sn defined in Equation (14). We also suppose that DNn,p < min lnn2 n , p and that (H1) and (H2) hold. Then we have n c P(G ) ≤ DNn,p exp − . 4(K12 + 1)2 ln n Proof. The proofs for this lemma and the succeeding one were developed by Brunel and Roche [4], based on their work in Lemma 6. We only adapt them to our setting. First of all, we have   [ X c b(p) < sn ), Gcm  ≤ P(λ P(G ) = P  m m∈Mn,p

m∈Mn,p

(p)

bm is the smallest eigenvalue of Φm and sn = 2α (1 − √ 1 ) with α > 2a. The last two where λ n ln n quantities are defined in Section 2.2. By Lemma B.4 we have ! sn (p) (p) b , P(λm < sn ) ≤ P µ bm < ej min1≤j≤Dm λ (p)

where µ bm is the smallest eigenvalue of the matrix Ψm defined in Lemma B.4. Since for all j ∈ {1, . . . , DNn,p } we have 2 ej = λj + λj + τ ≥ λj , λ p then using the fact that the eigenvalues decrease we get min

1≤j≤DNn,p

ej ≥ λ

min

1≤j≤DNn,p

λj = λDNn,p .

Using the assumption that λDNn,p ≥ n2α and the definition of sn given in (14) we obtain, sn

≤1− √

1 . ln n

ej min1≤j≤Dm λ √ We can then apply Lemma B.6 with ω = 1 − 1/ ln n and get n (p) b P(λm < sn ) ≤ 2 exp − . 4(K12 + 1)2 ln n As Nn,p ≤ DNn,p /2, X

b(p) < sn ) ≤ DN exp − P(λ m n,p

m∈Mn,p

n 4(K12 + 1)2 ln n

Combining this with the first inequality of this proof gives the desired result. bm be the smallest eigenvalue of the matrix Φm defined in Section Lemma B.4. For m ∈ Mn,p , let λ (p) 2.2, and µ bm be the smallest eigenvalue of the matrix   n X 1 x e x e qi,j qi,k  Ψm :=  . (51) n i=1 e ek λj λ 1≤j,k≤D m

Then, −1 b(p) λ m b(p) ej ≤µ b(p) , ≤ λ min λ m m e 1≤j≤Dm ρ(Γ) e is the spectral radius of the operator Γ. e where ρ(Γ) Proof. Now let us define

q q e eD ). Λm := diag( λ1 , . . . , λ m

(52)

We have that Φm = Λm Ψm Λm . (p) Recall that µ bm is the smallest eigenvalue of the matrix Ψm .

(p)

We notice that µ bm > 0 directly implies that Ψm is a positive definite matrix which by definition is invertible. As a consequence, (p) det (Ψm ) ̸= 0. Moreover, Λm is invertible as it is a diagonal matrix with positive coefficient. (p) Therefore det (Λm ) ̸= 0. As det (Φm ) = det (Λm ) · det (Ψm ) · det (Λm ), (p)

thus we get that det (Φm ) ̸= 0 and that Φm is invertible. Hence, µ bm > 0 implies that both Φm and Ψm are invertible and we have −1 −1 µ b(p) m = ρ(Ψm )

and

b(p) = ρ(Φ−1 )−1 , λ m m

where ρ is the spectral radius. We recall that for all square matrix A ∥A∥op = sup ∥Aa∥2 . ∥a∥2 =1

If A is symmetric, ρ(A) = ∥A∥op and we get, by using the definition of the spectral radius as well as the sub-multiplicative property of matrix norms, −1 ρ(Φ−1 m ) = ∥Φm ∥op −1 −1 = ∥Λ−1 m Ψm Λm ∥op 2 −1 ≤ ∥Λ−1 m ∥op ∥Ψm ∥op 2 −1 = ρ(Λ−1 m ) ρ(Ψm ).

Hence,

µ b(p) m ·

min

1≤j≤Dm

ej ≤ λ b(p) . λ m

−1 As Ψm = Λ−1 m Φm Λm , we have, using the same reasoning that

b(p)−1 max λ ej ≤ ρ(Γ) b(p)−1 . e λ µ b(p)−1 ≤λ m m m 1≤j≤Dm

The second inequality can be deduced from Lemma B.5. Lemma B.5. Let us assume that DNn,p < p. For all k ∈ {1, . . . , DNn,p } e k ϕk , e k=λ Γϕ ek ’s defined in (46) are the eigenvalues of Γ e associated to ϕk , and for all integer k > DN , where λ n,p e k = 0. Γϕ

Proof. For all s, t ∈ [0, 1] we have using the definition (21) and Lemma B.2, e t) = E X(s) e X(t) e K(s, DNn,p DNn,p

j=1

k=1

ϕj (t)ϕk (s)E(e xj x ek )

DNn,p

ej ϕj (s)ϕj (t). λ

j=1

Hence, for all integer k ∈ {1, ..., DNn,p }, Z 1 e k (s) = Γϕ

e t)ϕk (t)dt K(s, 0 DNn,p

Z 1 ϕj (t)ϕk (t)dt

ej ϕj (s) λ 0

j=1

ek ϕk (s), =λ and for all integer k > DNn,p , Z 1 e k (s) = Γϕ

e t)ϕk (t)dt K(s, 0 DNn,p

Z 1 ej ϕj (s) λ

ϕj (t)ϕk (t)dt 0

j=1

= 0.

Lemma B.6. Let ω be a real number such that 0 < ω < 1. For m ∈ Mn,p , consider the smallest (p) eigenvalue µ bm of the matrix Ψm . Then, under Assumptions (H1), (H2), we have (1 − ω)2 P(b µ(p) < ω) ≤ 2 exp −n . m 4(K12 + 1)2 Proof. We have

{b µ(p) b(p) m < ω} = {1 − µ m > 1 − ω}.

Since 1 − ω > 0,

{|1 − µ b(p) m | > 1 − ω} = {∥Ψm − I∥op > 1 − ω}.

Thus our goal is now to control P (∥Ψm − I∥op > 1 − ω). Let T x e x e i,D i,1 (m) Ui =  q , . . . , q m  ∈ RDm , eD e λ λ1 m 

and We can easily check that

i ∈ {1, ..., n},

T (m)T , . . . , Un(m)T ∈ Rn×Dm . Am = U1   n X x e x e i,j i,k q q  ATm Am =  ej λ ek i=1 λ

1≤j,k≤Dm

and thus Ψm =

1 T A Am . n m 33

(53)

(54)

Our aim is to apply Theorem 4.6.1 of Vershynin [16]. In order to do this, we must check that the (m) vectors Ui ’s are independent, mean-zero, isotropic and sub-gaussian. As we assumed that for all i ∈ {1, . . . , n} the true curves Xi are independent, that for all i ∈ {1, . . . , n}, h ∈ {0, . . . , p − 1} the noises ηi,h are also independent and that for all i ∈ {1, . . . , n}, h ∈ {0, . . . , p − 1} Xi and ηi,h are (m) independent we have that the Ui ’s are also independent. Moreover, as we made the assumption that the Xi ’s and the ηi,h ’s are centered we have that E(e xi,j ) = E

1 p

p−1

h=0

1X 1X Xi (th )ϕj (th ) + ηi,h ϕj (th ) p p p−1 X

p−1

E (Xi (th )) ϕj (th ) +

h=0

1X E(ηi,h )ϕj (th ) p h=0

= 0. Finally we can easily check that 

 x e x e i,j i,k (m) (m)T Ui Ui = q · q  ej ek λ λ

1≤j,k≤Dm

ej δjk . Therefore, and Lemma B.2 gives us that E(e xi,j x ei,k ) = λ (m) (m)T E Ui Ui = IDm . (m)

Lemma B.7 bellow gives us that the Ui ’s are sub-gaussian. Thus all the conditions needed to apply Theorem 4.6.1 of Vershynin [16] are satisfied and we get that e δe2 ) ≤ 2 exp −t2 , e 2 max(δ, P ∥Ψm − I∥op > K q e 2 = K 2 + 1. Let us now find which t allows us to have K e δe ≤ 1 − ω. where δe = C ′ Dnm + √tn and K 1 Replacing δe by its definition we get that r

1−ω Dm t +√ ≤ e2 n n K ! r √ 1−ω Dm ′ ⇐⇒ t ≤ n −C . e2 n K

e 2 δe ≤ 1 − ω ⇐⇒ C ′ K

n 1−ω We get that for n large enough ln1n < 2C ′K e 2 , therefore in this setting Dm < ln2 n < q which means that C ′ Dnm < 1−ω e 2 . Thus we can pick 2K

t= Then δe = C ′ Therefore,

Dm √t n + n ≤

1−ω e2 K

1 2

1−ω e2 K

√

1−ω e2 2C K

e δe2 ) = δ. e e 2 we have that 1−ω < 1 and max(δ, . As 1 − ω < 1 ≤ K e2 K

(1 − ω)2 n P (∥Ψm − I∥op > 1 − ω) ≤ 2 exp − e4 4K

(m)

Lemma B.7. Under hypotheses (H1) and (H2) we have for all i ∈ {1, ..., n} that Ui in (53) is sub-Gaussian with variance factor K12 + 1. 34

defined

(m)

Proof. Let v ∈ RDm such that ∥v∥2 = 1. By definition of the Ui given in (53) we have that for all t ∈ R    Dm X (m) (m) vj Ui,j  E et⟨v,Ui ⟩2 = E exp t j=1



 x e i,j = E exp t vj q   . ej j=1 λ Dm X

As defined in equation (47) we have that for all j ∈ {1, . . . , DNn,p } x ei,j = xi,j + η i,j . Let us focus P vj Dm on E exp t j=1 η i,j e . As we assumed that the ηi,h ’s are independent, we have that for all λj t∈R       p−1 Dm Dm X X X v t v ϕ (t ) j j j h  E exp t η i,j  = E exp  ηi,h ej ej p λ λ j=1 j=1 h=0    p−1 Dm Y X t v ϕ (t ) j j h  q = E exp  ηi,h , p ej j=1 h=0 λ and as we assumed (H2) we have that for all t′ ∈ R, i ∈ {1, . . . , n} and h ∈ {0, . . . , p − 1} 2 ′2 τ t ′ E (exp (t ηi,h )) ≤ exp . 2 Hence,



2  Dm X vj vj ϕj (th )   τ t q E exp t η i,j  ≤ exp  2   ej 2p λ ej j=1 j=1 h=0 λ   2  p−1 Dm 2 2 X X vj ϕj (th )   τ t  q = exp  2 . 2p ej j=1 h=0 λ 





Dm X

p−1 Y

2 2



Using Lemma B.1 we get that  2 p−1 p−1 Dm Dm 1 X X vj ϕj (th )  1 X X v v q q j q k ϕj (th )ϕk (th ) = 2 p2 p ej ej λ ek j=1 h=0 h=0 j,k=1 λ λ D

m 1 X v v q j q k ⟨ϕj , ϕk ⟩p = p ej λ ek j,k=1 λ

= Hence,



E exp t

Dm X j=1

m vj2 1X . ej p j=1 λ

 η i,j



vj   ≤ exp ej λ



Dm 2 2 2 X vj τ t 

j=1 λj

ej for all j ∈ {1, ..., DN } we have that τ ≤ 1, therefore As τp ≤ λ n,p e pλj



E exp t

Dm X j=1

 2 2 t vj  t 2 ∥v∥2 = exp . η i,j ≤ exp e 2 2 λj

PDm v Now let us focus on E exp t j=1 xi,j √ej . Using the definition of xi,j given in (47) and the λj P fact that for all i ∈ {1, ..., n} and for all s ∈ [0, 1] Xi (s) = j≥1 xi,j ϕj (s) we have,     p−1 Dm X vj vj  1 X X q xi,l ϕl (th ) ϕj (th ) xi,j q = p e e j=1 j=1 l≥1 h=0 λj λj

Dm X

Dm X v q j ⟨ϕj , ϕl ⟩p . = xi,l ej j=1 l≥1 λ

(55)

Let us consider cl :=

PDm

j=1



√vj ⟨ϕj , ϕl ⟩p . As we assumed that (H1) holds, we get that ej λ



    X v j E exp t xi,j q  = E exp t xi,l cl  ej j=1 l≥1 λ    2 2 X t K 1  λl c2l . ≤ exp  2 Dm X

l≥1

Using Fubini’s theorem we get that  2  X   X xi,l cl   = E(xi,l xi,r )cl cr E  l≥1

l,r≥1

λl c2l ,

l≥1

where we used the fact that E(xi,l xi,r ) = λl δlr for all l, r ≥ 1 (see Equation (49)) to get from the last equality. On an other hand, the same computations as in (55) give that Dm X X v q j xi,j = xi,l dl . ej j=1 l≥1 λ

Therefore, X

 2  X   λl c2l = E  xi,l cl   l≥1

l≥1

 2  D m  X vj  q xi,j   = E  e j=1 λj =

Dm X

v v q j q k E(xi,j xi,k ). ej λ ek j,k=1 λ

As for all i ∈ {1, . . . , n} and j ∈ {1, . . . , DNn,p }, xi,j = x ei,j − η i,j , that the ηi,h ’s are independent of everything else and centered, we get 2 ej − τ E(xi,j xi,k ) = E(e xi,j x ei,k ) − E(η i,j η i,k ) = δjk λ . p

The last inequality can be deduced from the reasoning in the proof of Lemma B.2. We then have X

λl c2l =

l≥1

≤

Dm 2 X vj τ2 e λj − e p j=1 λj Dm X

vj2

j=1

= ∥v∥22 = 1. Therefore,



 2 2 v t K1 j     q E exp t xi,j ≤ exp . 2 ej j=1 λ Dm X

Using the fact that the ηi,h ’s are independent of the Xi ’s, we get that for all i ∈ {1, . . . , n}       Dm Dm X X v v j j (m) E exp t⟨v, Ui ⟩2 = E exp t xi,j q  E exp t η i,j q  e e j=1 j=1 λj λj 2 2 t (K1 + 1) . ≤ exp 2 (m)

Therefore, for all i ∈ {1, . . . , n} Ui

is sub-gaussian with variance factor K12 + 1.

Lemma B.8. Let us assume that (H3) holds. Recall that for all j ∈ {1, . . . , DNn,p },  X  2 λ2qp ,     q≥1  X  λj+2qp + λ2qp−j+2 , λj =  q≥1  X    λj+2qp + λ2qp−j ,  

if j = 1, if j > 1 is odd, if j is even.

q≥1

Then there exist a positive constant C(a, c′ ) such that for every p ≥ 2 and every j ∈ {1, . . . , p − 1}, λj ≤ C(a, c′ )p−2a . Proof. We first note the following inequalities valid for every q ≥ 1 and 1 ≤ j < p : 2qp ≤ 2qp + j ≤ (2q + 1)p, (2q − 1)p ≤ 2qp − j ≤ 2qp, (2q − 1)p + 2 ≤ 2qp − j + 2 ≤ (2q + 1)p. Since (λk ) is non-increasing, if u ≤ v then λu ≥ λv . We repeatedly use this together with (H3) Case 1. j even. We have X λj = λ2qp+j + λ2qp−j . q≥1

Since 2qp + j ≥ 2qp and 2qp − j ≥ (2q − 1)p, λ2qp+j ≤ c′ (2qp)−2a , Hence

λj ≤ c′ p−2a

λ2qp−j ≤ c′ ((2q − 1)p)−2a .

(2q)−2a + (2q − 1)−2a = c′ ζ(2a)p−2a ,

q≥1

where ζ denotes the Riemann function. Thus λj ≤ C(a, c′ )p−2a . Case 2. j odd and j > 1. We have λj =

λ2qp+j + λ2qp−j+2 .

q≥1

Since 2qp + j ≥ 2qp and 2qp − j + 2 ≥ (2q − 1)p, λj ≤ c′ ζ(2a) p−2a . we conclude that λj ≤ C(a, c′ )p−2a . Case 3. j = 1. We have λ1 = 2

λ2qp .

q≥1

Using assumption (H3) we get, λ1 ≤ 2c′

X (2qp)−2a . q≥1

But

(2qp)−2a = p−2a 2−2a ζ(2a),

q≥1

therefore

λ1 ≤ C(a, c′ )p−2a .

Combining the three cases yields the result. Lemma B.9. Suppose that DNn,p < p, E⟨β, X1 ⟩4L2 < ∞ and that (H3) holds. Then, 1/2 2∥β∥2 2 E∥X e 1 − X 1 ∥4 2 eβ L L 1 C 1 ≤ C(a, c′ , τ )Cβ + + , n n np n

(56)

eβ is defined in (45) and Cβ in (31). where C e1 ⟩L2 as Proof. First we can rewrite ⟨β, X e1 ⟩L2 = ⟨β, X1 ⟩L2 + ⟨β, X e1 − X1 ⟩L2 . ⟨β, X By Minkowski’s inequality in L4 we have, 1/2 1/4 2 e1 ⟩4 2 e1 − X1 ⟩L2 |4 1/4 . E⟨β, X ≤ E⟨β, X1 ⟩4L2 + E|⟨β, X L Cauchy–Schwarz inequality gives, e1 − X1 ⟩L2 | ≤ ∥β∥L2 ∥X e1 − X1 ∥L2 , |⟨β, X Therefore, e1 − X1 ⟩L2 |4 E|⟨β, X Hence,

e 1 ⟩4 2 E⟨β, X L

1/2

≤

1/4

e1 − X1 ∥4 2 1/4 . ≤ ∥β∥L2 E∥X L

E⟨β, X1 ⟩4L2

1/4

e1 − X1 ∥4 2 + ∥β∥L2 E∥X L

1/4 2

Using now (u + v)2 ≤ 2u2 + 2v 2 , u, v ∈ R, we obtain the simpler bound

e 1 ⟩4 2 E⟨β, X L

1/2

1/2 1/2 e1 − X1 ∥4 2 ≤ 2 E⟨β, X1 ⟩4L2 + 2∥β∥2L2 E∥X . L 38

Using the definition of ∥.∥Γe given in (27), Lemma B.5 and Lemma B.8 we get DNn,p

e β⟩L2 = ⟨ ∥β∥2Γe = ⟨Γβ,

e j, βj Γϕ

j≥1

βk ϕ k ⟩

k≥1

e j , ϕk ⟩L2 = βj βk ⟨Γϕ

ej β 2 λ j

(57)

j=1

j,k≥1

2 ej ∥β∥2 2 ≤ ∥β∥2 2 τ + C1 + c′ ≤ C(a, c′ , τ )∥β∥2 2 1 + 1 . ≤ max λ L L L 1≤j≤DNn,p p p2a p Hence, 1/2 1/2 e 1 − X 1 ∥4 2 2 E⟨β, X1 ⟩4L2 2∥β∥2L2 E∥X eβ L C(a, c′ , τ )∥β∥2L2 1 C 1 ≤ + + +1 + . n n n n p n Therefore we have that 1/2 2∥β∥2 2 E∥X e 1 − X 1 ∥4 2 eβ L L C 1 1 ≤ C(a, c′ , τ )Cβ + + . n n np n

B.2

Proof of Theorem 3.1

Proof. We have that E(∥βe − β∥2Γe ) = E(∥βe − β∥2Γe 1Ω(n) ) + E(∥βe − β∥2Γe 1Ω(n)c ), where Ω(n) is such that

Ω(n) :=

(58) (59)

Ωm ,

m∈Mn

with Ωm :=

  

∥f ∥2Γe

  

n sup e , 2 −1 ≤ω    f ∈Sm ∥f ∥Γe

(60)

f ̸=0

where 0 < ω e < 1. The second term of the sum in Equation (58) is controlled by Lemma B.10. Let us focus now on the first term of the sum. Let β (m) be the projection of β onto Sm with respect to the semi-norm ⟨·, ·⟩Γ , for all m ∈ Mn,p . Then, ∥βe − β∥2Γe 1Ω(n) = ∥βe − β (m) + β (m) − β∥2Γe 1Ω(n) ≤ 2∥βe − β (m) ∥2Γe 1Ω(n) + 2∥β (m) − β∥2Γe 1Ω(n) 2 ≤ ∥βe − β (m) ∥2Γe + 2∥β (m) − β∥2Γe n 1−ω e 4 4 ≤ ∥βe − β∥2Γe + ∥β − β (m) ∥2Γe + 2∥β (m) − β∥2Γe , n n 1−ω e 1−ω e where we used the definition of Ω(n) to go from the second to third line. Taking the expectation, using the results of Proposition 3.1 we get that E(∥βe − β∥2Γe ) ≤ C1 min E(∥β (m) − β∥2Γe ) + pen(m) m∈Mn,p

e1 − X1 ∥2 2 ) + ∥β∥2L2 E(∥X L 1/2 ∥β∥2L2 e 1 1 4 + E∥X1 − X1 ∥L2 + Cβ + , n n np where Cβ = E(⟨β, X1 ⟩4 )1/2 + ∥β∥2L2 + 1 and C1 > 0 depends on θ, K, l, τl , δ, σ 2 .

Lemma B.10. Under the assumptions that DNn,p < min lnn2 n , p and that (H1), (H2) and (H3) hold, there exist a constant C0′ depending on ρ, K, σ, a and c′ such that 1 1 E ∥βe − β∥2Γe 1Ωcn ≤ C0′ + E(⟨β, X1 ⟩4 )1/2 + ∥β∥2L2 + 1 . n np Proof. The proof of this Lemma was developped by Brunel and Roche [4] in their Lemma 5. Here we just adapt it to our setting. Using the triangular inequality and the definition of βe we have, e 2 + ∥β∥2 )1Ω(n)c E ∥βe − β∥2Γe 1Ω(n)c ≤ 2E (∥β∥ e e Γ Γ 2 (n)c 2 ). = 2E ∥βbm ∥ 1 b Γ e Ω(n)c ∩G + 2∥β∥Γ e P(Ω Lemma B.4 and the definition of G allow us to derive the following inclusions : ( ) s n (p) b G ⊂ {λ µ bNn,p > , Nn,p > sn } ⊂ e ρ(Γ)

(61)

(62)

where ρ denotes the spectral radius of the operator. By using the same reasoning as detailed in the proof of Lemma B.11 bellow, we have inf

∥f ∥2Γe

f ∈SNn,p

∥f ∥2Γe

bT ΨNn,p b.

inf b̸=0,∥b∥=1

(63)

If we assume that µ bNn,p > 0, ΨNn,p is a symmetric matrix which is positive and there exists an orthogonal matrix U such that U T ΨNn,p U is a diagonal matrix whose diagonal entries are the eigenvalues of ΨNn,p . Therefore, µ bNn,p =

inf b̸=0,∥b∥=1

bT U T ΨNn,p U b =

inf c̸=0,∥c∥=1

cT ΨNn,p c.

(64)

Combining the results from Equations (62), (63) and (64), we have for all f ∈ SNn,p and f ̸= 0, ∥f ∥2Γe <

e ρ(Γ)∥f ∥2Γe

Taking f = βbm b we get ρ(Γ) e 2 2 bm E ∥βbm 1 ≤ (65) E ∥ β ∥ 1 (n)c (n)c b ∥Γ b e Ω en Ω ∩G ∩G . Γ sn T b b e e 2 2 As βbm is a mean-square-type estimator, the vector ⟨ β , X ⟩ , . . . , ⟨ β , X ⟩ can be seen 1 L n L b m b m b n as the orthogonal projection with respect to the euclidean scalar oproduct on R of the vector n T e1 ⟩L2 , . . . , ⟨f, X en ⟩L2 )T , f ∈ Sm (Y1 , . . . , Yn ) on the subspace (⟨f, X b . Therefore, n X

e 2 ⟨βbm b , Xi ⟩L2 ≤

i=1

n X

Yi2 ,

i=1

and 2 n∥βbm b ∥Γ e ≤

n X

Yi2 .

i=1

Replacing Yi by its definition, Yi = ⟨β, Xi ⟩L2 + ϵi we get n

2 2 ∥βbm b ∥Γ e ≤ 2∥β∥Γn + n

2X 2 ϵ . n i=1 i

(66)

Using Equations (65) and (66) we get 2ρ(Γ) e 2 E ∥βbm 1 E (n)c b ∥Γ e Ω ∩G ≤ sn

1 ∥β∥2Γn +

n X

n i=1

! ϵ2i

1Ω(n)c

(67)

ei ’s and that the set Ωc depends only on the X ei ’s we have As the ϵi ’s are independent of the X n ! n 1X 2 E 1Ω(n)c ϵ = σ 2 P(Ω(n)c ). (68) n i=1 i Cauchy–Schwarz inequality gives, 1/2 E ∥β∥2Γn 1Ω(n)c ≤ E ∥β∥4Γn P(Ω(n)c )1/2 . By definition of the norm ∥ · ∥Γn we have, " #2  n X 1 ⟨β, Xi ⟩2L2  E ∥β∥4Γn = E  n i=1 =E

n 1 X ⟨β, Xi ⟩4L2 n2 i=1



+ E

 2 n2

⟨β, Xi ⟩2L2 ⟨β, Xi′ ⟩2L2 

1≤i<i′ ≤n

n−1 1 = E ⟨β, X1 ⟩4L2 + ∥β∥4Γ . n n Hence,

1/2 n−1 1 4 4 E ≤ E ⟨β, X1 ⟩L2 + ∥β∥Γ P(Ω(n)c )1/2 . n n √ √ √ Using the fact that for all x, y ≥ 0 x + y ≤ x + y we have 1 2 4 1/2 2 + ∥β∥Γ P(Ω(n)c )1/2 . E ∥β∥Γn 1Ω(n)c ≤ √ E ⟨β, X1 ⟩L2 n ∥β∥2Γn 1Ω(n)c

(69)

Combining Equations (67), (68) and (69), 2P(Ω(n)c )1/2 1 2 4 1/2 2 2 b e E ∥β m ρ(Γ) √ E ⟨β, X1 ⟩L2 + ∥β∥Γ + σ . b ∥Γ e 1Ω(n)c ∩G ≤ sn n

As P(Ω(n)c ) ≤ P(Ω(n)c )1/2 and sn ≤ 2 we get, combining the previous result and Equation (61) : 4P(Ω(n)c )1/2 e √1 E ⟨β, X1 ⟩4 2 1/2 + ∥β∥2Γ + σ 2 + ∥β∥2 . (70) ρ(Γ) E ∥βe − β∥2Γe 1Ω(n)c ≤ e L Γ sn n Using Lemma B.11 and the assumption that DNn,p < n/ ln2 n we have P(Ω(n)c )1/2 1/2 ω e2n √ ≤ (ln n)−1 nα+1/2 exp − sn 8(K12 + 1)2 1 − 1/ ln n 1 ln n − C ′ n ≤ C exp α+ 2 C ′′ ≤ , n with C, C ′ , C ′′ depending on K1 . Now, using Lemma B.5 and Lemma B.8 we have that 2 ej = τ + C1 + c′ ≤ C2 1 + 1 , e = ρ(Γ) max λ 1≤j≤DNn,p p p2a p 41

(71)

where C1 , C2 > 0 depends on c′ , a and τ (for C2 ). We recall that here ρ denotes the spectral radius. Let’s now define the following quantity : 1/2 1 An := √ E ⟨β, X1 ⟩4L2 + ∥β∥2Γ + σ 2 . n Then using Equation (70) and (71) we get, 1 1 1 2 2 e E ∥β − β∥Γe 1Ω(n)c ≤ C3 An + An + ∥β∥Γe , n np n

ej ’s and the fact that the λj ’s where C3 depends on K1 , a and c′ . Using the definition of the λ decrease in a polynomial way (see (H3)), we get using the same computations as used in (57) DNn,p

∥β∥2Γe =

ej β 2 λ j

j=1

≤

max

1≤j≤DNn,p

ej ∥β∥2 2 λ L

2 τ C1 ′ + 2a + c p p 1 2 ≤ C4 ∥β∥L2 +1 . p

≤ ∥β∥2L2

Hence,

1 1 1 1 E ∥βe − β∥2Γe 1Ω(n)c ≤ C5 An + An + ∥β∥2L2 + ∥β∥2L2 , n np n np

where C5 > 0 depends on ω e , K1 , a and c′ . Therefore we obtain 1/2 1 1 2 e E ∥β − β∥Γe 1Ω(n)c ≤ C6 + E ⟨β, X1 ⟩4L2 + ∥β∥2Γ + ∥β∥2L2 + 1 , n np where C6 > 0 is a constant which depends on K1 , a and c′ . Finally, as X X X X ∥β∥2Γ = ⟨Γβ, β⟩L2 = ⟨ βj Γϕj , βk ϕk ⟩L2 = βj βk ⟨Γϕj , ϕk ⟩L2 = λj βj2 j≥1

k≥1

j≥1

j,k≥1

≤ max λj ∥β∥2L2 ≤ c′ ∥β∥2L2 , j≥1

we get that 1 1 2 ′ e + E(⟨β, X1 ⟩4 )1/2 + ∥β∥2L2 + 1 , E ∥β − β∥Γe 1Ω(n)c ≤ C0 n np

where C0′ > 0 depends on K1 , σ, a and c′ . Lemma B.11. Let us assume that (H1), (H2) hold and that DNn,p < p. Then,

P Ω

(n)c

≤ DNn,p exp −

where Ω(n)c is defined in (58).

ne ω2 4(K12 + 1)2

(72)

PDm fj ϕj . Therefore, by using the definition (26), we Proof. Let f ∈ Sm . We can write f as f = j=1 have n 1X 2 ei ⟩2 2 ∥f ∥Γe = ⟨f, X L n n i=1 D

DNn,p

Dm n X X

m X 1XX ⟨ fj ϕ j , x ei,k ϕk ⟩2L2 = n i=1 j=1

k=1

1 n i=1

Dm X

fj x ei,j

j=1

fj fk

n 1 X

n i=1

j,k=1

x ei,j x ei,k

= hT Φ(p) m h, (p)

where h = (f1 , . . . , fDm )T , and Φm was defined in (11). We used the fact that the ϕj ’s are orthonormal to get the third line. Moreover, using the definition (27) we get e f ⟩2 2 ∥f ∥2Γe = E ⟨X, L Nn,p Dm DX X fj ϕj ⟩2L2 x ek ϕk , =E ⟨

j=1

k=1

Dm X

fj fk E(e xk x ej )

j,k=1

Dm X

ej fj2 λ

j=1

= hT ΛTm Λm h, where Λm was defined by (52). We get the last line by using Lemma B.2. Using the matrix Ψm defined in (51), we get the following equation, ∥f ∥2Γe = hT Φm h = hT ΛTm Ψm Λm h = bT Ψm b, n

with b = Λm h, and

∥f ∥2Γe = hT ΛTm Λm h = bT b.

Therefore,

∥f ∥2Γe

∥f ∥2Γe and sup f ∈Sm f ̸=0

∥f ∥2Γe

bT Ψm b , bT b

(73)

bT (Ψm − I)b = ∥Ψm − I∥op , bT b b̸=0

− 1 = sup

as Ψm is symmetric. Therefore we can rewrite Ωm as Ωm = {∥Ψm − I∥op ≤ ω e} . As discussed in the proof of Lemma B.6 we can apply after some computations Theorem 4.6.1 of Vershynin [16] to get nw e2 c P (Ωm ) = P(∥Ψm − I∥op > ω e ) ≤ 2 exp − , 4(K12 + 1)2

and using the fact that Nn,p ≤ DNn,p /2 we get that

P Ω

(n)c

P(Ωcm ) ≤ DNn,p exp

m∈Mn

B.3

ne ω2 − 4(K12 + 1)2

Proof of Theorem 3.2

ej ’s which are defined in e in SN Proof. Lemma B.5 gives us that the eigenvalues of Γ are the λ n,p e (46). Now for all j ∈ {1, . . . , DNn,p } we have that λj ≤ λj , therefore for all f ∈ SNn,p , ∥f ∥Γ ≤ ∥f ∥Γe . For all m ∈ Mn,p and considering β (m) the projection of β on Sm we get that, ∥βe − β∥2Γ = ∥βe − β (m) + β (m) − β∥2Γ ≤ 2∥βe − β (m) ∥2 + 2∥β (m) − β∥2 Γ

≤ 2∥βe − β (m) ∥2Γe + 2∥β (m) − β∥2Γ ≤ 4∥βe − β∥2Γe + 4∥β − β (m) ∥2Γe + 2∥β (m) − β∥2Γ . Taking the expectation in the equation just above and using Theorem 3.1, we get that E(∥βe − β∥2Γ ) ≤ C2 min ∥β (m) − β∥2Γ + ∥β (m) − β∥2Γe + pen(m) m∈Mn,p

e − X∥2 2 ) + ∥β∥2L2 E(∥X L 1/2 ∥β∥2L2 e 1 1 4 + + Cβ + , E∥X − X∥L2 n n np To derive Equation (34) we combine this result with Lemma A.3 of Tsybakov [15]. Lemma B.12. Let us suppose that (H3) hold, DNn,p < p and that β ∈ W per (k, L). Then for all m ∈ Mn,p , 1 ∥β (m) − β∥2Γe ≤ C2′′ ∥β (m) − β∥2Γ + , p where C2′′ is a positive constant which depends on L, k, c′ , a, τ . Proof. We have, denoting as βj the coefficients of β in the Fourier basis, e − β (m) ), β − β (m) ⟩L2 ∥β − β (m) ∥2Γe = ⟨Γ(β DNn,p

ej β 2 , λ j

j=Dm +1

e associated with the elements of the Fourier since Lemma B.5 gives us that the eigenvalues of Γ e basis are the λj for j ∈ {1, . . . , DNn,p }, and are 0 for j > DNn,p . Using now the definition of the ej ’s defined in (46) and our assumption that β ∈ W per (k, L) for a certain positive integer k and a λ positive real number L we get that DNn,p

E(∥β − β (m) ∥2Γe ) =

X j=Dm +1

≤

X j>Dm

DNn,p

λj βj2 + λj βj2 +

λj βj2 +

j=Dm +1

sup Dm <l≤DNn,p

λl

X j>Dm

−2k −2a ≤ C3′ ∥β − β (m) ∥2Γ + Dm p +

βj2 +

τ2 2 β p j X τ2 β2 p j

j>Dm

τ 2 −2k D , p m

(74)

where C3′ > 0 depends on a, c′ , L, k. Let us give more P details on how we derived the last inequality. Let us focus first on the term supDm <l≤DNn,p λl j>Dm βj2 . Lemma B.8 allows us to deduce that λj = O(p−2a ) for all j = 1, ..., DNn . Hence by Lemma A.3 of Tsybakov [15] we have X −2k sup λl βj2 ≤ C ′ p−2a Dm , Dm <l≤DNn,p

j>Dm

where C is a positive constant which depends on c′ , a, L, k. Using Lemma A.3 of Tsybakov [15] 2 P 2 ′′ τ 2 −2k where C ′′ is a positive constant which depends on again we get that τp j>Dm βj ≤ C p Dm L, k. Hence, for all m ∈ Mn,p , ′

τ 2 −2k D . p l l∈Mn,p 2 2 Since for all l ∈ Mn,p Dl ≥ 1 we have supl∈Mn,p Dl−2k p−2a + τp Dl−2k = p−2a + τp . Now using E(∥β − β (m) ∥2Γe ) ≤ C3′ ∥β − β (m) ∥2Γ + sup Dl−2k p−2a +

the fact that p−2a ≤ p−1 as a > 1/2, we find that E(∥β − β (m) ∥2Γe ) ≤ C2′′ ∥β − β (m) ∥2Γ + p−1 .

Proof of Theorem 4.1

Using the result of Theorem 3.2, we get that E(∥βe − β∥2Γ ) ≤ C3 min ∥β (m) − β∥2Γ + pen(m) m∈Mn,p

e − X∥2 2 ) + E(∥X L 1/2 1 1 e 1 1 E∥X − X∥4L2 + + + . + n p n np With the assumption that the λj ’s decay in a polynomial way (see (H3)) and that β is in W per (k, L), we can use the result of the Theorem 2 of Brunel and Roche [4] which gives that X −2a−2k ∥β − β (m) ∥2Γ = βj2 λj ≤ CDm . j>Dm

Combining this reasoning with Lemmas C.1 and C.2 leads to the following equation, f (Dm ) + g(DNn,p ) , E(∥βe − β∥2Γ ) ≤ C min √ 1≤DNn,p <p∧ lnn 3 n, 1≤Dm ≤DNn,p

(75)

where for all x ≥ 1, f (x) = x

−2k−2a

σ2 +x n

and

1 1 1 x 1 −(2a−1) g(x) = + + + x + +1 . n p np p n

Let us focus on minimizing the function g first. As g is strictly convex, coercive and C 1 , g admits a unique minimum which is given by solving g ′ = 0. For all x ≥ 1, we have 1 1 ′ −2a g (x) = (1 − 2a)x + · +1 . p n 1/(2a) Therefore, g ′ (x) = 0 ⇐⇒ x = (2a − 1)p . Taking into account the fact that DNn,p must satisfy the following inequality DNn,p < min(p, lnn2 n ), we get that ( 1 2a n p 2a if p ≲ ln(n) 2 ∗ DNn,p ≈ n otherwise . ln(n)2 45

2a 1 n since if min(p, lnn2 n ) = p then p 2a ≤ p as a > 1/2, ln(n)2 ∗ and DN ≈ p1/(2a) in this situation. Otherwise, if min(p, lnn2 n ) = lnn2 n then n,p Remark : We derive the condition p ≲

p1/(2a) ≲ If p ≳

n ⇐⇒ p ≲ ln2 n

n ln2 n

2a .

2a n n then the best value that DNn,p can reach is ln(n) 2. ln2 n

Now let’s focus on f . We can easily verify that f is C 1 , strictly convex and coercive therefore the function admits a unique minimum. For all x ≥ 1 we have σ2 . n

f ′ (x) = −(2k + 2a)x−(2k+2a+1) + Therefore, f ′ (x) = 0 ⇐⇒ x =

n(2k+2a) 1/(2k+2a+1) . Thus, σ2 ∗ Dm ≈ n1/(2a+2k+1) .

∗ ∗ Now let’s check when Dm ≤ DN . We distinguish two cases : n,p

2a n ) then, ln(n)2

∗ • If DN ≈ p1/(2a) (or the case where p ≲ n,p

∗ ∗ Dm ≤ DN ⇐⇒ n 2a+2k+1 ≤ p 2a ⇐⇒ n 2a+2k+1 ≤ p. n,p n ∗ • If DN ≈ ln(n) 2 (or the case where p ≳ n,p

2a n ) then, ln(n)2 1

∗ ∗ Dm ≤ DN ⇐⇒ n 2a+2k+1 ≤ n,p

n . ln2 (n)

As a > 0 and k ≥ 1, the last inequality and the condition p ≳ for n large enough.

2a n are always compatible ln(n)2

From these two cases, we can deduce the following ones : 2a 2a n • If n 2a+2k+1 ≲ p ≲ ln(n) , then 2 −(2a−1) −(2a−1) −2k−2a 1 1 1 2 −1 e 2k+2a+1 2a 2a +p E(∥β − β∥Γ ) ≤ C1 n +p n + + + , n p np where C1 > 0. Since k ≥ 1, a > 0, (−2k − 2a)/(2k + 2a + 1) lies between -1 and 0. Therefore −2k−2a n 2k+2a+1 converges slower than n−1 . Moreover the fact that −1 ≤ −(2a − 1)/(2a) allows us to deduce that p−1 is dominated by p−(2a−1)/(2a) . We also have, n−1 p−1 n−1 p and

n−1 p−

(2a−1) 2a

2k+2a − 2k+2a+1

(2a−1) − 2a

= p− 2a −−−→ 0, p→∞

≲ n− 2a+2k+1 −−−−→ 0, n→∞

2a − 2a+2k+1

(2a−1)

2a−1

where we used the fact that p−1 ≲ n and thus that p− 2a ≲ n− 2a+2k+1 to derive (2a−1) the last inequality. This allows us to conclude that n−1 p− 2a and n−1 p−1 are dominated 2a+2k −2k−2a −(2a−1) by n− 2a+2k+1 . Hence the dominant rate is O(n 2k+2a+1 + p 2a ), and 2a+2k 2a−1 E ∥βe − β∥2Γ = O n− 2a+2k+1 + p− 2a .

• If p ≳

2a n , then ln(n)2 h −2k−2a E ∥βe − β∥2Γ ≤ C2 n 2k+2a+1 + n−(2a−1) ln−2(2a−1) (n) + n−2a ln−2(2a−1) (n) 1 n 1 1 n 1 1 i + + + , + + p ln2 (n) np ln2 (n) n p np

where C2 > 0. We now compare the different terms. First, let’s denote 2k+2a

T1 := n− 2k+2a+1 ,

T2 := n−(2a−1) ln−2(2a−1) (n),

T3 := n−2a ln−2(2a−1) (n),

1 1 1 1 n T5 := T6 := , T7 := , T8 := . 2 , 2 , n p np p ln n p ln n Then the bound may be rewritten as E ∥βe − β∥2Γ ≤ C2 (T1 + T2 + T3 + T4 + T5 + T6 + T7 + T8 ). T4 :=

We first observe that T3 = n−2a ln−2(2a−1) (n) =

1 −(2a−1) −2(2a−1) 1 n ln (n) = T2 . n n

Hence T3 = o(T2 ) and thus T3 is negligible compared with T2 . In a similar way we get that T5 =

1 n 1 1 = = T4 , 2 2 n n p ln n p ln n

therefore T5 = o(T4 ). Moreover, T8 =

1 1 1 1 = = T7 , np n p n

thus T8 = o(T7 ). Next, let’s compare T7 and T4 : T4 n n/(p ln2 n) = 2 −−−−→ ∞. = T7 1/p ln n n→∞ Therefore T7 = o(T4 ), and consequently T8 = o(T4 ) as well. Finally, let us compare T6 and 2k+2a 2k+2a T1 . Since 2k+2a+1 < 1, we have that T1 = n− 2k+2a+1 decays more slowly than n−1 . Hence T6 = o(T1 ). Therefore the bound simplifies to 2k+2a n − 2k+2a+1 2 −(2a−1) −2(2a−1) e E ∥β − β∥Γ = O n +n ln (n) + . p ln2 n 2a As we are in the situation where p ≳ lnn2 n we have that T4 =

n n n ≲ = n1−2a ln4a−2 (n). 2a 2 = 2a −4a 2 2 2 p ln n n ln n · ln n n/ ln n ln n

Since 1 − 2a = −(2a − 1), we obtain T4 = O n−(2a−1) ln4a−2 (n) . Therefore, 2k+2a E ∥βe − β∥2 = O n− 2k+2a+1 + n−(2a−1) ln−2(2a−1) (n) + n−(2a−1) ln4a−2 (n) . Γ

We compare now the last two terms. Since for all n ≥ 2, ln4a−2 (n) ≥ ln−2(2a−1) (n) the term n−(2a−1) ln4a−2 (n) is larger than n−(2a−1) ln−2(2a−1) (n). Thus T2 is dominated by T4 under the present lower bound on p. Therefore, 2k+2a E ∥βe − β∥2Γ = O n− 2k+2a+1 + n−(2a−1) ln4a−2 (n) . 47

We now compare

2k+2a

n− 2k+2a+1

and

n−(2a−1) ln4a−2 (n).

The dominant term is determined first by the powers of n. We therefore study whether 2k+2a 2k+2a+1 < 2a − 1. We have 2k + 2a < 2a − 1 ⇐⇒ 2k + 2a < (2a − 1)(2k + 2a + 1) 2k + 2a + 1 ⇐⇒ 2k + 2a < 4ak + 4a2 − 2k − 1 ⇐⇒ 0 < 4a2 + 4ak − 2a − 4k − 1 ⇐⇒ 0 < 4a(a − 1) + 4k(a − 1) + (2a − 1) ⇐⇒ 0 < (a − 1)(4a + 4k) + (2a − 1). Since a ≥ 1 and k ≥ 0, it follows that and

(a − 1)(4a + 4k) ≥ 0

2a − 1 > 0,

and thus (a − 1)(4a + 4k) + (2a − 1) > 0. Hence, 2k + 2a < 2a − 1. 2k + 2a + 1 2k+2a

Therefore n− 2k+2a+1 decays more slowly and is the dominant term. We conclude that 2k+2a E ∥βe − β∥2Γ = O n− 2k+2a+1 . 2a

∗ ∗ Let us focus now on the case where p ≲ n 2a+2k+1 . In this situation we can’t have Dm ≤ DN n,p ∗ and Dm ≈ n1/(2a+2k+1) . Here the best value possible that minimizes f is attained by taking ∗ ∗ ≈ p1/(2a) . We then get = DN Dm n,p

2a+2k 2a−1 2a−1 1 E ∥βe − β∥2Γ ≤ C3 p− 2a + n−1 p 2a + p− 2a + p− 2a n−1 + n−1 + p−1 + n−1 p−1 . 2a−1

Following the previous discussion, we have that p−1 is dominated by p− 2a . As a > 0 and k ≥ 1 2a+2k 2a−1 we have −(2a + 2k)/(2a) ≤ −(2a − 1)/(2a) and p− 2a is dominated by p− 2a . 2a

2a+2k+1 2a

Since we are in the framework of p ≲ n 2a+2k+1 , we have n−1 ≲ p− 2k + 1)/(2a) ≤ −(2a − 1)/(2a). Hence p−(2a−1)/(2a) dominates n−1 . 1

We also have, n−1 p 2a ≲ p− 1 n−1 p 2a . Moreover,

2a+2k 2a

2a−1

and using a previous argument, we get that p− 2a dominates

p−

(2a−1) 2a

p and

n−1

(2a−1) − 2a

n−1 p−1 p−

Therefore, p−

(2a−1) 2a

and obviously, −(2a +

(2a−1) 2a

≲ p−

2a+2k+1 2a

2a+2k 2a

−−−→ 0 p→∞

−−−→ 0. p→∞

(2a−1)

n−1 and n−1 p−1 are dominated by p− 2a . Hence, 2a−1 E ∥βe − β∥2Γ = O p− 2a .

Lemma C.1. Assume that (H3) holds. Then DNn,p −(2a−1) 2 ′ e E ∥X − X∥L2 ≤ C + DNn,p , p where C ′ is a positive constant which depends on τ, c′ , a. 48

Proof. We can express X in the Fourier basis : X = e we get that definition of X and X

1≤l xl ϕl , where xl = ⟨X, ϕl ⟩L2 .

Using the

Z 1 e − X∥2 2 ) = E e − X(t))2 dt E(∥X ( X(t) L 0 Nn,p Z 1 DX X =E | x ej ϕj (t) − xj ϕj (t)|2 dt

j=1

1≤j

Nn,p Z 1 DX

(e xj − xj )ϕj (t) −

j=1

xj ϕj (t)|2 dt .

DNn,p <j

As the (ϕj )1≤j are orthonormal, we then get Nn,p X DX 2 e (e x j − x j )2 + E E(∥X − X∥L2 ) = E

j=1

x2j .

DNn,p <j

As x ej = xj + η j for all j ∈ {1, . . . , DNn,p }, where xj and η j are defined in (47), we get that 2 (e xj − xj )2 = (xj − xj )2 + η 2j + 2η j (xj − xj ). Now, using the fact that E(η j ) = 0, E(η 2j ) = τp by Lemma B.2 and that (ηj )j=1,...,DNn are independent of X, we get the following equation : Nn,p DX X 2 2 e − X∥2 2 ≤ DNn,p τ + E E ∥X (x − x ) + E j j L p j=1

x2j .

(76)

DNn,p <j

Let us focus on the second term of the right part of the inequality. Using Fubini’s theorem we have, X X E x2j = E x2j . DNn,p <j

DNn,p <j

Now using Fubini’s theorem and the definition of the kernel K associated with the norm Gamma mentioned in (18) we get, E x2j = E ⟨X, ϕj ⟩2L2 Z 1Z 1 = ϕj (t)ϕj (s)E (X(s)X(t)) dsdt 0

Z 1 =

Z 1 ϕj (t)

K(s, t)ϕj (s)dsdt 0

= ⟨Γϕj , ϕj ⟩L2 = λj . Hence using (H3),

X E

x2j =

DNn,p <j

E x2j

DNn,p <j

λj

DNn,p <j ′

≤c

−2a

(77)

DNn,p <j

≤

c′ −(2a−1) D . 2a − 1 Nn,p

PD Nn,p 2 Let us focus now on E j=1 (xj − xj ) . In Lemma B.2 we defined for all j ∈ {1, . . . , DNn,p } ej = xj − x j , 49

and proved that

E e2j = λj .

Using now Lemma B.8 we get that λj = O(p−2a ).

∀j ∈ {1, . . . , DNn,p } Hence,





DNn,p

E

(78)

(xj − xj )2  ≤ CDNn,p p−2a .

j=1

Combining Equations (76), (77) and (78) and using the fact that p−2a ≤ p−1 as a > 1/2, we get DNn,p −(2a−1) 2 ′ e E ∥X − X∥L2 ≤ C + DNn,p . p

Lemma C.2. Assume that (H1), (H2), (H3) hold. Then, e 4 2 ≤ C(a, c′ , τ ) D−(2a−1) + DNn,p . E1/2 ∥X − X∥ L Nn,p p e1 we have Proof. By definition of X1 and X X e 22 = ∥X − X∥ (e x j − x j )2 + L

x2j .

j>DNn,p

j≤DNn,p

√ √ √ Using the inequality (x + y)2 ≤ 2x2 + 2y 2 first, and then the inequality x + y ≤ x + y we get that 1/2  X X √ 2 2 e 4 2 ≤ 2 E  E1/2 ∥X − X∥ (e xj − xj )2 x2j +E L j>DNn,p

j≤DNn,p

≤

√





2 E1/2

2 (e xj − xj )2 + E1/2

2 2

.

j>DNn,p

j≤DNn,p

P 2 1/2 Let us first study the term E xj − xj )2 . Let us recall that we can write x ej as j≤DNn,p (e x ej = xj + ej + η j as defined in Equations (47) and (48). Therefore, applying Cauchy–Schwarz inequality to the vector (1, 1, . . . , 1) ∈ RDNn,p and (ej + η j )2 j=1,...,D we get Nn,p

2



DNn,p

X 

DNn,p

2

(ej + η j )2 



(e xj − xj )2  = 

j=1

j=1 DNn,p

≤ DNn,p

(ej + η j )4

j=1 DNn,p

DNn,p



η 4j  .

 ≤ 8DNn,p 

j=1

Let us study the term E

Nn,p

j=1

e4j +

j=1

e4j . Hypothesis (H1) implies that the ej ’s are sub-Gaussian

with variance factor λj . Theorem 2.1 of Boucheron et al. [2] gives us that for all q ′ ≥ 1 and for all 1 ≤ j ≤ DNn,p , E e2q j

′

≤ q ′ !(4λj )q , 50

and taking q ′ = 2 leads to

2 E e4j ≤ 32λj .

Lemma B.8 gives that for all 1 ≤ j ≤ DNn,p , λj ≤ C(a, c′ )p−2a , hence E e4j ≤ C(a, c′ )p−4a . Now, let us focus on the term E η 4j . In a similar way, hypothesis (H2) gives us that η j is subGaussian with variance factor τ 2 /p. Theorem 2.1 of Boucheron et al. [2] gives that for all q ′ ≥ 1 and for all 1 ≤ j ≤ DNn,p , 2 q′ ′ τ 2q ′ ≤q! 4 . E ηj p Taking q ′ = 2 we get that

τ4 E η 4j ≤ 32 2 . p

This allows us to get that E1/2

(e xj − xj )2

≤ DNn,p

j≤DNn,p

C(c′ , a) C(τ ) + 2 p4a p

1/2

C(c′ , a, τ ) ≤ DNn,p , p

(79)

√ √ √ where we used the fact that x + y ≤ x + y and that a > 1/2 to get the last inequality. Lastly, 2 P x2j let’s focus on E1/2 . By using Tonelli’s theorem, we have j>DN n,p

 E

2 2

 X

= E

j>DNn,p

x2j x2k 

j>DNn,p k>DNn,p

E x2j x2k .

j>DNn,p k>DNn,p

Applying Cauchy–Schwarz inequality to x2j and x2k we find E

x2j

≤

j>DNn,p

E(x4j )1/2 E(x4k )1/2

j>DNn,p k>DNn,p

2

 X

=

E(x4j )1/2  .

j>DNn,p

Hypothesis (H1) and the fact that the λj ’s decrease in a polynomial way, give us that 2

 E

2 2

≤ 32 

j>DNn,p

λj 

j>DNn,p

2

 ≤ 32c′2 

j −2a 

j>DNn,p −2(2a−1)

≤ C(a, c′ )DNn,p and

E1/2

x2j

−(2a−1)

≤ C(a, c′ )DNn,p

j>DNn,p

(80)

Combining (79) and (80) we find that e 4 2 ≤ C(a, c′ , τ ) D−(2a−1) + DNn,p . E ∥X − X∥ L Nn,p p

Proof of Theorem 5.1

This section is divided into four parts. The first part presents the three conditions of Theorem 2.7 from Tsybakov [15], while the remaining three sections provide the results used to prove them. The proof follows the methodology introduced in Section 2.6 of Tsybakov [15]. The first step of the proof is to build test functions. We define M ∈ N∗ and for all l = 1, . . . , M we consider w(l) ∈ {0, 1}d . We set w(0) = (0, . . . , 0) ∈ Rd . Then we consider the following test functions βw(l) (t) =

d X

(l)

wj uj ϕj (t)

for all

l = 0, . . . , M,

(81)

j=1

where u2j =

C nλj

with

α′ log(2) . C = min L2 (2π)−2k c′−1 , σ 2 4

(82)

We set 0 < α′ < 1/8 and 0 < κ < 1 such that n ≥ (1 − κ)−(2a+2k+1) . We also define

d = ⌈κn 2a+2k+1 ⌉

(83) (84)

with n such that d ≥ 8. Remark 3. The parameter M corresponds to the number of hypotheses in the packing set. Its introduction is motivated by the Varshamov–Gilbert lemma (Lemma 2.9 in Tsybakov [15]), which ensures the existence of a large subset of binary vectors with pairwise Hamming distance bounded below. This large family of well-separated alternatives is the key ingredient for applying Tsybakov’s lower-bound theorem. We then consider the following setting : • A class of functions containing the true slope function β that we want to estimate : W per (k, L) which is given by Equation (3). • A sample (Zi , Yi )ni=1 which is such that for all h = 0, . . . , p − 1, for all i = 1, . . . , n, Zi (th ) = Xi (th ) + ηi,h ,

(85)

where the ηi,h ’s are i.i.d, Gaussian, centered and of variance τ 2 . For each l = 0, . . . , M we consider Yi = ⟨Xi , βw(l) ⟩L2 + ϵi , (86) where βw(l) ∈ W per (k, L) and the ϵi ’s are i.i.d, are Gaussian, centered and of variance σ 2 . Both the ϵi ’s and ηi,h ’s are independent of everything else. We denote the joint distribution of this sample by Pl = Pw(l) . • A semi distance ∥ · ∥Γ which is defined in (28). In order to apply Theorem 2.7 of Tsybakov [15] we need to check the three following conditions: 1. ∀l = 0, . . . , M , βw(l) ∈ W per (k, L). 52

2. Pj ≪ P0 for all j = 1, . . . , M and M

1 X K(Pj ∥P0 ) ≤ α′ log(M ), M j=1 with 0 < α′ < 1/8 and Pj = Pw(j) for j = 0, 1, . . . , M . Here K(·∥·) denotes the Kullback– Leibler divergence. 3. ∀ 0 ≤ l < l′ ≤ M

2k+2a

∥βw(l) − βw(l′ ) ∥2Γ ≥ 2C4 n− 2k+2a+1 , where C4 > 0 is a positive constant which will be defined bellow. Proposition D.1, D.2 and D.3 allow us to apply this theorem and therefore to prove the result.

D.1

Condition 1. of the proof

Proposition D.1. Consider M ∈ N\{0} and for all l = 1, . . . , M , define w(l) ∈ {0, 1}d , as well as w(0) = (0, 0, . . . , 0) ∈ Rd where d is defined by (84). Consider for all l = 0, 1, . . . , M , the test functions βw(l) given by (81). Then, for all l = 0, 1, . . . , M , βw(l) ∈ W per (k, L). Proof. As for all t ∈ [0, 1] βw(0) (t) = 0 we have that βw(0) is trivially in W per (k, L). Let us focus (s) on the case where l = 1, 2, . . . , M . We need to prove that for all s = 1, . . . , k, βw(l) is 1-periodic (s)

and that ∥βw(l) ∥2L2 ≤ L2 . For all s = 1, . . . , k we have, by Lemma D.1 ⌊d/2⌋ (s) βw(l) (t) =

(2πj)s

(l) (l) as w2j u2j + cs 1{2j+1≤d} w2j+1 u2j+1 ϕ2j (t)

j=1

(l) (l) + bs w2j u2j + as 1{2j+1≤d} w2j+1 u2j+1 ϕ2j+1 (t) . (s)

As each real Fourier basis function is 1-periodic we have that βw(l) is 1-periodic for all s = 1, ..., M . With the same argument we have, for s = 0 that the function is also 1-periodic. To conclude the (k) proof we then need to prove that ∥βw(l) ∥2L2 ≤ L2 . By Lemma D.1 we have (k)

∥βw(l) ∥2L2 ≤ (2π)2k

d X

j 2k (wj uj )2 .

j=1

Using the definition of uj given by (82) and the fact that λj decrease in a polynomial way we have d

(k)

∥βw(l) ∥2L2 ≤

L2 X 2k+2a j . n j=1

As a > 1/2 and k is a positive integer, 2k + 2a ≥ 1 and we get d

L2 2k+2a+1 L2 X 2k+2a j ≤ d . n j=1 n Using now the definition of d and equation (83) we find for n large enough (k)

∥βw(l) ∥2L2 ≤ L2 . Therefore, for all l = 0, 1, . . . , M ,

βw(l) ∈ W per (k, L).

Lemma D.1. Consider M ∈ N\{0} and for all l = 1, . . . , M , define w(l) ∈ {0, 1}d where d is defined by (84). Consider for all l = 1, . . . , M , the test functions βw(l) given by (81). Then, for all l = 1, . . . , M and for all integer s ≥ 1, βw(l) is s-times differentiable and ⌊d/2⌋ (s)

βw(l) (t) =

(2πj)s

(l) (l) as w2j u2j + cs 1{2j+1≤d} w2j+1 u2j+1 ϕ2j (t)

j=1

(l) (l) + bs w2j u2j + as 1{2j+1≤d} w2j+1 u2j+1 ϕ2j+1 (t) , sπ sπ with as = cos ( sπ 2 ), bs = − sin ( 2 ) and cs = sin ( 2 ). Moreover, (s)

∥βw(l) ∥2L2 ≤ (2π)2k

d X

j 2k (wj uj )2 .

j=1

Proof. Let s ∈ {1, . . . , k}. Then, βw(l) is s-times differentiable as a sum of s-times differentiable functions and (s) βw(l) (t) =

d X

⌊d/2⌋ (l) (s) wj uj ϕj (t) =

j=1

⌊(d−1)/2⌋ (l) (s) w2j u2j ϕ2j (t) +

j=1

(l)

(s)

w2j+1 u2j+1 ϕ2j+1 (t).

j=1

Now,

√ ds cos(2πjt) √ sπ s 2 = 2(2πj) cos 2πjt + , dts 2 √ ds sin(2πjt) √ sπ s 2 = 2(2πj) sin 2πjt + . dts 2 The trigonometric formulas lead to, sπ sπ √ √ sπ √ = 2 cos cos(2πjt) − 2 sin sin(2πjt) 2 cos 2πjt + 2 2 2 = as ϕ2j (t) + bs ϕ2j+1 (t), and

√

sπ sπ √ sπ √ = 2 sin cos(2πjt) + 2 cos sin(2πjt) 2 sin 2πjt + 2 2 2 = cs ϕ2j (t) + as ϕ2j+1 (t),

sπ sπ with as = cos ( sπ 2 ), bs = − sin ( 2 ) and cs = sin ( 2 ). More particularly these coefficients have the following values depending on s:

s mod 4 0 1 2 3

as 1 0 -1 0

bs 0 -1 0 1

cs 0 1 0 -1

Using the previous result, we get that ⌊d/2⌋ (s)

βw(l) (t) =

(l) (l) (2πj)s as w2j u2j ϕ2j (t) + bs w2j u2j ϕ2j+1 (t)

j=1 ⌊(d−1)/2⌋

(l) (l) (2πj)s cs w2j+1 u2j+1 ϕ2j (t) + as w2j+1 u2j+1 ϕ2j+1 (t)

j=1 ⌊d/2⌋

(2πj)s

(l) (l) as w2j u2j + cs 1{2j+1≤d} w2j+1 u2j+1 ϕ2j (t)

j=1

(l) (l) + bs w2j u2j + as 1{2j+1≤d} w2j+1 u2j+1 ϕ2j+1 (t) , 54

which proves the first point of the Lemma. Now let us focus on the second one. Let’s consider (l) (l) Aj := (2πj)k ak w2j u2j + ck 1{2j+1≤d} w2j+1 u2j+1 , (l) (l) Bj := (2πj)k bk w2j u2j + ak 1{2j+1≤d} w2j+1 u2j+1 , as well as,

(l)

xj := w2j u2j ,

yj := 1{2j+1≤d} w2j+1 u2j+1 .

This gives the following expressions for Aj and Bj : Aj = (2πj)k (ak xj + ck yj ),

Bj = (2πj)k (bk xj + ak yj ).

From this we can deduce that A2j + Bj2 = (2πj)2k (ak xj + ck yj )2 + (bk xj + ak yj )2 = (2πj)2k (a2k + b2k )x2j + (c2k + a2k )yj2 + 2xj yj (ak ck + ak bk ) . Now we can notice that a2k + b2k = cos2

kπ 2

+ sin2

kπ 2

c2k + a2k = sin2

= 1,

kπ 2

+ cos2

kπ 2

= 1,

and ak ck + ak bk = ak (ck + bk ) = 0. Therefore the cross term vanishes and both quadratic coefficients are equal to 1, which gives (l) (l) A2j + Bj2 = (2πj)2k (x2j + yj2 ) = (2πj)2k (w2j u2j )2 + 1{2j+1≤d} (w2j+1 u2j+1 )2 . (87) By orthonormality of the real Fourier basis {ϕj }j≥1 in L2 (0, 1), we have ⟨ϕi , ϕj ⟩L2 = δij . Hence, (k) when βw(l) is written as a linear combination ⌊d/2⌋ (k)

βw(l) (t) =

Aj ϕ2j (t) + Bj ϕ2j+1 (t) ,

j=1

its squared L2 norm is simply the sum of the squared coefficients : ⌊d/2⌋ (k) ∥βw(l) ∥2L2 =

A2j + Bj2 ,

j=1

since all cross terms ⟨ϕi , ϕj ⟩L2 with i ̸= j vanish. Substituting equation (87) we obtain ⌊d/2⌋ (k)

∥βw(l) ∥2L2 = (2π)2k

j 2k

(l)

w2j u2j

(l)

+ 1{2j+1≤d} w2j+1 u2j+1

j=1



⌊d/2⌋

⌊(d−1)/2⌋

= (2π)2k 

j 2k (w2j u2j )2 +

j=1



(88)

j 2k (w2j+1 u2j+1 )2  ,

j=1

where the indicator function has been replaced by the upper limit ⌊(d − 1)/2⌋. Let us treat the first sum. We set r = 2j. Then j = r/2, and when j = 1 we have r = 2, while when j = ⌊d/2⌋, r = 2⌊d/2⌋ ≤ d. Therefore ⌊d/2⌋

X j=1

j 2k (w2j u2j )2 =

X r 2k (wr ur )2 . 2 r=2,4,... r≤d

Since r/2 ≤ r, we get the bound ⌊d/2⌋

j 2k (w2j u2j )2 ≤

r2k (wr ur )2 .

r=2,4,... r≤d

j=1

Let us now focus on the second sum of equation (88). We set r = 2j + 1. Then j = (r − 1)/2, and as j runs from 1 to ⌊(d − 1)/2⌋, r runs over all odd integers r = 3, 5, . . . , 2⌊(d − 1)/2⌋ + 1 ≤ d. Hence, ⌊(d−1)/2⌋ X X r − 1 2k (wr ur )2 . j 2k (w2j+1 u2j+1 )2 = 2 r=1,3,... j=1 r≤d

Since (r − 1)/2 ≤ r, we obtain ⌊(d−1)/2⌋

j 2k (w2j+1 u2j+1 )2 ≤

r2k (wr ur )2 .

r=1,3,... r≤d

j=1

Adding the even and odd contributions gives 



X  X 2k  (k) ∥βw(l) ∥2L2 ≤ (2π)2k  r (wr ur )2 + r2k (wr ur )2   . r=2,4,... r≤d

r=1,3,... r≤d

The union of even and odd indices from 1 to d is simply {1, 2, . . . , d}, therefore we can merge the sums: d X (k) 2 2k ∥βw(l) ∥L2 ≤ (2π) r2k (wr ur )2 . r=1

D.2

Condition 2. of the proof

Proposition D.2. Let M ∈ N \ {0} and, for each l = 1, . . . , M , let w(l) ∈ {0, 1}d and w(0) = (0, . . . , 0), where d is defined in (84). For j ∈ {0, l}, let βw(j) be defined by (81). Let Pj denote the joint law of the observed sample {(Zi , Yi )}ni=1 under the model associated with βw(j) . Then, for every l = 1, . . . , M , K(P0 ∥Pl ) ≤ α′ log M, for some 0 < α′ < 81 . Proof. We first work at the level of a single observation (X, Z, Y ), and later use the fact that (Xi , Zi , Yi )i=1,...,n are i.i.d to recover the result for the full sample. Using Lemma D.2, we have h i (0),1 (l),1 K P01 ∥Pl1 ≤ EX∼P (0),1 (X) K Pfull (Y | X)∥Pfull (Y | X) , (89) full

(l),1

where Pfull is the probability of the triplet (X, Z, Y ) under model l. We now compute the right(l),1 hand side. Under Pfull , conditionally on X, we have Y | X ∼ N ⟨X, βw(l) ⟩L2 , σ 2 . Define ν (l) := ⟨X, βw(l) ⟩L2 =

d X j=1

and

ν (0) := 0. 56

(l)

wj uj ⟨X, ϕj ⟩L2 .

(90)

Since the conditional distributions are Gaussian with the same variance, we obtain (0),1 i dPfull (Y | X) 1 h (l) 2 (0) 2 , (Y − ν ) − (Y − ν ) = exp (l),1 2σ 2 dPfull (Y | X) and after developing the square we get (0),1

dPfull (Y | X) (l),1

dPfull (Y | X)

= exp

1 (l) 2 (l) ν − 2Y ν = exp 2σ 2

! 2 ν (l) Y ν (l) − . 2σ 2 σ2

(0),1

By taking expectation under Pfull (Y | X), we obtain " # (0),1 i dPfull (Y | X) 1 h (l) 2 (l) X − 2ν EP (0),1 log = ν E(Y | X) . (l),1 full 2σ 2 dP (Y | X) full

(0),1 Now, under Pfull , we have E(Y

| X) = ν (0) = 0, hence " # (0),1 dPfull (Y | X) 1 (l) 2 X EP (0),1 log = ν . (l),1 full 2σ 2 dPfull (Y | X)

Using the definition of ν (l) given in (90) we have that d 2 X (l) (l) ν (l) = wj wk uj uk ⟨X, ϕj ⟩L2 ⟨X, ϕk ⟩L2 . j,k=1 (0),1

Taking expectation with respect to X ∼ Pfull (X), we obtain E

(l)

d X

(l) (l) wj wk uj uk E ⟨X, ϕj ⟩L2 ⟨X, ϕk ⟩L2 .

j,k=1

Using now equation (49) we get that E

ν (l)

d X

(l) 2

λj u2j wj

j=1

Combining with (89), we get for one observation d 1 X (l) 2 K P01 ∥Pl1 ≤ λj u2j wj , 2 2σ j=1

Since the observations are independent, we have K P0 ∥Pl = nK P01 ∥Pl1 , and therefore K(P0 ∥Pl ) ≤

d n X (l) 2 λj u2j wj . 2 2σ j=1 ′

C Using the definition u2j = nλ and the bound C ≤ σ 2 α log(2) , we obtain 4 j d n X α′ log(2)d (l) 2 2 λ u w ≤ . j j j 2σ 2 j=1 8 log(M ) Finally, by the Varshamov–Gilbert bound (Lemma 2.9 in [15]), M ≥ 2d/8 , hence d ≤ 8 log(2) , which concludes the proof.

Lemma D.2. Let l ∈ {1, . . . , M }. Let P01 and Pl1 denote the joint laws of (Z, Y ) under the models (0),1 (l),1 associated with βw(0) and βw(l) , respectively. Let Pfull and Pfull denote the corresponding joint laws of (X, Z, Y ). Then h i (0),1 (l),1 K(P01 ∥Pl1 ) ≤ EX∼P (0),1 (X) K Pfull (Y |X)∥Pfull (Y |X) . full

(j),1

Proof. We work with the full joint model for each j ∈ {0, l}, denoted Pfull , governing the triplet (j),1 (X, Z, Y ). The observed law Pj1 is the (Z, Y )-marginal of Pfull . Using Lemma D.3 stated below and its associated notation, we have K P01 ∥Pl1 = K P01,Z ∥Pl1,Z + EZ∼P 1,Z K P01 (· | Z)∥Pl1 (· | Z) . 0

(91)

Since the marginal law of Z does not depend on j for j ∈ {0, l} (the distribution of (X, η) being identical across models), we have P01,Z = Pl1,Z and thus K P01,Z ∥Pl1,Z = 0. (j),1

Now fix z. For j ∈ {0, l} we can define under the full model Pfull , (j),1

µz (dx) := Pfull (X ∈ dx | Z = z) be a regular conditional distribution of X given Z = z. Note that µz does not depend on j, since the joint law of (X, Z) is the same for all j. Let us denote by µ(dx) the law of X. We have using Bayes’s formula that the density of µz (dx) with respect to µ(dx) is µz (dx) = R

fZ|X=x (z) µ(dx), fZ|X=u (z)µ(du)

(92)

where fZ|X=x is defined in equation (95). Using the definition of the conditional density (98) we have for j ∈ {0, l} that pj (z, y) pj (y | z) = Z = pj (z)

(j)

fZ|X=x (z)pfull (y | x)µ(dx) R . fZ|X=x (z)µ(dx)

Now using the definition of µz given in equation (92) we find for j ∈ {0, l}, Z Z fZ|X=x (z) (j) (j) pfull (y | x)µz (dx) = pfull (y | x) R µ(dx) fZ|X=u (z)µ(du) R (j) fZ|X=x (z)pfull (y | x)µ(dx) R = fZ|X=x (z)µ(dx) = pj (y | z). Let us still consider z fixed, and let us compute K P01 (· | Z = z)∥Pl1 (· | Z = z) . Using the definition of the Kullback–Leibler divergence and the previous representation for pj (y | z), we have Z p0 (y | z) 1 1 K P0 (· | Z = z)∥Pl (· | Z = z) = p0 (y | z) log dy pl (y | z) R (0) Z Z p (y | x)µz (dx) (0) = pfull (y | x)µz (dx) log R full dy. (l) pfull (y | x)µz (dx) Now let us fix y and set (0)

a(x) = pfull (y | x),

(l)

b(x) = pfull (y | x),

Z s=

b(x)µz (dx).

Let us define the probability measure π(dx) = b(x)µsz (dx) and the ratio r(x) = a(x)/b(x). For φ(t) = t log t which is a convex function on (0, ∞) we have by Jensen’s inequality, Z Z a(x) µz (dx) = b(x)φ(r(x))µz (dx) a(x) log b(x) Z = s φ(r(x))π(dx) Z ≥ sφ r(x)π(dx) R R a(x)µz (dx) a(x)µz (dx) =s log . s s R As s = b(x)µz (dx), we obtain for each y, Z

Integrating over y and applying Fubini’s theorem gives K

P01 (Y

| Z = z)∥Pl1 (Y

| Z = z) ≤

(0) pfull (y | x) (0) dy pfull (y | x) log (l) pfull (y | x)

! µz (dx).

Taking expectations over Z ∼ P01,Z and using (91) we get, h i (0),1 (l),1 K P01 ∥Pl1 ≤ EX∼P (0),1 (X) K Pfull (Y | X)∥Pfull (Y | X) . full

(93)

Lemma D.3. Let P01 and Pl1 , l = 1, . . . , M , be two probability measures on Rp × R corresponding to the joint distribution of (Z, Y ). In particular, for all model j ∈ {0, l}, Y = ⟨X, βw(j) ⟩L2 + ϵ,

(94)

where βw(j) given by (81) and ϵ ∼ N (0, σ 2 ). Assume that P01 ≪ Pl1 and let us denote by P01,Z and Pl1,Z the marginal laws of Z. Then, K(P01 ∥Pl1 ) = K(P01,Z ∥Pl1,Z ) + EZ∼P 1,Z K P01 (· | Z)∥Pl1 (· | Z) . 0

Proof. We first show that P01 and Pl1 admit densities with respect to the Lebesgue measure on Rp+1 . By definition of Z and using the fact that the ηi,h ’s are Gaussian, centered and of variance τ 2 , we have that Z | X = x ∼ N (xgrid , τ 2 Ip ), where xgrid = (x(h/p))h=0,...,p−1 . Therefore Z | X = x admits a density fZ|X=x which is such that fZ|X=x (z) =

∥z − xgrid ∥22 1 exp − . 2τ 2 (2πτ 2 )p/2

(95)

Moreover, under Pj1 , j ∈ {0, l}, the response variable Y is described by equation (94). Therefore for all j ∈ {0, l} we have conditional on X = x, Y | X = x ∼ N (⟨x, βw(j) ⟩L2 , σ 2 ). 59

(j)

Hence it admits a density pfull (· | x) which is such that (y − ⟨x, βw(j) ⟩L2 )2 1 (j) exp − . pfull (y | x) = 2σ 2 (2πσ 2 )1/2 As the ϵi ’s and ηi,h ’s are assumed to be independent we have that for all j ∈ {0, l}, l = 1, . . . , M , (Z, Y ) | X = x admits a density which is given by (j)

(j)

f(Z,Y )|X=x (z, y) = fZ|X=x (z)pfull (y | x). Let us now integrate out with respect to the law of X that we denote as µ(dx). We obtain that the density of Pj1 is given by Z (j) pj (z, y) = f(Z,Y )|X=x (z, y)µ(dx) Z (96) (j) = fZ|X=x (z)pfull (y | x)µ(dx). Finally, we get that Pj1,Z admits a density too which is given by pZ j (z) =

pj (z, y)dy Z Z (j) = fZ|X=x (z)pfull (y | x)µ(dx)dy Z = fZ|X=x (z)µ(dx),

(97)

where we used Fubini’s theorem and the definition of a density to derive the last line. Thus the conditional density of Y given Z under Pj1 is given by pj (z, y) pZ j (z) R (j) fZ|X=x (z)pfull (y | x)µ(dx) R , = fZ|X=x (z)µ(dx)

pj (y | z) =

(98)

By definition of the Kullback–Leibler divergence, we have Z p0 (z, y) 1 1 K P0 ∥Pl = p0 (z, y) log dzdy pl (z, y) Z p0 (y | z)pZ 0 (z) = p0 (z, y) log dzdy pl (y | z)pZ l (z) Z Z Z p0 (y | z) p0 (z) dzdy + p (z, y) log dzdy, = p0 (z, y) log 0 pl (y | z) pZ l (z) where we used the definition of the conditional density given by equation (98) to get the second line and the property of the log to get the last one. Now let’s deal with these two terms one by one. For the first one we have, using Fubini’s theorem and the definition of the marginal density that Z Z Z Z Z p0 (z) p0 (z) p0 (z, y) log dzdy = log p (z, y)dy dz 0 pZ pZ l (z) l (z) Z Z p0 (z) = pZ (z) log dz 0 pZ l (z) = K P01,Z ∥Pl1,Z .

Combining these two results yields the desired property.

D.3

Condition 3. of the proof

Proposition D.3. Consider d defined by Equation (84), the test functions given by Equation (81), the semi-norm described by Equation (28). Then, 2a+2k

∥βw(l) − βw(l′ ) ∥2Γ ≥ 2C4 n− 2a+2k+1 , where C4 is a positive constant which depends on k, L, c′ , σ. Proof. Using the definition of Γ given in (17) we find that ∥βw(l) − βw(l′ ) ∥2Γ = ⟨Γ(βw(l) − βw(l′ ) ), βw(l) − βw(l′ ) ⟩L2 d d X X (l) (l′ ) (l) (l′ ) =⟨ uj (wj − wj )Γϕj , uk (wk − wk )ϕk ⟩L2 j=1

d X

k=1 (l′ )

(l)

(l′ )

(l)

uj uk (wj − wj )(wk − wk )⟨Γϕj , ϕk ⟩L2

(99)

j,k=1

d X

(l)

(l′ )

λj (wj − wj )2 u2j ,

j=1

as the λj ’s are eigenvalues of Γ associated to the eigenvectors ϕj ’s. Now using the definition of uj given by (82) we get d C X (l) (l′ ) (wj − wj )2 . ∥βw(l) − βw(l′ ) ∥2Γ ≥ n j=1 Now, as d d X X (l) (l′ ) (wj − wj )2 = 1{w(l) ̸=w(l′ ) } , j=1

j=1

we can use the Varshamov–Gilbert bound (Lemma 2.9 of Tsybakov [15]) and get that ∥βw(l) − βw(l′ ) ∥2Γ ≥

Cd . 8n

Using the definition of d we find that 2a+2k

∥βw(l) − βw(l′ ) ∥2Γ ≥ 2C4 n− 2a+2k+1 , where C4 = Cκ 16 which is the desired result.

Orthogonal projection of β onto Sm with respect to ⟨·, ·⟩Γ and ⟨·, ·⟩Γe

As β is in L2 ([0, 1]) and the Fourier basis forms a basis of this space, we can write X β= βj ϕ j , j≥1

(m)

where βj = ⟨β, ϕj ⟩L2 . We first determine βΓ the orthogonal projection of β onto Sm with respect to ⟨·, ·⟩Γ . Using the definition of Sm defined by equation (5) we have that (ϕ1 , ϕ2 , . . . , ϕDm ) form (m) a basis of Sm . By definition of the orthogonal projection βΓ is in Sm , there exist real numbers µ1 , µ2 , . . . , µDm , such that Dm X (m) βΓ = µj ϕj , j=1 (m)

and for the same argument we have that β − βΓ implies that for all k = 1, . . . , Dm ,

⊥ ∈ Sm . As (ϕ1 , ϕ2 , . . . , ϕDm ) is a basis of Sm this

(m)

⟨β − βΓ , ϕk ⟩Γ = 0. Using the fact that for all j, k = 1, . . . , Dm ⟨ϕj , ϕk ⟩Γ = ⟨Γϕj , ϕk ⟩L2 = ⟨λj ϕj , ϕk ⟩L2 = λj δjk , with λj > 0 for all j = 1, . . . , Dm , and Fubini’s theorem to justify the following computation, we obtain (m)

⟨β − βΓ , ϕk ⟩Γ = ⟨

βj Γϕj , ϕk ⟩L2 + ⟨

Dm X

(βj − µj )Γϕj , ϕk ⟩L2

= λk (βk − µk ).

j=1

j>Dm (m)

Therefore, ⟨β − βΓ , ϕk ⟩Γ = 0 ⇐⇒ λk (βk − µk ) = 0 for all k = 1, . . . , Dm . And as λk > 0 we find PDm (m) (m) that ⟨β − βΓ , ϕk ⟩Γ = 0 ⇐⇒ βk = µk . Hence, βΓ = j=1 βj ϕj . We proceed in a similar way (m)

to determine βΓe

the orthogonal projection of β onto Sm with respect to ⟨·, ·⟩Γe . By the definition (m)

of the orthogonal projection, we must have βΓe (m)

βΓe

∈ Sm , therefore there exist µ e1 , . . . , µ eDm such that Dm X

µ ej ϕj .

j=1

We also have that for all k = 1, . . . , Dm , (m)

β − βΓe

= 0.

Using Lemma B.5 and Fubini’s theorem we obtain, (m)

⟨β − βΓe , ϕk ⟩Γe = ⟨

j>Dm

e j , ϕk ⟩L2 + ⟨ βj Γϕ

Dm X

e j , ϕk ⟩L2 (βj − µ ej )Γϕ

ek (βk − µ =λ ek ).

j=1

(m) ek > 0 we find Hence, ⟨β − βΓe , ϕk ⟩Γe = 0 ⇐⇒ λk (βk − µ ek ) = 0 for all k = 1, . . . , Dm . And as λ P (m) (m) (m) (m) Dm that ⟨β − βΓe , ϕk ⟩Γe = 0 ⇐⇒ βk = µ ek . Therefore, βΓe = j=1 βj ϕj and βΓe = βΓ .

Acknowledgments I am deeply grateful to my supervisors, Gaëlle Chagny, Vincent Rivoirard and Angelina Roche, for their insightful comments and constructive feedback. The author also acknowledge the support of the French Agence Nationale de la Recherche (ANR) under reference ANR-24-CE40-2439 (FUNMathStat project).

References [1]

Yannick Baraud. “Model Selection for Regression on a Fixed Design”. In: Probability Theory and Related Fields 117 (2000), pp. 467–493. doi: 10.1007/PL00008731. 62

[2]

Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford, UK: Oxford University Press, 2013. isbn: 9780199535255. doi: 10.1093/acprof:oso/9780199535255.001.0001.

[3]

Elodie Brunel, André Mas, and Angelina Roche. “Non-Asymptotic Adaptive Prediction in Functional Linear Models”. In: Journal of Multivariate Analysis 143 (2016), pp. 208–232. issn: 0047-259X. doi: 10.1016/j.jmva.2015.09.008.

[4]

Elodie Brunel and Angelina Roche. “Penalized Contrast Estimation in Functional Linear Models with Circular Data”. In: Statistics 49.6 (2015), pp. 1298–1321. doi: 10 . 1080 / 02331888.2014.993986.

[5]

Hervé Cardot, Frédéric Ferraty, and Pascal Sarda. “Functional Linear Model”. In: Statistics & Probability Letters 45.1 (1999), pp. 11–22. doi: 10.1016/S0167-7152(99)00036-X.

[6]

Hervé Cardot and Jan Johannes. “Thresholding Projection Estimators in Functional Linear Models”. In: Journal of Multivariate Analysis 101.2 (2010), pp. 395–408. issn: 0047-259X. doi: 10.1016/j.jmva.2009.03.001.

[7]

Hervé Cardot et al. “Smoothing Splines Estimators in Functional Linear Regression with Errors-in-Variables”. In: Computational Statistics & Data Analysis 51.10 (2007).

[8]

F. Comte and J. Johannes. “Adaptive Estimation in Circular Functional Linear Models”. In: Mathematical Methods of Statistics 19.1 (2010), pp. 42–63.

[9]

S. Dieng et al. “Application of Functional Data Analysis to Identify Patterns of Malaria Incidence, to Guide Targeted Control Strategies”. In: International Journal of Environmental Research and Public Health 17.11 (2020), p. 4168. doi: 10.3390/ijerph17114168.

[10]

K. F. Frøslie, J. Røislien, E. Qvigstad, et al. “Shape Information from Glucose Curves: Functional Data Analysis Compared with Traditional Summary Measures”. In: BMC Medical Research Methodology 13.6 (2013). doi: 10.1186/1471-2288-13-6.

[11]

Paul-Marie Grollemund et al. “Bayesian Functional Linear Regression with Sparse Step Functions”. In: Bayesian Analysis 14.1 (2019), pp. 111–135. doi: 10.1214/18-BA1095.

[12]

Gareth M. James, Jing Wang, and Ji Zhu. “Functional Linear Regression That’s Interpretable”. In: The Annals of Statistics 37.5A (2009), pp. 2083–2108. doi: 10 . 1214 / 08 AOS641.

[13]

Piotr Kokoszka, Hong Miao, and Ben Zheng. “Testing for Asymmetry in Betas of Cumulative Returns: Impact of the Financial Crisis and Crude Oil Price”. In: Statistics & Risk Modeling 34.1–2 (2017), pp. 33–53. doi: 10.1515/strm-2016-0010.

[14]

J. O. Ramsay and B. W. Silverman. Functional Data Analysis. 2nd ed. Springer Series in Statistics. Springer, 2005.

[15]

Alexandre B. Tsybakov. Introduction to Nonparametric Estimation. Springer Series in Statistics. New York, NY: Springer, 2009. isbn: 978-0-387-79051-0. doi: 10.1007/b13794.

[16]

Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science. 2nd ed. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge: Cambridge University Press, 2026.

Record · ID 668060 · SHA-256 54c39d40243deaff
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.