Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexity

June 25, 2026 Abstract We study the fundamental problem of learning a high-dimensional Gaussian truncated to an unknown halfspace. Lee, Mehrotra and Zampetakis (FOCS’24) recently obtained the first polynomial time algorithm for this problem, but their resulting sample and time complexity bounds are not optimal. Under non-trivial truncation, for any target accuracy 𝜀 > 0 and ˜ 2 /𝜀2 ) samples and learns the dimension 𝑑 we give an efficient algorithm that uses 𝑛 = 𝑂(𝑑 underlying Gaussian to error 𝜀 in total variation distance. Our algorithm is also fast: its runtime is dominated by the cost of computing the empirical covariance matrix. Both our sample and time complexity are optimal in terms of 𝑑 and 𝜀 even without truncation: in this regard, we can learn a Gaussian under halfspace truncation for free. The key ingredient behind our result is a novel reinterpretation of the low-degree moments of the truncated Gaussian in terms of a relative truncation parameter. This relative truncation parameter uniquely determines the parameters of the untruncated Gaussian and enables direct parameter recovery. This reinterpretation allows us to circumvent the time intensive projected stochastic gradient descent procedure that is widely used in learning under truncation.

Contents 1

Introduction 1.1 Main contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 2 5

Technical overview

Preliminaries 3.1 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Inverse Mill’s ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Gaussian concentration and anti-concentration . . . . . . . . . . . . . . . . . . . . . . 3.4 Concentration bounds for sub-Gaussian random variables . . . . . . . . . . . . . . .

12 12 12 19 19

Properities of halfspace truncated Gaussians 4.1 The first three moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Sub-Gaussianity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Preconditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21 22 23 24

Proof of the main result 5.1 Recovering the direction of Σ𝑤 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Relative truncation parameter recovery . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Relative truncation parameter recovery implies parameter recovery . . . . . . . . . . 5.4 Proof of Theorem 5.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

26 28 31 35 42

Robustness 50 6.1 Preliminaries on robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 6.2 Learning parameters under halfspace truncation and robustness . . . . . . . . . . . . 51

Extension to intersections of two orthogonal halfspaces 7.1 Moments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Relative truncation parameter estimation . . . . . . . . . . . . . . . . . . . . . . . . . 7.3 Proof of Theorem 7.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57 58 59 62

Acknowledgments

A Deferred proofs from the main text A.1 Cumulants of the one-dimensional truncated distribution . . . . . . . . . . . . . . . . A.2 Matrix and vector calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 Concentration results and error analysis for sample moments . . . . . . . . . . . . .

65 65 74 76

B Computations

C Determining 𝑤

Introduction

We study statistical estimation under truncation. In this setting, data is only observed when it falls in some survival set. There is a long line of work on learning based on truncated or censored samples; dating back to at least the work of Daniel Bernoulli [Ber60]. Bernoulli studied the effectiveness of the smallpox vaccination and encountered the problem of missing data from people who died of smallpox; thus he only had access to ‘truncated’ data. Since then, statistical estimation from truncated or censored samples has been studied by numerous scientists, including [Gal98], [Pea02], [PL08], [Lee14], [Fis31], and [BS37]; we refer the reader to the book [Coh91] for a more comprehensive treatment. Importantly however, all these works either study the onedimensional setting or utilize methods that are not computationally efficient in higher dimensions. In the last decade, the following setting has received renewed attention: Given samples from a multivariate Gaussian with unknown parameters truncated to some set 𝑆, learn the Gaussian in total variation (TV) distance. We note that for this, it is enough to learn the mean and covariance up to small error. [DGTZ18] showed that, given oracle access to the truncation set, the parameters can ˜ 2 /𝜀2 ). This be learned in TV-distance up to error 𝜀 in polynomial time and in sample complexity 𝑂(𝑑 sample complexity is optimal up to logarithmic factors since already for learning the covariance of a Gaussian, even without truncation, 𝑛 = Ω(𝑑2 /𝜀2 ) samples are information-theoretically necessary (cf. [CZZ10]). If the truncation set is unknown and arbitrary, [DGTZ18] also showed that learning up to arbitrarily small error in TV-distance is impossible. [KTZ19] studied the more tractable setting where the truncation set is unknown but belongs to some well-behaved class of possible truncation sets. They are able to learn the parameters of a Gaussian with diagonal covariance matrix given a bound on the Gaussian surface area of the truncation set. If the Gaussian surface area is at most Γ, then 2 they can learn the Gaussian up to TV-error 𝜀 with time and sample complexity 𝑑 poly(1/𝜀)·Γ . Without further assumptions on the truncation set, [DKPZ24] showed that the exponential dependence on 1/𝜀 is necessary in the SQ model. More recently, [LMZ24] extended the result of [KTZ19] to general Gaussians. Furthermore, they showed that for two simple classes of truncation sets, namely, axis-aligned rectangles and halfspaces, one can even learn in time and sample complexity poly(𝑑/𝜀). This is the first algorithm that achieves fully polynomial runtime for learning a Gaussian under truncation to some unknown survival set. In this work, we continue the investigation of learning from truncated samples for ‘simple’ survival sets. We focus on the case of halfspace truncation. For halfspaces, the result by [LMZ24] needs at least Ω(𝑑3 ) samples and might not be practical. This is because an important step of their algorithm is based on projected stochastic gradient descent, for which they use the ellipsoid method for the projection step. It is important to note that the projection is not to the survival set, but instead to some set in the parameter space. Therefore, even for simple classes it is unclear whether this step can be made fast. Concretely, a direct application of the ellipsoid method (without using any specifics about the problem) yields a lower bound of Ω(𝑑 10 ) on the runtime of their algorithm ([LMZ24, Algorithm 3]).1 Thus, their paper leaves open the following natural question for halfspace truncation: Are there fast algorithms with optimal sample complexity, matching the untruncated setting? 1 Algorithm 3 in [LMZ24] runs the ellipsoid method on a problem in 𝑚 = Θ(𝑑 2 ) dimensions. The ellipsoid method needs Ω(𝑚 2 ) iterations to terminate and each iteration takes Ω(𝑚 2 ) operations (as we need to update the 𝑚 × 𝑚 matrix describing the ellipsoid). Thus, one run of the ellipsoid method needs Ω((𝑑2 )4 ) = Ω(𝑑 8 ) time. Finally, they run the ellipsoid method Ω(𝑑2 ) times, giving a runtime lower bound of Ω(𝑑10 ).

Naturally, the best one could hope for is to match the results of the untruncated setting — which has been studied for decades and is well-understood. There, the best known algorithm is to compute the sample mean and the sample covariance; see e.g. [CZZ10]. To achieve error 𝜀, one needs 𝑁 = Θ(𝑑2 /𝜀2 ) many samples. The time needed to compute this estimator is asymptotically the same as the time needed to multiply a 𝑑 × 𝑁 matrix with its transpose, which we denote by 𝑇(𝑁 , 𝑑). Currently, the best-known runtime for this is 𝑂(𝑑3.250035 /𝜀2 ) as a result of recent advances in fast matrix multiplication due to [ADVW+ 25].

1.1

Main contributions

In this paper, we answer the above question in the affirmative by designing an algorithm that acheives the following guarantees. Theorem 1.1 (Informal, see Theorem 5.1). There is an algorithm with the following guarantee: The algorithm takes as input 𝑁 samples from a 𝑑-dimensional Gaussian 𝒩 𝜇, Σ truncated to some halfspace of mass 𝛼 as well as a lower bound 𝛼0 on 𝛼. If 𝑁 = 𝑑 2 · poly(1/𝜀, log(1/𝛼0 )), the algorithm outputs 𝜇ˆ and b Σ such that, with probability at least 0.99, we have:

ˆ b 𝑑TV 𝒩 𝜇, Σ , 𝒩 𝜇, Σ

⩽ 𝜀.

The time complexity of the algorithm is 𝑂(𝑇(𝑁 , 𝑑)) ⩽ 𝑂(𝑑3.250035 ) · poly(1/𝜀, log(1/𝛼 0 )). Remark. If 𝛼 ⩽ 0.99, then our algorithm needs only 𝑁 = 𝑑2 /𝜀2 · polylog(1/𝛼0 ) samples; otherwise it needs 𝑁 = 𝑂(𝑑2 /𝜀4 ) many samples. The algorithm is the same for both cases. We also note that the ‘0.99’ both in the condition on 𝛼 and in the success probability can be replaced by arbitrary constants 𝑐 < 1. Our algorithm is the first algorithm that achieves optimal sample complexity dependence on 𝑑. The runtime of our algorithm matches the runtime of computing the sample covariance in the untruncated setting and hence is also optimal in terms of 𝑑. Furthermore, the dependence of the sample and time complexity on 𝜀 are optimal whenever there is non-trivial truncation (meaning, 𝛼 ⩽ 0.99). Regarding our dependence of the sample complexity on 𝛼 0 , we remark that all prior works, including inefficient algorithms, also have a dependence on 𝛼0 . In fact, under the assumption that the set is unknown, all previous efficient algorithms incur either an exponential dependence on 1/𝛼0 or work under the assumption that 𝛼 0 ⩾ Ω(1) (information-theoretically, [KTZ19] showed that one can achieve polynomial dependence). Interestingly, in the case where the set is known, more recently, [KKT25] showed that one can achieve polylog(1/𝛼 0 ) dependence on 𝛼 0 for the sample complexity (their runtime dependence on 1/𝛼0 is polynomial). In the case of halfspaces, our algorithm matches this guarantee even when the set is unknown. This is the first efficient algorithm to get better than exponential dependence on 1/𝛼 0 when the set is unknown. An overview of the known algorithms for learning a Gaussian under halfspace truncation can be found in Table 1. We note that all of these cited results are more general than halfspaces; in order to be able to compare them to our result, the results mentioned in the table are specialized to the case of halfspace truncation.

Table 1: Overview of efficient algorithms for learning a Gaussian under halfspace truncation Truncation set (known/unknown)

Underlying distribution

Sample complexity (in 𝑑 and 𝜀)

[DGTZ18]

Known

𝒩 𝜇, Σ

˜ 2 /𝜀2 ) 𝑂(𝑑

[KTZ19]

Unknown

[LMZ24] [KKT25]

Unknown Known

𝒩 𝜇, Σ for Σ diagonal 𝒩 𝜇, Σ 𝒩 𝜇, Σ

This work

Unknown

𝒩 𝜇, Σ

Paper

𝑑poly(1/𝜀)

Dependence on 𝛼 0 Assumption: 𝛼0 ⩾ Ω(1) exp(1/𝛼0 )

poly(𝑑, 1/𝜀) exp(1/𝛼0 ) 2 2 𝑂(𝑑 /𝜀 ) poly log(1/𝛼0 ) 𝑂(𝑑 2 /𝜀2 ) if 𝛼 ⩽ 0.99 poly log(1/𝛼0 ) 𝑂(𝑑 2 /𝜀4 ) otherwise

High-level overview of our techniques and comparison to prior work. At a high level, our algorithm is based on the method of moments. We use the first three moments of the distribution 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 2 to estimate a relative truncation parameter. Our key insight is that this relative truncation parameter fully characterizes how truncation changes the moments compared to the untruncated distribution. This enables us to directly estimate the parameters 𝜇 and Σ of the Gaussian from the moments themselves. We now compare our approach to [LMZ24], who gave the first efficient algorithm for learning the parameters of a Gaussian under halfspace truncation. They use the method of moments to first learn the truncation set. Then, they apply projected stochastic gradient descent to estimate the parameters. At a high level, their proof can be divided into the following four steps: Σ𝑤 1. First, they use the third central moment to find a vector close to 𝑢 = ∥Σ𝑤∥ . 3 2. Then, they compute a vector close to the vector 𝑤 of the halfspace. 3. Next, they find a value close to 𝜏 by picking the smallest threshold such that the resulting set contains all the samples. 4. Finally, given the estimate for the truncation set, they use their general projected stochastic gradient descent procedure to learn the parameters.

While our approach is also based on the method of moments, we use the moments for a different purpose: rather than estimating the truncation set as in [LMZ24], we use the moments to directly estimate the parameters of the underlying Gaussian. By using the method of moments for direct parameter recovery, we are able to get a fast algorithm: crucially, we can avoid the time-consuming projected stochastic gradient step (step 4) and achieve optimal runtime in terms of 𝑑. 2 Here, 𝒩 (𝜇, Σ)| ⟨𝑤,𝑥⟩⩽ 𝜏 denotes the Gaussian distribution 𝒩 (𝜇, Σ) truncated to the halfspace {𝑥 : ⟨𝑤, 𝑥⟩ ⩽ 𝜏} for some unit vector 𝑤. 3 In their paper, [LMZ24] claim that the third central moment 𝑀 is proportional to 3

𝑀3 B E[(𝑋 − E[𝑋])⊗3 ] ∝ 𝑤 ⊗3 . They use this to determine 𝑤. However, there is small gap in this argument: As we show later, the third central moment is in fact proportional to (Σ𝑤)⊗3 (and not 𝑤 ⊗3 ); see Lemma 4.2. Since our goal is to estimate Σ, the third central moment therefore cannot be directly used to estimate 𝑤. Instead, one can compute a vector 𝑢 proportional to Σ𝑤 using the third central moment. Then, one can solve the following linear system involving the second central moment 𝑀2 to recover 𝑤. Namely, it holds that 𝑀2−1 𝑢 ∝ 𝑤. We include a proof of this for completeness in Appendix C.

As a first step in our algorithm, we also estimate a vector 𝑢ˆ as [LMZ24] do in step 1. However, we then do not use it to estimate the vector 𝑤. Instead, the key insight in our algorithm is that the first non-central moment and the second and third central moments all share an important structural property: They are equal to the untruncated moments plus an additional term involving only the following vector Σ𝑤/ Σ1/2 𝑤 and the same relative truncation parameter 𝛾. Importantly, we show that the first three moments uniquely determine 𝛾 and ∥Σ𝑤∥/∥Σ1/2 𝑤∥. This allows us to use the moments to get an estimate for 𝛾 and ∥Σ𝑤∥/∥Σ1/2 𝑤∥. Combining this with our first step enables us to directly compute estimates for the parameters. Together, these ideas yield a simple and fast algorithm for estimating the parameters of a halfspace truncated Gaussian. Regarding the sample complexity, [LMZ24] require a high-accuracy estimate of the third moment in their step 2. This step requires at least Ω(𝑑 3 ) samples. In contrast, for our algorithm, when computing a vector close to 𝑢 = Σ𝑤/∥Σ𝑤∥, we use a random contraction of the third central moment and then use a spectral decomposition of the resulting matrix, which allows us to get optimal sample complexity. Robustly learning the parameters of a Gaussian under halfspace truncation. In contrast to the algorithm by [LMZ24], we utilize the moments of the truncated distribution to directly estimate the parameters. Motivated by this, we introduce the problem of robust estimation of a truncated Gaussian. We consider the following corruption model: First, samples from a halfspace truncated Gaussian 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 are drawn. Then, an adversary is allowed to modify an 𝜂-fraction of the samples arbitrarily; in particular the samples do not need to lie in the set {𝑥 ∈ R𝑑 | ⟨𝑤, 𝑥⟩ ⩽ 𝜏} anymore. The adversary is computationally unbounded, knows the truncation set as well as the parameters and can adaptively change the samples after they have been drawn. We believe that this corruption model, which is analogous to the strong contamination model in robust statistics [DK23], might be of independent interest for learning under truncation. Given that our algorithm only uses the first three moments, we can robustify it by using robust estimates for the moments instead of relying on the sample moments. Our robust algorithm uses a black-box robust estimator due to [KSS18] and the fact that the truncated distribution is sub-Gaussian. We prove the following result. To simplify the exposition, we state our result for constant 𝛼 and for Σ ⪰ Ω(1) · 𝐼 𝑑 . The full result can be found in Theorem 6.8 in Section 6. Theorem 1.2 (Informal, see Theorem 6.8). Consider the truncated distribution 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 , where the mass 𝛼 of the survival set satisifies Ω(1) ⩽ 𝛼 ⩽ 0.99. Assume Σ ⪰ Ω(1) · 𝐼 𝑑 . Let 0 < 𝛿 < 1/2 be an arbitrary constant. Let 𝜂 > 0 be the corruption parameter and assume that 𝜂 ⩽ 1/𝑑 1−2𝛿 . For 𝑛 = poly(𝑑), given an 𝜂-corruption 𝑋1 , . . . , 𝑋𝑛 of 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽ 𝜏} , we can compute, in polynomial time in 𝑑 and with probability at least 0.99, estimators 𝜇ˆ and b Σ such that Σ−1/2 (𝜇ˆ − 𝜇) ⩽ 𝑂 𝛿

√

and ∥Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑 ∥𝐹 ⩽ 𝑂 𝛿

𝑑 · 𝜂1/2−𝛿

√

𝑑 · 𝜂1/2−𝛿 .

While our algorithm can only tolerate a very small amount of corruption, we believe that already this is non-trivial because we are not trying to estimate the moments of the distribution to which we have (sample) access to (which would be the typical goal in robust statistics) but instead of the underlying untruncated distribution. In particular, the adversary can place samples outside the survival set. Thus, it is unclear if one can even estimate the truncation set. We remark that previous work in truncated statistics cannot (easily) be made robust. For example, even if one 4

could learn the truncation set under adversarial corruption, it is unclear whether the algorithm of [LMZ24] can be made to succeed, since it is not clear how to robustify their final step involving projected stochastic gradient descent. That being said, we believe that it is likely that the amount of corruption and the error rate can be improved. We leave it as an open question for future work whether using a more specialized robust estimator for the moments can improve Theorem 1.2. We believe that our result can act as a starting point for further investigation of robust estimation given only truncated samples. In particular, our result also leaves open what happens in weaker corruption models (e.g., a model where only oblivious corruptions are allowed). Extensions to other truncation sets. Finally, we show that our techniques can be used beyond halfspace truncation. In Theorem 7.1, we show that using similar ideas we can also learn the mean of an identity-covariance Gaussian 𝒩 𝜇, 𝐼 𝑑 truncated to an intersection of two orthogonal halfspaces.

Similar to our main result, we can achieve sample complexity 𝑂

𝑑2 𝜀2

· polylog(1/𝛼 0 ) + 𝑂

𝑑2 𝜀4

. Our

algorithm for this case also works using only the first three moments of the truncated distribution. At a very high level, instead of recovering one relative truncation parameter, for the case of truncation by an intersection of two orthogonal halfspaces, we recover two relative truncation parameters; one for each truncation direction. For details, we refer to Section 7. While we only prove the case of truncation by an intersection of 2 orthogonal halfspaces, we believe that our proof can be extended to an intersection of 𝑘 (pairwise) orthogonal halfspaces, yielding a sample complexity of 𝑂

1.2

𝑑2 ·poly(𝑘) 𝜀2

· polylog(1/𝛼 0 ) + 𝑂

𝑑2 ·poly(𝑘) 𝜀4

; see Remark 7.2.

Related work

Method of Moments. Solving problems in statistical estimation using the Method of Moments (MoM) has a long history. It can be dated back to the famous statistician Karl Pearson [Pea94]. This method has led to several recent advances in high-dimensional parameter estimation tasks, including those involving learning mixture models [MV10, LL22] and independent component analysis (ICA) [BRV13, BCV14]. In this work we adopt the MoM for learning the parameters of a halfspace truncated Gaussian. Truncated Learning. Beyond the results described earlier, estimation under truncation has seen applications to differentially private statistical estimation [ZZ25]. There has also been recent work on mean testing under truncation [CGWY25] and on detecting truncation [DNS23, DLNS24]. A recent work [LMZ26] studies learning under positive and imperfectly labeled samples and extends the general learning algorithm of [LMZ24] for learning under general truncation sets. Robust Estimation. Robust parameter estimation has a long history in statistics [Hub92, HRVF04]. Recent algorithmic breakthroughs by [LRV16, DKK+ 19] have led to the new subfield of algorithmic robust statistics, which primarily deals with parameter estimation in the presence of adversarial outliers (see the textbook [DK23] for a comprehensive treatment). A very useful method that has emerged for robust statistical estimation is the powerful Sum-of-Squares hierarchy of semi-definite programs [BS16] that often efficiently achieves the best possible statistical guarantees for a large range of robust estimation tasks (see e.g. [KSS18]). In this work, we use Sum-of-Squares based robust moment estimation algorithms as a black-box to solve the robust version of the learning under truncation problem that we introduce. 5

Technical overview

In order to prove our main result Theorem 1.1, we want to use the moments of the truncated distribution to estimate the parameters. Consider the distribution 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽𝜏} , where 𝑤 is a unit vector. Recall that our goal is to find estimates 𝜇ˆ and b Σ such that

ˆ b Σ 𝑑TV 𝒩 𝜇, Σ , 𝒩 𝜇,

⩽ 𝜀.

To show closeness in total variation distance as above, it is enough to show ∥Σ−1/2 (𝜇ˆ − 𝜇)∥ ⩽ 𝜀

∥Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑 ∥𝐹 ⩽ 𝜀,

and

see e.g. [DKK+ 19]. To achieve this, on a high level, we first determine the moments of the truncated distribution. Then, we observe that the moments are determined by 𝜇 and Σ as well as a relative truncation parameter 𝛾 and a certain vector. We show that if we can determine this parameter and vector, then based on the first non-central and the second central moment of the truncated distribution, we would be able to determine the 𝜇 and Σ. Finally, we show how to determine 𝛾 and this vector based on the second and third central moment. The moments of the truncated distribution. As a warm up, we start by considering the truncated distribution 𝑌 ∼ 𝒩 (0, 𝐼 𝑑 )|{𝑒 ⊤ 𝑥 ⩽ 𝛾} . Intuitively, the moments of this distribution should be the ones 1

of the standard Gaussian, but shifted in the direction of 𝑒1⊗𝑘 since 𝑒1 is the direction in which the distribution is truncated.4 For example, E[𝑌] = 𝑐 1 𝑒1 since in all but the first coordinate, 𝑌 is a standard Gaussian. More precisely, 𝑐 1 = 𝜅1 (𝛾) is the first cumulant5 of a one-dimensional standard Gaussian truncated to (−∞, 𝛾]. More generally, higher moments can be related to the higher order cumulants of this one-dimensional distribution. To compute the mean (and more generally the moments) of a general truncated Gaussian 𝑋 ∼ 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽𝜏} , our goal is to relate it to the mean (and the corresponding moments) of 𝑌. To this end, consider the linear transformation 𝑦 ↦→ 𝜓(𝑦) = 𝜇 + Σ1/2𝑈 𝑦, where 𝑈 is a rotation matrix. For any 𝑈, this map transforms a standard Gaussian 𝒩 (0, 𝐼 𝑑 ) to 𝒩 𝜇, Σ . We want to pick 𝑈 and 𝛾 in a way such that the truncation set of 𝜓(𝑌) is equal to the truncation set of 𝑋. This will show that 𝜓(𝑌) and 𝑋 have the same distribution, thus allowing us to express the moments of 𝑋 in terms of the moments of 𝑌. First, we show how to pick 𝑈 such that the direction of truncation of 𝜓(𝑌) is equal to 𝑤. Since 𝑌 is truncated to {𝑥 | 𝑒1⊤ 𝑥 ⩽ 𝛾}, 𝜓(𝑌) is truncated to the set {𝜓(𝑦) | 𝑦 such that 𝑒1⊤ 𝑦 ⩽ 𝛾} = {𝑥 | 𝑒1⊤ 𝜓−1 (𝑥) ⩽ 𝛾} = {𝑥 | 𝑒1⊤𝑈 −1 Σ−1/2 (𝑥 − 𝜇) ⩽ 𝛾} = {𝑥 | 𝑒1⊤𝑈 ⊤ Σ−1/2 𝑥 ⩽ 𝛾 + 𝑒1⊤𝑈 ⊤ Σ−1/2 𝜇} 4 We remark that this is only true since the coordinates are independent. As we will see later, for 𝒩 (𝜇, Σ)|

moments would be shifted in the direction of (Σ𝑒1 )⊗𝑘 and not 𝑒1⊗𝑘 . 5 The first and second cumulants are just the mean and the covariance. moment. For more details, see Definition 3.1.

(2.1) {𝑒1⊤ 𝑥 ⩽ 𝛾} , the

The third cumulant is the third central

Thus, we want to pick 𝑈 such that Σ−1/2𝑈 𝑒1 ∝ 𝑤 ⇐⇒ 𝑈 𝑒1 ∝ Σ1/2 𝑤. As a rotation matrix, 𝑈 preserves the norm and hence 𝑈 𝑒1 =

Σ1/2 𝑤

(2.2)

In order to determine the value of 𝛾 needed such that 𝜓(𝑌) has the same distribution as 𝑋, we want to write (2.1) in the form {𝑥 | 𝑤 ⊤ 𝑥 ⩽ 𝜏}. By (2.2), we can write the truncation set for 𝑌 as {𝜓(𝑦) | 𝑦 such that 𝑒1⊤ 𝑦 ⩽ 𝛾} = {𝑥 | 𝑤 ⊤ 𝑥 ⩽ ∥Σ1/2 𝑤∥𝛾 + 𝑤 ⊤ 𝜇}. Hence, we need that 𝛾=

𝜏 − 𝑤⊤𝜇 ∥Σ1/2 𝑤∥

(2.3)

For 𝑈 as in (2.2) and 𝛾 as in (2.3), 𝜓(𝑌) ∼ 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽ 𝜏} . In particular, we can now compute E[𝑋] using this transformation 𝜓 and E[𝑌]:

E[𝑋] = 𝜇 + Σ1/2𝑈 E[𝑌] = 𝜇 + 𝜅1 (𝛾) · Σ1/2𝑈 𝑒1 = 𝜇 + 𝜅1 (𝛾) ·

Σ𝑤 . ∥Σ1/2 𝑤∥

Analogously we can also compute the higher moments of 𝑌 and use them, together with the the transformation 𝜓 to compute the moments of 𝑋. Namely, we have that

E[𝑋] = 𝜇 + 𝜅1 (𝛾) ·

Σ𝑤

Σ𝑤 E[(𝑋 − E[𝑋]) ] = Σ − (1 − 𝜅2 (𝛾)) · ∥Σ1/2 𝑤∥ ⊗2

Σ𝑤 E[(𝑋 − E[𝑋]) ] = 𝜅3 (𝛾) · ∥Σ1/2 𝑤∥ ⊗3

(2.4)

Σ1/2 𝑤

⊗3

⊗2

(2.5) (2.6)

For more details on this computation, see Lemma 4.2. In particular, these moments are determined by the parameters of the untruncated Gaussian, the vector Σ𝑤/∥Σ1/2 𝑤∥ and 𝛾 defined as in (2.3). We will refer to 𝛾 = 𝛾(𝜇, Σ, 𝑤, 𝜏) as the relative truncation parameter of the distribution 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽ 𝜏} . In particular, since the third moment is rank 1, if we would know the relative truncation parameter, then we could use (2.6) to estimate Σ𝑤/∥Σ1/2 𝑤∥. Once we determined this vector, we could then use (2.4) and (2.5) respectively to estimate 𝜇 and Σ. To summarize, since we can approximate the moments of 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽𝜏} by the sample moments, the main question is how to estimate the relative truncation parameter defined in (2.3). First three moments determine the relative truncation parameter. Before showing how we estimate the relative truncation parameter 𝛾, we want to establish that the information in the first three moments is in fact enough to uniquely determine 𝛾. For this, consider two truncated Gaussians 𝒩 (𝜇1 , Σ1 )|{𝑤 ⊤ 𝑥 ⩽ 𝜏1 } and 𝒩 (𝜇2 , Σ2 )|{𝑤 ⊤ 𝑥 ⩽ 𝜏2 } . We want to show that if the first non-central 2 1 moment as well as the second and third central moments of these two distributions are identical (cf. (2.4), (2.5) and (2.6)), then we need to have 𝛾1 = 𝛾2 . This will show that it is information-theoretically possible to determine the relative truncation parameter based only on these low-degree moments. 7

To formally show that the first three moments of the two distributions 𝒩 (𝜇1 , Σ1 )|{𝑤 ⊤ 𝑥 ⩽ 𝜏1 } and 1 𝒩 (𝜇2 , Σ2 )|{𝑤 ⊤ 𝑥 ⩽ 𝜏2 } being equal implies that 𝛾1 = 𝛾2 , consider first the second moment. By assump2 tion, we have that ⊗2

⊗2

= Σ2 − (1 − 𝜅2 (𝛾2 )) ·

Σ1 − (1 − 𝜅2 (𝛾1 )) ·

By multiplying from the left by 𝑤 1⊤ and from the right by 𝑤2 as well as identifying 𝑎 ⊗2 with 𝑎𝑎 ⊤ , we get that 𝑤 1⊤ Σ1 𝑤2 −

(1 − 𝜅 2 (𝛾1 )) 1/2 Σ1 𝑤 1

· 𝑤 1⊤ Σ1 𝑤 1 · 𝑤 1⊤ Σ1 𝑤 2 = 𝑤1⊤ Σ2 𝑤 2 −

(1 − 𝜅 2 (𝛾2 )) 1/2 Σ2 𝑤 2

· 𝑤 1⊤ Σ2 𝑤2 · 𝑤 2⊤ Σ2 𝑤2 .

1/2

Using that 𝑤 ⊤ Σ 𝑤 = Σ𝑖 𝑤 𝑖 , the above simplifies to 𝑖 𝑖 𝑖 𝜅2 (𝛾1 ) · 𝑤 1⊤ Σ1 𝑤 2 = 𝜅2 (𝛾2 ) · 𝑤 1⊤ Σ2 𝑤2 .

(2.7)

Our goal now is to relate the two numbers 𝑤1⊤ Σ1 𝑤 2 and 𝑤1⊤ Σ2 𝑤 2 . We do so using the third central moment (cf. (2.6)). Since we assumed that the third moments also agrees, we have that 𝜅 3 (𝛾1 ) ·

! ⊗3

Σ1 𝑤 1

= 𝜅3 (𝛾2 ) ·

1/2

∥Σ1 𝑤1 ∥

Σ2 𝑤 2

! ⊗3

1/2

∥Σ2 𝑤 2 ∥

This is a rank-1 tensor, so by considering the diagonal elements, we get that also the corresponding vectors need to agree, i.e., that Σ1 𝑤 1 1/2

Σ1 𝑤 1

𝜅3 (𝛾2 ) = 𝜅3 (𝛾1 )

1/3 ·

Σ2 𝑤 2 1/2

Σ2 𝑤 2

By multiplying by 𝑤2⊤ from the left, we can get an expression for 𝑤 1⊤ Σ1 𝑤 2 = 𝑤2⊤ Σ1 𝑤1 . By multiplying instead by 𝑤1⊤ from the left, we can get an expression for 𝑤1⊤ Σ2 𝑤 2 . Using again that 1/2

𝑤⊤ Σ 𝑤 = Σ𝑖 𝑤 𝑖 𝑖 𝑖 𝑖

and combining these we get that

𝜅 3 (𝛾1 ) 𝜅 3 (𝛾2 )

1/3

𝑤 2⊤ Σ1 𝑤 1 =

1/2 Σ1 𝑤 1

1/2 Σ1 𝑤 2

𝜅 3 (𝛾2 ) = 𝜅3 (𝛾1 )

1/3

𝑤1⊤ Σ2 𝑤 2 .

We can now use this equality in (2.7) to get 𝜅 2 (𝛾1 )

𝜅2 (𝛾2 ) = 𝜅 3 (𝛾1 )2/3 𝜅3 (𝛾2 )2/3

𝜅3 (𝛾1 )

𝜅3 (𝛾2 ) = . 𝜅 2 (𝛾1 )3/2 𝜅2 (𝛾2 )3/2

⇐⇒

We now observe that the function 𝜅3 (𝛾)/(𝜅2 (𝛾))3/2 is the skewness of the one-dimensional standard Gaussian truncated to (−∞, 𝛾]. We denote this function by skew(𝛾). Thus, we have that if the first three moments of 𝒩 (𝜇1 , Σ1 )|{𝑤 ⊤ 𝑥 ⩽ 𝜏1 } and 𝒩 (𝜇2 , Σ2 )|{𝑤 ⊤ 𝑥 ⩽𝜏2 } agree, then we need to have 2

skew(𝛾1 ) = skew(𝛾2 ). The key argument in this proof now is that for our specific case this function skew is monotonic (cf. Lemma 3.17) and thus the above implies that also 𝛾1 = 𝛾2 . 8

Estimating the relative truncation parameter algorithmically. We now want to show how we can use the proof above to determine the parameters 𝜇 and Σ given samples from the distribution 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽ 𝜏} . We showed above that the first three moments uniquely determine 𝛾 =

𝜏−𝑤 ⊤ 𝜇

. ∥Σ1/2 𝑤 ∥ The above also lays down a clear path: first, estimate 𝛾 and then determine the parameters 𝜇 and Σ. To do this, we first compute skew(𝛾) based on the moments. Then, using the fact that skew is monotonic, we can invert this one-dimensional function and recover the relative truncation parameter. This then allows us to use (2.4) and (2.5) to recover the parameters. It thus remains to show how to determine 𝛾 algorithmically based on the first three moments. Define the vector Σ𝑤 𝑢 ∗ = 𝜅3 (𝛾)1/3 · . Σ1/2 𝑤 The third central moment equals 𝑢 ∗⊗3 and hence, we can use it to obtain a unit vector 𝑢˜ in the direction of 𝑢 ∗ , that is, 𝑢˜ = (Σ𝑤)/∥Σ𝑤∥. By computing the inner product of the third central moment with the tensor 𝑢˜ ⊗3 , we can even get the norm of 𝑢 ∗ since ˜ 𝑢 ∗ ⟩3 = 𝜅3 (𝛾) · 𝑢˜ ⊗3 , E[(𝑋 − E[𝑋])⊗3 ] = ⟨𝑢,

∥Σ𝑤∥3 . ∥Σ1/2 𝑤∥3

(2.8)

We now want to use the second central moment to compute a similar expression involving the same norms but 𝜅 2 (𝛾) instead of 𝜅 3 (𝛾). Combining this with the above then allows us to compute ˜ where 𝑀2 is the second central skew(𝛾). It turns out that we can do so by computing 𝑢˜ ⊤ 𝑀2−1 𝑢, moment. Since 𝑀2 differs from Σ only by a rank-1 term, we can use the Sherman-Morrison formula to compute its inverse Σ−1 ΣΣ𝑤 · 𝑤 Σ Σ−1 ∥ 1/2 𝑤 ∥ ∥Σ1/2 𝑤 ∥ ⊤

𝑀2−1 = Σ−1 + (1 − 𝜅2 (𝛾)) ·

𝑤⊤Σ

1 − (1 − 𝜅2 (𝛾)) ·

∥Σ1/2 𝑤 ∥

𝑤𝑤 ⊤

= Σ−1 + (1 − 𝜅2 (𝛾)) ·

Σ1/2 𝑤

Σ−1 ΣΣ𝑤 ∥ 1/2 𝑤 ∥

· 𝜅 2 (𝛾)

˜ we get that Now, left-multiplying by 𝑢˜ ⊤ and right-multiplying by 𝑢, 𝑢˜ ⊤ 𝑀2−1 𝑢˜ = =

𝑤 ⊤ ΣΣ−1 Σ𝑤 ∥Σ𝑤∥2 Σ1/2 𝑤 ∥Σ𝑤∥2

+ (1 − 𝜅2 (𝛾)) ·

𝑤 ⊤ Σ𝑤𝑤 ⊤ Σ𝑤 ∥Σ𝑤∥2 · Σ1/2 𝑤 Σ1/2 𝑤

· 𝜅 2 (𝛾)

∥Σ𝑤∥2 · 𝜅 2 (𝛾)

Σ1/2 𝑤 1 = . · 𝜅 2 (𝛾) ∥Σ𝑤∥2

(2.9)

Now, combining (2.9) with (2.8), we get that 𝑢˜ ⊗3 , E[(𝑋 − E[𝑋])⊗3 ] · 𝑢˜ ⊤ 𝑀2−1 𝑢˜

3/2

𝜅 3 (𝛾) 𝜅2 (𝛾)3/2

= skew(𝛾).

We remark that we can compute both quantities on the left-hand side of the above equation using 𝑢˜ (that we computed using the third moment) and the moments. Hence, we can compute skew(𝛾). 9

We now crucially use the fact that skew is monotonic, which allows us to recover 𝛾. As already discussed before, this allows us to recover the parameters: we can use the third moment to recover the vector (Σ𝑤)/ Σ1/2 𝑤 (which we do implicitly in our algorithm) and then use the first and second moment to recover 𝜇 and Σ. Our algorithm is summarized in Algorithm 1. Algorithm 1: Parameter estimation of a halfspace-truncated Gaussian based on first three moments Input: First non-central moment 𝜇t , second central moment 𝑀2 and third central moment 𝑀3 of the distribution 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽ 𝜏} Output: Mean 𝜇 and covariance Σ. 1. Compute 𝑢˜ from the third central moment 𝑀3 . 2. Compute the relative truncation parameter: 𝛾 = skew−1 𝜅 (𝛾)

3. Compute the mean of the Gaussian: 𝜇 = 𝜇t − √ 1

𝑢˜ ⊤ 𝑀2−1 𝑢˜

𝜅2 (𝛾)

4. Compute the covariance of the Gaussian: Σ = 𝑀2 +

𝑢˜ ⊤ 𝑀2−1 𝑢˜

1 −1 𝜅2 (𝛾)

−1/2

3/2

· 𝑢˜ ⊗3 , 𝑀3

˜ 𝑢.

· 𝑢˜ ⊤ 𝑀2−1 𝑢˜

−1

𝑢˜ 𝑢˜ ⊤ .

We emphasize here that our particular choice of expressing both the second and third moment in (2.5) and (2.6) in terms of the same relative truncation parameter 𝛾 is precisely what enables direct parameter recovery without having to learn the truncation set — deviating from existing approaches in the truncated statistics literature [DGTZ18, LMZ24]. An important upshot is that this also enables us to design simple and fast algorithms, making learning under truncation potentially practical. Sample complexity. In Section 5, we will show how to implement the above strategy when only having access to the sample moments. We now give a brief overview how the errors in estimating the sample moments influence the sample complexity of our algorithm. The main bottlenecks for the sample complexity are the approximation 𝑢ˆ to 𝑢˜ and the approxb 2 to 𝑀2 . We need to ensure that the expressions in (2.8) and (2.9) are close to their true imation 𝑀 values. If we have sufficiently good approximations to these, then with 𝑂 𝛼 (𝑑 2 /𝜀2 ) extra samples we can get good approximations to the mean and covariance of the truncated distribution and use the strategy described above to get estimators for 𝜇 and Σ with error 𝜀. We show in Section 5 that it is sufficient to get estimates 𝑢ˆ −

𝑢∗ b 2 𝑀 −1/2 − 𝐼 𝑑 ∥𝐹 ⩽ 𝑂 𝛼 (𝜀). ⩽ 𝑂 𝛼 (𝜀) and ∥𝑀2−1/2 𝑀 2 ∥𝑢 ∗ ∥ 2

(2.10)

We note that our algorithm contains a preconditioning step that allows us to reduce our problem b 2 without incurring an error that to a well-conditioned one. This allows us to invert the matrix 𝑀 depends on the condition number of Σ. b 2 as in (2.10) is possible with 𝑂 𝛼 (𝑑2 /𝜀2 ) samples by standard Achieving the estimate of 𝑀 ˜ we could estimate it by estimating the diagonal concentration bounds. For the estimate of 𝑢, elements of the third moment tensor. However, the diagonal elements of the third moment tensor are (𝑢𝑖∗ )3 (and not 𝑢𝑖∗ ). Using this strategy would lead to suboptimal sample complexity in 𝑑 since we would need a very good approximation to each entry (𝑢𝑖∗ )3 . This is because (𝑢𝑖∗ )3 can be small and thus taking the cube-root can amplify the error. 10

To improve the sample complexity, we instead compute a random contraction of the third moment tensor.6 Then, on the resulting matrix we can compute the largest eigenvalue (in absolute value) and compute the corresponding vector. This vector is close to 𝑢˜ since in expectation (over the samples) this random contraction will yield a rank one matrix proportional to 𝑢˜ 𝑢˜ ⊤ . We show in Lemmas 5.5 and 5.8 that this strategy allows us to get an estimate 𝑢ˆ satisfying the guarantee in (2.10) using 𝑂 𝛼 (𝑑2 /𝜀2 ) samples. This allows us to get optimal sample complexity in 𝑑. 2 Putting it together, we show that using 𝑑𝜀2 · poly(1/𝜅2 (𝛾), 1/𝜅3 (𝛾)) samples, we can get estimates 𝜇ˆ and b Σ such that

Σ−1/2 (𝜇ˆ − 𝜇) ⩽ 𝜀

𝐼 𝑑 − Σ−1/2 b ΣΣ−1/2

and

𝐹

⩽ 𝜀.

We show in Section 3.2 that for 𝛼 ⩽ 0.99 we can bound 1/𝜅 2 (𝛾), 1/𝜅3 (𝛾) ⩽ poly log(1/𝛼 0 ). For 𝛼 → 1, 𝜅3 (𝛾) → 0, which leads to an increase in the sample complexity. If 𝛼 is very close to 1 ˜ (meaning 𝛼 ⩾ 1 − 𝑂(𝜀)), then the sample mean and sample covariance of the truncated distribution are good enough estimates and we show that our estimators are close to these. For the in-between ˜ case (i.e., 0.99 ⩽ 𝛼 < 1 − 𝑂(𝜀)), we can bound 𝜅 3 (𝛾) ⩾ Ω(𝜀), which gives the sample complexity 2 4 𝑂(𝑑 /𝜀 ). This shows the sample complexity as described in Theorem 1.1. The fact that we need more samples for the case 𝛼 > 0.99 seems somewhat artificial. The problem is that, when 𝛼 is close to 1, the norm of the third central moment tensor 𝑀3 goes to 0, ˜ On the other hand it seems that the problem of estimating which makes it harder to estimate 𝑢. the parameters of a truncated Gaussian should become easier if there is less truncation. However, it is unclear to us whether and how the sample complexity could be improved with our approach. Runtime. We now briefly discuss the runtime of our algorithm. As a first step, we need to ˆ For this, we need to compute a random contraction of the third central compute the vector 𝑢. sample moment tensor. If we first perform the contraction and then average over the samples, we need to add 𝑁 rank-1 matrices of size 𝑑 2 , which can be done in time 𝑂(𝑇(𝑁 , 𝑑)) using fast rectangular matrix multiplication. Next, we need to compute the spectral decomposition of the b 2 we need to resulting matrix, which can be done in time 𝑂(𝑑3 ). For computing the estimate 𝑀 compute the sample covariance, which again can be done in time 𝑂(𝑇(𝑁 , 𝑑)). We then need to b 2 , which can be done in time 𝑂(𝑑3 ). Next, we compute the expressions in (2.8) invert the matrix 𝑀 and (2.9). If we again first compute the inner product and then average over the samples (for (2.8)), these can be done in time 𝑂(𝑁 · 𝑑) and 𝑂(𝑑3 ) respectively. Since 𝑇(𝑁 , 𝑑) ⩾ Ω(𝑁 · 𝑑) ⩾ Ω(𝑑3 ), the terms involving 𝑇(𝑁 , 𝑑) dominate and we can thus compute an estimate for skew(𝛾) in time 𝑂(𝑇(𝑁 , 𝑑)). ˆ We note Once we know an estimate of skew(𝛾), we use the inverse of skew to compute 𝛾. that since this is a one-dimensional monotonic function, we can compute an approximate inverse using binary search. The computational complexity of this step is negligible relative to the primary calculations (see Remark 5.16). The final step requires adding a single term to the first and second moments, which is achievable in time 𝑂(𝑑2 ). Hence, the dominating term in the overall runtime is the computation of the sample covariance, which, as argued in Section 1, is 𝑂(𝑇(𝑁 , 𝑑)) ⩽ 𝑂(𝑑3.250035 ) · poly(1/𝜀, log(1/𝛼 0 )). Organization. In Section 3, we cover the necessary preliminaries for our paper and in particular prove that the function skew is monotonic. In Section 4, we compute the moments of the truncated 6 Random contractions are typically used in decomposition of higher-order tensors and have seen applications in

learning Gaussian Mixture Models, Independent Component Analysis and Dictionary Learning. The specific type of random contraction we utilize is attributed to Jennrich [Har70, LRA93].

distribution and discuss our preconditioning. Then, in Section 5, we give a complete proof of our main result. In Section 6, we give our result on robustly estimating the parameters of a Gaussian under halfspace truncation. In Section 7, we prove the extension of our result to an intersection of two orthogonal halfspaces. In Appendices A and B, we prove some facts and computations needed for our results. Finally, in Appendix C, we show how one could get 𝑤 based on the moments. This is not needed in our proof, but we include it for completeness since this is necessary for the proof of [LMZ24].

Preliminaries

3.1

Notation

We use the following convention: We denote the unit sphere in R𝑑 by 𝒮 𝑑−1 . For 𝑛 ∈ N, [𝑛] denotes the set {1, 2, . . . , 𝑛}. Unless explicitly stated, the base of the logarithm is 𝑒. For two vectors 𝑎, 𝑏 ∈ R𝑑 , we denote their inner product by ⟨𝑎, 𝑏⟩ or 𝑎 ⊤ 𝑏. Unless otherwise specified, all vector norms (∥ · ∥ or ∥ · ∥2 ) are the Euclidean norm. For matrices, ∥ · ∥ denotes the spectral norm and ∥ · ∥𝐹 denotes the Frobenius norm, ∥ · ∥∗ denotes the nuclear norm. We use the notation 𝑂(·), Θ(·), Ω(·), ≲ , ≳ to hide e to hide logarithmic factors. We use 1[·] for the indicator variable. absolute constants. We use 𝑂(·) For positive semi-definite (PSD) matrices 𝐴, 𝐵 ∈ R𝑑×𝑑 , we use 𝐴 ≼ 𝐵 to mean that 𝐴 is at most 𝐵 in the Löwner order (meaning 𝐵 − 𝐴 is PSD). We use 𝜆min (𝐴) to denote the minimum eigenvalue of a square matrix 𝐴. We denote the total variation distance by 𝑑TV . We denote the density function of the standard Gaussian by 𝜑(·) and its cumulative density function by Φ(·). Tensor norm. For two order-3 tensors 𝑇, 𝑆 ∈ R𝑑×𝑑×𝑑 , we define ⟨𝑇, 𝑆⟩ = 𝑖𝑗 𝑘 𝑇𝑖,𝑗,𝑘 𝑆 𝑖,𝑗,𝑘 . Further more, we define the injective tensor norm of 𝑇 as ∥𝑇∥ = sup∥𝑥∥=∥𝑦∥=∥𝑧∥=1 ⟨𝑥 ⊗ 𝑦 ⊗ 𝑧, 𝑇⟩ . Note that if 𝑇 is a symmetric tensor, as a consequence of Banach’s Theorem (see e.g. [Fri13, Theorem 1] or [CHLZ12, Corollary 4.2]), it holds that

∥𝑇∥ =

sup

⟨𝑥 ⊗ 𝑦 ⊗ 𝑧, 𝑇⟩ = sup

∥𝑥∥=∥𝑦∥=∥𝑧∥=1

𝑥 ⊗3 , 𝑇

∥𝑥∥=1

Truncated Gaussian. For a measurable set 𝑆 ⊆ R𝑑 , let 𝛼 > 0 to be the probability mass of the set 𝑆 under the true Gaussian distribution 𝒩 (𝜇, Σ), i.e., 𝛼 = E𝑥∼𝒩 (𝜇,Σ) 1[𝑥 ∈ 𝑆]. We say that a random vector 𝑋 follows a Gaussian 𝒩 (𝜇, Σ) truncated to 𝑆, if it follows the distribution 𝒩 (𝜇, Σ) conditioned on the event 𝑥 ∈ 𝑆. For a halfspace 𝐻 = {𝑥 : ⟨𝑤, 𝑥⟩ ⩽ 𝜏}, we denote a Gaussian 𝒩 (𝜇, Σ) truncated to a halfspace 𝐻 by 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏

3.2

Inverse Mill’s ratio

For 𝛾 ∈ R, consider the standard Gaussian random variable 𝒩 (0, 1) conditioned on (−∞, 𝛾], denoted by 𝑍 𝛾 . In this section, we introduce the first three cumulants of 𝑍 𝛾 , show how to describe them in terms of the density function and the cumulative density function of the standard Gaussian and give several bounds that will be needed later in our proofs. For a general random variable 𝑋 on R, the cumulants are defined as follows.

Definition 3.1. Let 𝑋 be a random variable on R. We define its cumulant generating function by 𝐾(𝑡) = log E 𝑒 𝑡𝑋 . If 𝐾(𝑡) exists and is analytic, we define the 𝑛-th cumulant of 𝑋, denoted by 𝜅 𝑛 (𝑋) as the 𝑛-th order derivative of 𝐾(𝑡) evaluated at 𝑡 = 0, i.e., 𝜅 𝑛 (𝑋) = 𝐾 (𝑛) (0). In particular, one can compute that the first cumulant is the mean of 𝑋, the second cumulant is the variance of 𝑋 and the third cumulant is the third central moment. Thus, the first three cumulants of 𝑍 𝛾 (which we denote by 𝜅1 (𝛾), 𝜅 2 (𝛾), and 𝜅 3 (𝛾) respectively) are as follows: 𝜅1 (𝛾) = E 𝑍 𝛾 , 𝜅2 (𝛾) = E 𝑍 𝛾 − E 𝑍 𝛾

𝜅3 (𝛾) = E 𝑍 𝛾 − E 𝑍 𝛾

(3.1)

They can be characterized in terms of 𝜑(𝛾) and Φ(𝛾) as follows. Fact 3.2. Let 𝛾 ∈ R. We have that

∫ 𝛾

1 𝜅1 (𝛾) = · Φ(𝛾)

𝑡𝜑(𝑡)𝑑𝑡 = −

−∞

𝜑(𝛾) 𝜑(𝛾) 𝜅2 (𝛾) = 1 − 𝛾 − Φ(𝛾) Φ(𝛾)

𝜑(𝛾) , Φ(𝛾)

𝜑(𝛾) 𝜑(𝛾) 𝜑(𝛾) · 1 − 𝛾 2 − 3𝛾 · −2· 𝜅3 (𝛾) = Φ(𝛾) Φ(𝛾) Φ(𝛾)

The proof of Fact 3.2 follows by direct computation. We include a proof in Section A.1. Moreover, the skewness of a random variable 𝑋, denoted by skew(𝑋) on R, is defined as the 𝑋−𝜇 third moment of the normalized random variable 𝜎 , where 𝜇 denotes the mean of 𝑋 and 𝜎 the standard deviation. This can also be written in terms of the cumulants. Namely, we have that 𝜅3 (𝑋) . 𝜅2 (𝑋)3/2

skew(𝑋) =

We denote the skewness of 𝑍 𝛾 by skew(𝛾). Using Fact 3.2, we immediately get that 𝜑(𝛾) 𝜑(𝛾) 𝜑(𝛾) skew(𝛾) = 1 − 𝛾 2 − 3𝛾 −2 Φ(𝛾) Φ(𝛾) Φ(𝛾)

𝜑(𝛾) 𝜑(𝛾) 1−𝛾 − Φ(𝛾) Φ(𝛾)

2 ! − 23

(3.2)

We now discuss some properties of 𝜅1 , 𝜅2 , 𝜅3 and give some estimates for them that we will need 𝜑(𝑥) for our analysis later. Observe that 𝜅 1 (𝑥), 𝜅2 (𝑥), 𝜅3 (𝑥) can be viewed as polynomials in 𝑥 and Φ(𝑥) . 𝜑(𝑥)

We have that the ratio Φ(𝑥) ≈ 𝜑(𝑥) as 𝑥 → +∞ and thus the non-trivial case is when 𝑥 → −∞. To address this latter case, as in [LMZ24], we introduce the following function, commonly known as the inverse Mills ratio or hazard function: ℎ(𝑥) =

𝜑(𝑥) . 1 − Φ(𝑥) 𝜑(−𝑥)

Observe that bounds on ℎ(𝑥) (which typically only work for 𝑥 ⩾ 0) can be related to Φ(−𝑥) as follows 𝜑(−𝑥) 𝜑(𝑥) = = ℎ(𝑥). Φ(−𝑥) 1 − Φ(𝑥) We will need the following three facts. 13

Fact 3.3 ([GU14]). Define the following functions for 𝑘 ∈ N0 and 𝑥 > 0 𝑟 𝑘+1 (𝑥) B 𝑥 +

1 𝑥 + 𝑥+ 2 3 .. .

𝑥+ 𝑥𝑘

Then, for any 𝑚 ∈ N and 𝑥 ⩾ 0 we have 𝑟2𝑚−1 (𝑥) < ℎ(𝑥) < 𝑟2𝑚 (𝑥). In particular, we have 𝑥 < ℎ(𝑥) < 𝑥 +

1 . 𝑥

We have that 𝑟2𝑚 (𝑥) ↘ ℎ(𝑥) and 𝑟2𝑚+1 ↗ ℎ(𝑥) as 𝑚 → ∞ for all 𝑥 > 0. In particular, the above estimates are accurate for large 𝑥 > 0. Intuitively, this is because the higher-order term 𝑥 + 𝑥𝑘 ≈ 𝑥 for large 𝑥. However, for small fixed 𝑘, the q approximation quality of 𝑟 𝑘 (𝑥) can be extremely poor near 𝑥 = 0. In particular, despite ℎ(0) = 𝜋2 , the upper bound 𝑟2𝑚 (𝑥) → ∞ as 𝑥 → 0 for all fixed integers 𝑚 > 0. For 𝑥 close to 0, we can instead use the following two estimates of the inverse Mills ratio, where the second one is exact at 𝑥 = 0. Fact 3.4 ([Sam53, Bir42]). For 𝑥 > 0, we have 1 √ 1 √ 2 ( 𝑥 + 8 + 3𝑥) < ℎ(𝑥) < ( 𝑥 2 + 4 + 𝑥). 4 2 Fact 3.5 ([Boy59]). For 𝑥 > 0, we have

(𝜋 − 2)2 𝑥 2 + 2𝜋 + 2𝑥 < ℎ(𝑥) < 𝜋

√

𝑥 2 + 2𝜋 + (𝜋 − 1)𝑥 . 𝜋

Based on these facts, we prove the following claims that are necessary for our analysis. We provide proofs of these claims in Section A.1. Claim 3.6. The function 𝜅1 as defined in (3.1) is 1-Lipschitz on R. Claim 3.7. For every 𝑡 ∈ R, we have that 0 < 𝜅 2 (𝑡) ⩽ 1. d Claim 3.8. For 𝜅2 and 𝜅 3 as in (3.1), we have d𝑡 𝜅2 (𝑡) = −𝜅3 (𝑡). Moreover, 𝜅 3 (𝑡) < 0 for every 𝑡 ∈ R and thus 𝜅2 (𝑡) is monotonically increasing.

Claim 3.9. For every 𝑡 ∈ R, we have |𝜅1 (𝑡)| · 𝜅 2 (𝑡) ⩽ 𝑂(1). d Claim 3.10. For every 𝑡 ∈ R, we have | d𝑡 𝜅 2 (𝑡)| = |𝜅3 (𝑡)| ⩽ 𝑂(1) and 0 < 𝜅1 (𝑡) · 𝜅 3 (𝑡) ⩽ 𝑂(1).

Claim 3.11. For 𝑡 ⩾ 1, we have |𝜅1 (𝑡)| ⩽ 1 − 𝜅2 (𝑡). Claim 3.12. For 𝑡 ⩽ −3, we have 𝜅2 (𝑡) ⩾ 4𝑡12 . 1 Claim 3.13. For 𝑡 ⩽ −2, we have |𝜅3 (𝑡)| ⩾ 50·|𝑡| 3.

Claim 3.14. For 𝑡 ⩾ 1, we have 1 − 𝜅 2 (𝑡) ⩽ 16 · (1 − Φ(𝑡)) · log 1/(1 − Φ(𝑡)). 14

Claim 3.15. For 𝑡 ⩾ 2, we have 𝑡2 ⩽ log (1/(1 − Φ(𝑡))) ⩽ 2𝑡 2 . Claim 3.16. There are absolute constants 𝑚, 𝑀 > 0 such that 𝑚⩽

|skew(𝑡)| (1 − Φ(𝑡)) log3/2 (1/(1 − Φ(𝑡)))

⩽𝑀

for all 𝑡 ⩾ 0. The following lemma is central for our proof. Lemma 3.17. The function skew as defined in (3.2) is monotonically increasing on R. d Proof. From Claim 3.8 we know that d𝑡 𝜅2 (𝑡) = −𝜅3 (𝑡). It follows that

d skew(𝑡) = d𝑡

d d𝑡 𝜅 3 (𝑡)

d 𝜅 2 (𝑡)3/2 − 32 𝜅 2 (𝑡) d𝑡 𝜅2 (𝑡) · 𝜅 3 (𝑡)

𝜅2 (𝑡)3

d 2 d𝑡 𝜅3 (𝑡) 𝜅2 (𝑡) + 3𝜅 3 (𝑡)2

2𝜅 2 (𝑡)5/2

Since 𝜅2 (𝑡) > 0 for every 𝑡 ∈ R by Claim 3.7, it suffices to show that the numerator is positive. Φ(𝑡) Φ(𝑡) d Defining 𝑔(𝑡) = 𝜑(𝑡) · 2 d𝑡 𝜅3 (𝑡) 𝜅 2 (𝑡) + 3𝜅 3 (𝑡)2 , since 𝜑(𝑡) > 0, it is enough to show 𝑔(𝑡) > 0 for all 𝑡 ∈ R. After some calculation we obtain 𝜑(𝑡) 𝜑(𝑡) 𝑔(𝑡) = 2𝑡 − 6𝑡 + (𝑡 + 14𝑡 − 5) + (2𝑡 3 + 20𝑡) Φ(𝑡) Φ(𝑡) 3

𝜑(𝑡) + (𝑡 + 8) Φ(𝑡) 2

(3.3)

In order to show that 𝑔(𝑡) > 0 for all 𝑡 ∈ R, we consider the following function 𝑝 𝑡 (𝑥) = 2𝑡 3 − 6𝑡 + (𝑡 4 + 14𝑡 2 − 5)𝑥 + (2𝑡 3 + 20𝑡)𝑥 2 + (𝑡 2 + 8)𝑥 3 . Note that 𝑔(𝑡) = 𝑝 𝑡

𝜑(𝑡) Φ(𝑡)

. Our argument is now structured as follows: For fixed 𝑡, we will define

𝜑(𝑡)

values 𝑥ℓ (𝑡) ⩽ Φ(𝑡) ⩽ 𝑥 𝑢 (𝑡). We then show that 𝑝 𝑡 is monotonically increasing on [𝑥ℓ (𝑡), 𝑥 𝑢 (𝑡)]. Then we lower bound 𝜑(𝑡) 𝑔(𝑡) = 𝑝 𝑡 ⩾ 𝑝 𝑡 (𝑥ℓ (𝑡)) Φ(𝑡) and argue that 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0. In order to show that 𝑝 𝑡 is monotonically increasing, we compute the derivative of 𝑝 𝑡 with respect to 𝑥:

𝜕 𝑝 𝑡 (𝑥) = 𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 + 3(𝑡 2 + 8) · 𝑥 2 𝜕𝑥

and show that this is positive for 𝑥 ∈ [𝑥ℓ (𝑡), 𝑥 𝑢 (𝑡)]. 4 4 We distinguish the following cases: (i) 0 < 𝑡; (ii) − 10 < 𝑡 ⩽ 0; and (iii) 𝑡 < − 10 . For each of the 𝜑(𝑥)

cases we give values for 𝑥ℓ (𝑡) and 𝑥 𝑢 (𝑡) based on approximations for Φ(𝑥) and show that 𝑔(𝑡) > 0 using the strategy described above. We will defer some of the computations to Appendix B, where we give Mathematica code to verify the statements we need.

Case (i): 0 < 𝑡. For 𝑡 > 0, we define 𝑥ℓ (𝑡) = √𝑒 𝜑(𝑡)

−𝑡 2 /2

𝜋/2+𝑡

> 0. Note that since for 𝑡 > 0 we have

Φ(𝑡) ⩽ 12 + √𝑡 , we get that Φ(𝑡) ⩾ 𝑥ℓ (𝑡) for every 𝑡 > 0. Since we have 4(𝑡 3 +10𝑡) > 0 and 3(𝑡 2 +8) > 0 2𝜋

for 𝑡 > 0, we get that for any 𝑥 ⩾ 𝑥ℓ (𝑡)7

We show in Fact B.1 that

𝜕 𝜕 𝑝 𝑡 (𝑥) ⩾ 𝑝 𝑡 (𝑥ℓ (𝑡)). 𝜕𝑥 𝜕𝑥

𝜕 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0 𝜕𝑥

for all 𝑡 > 0. Hence, we get that for 𝑡 > 0, 𝑝 𝑡 is monotonically increasing on [𝑥ℓ (𝑡), ∞). Thus, we get that 𝜑(𝑡) 𝑔(𝑡) = 𝑝 𝑡 ⩾ 𝑝 𝑡 (𝑥ℓ (𝑡)) . Φ(𝑡) In Fact B.1, we also show that 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0 for 𝑡 > 0, which shows 𝑔(𝑡) > 0 for 𝑡 > 0. 4 Case (ii): − 10 < 𝑡 ⩽ 0. In this case, we define

p 𝑥ℓ (𝑡) =

(𝜋 − 2)2 𝑡 2 + 2𝜋 − 2𝑡 𝜋

√ and

𝑥 𝑢 (𝑡) =

𝑡 2 + 2𝜋 − (𝜋 − 1)𝑡 . 𝜋

𝜑(𝑡)

For 𝑡 ⩽ 0, we have 𝑥ℓ (𝑡) > 0. By Fact 3.5 we have 𝑥ℓ (𝑡) ⩽ Φ(𝑡) ⩽ 𝑥 𝑢 (𝑡). Since for 𝑡 ⩽ 0 we have 4(𝑡 3 + 10𝑡) ⩽ 0 and 3(𝑡 2 + 8) > 0, we get that, for 𝑥ℓ (𝑡) ⩽ 𝑥 ⩽ 𝑥 𝑢 (𝑡) we have8

𝜕 𝑝 𝑡 (𝑥) ⩾ 𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 𝑢 (𝑡) + 3(𝑡 2 + 8) · 𝑥ℓ (𝑡)2 . 𝜕𝑥

We show in Fact B.2 that 𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 𝑢 (𝑡) + 3(𝑡 2 + 8) · 𝑥ℓ (𝑡)2 > 0 4 for − 10 < 𝑡 ⩽ 0. Thus, we have that 𝑝 𝑡 is monotonically increasing on [𝑥ℓ (𝑡), 𝑥 𝑢 (𝑡)] and thus in particular 𝜑(𝑡) 𝑔(𝑡) = 𝑝 𝑡 ⩾ 𝑝 𝑡 (𝑥ℓ (𝑡)) . Φ(𝑡) 4 4 In Fact B.2 we also show that 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0 for − 10 < 𝑡 ⩽ 0 and thus 𝑔(𝑡) > 0 for − 10 < 𝑡 ⩽ 0. 4 Case (iii): 𝑡 ⩽ − 10 . In this case, we define 𝑥ℓ (𝑡) = 𝑟81 (−𝑡) and 𝑥 𝑢 = 𝑟20 (−𝑡). We verify in Fact B.3 𝜑(𝑡)

4 that for 𝑡 ⩽ − 10 we have 𝑥ℓ (𝑡) > 0. By Fact 3.3, we have that 𝑥ℓ (𝑡) ⩽ Φ(𝑡) ⩽ 𝑥 𝑢 (𝑡). We again get that8

𝜕 𝑝𝑡 𝜕𝑥

𝜑(𝑡) ⩾ 𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 𝑢 (𝑡) + 3(𝑡 2 + 8) · 𝑥ℓ (𝑡)2 . Φ(𝑡)

In Fact B.3 we show that 𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 𝑢 (𝑡) + 3(𝑡 2 + 8) · 𝑥ℓ (𝑡)2 > 0 7 Since 𝑥 (𝑡) > 0, this also implies 𝑥 2 ⩾ 𝑥 (𝑡)2 . ℓ ℓ 8 We are again using that since 𝑥 (𝑡) > 0 we also have 𝑥 (𝑡)2 ⩽ 𝑥 2 . ℓ ℓ

4 . We thus again get that 𝑝 𝑡 is monotonically increasing on [𝑥ℓ (𝑡), 𝑥 𝑢 (𝑡)] and thus for 𝑡 ⩽ − 10

𝑔(𝑡) = 𝑝 𝑡

𝜑(𝑡) ⩾ 𝑝 𝑡 (𝑥ℓ (𝑡)) . Φ(𝑡)

4 Finally, we also show in Fact B.3 that 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0 for 𝑡 ⩽ − 10 and thus also 𝑔(𝑡) > 0 for these 𝑡. This completes the proof. □

In Appendix B, we will show the following bound on skew. Claim 3.18. We have lim𝑡→−∞ skew(𝑡) = −2 and lim𝑡→+∞ skew(𝑡) = 0. Thus, for all 𝑡 ∈ R we have skew(𝑡) ∈ [−2, 0] and in particular |skew(𝑡)| ⩽ 2. We can even strengthen Lemma 3.17. In Appendix B, we will show the following two claims that we need for the proofs later. For both of the claims, the idea is that the involved function are point-wise strictly positive. Thus, since the functions are continuous, it is enough to show that the limits as 𝑡 → ±∞ are positive as well to conclude that we can lower bound them by a constant 𝐶 > 0. In Appendix B, we compute these limits and thus prove these claims. Claim 3.19. There exists a constant 𝐶 > 0 such that for all 𝑡 ∈ R we have d d𝑡 skew(𝑡) ⩾ 𝐶. d | d𝑡 𝜅2 (𝑡)|

Moreover, |𝜅2 (𝑎) − 𝜅 2 (𝑏)| ⩽ 𝐶1 · |skew(𝑎) − skew(𝑏)| for every 𝑎, 𝑏 ∈ R. 𝜅 (𝑡)

1 Claim 3.20. Define 𝜓(𝑡) B 𝜅 (𝑡) 1/2 . Then, there exists a constant 𝐶 > 0 such that for all 𝑡 ∈ R we have 2

d d𝑡 skew(𝑡) ⩾ 𝐶 · 𝜅2 (𝑡)2 . d | d𝑡 𝜓(𝑡)|

Moreover, 𝜓(𝑎) − 𝜓(𝑏) ⩽ 𝐶1 · |skew(𝑎) − skew(𝑏)| · max{𝜅 2 (𝑎)−2 , 𝜅2 (𝑏)−2 } for every 𝑎, 𝑏 ∈ R. Furthermore, we can also show an analogous statement for the relation between 𝜅1 and 𝜅2 , which we will also prove in Appendix B. The proof idea is the same as for the above two claims (note that 𝜅2 is also monotonic by Claim 3.8). Claim 3.21. There exists a constant 𝐶 > 0 such that for all 𝑡 ∈ R we have d d𝑡 𝜅 2 (𝑡) ⩾ 𝐶 · 𝜅2 (𝑡)2 . d | d𝑡 𝜅 1 (𝑡)|

Moreover, |𝜅1 (𝑎) − 𝜅 1 (𝑏)| ⩽ 𝐶1 · |𝜅 2 (𝑎) − 𝜅 2 (𝑏)| · max{𝜅 2 (𝑎)−2 , 𝜅2 (𝑏)−2 } for every 𝑎, 𝑏 ∈ R. Finally, we need tight estimates of 𝜅 2 (𝛾) and 𝜅 3 (𝛾) for our proofs later. In order to relate these bounds to the mass 𝛼 of the truncation set (the halfspace), we use the relation 𝛼 = Φ(𝛾). We defer its proof to Lemma 4.2. Claim 3.22. Let 𝛾 ∈ R. For 𝛼 = Φ(𝛾), we have 1 1 , . 𝜅 2 (𝛾) ⩾ min 20 8 log 1/𝛼

Proof. From Claim 3.8 we know that 𝜅 2 (𝛾) monotonically increases on R. For 𝛾 → ∞ the distribution converges to standard Gaussian, and the variance goes to 1. For the case 𝛾 → −∞, we can use the rational approximation of the Mills ratio from Fact 3.3. Concretely, we distinguish the two cases 𝛾 > −3 and 𝛾 ⩽ −3: Case (i): 𝛾 > −3. For 𝛾 > −3 we have 𝜅 2 (𝛾) ⩾ 𝜅 2 (−3) ⩾ 1/20 by monotonocity of 𝜅 2 . Case (ii): 𝛾 ⩽ −3. For 𝛾 ⩽ −3, by Claim 3.12 we have that 𝜅2 (𝛾) ⩾ 4𝛾1 2 . It remains to get a 𝜑(𝛾)

bound on 𝛾 in terms of 𝛼. By Fact 3.3, we know that 𝛼 = Φ(𝛾) ⩽ −𝛾 . This implies that √ √ 2 2 2 1 ⩾ − 2𝜋𝛾 · 𝑒 𝛾 /2 ⩾ 3 2𝜋 · 𝑒 𝛾 /2 ⩾ 𝑒 𝛾 /2 𝛼 and thus we get that log 1/𝛼 ⩾ 𝛾 2 /2. Combining this with bound on 𝜅2 (𝛾) from above, we get that, for 𝛾 ⩽ −3, 1 1 . 𝜅2 (𝛾) ⩾ · 8 log 1/𝛼 Combining the two cases, we get the lower bound as claimed.

□

Claim 3.23. Let 𝛾 ∈ R. For 𝛼 = Φ(𝛾), we have

(

)

1 1 1 . |𝜅3 (𝛾)| ⩾ min , , √ (1 − 𝛼) log3/2 3/2 20 1 − 𝛼 4 2 50 log 1/𝛼 Proof. Unlike 𝜅2 (𝛾), the function 𝜅3 (𝛾) is not monotonic in 𝛾. Moreover, the function approaches 0 both for 𝛾 → +∞ and 𝛾 → −∞. Thus, we split our analysis in three parts: First, we consider the two cases 𝛾 → ±∞ where we use the rational approximations of the Mills ratio from Fact 3.3. Finally, we bound 𝜅2 (𝛾) on a small interval around 0 by an explicit computation. 1 Case (i): 𝛾 ⩽ −2. By Claim 3.13, for 𝛾 ⩽ −2, we have the lower bound |𝜅 3 (𝛾)| ⩾ 50·|𝛾| 3 . From

the proof of Claim 3.22 we already know that log 1/𝛼 ⩾ 𝛾2 /2 for 𝛾 ⩽ −3 ⩽ −2. So it holds that 1 𝜅3 (𝛾) ⩾ for 𝛾 ⩽ −2. 3/2 50 log

1/𝛼

Case (ii): 𝛾 ⩾ 2. From Claim 3.8 we know 𝜅3 (𝛾) < 0 and thus we have that |𝜅3 (𝛾)| = ℎ(𝛾) 𝛾 2 − 1 + 3𝛾ℎ(𝛾) + 2ℎ(𝛾)2 . √ Noting that ℎ(𝛾) > 0 and Φ(𝛾) ⩽ 1 for all 𝛾, we have for 𝛾 ⩾ 2 > 2 that

|𝜅3 (𝛾)| ⩾ 𝜑(𝛾)(𝛾 2 − 1) ⩾

1 𝜑(𝛾)𝛾2 . 2

𝜑(𝛾)

By Fact 3.3, we have for 𝛾 ⩾ 2 > 0 that 𝛾 ⩽ 1−Φ(𝛾) . Combining this with the above bound on |𝜅3 (𝛾)| and using 𝛼 = Φ(𝛾) we get 1 |𝜅 3 (𝛾)| ⩾ 𝛾3 (1 − 𝛼). 2 It again remains to give a bound on 𝛾 in terms of 𝛼. Note that, for 𝛾 ⩾ 2, 𝛼 = Φ(𝛾), by Claim 3.15 we have log 1/(1 − 𝛼) ⩽ 2𝛾 2 Combining this with the inequality above, we get that 1 1 1 |𝜅3 (𝛾)| ⩾ log 2 2 1−𝛼

3/2

1 1 (1 − 𝛼) = √ (1 − 𝛼) log3/2 1−𝛼 4 2 18

Case (iii): 𝛾 ∈ (−2, 2). For the last case, we verify in Fact B.4 in Appendix B that for 𝛾 ∈ (−2, 2), 1 . we have |𝜅3 (𝛾)| ⩾ 20 Putting all the three cases together gives the result. □

3.3

Gaussian concentration and anti-concentration

We will need the following two results on the concentration and anti-concentration of a Gaussian random variable. We note that the first fact follows immediately from the boundedness of the Gaussian density. Fact 3.24 (Gaussian anti-concentration). For every 𝛿 > 0, we have that

r P

𝑋∼𝑁(0,𝜎 2 )

[−𝛿 ⩽ 𝑋 ⩽ 𝛿] ⩽

2 𝛿 · 𝜋 𝜎

Fact 3.25 (Gaussian concentration, see e.g. [vH16, Theorem 3.25]). Let 𝜓 : R𝑑 → R be a 𝜆-Lipschitz function. Let 𝑔 ∼ 𝑁(0, 𝐼 𝑑 ). Then for all 𝑡 > 0, 𝑡2 P 𝜓(𝑔) − E 𝜓(𝑔) ⩾ 𝑡 ⩽ 2 exp − 2 2𝜆

3.4

Concentration bounds for sub-Gaussian random variables

In this section, we state two concentration results for sub-Gaussian random variables. To do so, we define the following (quasi-)norm on a random variable 𝑋. Let 𝜓 : R⩾0 → R⩾0 be a nondecreasing function that satisfies 𝜓(0) = 0 and 𝜓(𝑥) → ∞ as 𝑥 → ∞. Then we define the following (quasi)-norm of 𝑋 as |𝑋| ∥𝑋∥𝜓 = inf 𝑡 > 0 | E𝜓 ⩽1 𝑡 For a constant 𝜃 ∈ (0, 1) ∪ {2}, let 𝜓𝜃 (𝑥) = exp(𝑥 𝜃 ) − 1. Setting 𝜓 = 𝜓2 in the above results in the sub-Gaussian norm. Recall that a random vector 𝑋 is 𝑟-sub-Gaussian if ⟨𝑣, 𝑋⟩ is 𝑟-sub-Gaussian for every unit vector 𝑣 (meaning that ∥⟨𝑣, 𝑋⟩ − E[⟨𝑣, 𝑋⟩]∥𝜓2 ⩽ 𝑟).9 The following fact provides a standard concentration bound for the covariance estimation of sub-Gaussian √ distributions. Note that the second part follows directly from the first one by bounding ∥𝐴∥𝐹 ⩽ 𝑑 · ∥𝐴∥. Fact 3.26 ([Ver18, Theorem 4.6.1]). Let 𝑦1 , . . . , 𝑦𝑛 ∈ R𝑑 be i.i.d. samples with covariance Σ ≻ 0 such Í that Σ−1/2 𝑦 𝑖 is 𝑟-sub-Gaussian. Let the sample mean be 𝑦¯ = 𝑛1 𝑛𝑖=1 𝑦 𝑖 and let the sample covariance be Í b ¯ ⊗2 . Then, with probability at least 1 − 𝛿, we have that Σ = 𝑛1 𝑛𝑖=1 (𝑦 𝑖 − 𝑦) ∥Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑 ∥ ≲ 𝑟 2 ·

𝑑 + log 1/𝛿 𝑑 + log 1/𝛿 + . 𝑛 𝑛

Furthermore, there exists some absolute constant 𝐶 > 0 such that for 𝑛 ⩾ 𝐶(𝑑 + log 1/𝛿) we have with probability at least 1 − 𝛿 p 𝑑 + 𝑑 log 1/𝛿 ∥Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑 ∥𝐹 ≲ 𝑟 2 · . √ 𝑛 9 Often, such a random variable is also called 𝜎 2 -sub-Gaussian, where 𝜎 = 𝑟 is an upper bound on the 𝜓 -norm of 2

the one-dimensional projections.

In the remainder of this section, we show a concentration result p for cubes of sub-Gaussian −1 random variables. As in [KC22], we define the function 𝜓𝜃,𝐿 (𝑦) = log(1 + 𝑦) + 𝐿 log(1 + 𝑦)1/𝜃 for −1 all 𝑦 ⩾ 0, and let 𝜓𝜃,𝐿 be the inverse function of 𝜓𝜃,𝐿 . Note that when 𝜃 ∈ (0, 1), ∥·∥𝜓𝜃,𝐿 is not a norm since it only satisfies sub-additivity up to a multiplicative constant depending on 𝜃. The norm ∥·∥𝜓𝜃,𝐿 can be related to the norm ∥·∥𝜓𝜃 via the following theorem. Theorem 3.27 (Theorem 3.1 in [KC22]). Fix 𝜃 ∈ (0, 1). Let 𝑘 ⩾ 1 and consider a sequence of independent 𝑘 zero-mean random variables {𝑋𝑖 }𝑖=1 such that ∥𝑋𝑖 ∥𝜓𝜃 < ∞ for all 𝑖 ∈ [𝑘]. Then there exist constants ′ 𝑄 𝜃 , 𝑄 𝜃 > 0 that only depend on 𝜃 such that 𝑘 Õ 𝑖=1

𝑋𝑖

⩽ 𝑄𝜃 ·

v u t 𝑘 Õ 𝑖=1

𝜓𝜃,𝐿

∥𝑋𝑖 ∥2𝜓𝜃

𝑄′

where 𝐿 = √𝜃 . 𝑘

Finally, we have the following tail bound for a random variable with finite ∥·∥𝜓𝜃,𝐿 . Fact 3.28 (Proposition A.3 in [KC22]). Fix 𝜃 ∈ (0, 1) and 𝐿 > 0. Then, for a random variable 𝑋 with ∥𝑋∥𝜓𝜃,𝐿 < ∞, we have that

h i √ P |𝑋| ⩾ ∥𝑋∥𝜓𝜃,𝐿 ( 𝑡 + 𝐿𝑡 1/𝜃 ) ⩽ 2 exp(−𝑡) for all 𝑡 > 0. We will now put the above results together to get the following corollary about the concentration of cubes of sub-Gaussian random variables. Corollary 3.29 (Concentration inequality for cubes of sub-Gaussians). Let 𝑍1 , . . . , 𝑍 𝑛 be i.i.d. samples of a real-valued random variable 𝑍 such that ∥𝑍∥𝜓2 ⩽ 𝑟. Then there exists an absolute constant 𝐾 > 0 such that, for every 𝑡 > 0, !! 𝑛

1Õ 3 𝑍 𝑖 − E 𝑍 3 ⩾ 𝐾𝑟 3 𝑛 𝑖=1

𝑡 𝑡 3/2 + 𝑛 𝑛

⩽ 2𝑒 −𝑡 .

Proof. Since ∥𝑍∥𝜓2 ⩽ 𝑟, we have from the definition that ∥𝑍 3 ∥𝜓2/3 ⩽ ∥𝑍∥3𝜓2 ⩽ 𝑟 3 . Also, the 𝜓2/3 -quasi-norm satisfies the triangle inequality up to an absolute constant, that is, ∥𝑍 3 − E 𝑍 3 ∥𝜓2/3 ≲ ∥𝑍 3 ∥𝜓2/3 + | E 𝑍 3 |. We will show that | E 𝑍 3 | is in fact bounded by 𝑂 of the 𝜓2/3 norm that

∥𝑍 3 ∥𝜓2/3

. To show this we first note by definition

! 2/3   3|   |𝑍  ⩽ 2. E exp  ∥𝑍 3 ∥𝜓2/3    

Using this and applying Markov’s inequality we get

P |𝑍 3 | ⩾ 𝑢 ∥𝑍 3 ∥𝜓2/3

|𝑍 3 | © = P exp ∥𝑍 3 ∥𝜓2/3

! 2/3

ª ⩾ exp 𝑢 2/3 ® « ¬ ! 2/3     |𝑍 3 |  · exp 𝑢 −2/3 ⩽ E exp  ∥𝑍 3 ∥𝜓2/3     −2/3 ⩽ 2 exp 𝑢

Obtaining the above bound will now enable us to apply the integral identity for expectations. We therefore have 3

∫ ∞

E |𝑍 | =

P(|𝑍 | ⩾ 𝑠) d𝑠 ⩽ ∥𝑍 ∥𝜓2/3

∫ ∞ 0

2𝑒 −𝑢

2/3

d𝑢 ≲ ∥𝑍 3 ∥𝜓2/3 .

where in the first inequality we used the substitution 𝑠 = 𝑢 · ∥𝑍 3 ∥𝜓2/3 . Hence, we have that ∥𝑍 3 − E 𝑍 3 ∥𝜓2/3 ≲ 𝑟 3 . Now define

𝑌𝑖 := 𝑍 3𝑖 − E 𝑍 3𝑖 .

Then, the 𝑌1 , . . . , 𝑌𝑛 are i.i.d., mean-zero, and satisfy ∥𝑌𝑖 ∥𝜓2/3 ≲ 𝑟 3 . Applying the sum-of-sub-Weibulls concentration from Theorem 3.27 we get 𝑛 Õ 𝑖=1

√

𝑌𝑖

≲

𝑛 𝑟3,

𝜓2/3,𝐿

√ where 𝐿 = 𝐶/ 𝑛 for some absolute constant 𝐶 > 0. Using the corresponding tail inequality from Fact 3.28 we have that ! 𝑛 Õ √ 3 √ 𝑡 3/2 P 𝑡+ √ ⩽ 2𝑒 −𝑡 . 𝑌𝑖 ⩾ 𝐾 𝑛 𝑟 𝑛 𝑖=1 Finally, dividing by 𝑛, we conclude that 𝑛

1Õ 3 𝑍 𝑖 − E 𝑍 3 ⩾ 𝐾𝑟 3 𝑛 𝑖=1

𝑡 𝑡 3/2 + 𝑛 𝑛

which completes the proof.

⩽ 2𝑒 −𝑡 , □

Properities of halfspace truncated Gaussians

In this section, we prove several statements that we need in the proofs later. In Section 4.1, we compute the first three moments of a truncated Gaussian. In Section 4.2, we show that halfspace truncated Gaussian are sub-Gaussian. And finally, in Section 4.3, we show how to precondition a truncated Gaussian such that the covariance is well-conditioned. 21

4.1

The first three moments

In order to compute the first three moments, we need the following claim that shows that linear transformations of Gaussians truncated to a halfspace continue to remain Gaussians truncated to a halfspace albeit with different parameters. We defer the proof of this claim to Section A.2. Claim 4.1 (Linear transformations preserve halfspace truncation). Let 𝐴 ∈ R𝑑×𝑑 be an invertible matrix and let 𝑏 ∈ R𝑑 . Consider the random variable 𝑥 ∼ 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Then, the random variable 𝑦 = 𝐴𝑥 + 𝑏 follows the distribution 𝒩 (𝐴𝜇 + 𝑏, 𝐴Σ𝐴⊤ )|⟨𝑤 ′ ,𝑦 ⟩⩽ 𝜏′ , where 𝑤′ =

𝐴−⊤ 𝑤 ∥𝐴−⊤ 𝑤∥

𝜏′ =

and

𝜏 + ⟨𝐴−⊤ 𝑤, 𝑏⟩ . ∥𝐴−⊤ 𝑤∥

Using this claim, we can now compute the structure of the first moment as well as the second and third central moments of the Gaussian distribution under arbitrary halfspace truncation. Each of them can be viewed as the corresponding moment of an untruncated Gaussian and an additive rank-1 term due to truncation. Lemma 4.2. Let 𝑋 ∼ 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 for some unit vector 𝑤. Then, we have

E𝑋 = 𝜇 + 𝜅1 (𝛾)

where 𝛾 =

𝜏−⟨𝑤,𝜇⟩

∥Σ1/2 𝑤 ∥

Σ𝑤 , ∥Σ1/2 𝑤∥

𝑀2 B E(𝑋 − E𝑋)

⊗2

𝑀3 B E(𝑋 − E𝑋)

⊗3

⊤

Σ𝑤 Σ𝑤 = Σ − (1 − 𝜅2 (𝛾)) · 1/2 , ∥Σ 𝑤∥ ∥Σ1/2 𝑤∥

Σ𝑤 = 𝜅3 (𝛾) · ∥Σ1/2 𝑤∥

⊗3

. Furthermore, we have that 𝛼 = Φ(𝛾).

Proof. Consider a random vector 𝑌 that follows a 𝑑-dimensional Gaussian 𝒩 (0, 𝐼 𝑑 )|⟨𝑒1 ,𝑦⟩⩽ 𝛾 . Since only the first coordinate of 𝑌 is truncated (and equal to 𝑍 𝛾 as in Section 3.2), we have that

E 𝑌 = 𝜅1 (𝛾)𝑒1 , E(𝑌 − E 𝑌)⊗2 = 𝐼 𝑑 − (1 − 𝜅2 (𝛾)) · 𝑒1⊗2 , E(𝑌 − E 𝑌)⊗3 = 𝜅3 (𝛾) · 𝑒1⊗3 . 1/2

1/2

Σ 𝑤 ⊤ Σ 𝑤 . By Consider an orthonormal matrix 𝑈 such that 𝑈 𝑒1 = ∥Σ 1/2 𝑤∥ or equivalently 𝑒 1 = 𝑈 ∥Σ1/2 𝑤 ∥ Claim 4.1, we have that that 𝑈𝑌 follows a standard Gaussian 𝒩 (0, 𝐼 𝑑 ) truncated to the set

( 𝑥:

Σ1/2 𝑤 Σ1/2 𝑤

!⊤

) 𝑥⩽𝛾 .

By scaling 𝑈𝑌 by Σ1/2 and shifting by 𝜇, again by Claim 4.1, we obtain a Gaussian 𝒩 (𝜇, Σ) truncated to the halfspace 𝐻 = 𝑥 : ⟨𝑤, 𝑥⟩ ⩽ 𝛾 Σ1/2 𝑤 + ⟨𝑤, 𝜇⟩ . Thus, we get a direct relationship between 𝑌 and 𝑋. Namely, if we have 𝜏 = 𝛾 Σ1/2 𝑤 + ⟨𝑤, 𝜇⟩ or equivalently 𝛾 = 𝛼=

𝜏−⟨𝑤,𝜇⟩

∥Σ1/2 𝑤 ∥

𝑥∼𝒩 (𝜇,Σ)

, then we can get 𝑋 by 𝑋 = Σ1/2𝑈𝑌 + 𝜇. By construction, this also implies 1[𝑥 ∈ 𝐻] =

𝑦∼𝒩 (0,𝐼𝑛 )

1[{𝑒1⊤ 𝑦 ⩽ 𝛾}] = 22

𝑦1 ∼𝒩 (0,1)

1[𝑦1 ⩽ 𝛾] = Φ(𝛾).

Finally, from the relationship 𝑋 = Σ1/2𝑈𝑌 + 𝜇 we compute the moments as follows. First, the mean of 𝑋 is Σ𝑤 . E 𝑋 = E 𝜇 + Σ1/2𝑈𝑌 = 𝜇 + Σ1/2𝑈 · 𝜅1 (𝛾)𝑒1 = 𝜇 + 𝜅1 (𝛾) Σ1/2 𝑤 Next, we can compute the second central moment of 𝑋 as

E (𝑋 − E 𝑋)⊗2 = (Σ1/2𝑈)⊗2 E (𝑌 − E 𝑌)⊗2 = (Σ1/2𝑈)⊗2 𝐼 𝑑 − (1 − 𝜅2 (𝛾))𝑒1⊗2

= Σ − (1 − 𝜅2 (𝛾)) Σ1/2𝑈 𝑒1 = Σ − (1 − 𝜅2 (𝛾))

Σ𝑤 Σ1/2 𝑤

⊗2 ! ⊗2 .

Finally, the third central moment equals

E (𝑋 − E 𝑋)⊗3 = (Σ1/2𝑈)⊗3 E (𝑌 − E 𝑌)⊗3 = (Σ1/2𝑈)⊗3 𝜅3 (𝛾)𝑒1⊗3 = 𝜅3 (𝛾)

Σ𝑤

! ⊗3 ,

Σ1/2 𝑤

which completes the proof.

4.2

□

Sub-Gaussianity

For our algorithm we need high accuracy estimates of the moments of a halfspace truncated Gaussian. To get those, we first establish the following two claims about sub-Gaussianity of the truncated Gaussian. −1/2

Claim 4.3. Suppose 𝑋 ∼ 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Then Σ−1/2 𝑋 is 1-sub-Gaussian and 𝑀2 Gaussian.

𝑋 is 1/ 𝜅 2 (𝛾)-sub-

Corollary 4.4. Suppose 𝑋 ∼ 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Then 𝑋 is ∥Σ∥1/2 -sub-Gaussian. For the special case of the standard one-dimensional Gaussian, this was already shown in [BMA26]. We use this fact to prove the general case. Fact 4.5 (cf. Theorem 2.1 in [BMA26]). A random variable 𝑍 𝛾 from the standard Gaussian distribution 𝒩 (0, 1) truncated to (−∞, 𝛾] is 1-sub-Gaussian. Proof of Claim 4.3 and Corollary 4.4. A random vector is 𝑟-sub-Gaussian if its univariate projection along every direction 𝑣 is 𝑟-sub-Gaussian. For the first claim it therefore suffices to show that the random variable ⟨𝑣, Σ−1/2 𝑋⟩ is 1-sub-Gaussian. Define 𝑌 B 𝑈 ⊤ Σ−1/2 𝑋. We first show that this is 1-sub-Gaussian. Let 𝑣 be a unit vector. Then, we need to show that ⟨𝑣, 𝑌⟩ is 1-sub-Gaussian. 𝑑 Consider the canonical basis for R𝑑 defined using the vectors 𝑒 𝑗 𝑗=1 . Then, by expressing 𝑣 in this canonical basis we have ⟨𝑣, 𝑌⟩ =

𝑑 Õ

𝑣 𝑗 ⟨𝑒 𝑗 , 𝑌⟩ =

𝑗=1

𝑑 Õ

𝑣 𝑗 𝑌𝑗 .

𝑗=1

By construction, 𝑌1 is distributed identical to 𝑍𝜏 . Since one-dimensional truncated Gaussians are 1-sub-Gaussian due to Fact 4.5, 𝑌1 is 1-sub-Gaussian. Now, 𝑌2 , . . . , 𝑌𝑑 are independent onedimensional standard Gaussians. Therefore they are also 1-sub-Gaussian. Since the density 23

function is a product distribution over the coordinates, we have independence between the {𝑌𝑖 }𝑛𝑖=1 . Using the fact that addition of independent sub-Gaussian random variables is sub-Gaussian, and the fact that 𝑣 is a unit vector, it follows that 𝑌 is 1-sub-Gaussian. Multiplying a random vector by a matrix 𝐴 scales the sub-Gaussian parameter by ∥𝐴𝑇 ∥ (because |⟨𝑣, 𝐴𝑌⟩| ⩽ ∥𝐴⊤ ∥ · |⟨𝑣 ′ , 𝑌⟩| for 𝑣 ′ = 𝐴⊤ 𝑣/∥𝐴⊤ 𝑣∥). Thus, we have that Σ−1/2 𝑋 is also 1-sub-Gaussian (because ∥𝑈 ⊤ ∥ = 1). Furthermore, we also get that 𝑋 is ∥Σ1/2 ∥-sub-Gaussian. Finally, by Lemma 4.2, we have that 1/2 1/2 ⊤ ! Σ 𝑤 Σ 𝑤 𝑀2 = Σ1/2 · 𝐼 𝑑 − (1 − 𝜅2 (𝛾)) ·Σ1/2 . ∥Σ1/2 ∥ ∥Σ1/2 ∥

}

𝐵B

We have that 𝐵−1/2 Σ−1/2 𝑋 is ∥𝐵−1/2 ∥-sub-Gaussian. Since we get 𝐵 by removing a rank-1 component from the identity matrix, the eigenvalues of 𝐵 are 1 with multiplicity 𝑑 − 1 and 𝜅 2 (𝛾). Thus, p −1/2 ∥𝐵−1/2 ∥ = 1/ 𝜅2 (𝛾). Finally, note that 𝐵−1/2 Σ−1/2 and 𝑀2 differ by an orthogonal matrix (namely 1/2

−1/2

𝐵−1/2 Σ−1/2 𝑀2 ) and thus also 𝑀2

4.3

𝑋 is 1/ 𝜅2 (𝛾)-sub-Gaussian, which completes the proof.

□

Preconditioning

For our algorithm, we need to estimate the inverse of 𝑀2 . A straight-forward application of b 2 for 𝑀2 and then taking the inverse of 𝑀 b 2 would yield a sample Fact 3.26 to get an estimate 𝑀 complexity that depends on the condition number of 𝑀2 . The reason for this is that from Fact 3.26 −1/2 b −1/2 b −1 − 𝑀 −1 ∥. We can relate we get a bound on ∥𝑀2 𝑀 − 𝐼 𝑑 ∥ but we will need a bound on ∥ 𝑀 2 𝑀2 2 2 these two via the following inequality 1/2

−1/2 b

b −1 − 𝑀 −1 ∥ ⩽ ∥ 𝑀 b −1 ∥ · ∥𝑀 ∥ · ∥𝐼 𝑑 − 𝑀 ∥𝑀 2 2 2 2 2

−1/2

𝑀2 𝑀2

−1/2

∥ · ∥𝑀2

∥,

but in order to make this norm small, the sample complexity will depend on the condition number of 𝑀2 , which can be arbitrarily large. To avoid this additional dependency, we compute a natural preconditioner to ensure that our problem becomes well-conditioned. The guarantee of this preconditioning is the following, where, as before, 𝛼 denotes the probability mass of the halfspace under the target Gaussian distribution. Lemma 4.6. Without loss of generality, given 𝑛 = Θ(𝑑 · max{1, log4 1/𝛼}) samples from 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 , we can assume that 21 𝐼 𝑑 ⪯ 𝑀2 ⪯ 2𝐼 𝑑 and 21 𝐼 𝑑 ⪯ Σ ⪯ 𝜅22(𝛾) 𝐼 𝑑 with high probability. Using Lemma 4.6, we can obtain an inverse for 𝑀2 without incurring a condition number factor in the sample complexity. Throughout the rest of the paper, we will work with the assumption that we have preconditioned the data as described in this lemma. The proof follows directly from the following two claims. Claim 4.7. Given 𝑛 = Θ(𝑑 · max{1, log4 1/𝛼}) i.i.d. samples from 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 , with high probability e 2 = 1 Í𝑛 (𝑥 𝑖 − 𝑥)(𝑥 ¯ 𝑖 − 𝑥) ¯ ⊤ satisfies the sample covariance 𝑀 𝑖=1 𝑛 −1/2 e

𝐼 𝑑 − 𝑀2

−1/2 e

e 2 ≻ 0 satisfy 𝐼 𝑑 − 𝑀 Claim 4.8. Let 𝑀 2

−1/2

𝑀2 𝑀2

⩽

−1/2

𝑀2 𝑀2

⩽

assume that 21 𝐼 𝑑 ⪯ 𝑀2 ⪯ 2𝐼 𝑑 and 21 𝐼 𝑑 ⪯ Σ ⪯ 𝜅22(𝛾) 𝐼 𝑑 . 24

𝜅2 (𝛾) . 2

𝜅2 (𝛾) 2 . Then, without loss of generality, we can

Proof of Claim 4.7. As long as we have 𝑛 = Ω(𝑑/𝛽2 ) samples, we know from Fact 3.26 that −1/2 e

∥𝐼 𝑑 − 𝑀2

−1/2

𝑀2 𝑀2

∥≲

1 𝛽, 𝜅2 (𝛾)

since for every sample 𝑥 𝑖 , 𝑀2−1 𝑥 𝑖 is 1/ 𝜅2 (𝛾)-sub-Gaussian by Claim 4.3. Setting 𝛽 = Ω(𝜅 2 (𝛾)2 )

and using Claim 3.22 which lower bounds 𝜅2 (𝛾) in terms of 𝛼 as 𝜅 2 (𝛾) ⩾ min obtain the result.

1/20, 8 log1 1/𝛼

, we

□ −1/2

e Proof of Claim 4.8. The idea of our preconditioning is to left-multiply the samples by 𝑀 , then 2 apply our algorithm and in the end undo the preconditioning by changing the estimates appropriately. We will first argue that the moments after this preconditioning satisfy what we claim and then explain how to undo the preconditioning while keeping the error the same. Moments are well-conditioned. By Claim 4.1, the covariance of the untruncated Gaussian e −1/2 Σ 𝑀 e −1/2 . The second central moment of the truncated after this preconditioning is Σ′ = 𝑀 2 2 e e 𝑀2 𝑀 . We want to argue that 21 𝐼 𝑑 ⪯ 𝑀2′ ⪯ 2𝐼 𝑑 and 21 𝐼 𝑑 ⪯ Σ′ ⪯ 𝜅22(𝛾) 𝐼 𝑑 . Gaussian is 𝑀2′ = 𝑀 2 2 From our assumption, we get that (since 𝜅2 (𝛾) ⩽ 1 by Claim 3.7) −1/2

−1/2

1 3 −1/2 e −1/2 𝐼 𝑑 ⪯ 𝑀2 𝑀 ⪯ 𝐼𝑑 . 2 𝑀2 2 2 −1/2

1/2

e Left multiplying by 𝑀 𝑀2 and right multiplying by the transpose of this matrix (which 2 preserves the Loewner order), we get 2 1 e −1/2 ⪯ 2𝐼 𝑑 , e −1/2 𝑀2 𝑀 𝐼 𝑑 ⪯ 𝐼 𝑑 ⪯ 𝑀2′ = 𝑀 2 2 2 3 which proves that 𝑀2′ is well-conditioned. For Σ′ , recall from Lemma 4.2 that 𝑀2 = Σ − (1 − 𝜅2 (𝛾)) · Σ1/2 = Σ1/2 𝐼 𝑑 − (1 − 𝜅2 (𝛾)) ·

Σ1/2 𝑤

!⊤!

We have 𝜅 2 (𝛾)𝐼 𝑑 ⪯ 𝐼 𝑑 − (1 − 𝜅2 (𝛾)) ·

Σ1/2

!⊤! Σ1/2 .

!⊤ ⪯ 𝐼𝑑

−1/2 1/2 Σ and right multiplying by the transpose of this matrix,

e and thus, by left multiplying by 𝑀 2 we get Hence, it follows that

𝜅2 (𝛾)Σ′ ⪯ 𝑀2′ ⪯ Σ′ .

1 2 1 𝐼 𝑑 ⪯ 𝑀2′ ⪯ Σ′ ⪯ 𝑀2′ ⪯ 𝐼𝑑 . 2 𝜅2 (𝛾) 𝜅2 (𝛾)

Undoing the preconditioning. Assume we have estimates 𝜇ˆ 𝑇 and b Σ𝑇 in the transformed space. 1/2 1/2 b e 1/2 e e b We claim that 𝜇ˆ B 𝑀2 𝜇ˆ 𝑇 and Σ B 𝑀2 Σ𝑇 𝑀2 have the same error in the original space as 𝜇ˆ 𝑇 and b Σ𝑇 have in the transformed space. This relies on the fact that the Mahalanobis norm, in which we measure the error, is affine-invariant, as the following claim shows. We defer the proof to Section A.2. 25

Fact 4.9. Let 𝐴, 𝐵 ∈ R𝑑×𝑑 be positive definite matrices. Let 𝐿 ∈ R𝑑×𝑑 be an invertible matrix. Then, we have that 1 1 1 1 𝐼 𝑑 − 𝐴− 2 𝐵𝐴− 2 = 𝐼 𝑑 − (𝐿𝐴𝐿⊤ )− 2 𝐿𝐵𝐿⊤ (𝐿𝐴𝐿⊤ )− 2 𝐹

𝐹

e −1/2 , we have By Fact 4.9 for 𝐴 = Σ, 𝐵 = e Σ and 𝐿 = 𝑀 𝐼 𝑑 − Σ−1/2 e ΣΣ−1/2

𝐹

= 𝐼 𝑑 − Σ′−1/2 e Σ𝑇 Σ′−1/2 −1/2

e Furthermore, the mean after preconditioning is 𝜇′ = 𝑀 2 Σ′−1/2 (𝜇ˆ 𝑇 − 𝜇′ ) =

e −1/2 Σ 𝑀 e −1/2 𝑀 −1/2

e = 𝑀 2

−1/2

𝜇 by Claim 4.1 and thus we also have

. 𝐹

−1/2

e 𝑀 2

−1/2 −1/2 −1/2 e −1/2 −1/2 e e e 𝑀 (𝜇ˆ − 𝜇), 𝑀 Σ𝑀 𝑀 (𝜇ˆ − 𝜇) 2

−1/2

e Σ𝑀 2

−1

−1/2

e 𝑀 2

(𝜇ˆ − 𝜇), (𝜇ˆ − 𝜇)

= Σ−1 (𝜇ˆ − 𝜇), 𝜇ˆ − 𝜇 = Σ−1/2 (𝜇ˆ − 𝜇)

Thus, both the mean estimator 𝜇ˆ and the covariance estimator b Σ have the same error as 𝜇ˆ 𝑇 and b Σ𝑇 . −1/2 e Hence, we can indeed precondition the samples by left-multiplying them by 𝑀2 , then run our algorithm on the well-conditioned instance and output 𝜇ˆ and b Σ. The error does not change by this and thus we can without loss of generality assume that our instance is well-conditioned. □

Proof of the main result

In this section, we prove our main result, Theorem 1.1, that we restate below. Theorem 5.1 (Full statement of Theorem 1.1). Consider a halfspace truncated Gaussian 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 , where Σ ≻ 0. Let 𝛼 be the mass of the truncation set. Then, given

2  𝑑  polylog(𝛼 −1 ) if 𝛼 ⩽ 0.99 𝑂   𝜀2 𝑛= 2  if 𝛼 > 0.99  𝑂 𝑑𝜀4  ˆ b samples, Algorithm 2 computes estimators 𝜇, Σ such that with probability at least 0.99

ˆ b 𝑑TV 𝒩 (𝜇, Σ), 𝒩 (𝜇, Σ) ⩽ 𝜀 The runtime of the algorithm is 𝑂(𝑇(𝑛, 𝑑)), where 𝑇(𝑛, 𝑑) is the time needed to multiply a 𝑑 × 𝑛 matrix with its transpose. Remark 5.2. We do not need to know 𝛼 to run our algorithm. It is sufficient that we get as input a lower bound 𝛼 0 on 𝛼. Then we can replace 𝛼 by 𝛼0 in the sample complexity bound of Theorem 5.1 (which makes the sample 2 /𝜀4 samples. If complexity bound only larger) and run the algorithm with 𝑂 𝑑2 /𝜀2 · polylog(𝛼 −1 ) + 𝑑 0 we additionally know whether 𝛼 ⩽ 0.99 or not, we can run the algorithm with 𝑂 𝑑 2 /𝜀2 · polylog(𝛼 −1 ) or 0 𝑂 𝑑 2 /𝜀4 samples respectively. 26

Algorithm 2: Algorithm for estimating the mean and covariance of a halfspace truncated Gaussian Input: Dimension 𝑑, sample size 𝑛, samples {𝑥 𝑖 }𝑛𝑖=1 , target accuracy 𝜀. Output: Estimated mean 𝜇ˆ ∈ R𝑑 and covariance b Σ ∈ R𝑑×𝑑 . 1. Compute sample mean: 𝑥¯ = 𝑛1

Í𝑛

𝑖=1 𝑥 𝑖 .;

b2 = 1 2. Compute sample second central moment: 𝑀 𝑛

Í𝑛

¯ 𝑖=1 (𝑥 𝑖 − 𝑥)

⊗2 .;

3. Draw a standard Gaussian vector: 𝑔 ∼ 𝒩 (0, 𝐼 𝑑 ).;

b3 = 1 4. Compute the random contraction: 𝑅 = (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀 𝑛

Í𝑛

¯ 𝑖=1 (𝑥 𝑖 − 𝑥)

⊗2 · ⟨𝑔, 𝑥

¯ 𝑖 − 𝑥⟩.

ˆ the eigenvector of 𝑅 corresponding to the largest eigenvalue in absolute 5. Compute 𝑢, value (with unit norm). −1

6. Estimate the relative truncation parameter: 𝛾ˆ = skew

−

b −1 𝑢ˆ 𝑢ˆ ⊤ 𝑀 2

3/2 D ·

b 𝑢ˆ ⊗3 , 𝑀

b 3 < 0. 7. Flip the sign of 𝑢ˆ if 𝑢ˆ ⊗3 , 𝑀 ˆ 𝜅 ( 𝛾)

8. Estimate the mean of the Gaussian: 𝜇ˆ = 𝑥¯ + √ 1

b2 + 9. Estimate the covariance of the Gaussian: b Σ=𝑀

ˆ 𝜅2 (𝛾)

b −1 𝑢ˆ 𝑢ˆ ⊤ 𝑀 2

−1/2

1 ˆ −1 𝜅2 (𝛾)

ˆ 𝑢.

b −1 𝑢ˆ · 𝑢ˆ ⊤ 𝑀 2

−1

𝑢ˆ 𝑢ˆ ⊤ .

ˆ b 10. Return estimates 𝜇, Σ .

Note that we argued in Section 2 that if we would have access to the exact moments, then the estimators 𝜇ˆ and b Σ are equal to 𝜇 and Σ respectively. In this section, we prove that even if we only have access to the sample moments, the above estimators work. The proof of this theorem is structured as follows: Σ𝑤 • In Section 5.1, we prove that 𝑢ˆ (after the potential sign flip) is good estimator for − ∥Σ𝑤∥ ,

b 3 is a good estimator for 𝑀3 . assuming that 𝑀 • In Section 5.2, we prove that 𝛾ˆ is a good estimator for 𝛾, given that we have a good estimator b 2 is a good estimator for 𝑀2 . for 𝑢ˆ and that 𝑀 b 2 , 𝑢ˆ and • In Section 5.3, we prove that 𝜇ˆ and b Σ are good estimators for 𝜇 and Σ, given that 𝑀 𝛾ˆ are good estimators. • Finally, in Section 5.4, we put everything together: We determine the sample complexity such b 2 and 𝑀 b 3 are good enough estimators such that the lemmas proved in Sections 5.1 that 𝑀 to 5.3 imply the guarantees of Theorem 5.1. We remark that the assumption Σ ≻ 0 is not necessary. Suppose Σ is of rank 0 < 𝑟 < 𝑑, then all samples would live in a subspace of R𝑑 of dimension 𝑟. One can identify this subspace with 𝑂(𝑑) samples with high probability, and this reduces the original problem to an 𝑟-dimensional problem via an appropriate linear transform (e.g., an orthogonal projection from R𝑑 to the subspace).

5.1

Recovering the direction of Σ𝑤

In order to find the direction of Σ𝑤, a natural attempt would be to consider the tensor analogue b 3 . However, unlike the matrix case, it of low-rank approximations of the sample third moment 𝑀 is NP-hard to find the best rank-1 approximation of a general symmetric order-3 tensor [HL13]. To bypass this computational barrier, we reduce the problem to that of finding the eigenvector corresponding to the largest absolute eigenvalue in a certain matrix. We obtain such a matrix by contracting the tensor with a random vector. Throughout this section we will use the notation ∗ ⊗3 . Our 𝑢 ∗ = 𝜅3 (𝛾)1/3 ∥ΣΣ𝑤 1/2 𝑤∥ . It then follows that the population third central moment 𝑀3 = (𝑢 ) Σ𝑤 . We will require the following claims for our proof that estimator 𝑢ˆ will approximate ∥𝑢𝑢 ∗ ∥ = − ∥Σ𝑤∥ will follow next. We defer the proofs to Section A.2. ∗

Claim 5.3. Let 𝐴, 𝐵 ∈ R𝑑×𝑑 be symmetric matrices with 𝐵 being an arbitrary rank-1 matrix. Consider the spectral decomposition of 𝐴, 𝐴=

𝑑 Õ

𝜆𝑖 𝑣 𝑖 𝑣 ⊤ 𝑖

𝑖=1

where the vectors {𝑣 𝑖 }𝑑𝑖=1 form an orthonormal basis of R𝑑 and the singular values {𝜆 𝑖 }𝑑𝑖=1 satisfy |𝜆1 | ⩾ |𝜆2 | ⩾ · · · ⩾ |𝜆 𝑑 |. We then have that 2

𝜆1 𝑣 1 𝑣 1⊤ − 𝐵 𝐹 ⩽ 2⟨𝜆1 𝑣 1 𝑣 1⊤ − 𝐵, 𝐴 − 𝐵⟩. 𝑎 and Claim 5.4. Let 𝑎, 𝑏 ∈ R𝑑 . Let 𝛽 > 0. Suppose that ∥𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ ∥𝐹 ⩽ 𝛽. Then, we have that, for 𝑢 = ∥𝑎∥ 𝑏 𝑣 = ∥𝑏∥ ,

min{∥𝑢 + 𝑣∥2 , ∥𝑢 − 𝑣∥2 } ⩽ 8

𝛽2 ∥𝑎∥4

The main result of this section is the following lemma that shows that given an accurate estimate for 𝑀3 , we can recover the direction of 𝑢 ∗ . We will show afterwards how many samples we need to get such an estimate for 𝑀3 .

b 3 ∈ R𝑑×𝑑×𝑑 satisfying Lemma 5.5. Given 𝛽 > 0 and a symmetric tensor 𝑀 D

b 3 ⩽ 𝛽, sup 𝑣 ⊗3 , 𝑀3 − 𝑀 ∥𝑣∥=1

one can compute a unit vector 𝑢ˆ ∈ R𝑑 such that with probability at least 1 − 10−4 − 𝑒 −2𝑑 √ 𝑑 𝑢∗ 𝑢∗ 𝛽 . min 𝑢ˆ + ∗ , 𝑢ˆ − ∗ ⩽𝑂 ∥𝑢 ∥ 2 ∥𝑢 ∥ 2 |𝜅3 (𝛾)| ˆ we first draw 𝑔 ∼ 𝒩 (0, 𝐼 𝑑 ) and compute the matrix Proof. To compute the unit vector 𝑢,

b3 𝑅 = (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀 We then compute the eigenvector corresponding to the largest (absolute) eigenvalue of 𝑅. We will show that this gives an estimator 𝑢ˆ with the claimed guarantees. b 3 is a symmetric matrix. Let 𝑢ˆ be the (unit) We begin by observing that (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀 eigenvector associated with its largest eigenvalue in absolute value, and 𝜆 be the corresponding 28

b 3 . Our goal is to show eigenvalue. Then 𝜆 · 𝑢ˆ 𝑢ˆ ⊤ is the best rank-1 approximation of (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀 𝑢∗ that 𝑢ˆ is close to ∥𝑢 ∗ ∥ . b 3 and (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀3 are close in Frobenius norm. First, we will show that (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀 Then, since (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀3 = 𝑔, 𝑢 ∗ · 𝑢 ∗ (𝑢 ∗ )⊤ has rank 1, we will be able to use Claim 5.4 that relates closeness in Frobenius norm of rank-1 matrices to the closeness of the vectors themselves to conclude the lemma. Invoking Claim 5.3, we obtain b 3 − 𝑀3 )⟩ ∥𝜆𝑢ˆ 𝑢ˆ ⊤ − 𝑔, 𝑢 ∗ 𝑢 ∗ (𝑢 ∗ )⊤ ∥2𝐹 ⩽ 2⟨𝜆𝑢ˆ 𝑢ˆ ⊤ − 𝑔, 𝑢 ∗ · 𝑢 ∗ (𝑢 ∗ )⊤ , (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · ( 𝑀 (i)

b 3 − 𝑀3 )∥ ⩽ 2∥𝜆𝑢ˆ 𝑢ˆ ⊤ − 𝑔, 𝑢 ∗ · 𝑢 ∗ (𝑢 ∗ )⊤ ∥∗ · ∥(𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · ( 𝑀 (ii)

√

b 3 − 𝑀3 ) ⩽ 2 2∥𝜆𝑢ˆ 𝑢ˆ ⊤ − 𝑔, 𝑢 ∗ · 𝑢 ∗ (𝑢 ∗ )⊤ ∥𝐹 · (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · ( 𝑀

In inequality (i) we used matrix Hölder’s inequality. In (ii) we used that 𝜆𝑢ˆ 𝑢ˆ ⊤ − 𝑔, 𝑢 ∗ 𝑢 ∗ (𝑢 ∗ )⊤ has √ rank at most 2 and thus its nuclear norm is bounded by 2 times its Frobenius norm. We can do the exact same computation with −𝑔 instead of 𝑔 and thus we get √ b 3 − 𝑀3 ) . ∥𝜆𝑢ˆ 𝑢ˆ ⊤ ± 𝑔, 𝑢 ∗ 𝑢 ∗ (𝑢 ∗ )⊤ ∥𝐹 ⩽ 2 2 (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · ( 𝑀 From our assumption we have

b 3 − 𝑀3 ) ⩽ ∥𝑔∥ · sup ⟨𝑣 ⊗3 , 𝑀 b 3 − 𝑀3 ⟩ ⩽ 𝑔 · 𝛽. (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · ( 𝑀 ∥𝑣∥=1

Using Fact 3.25 about the norm of a Gaussian random vector we have

h h √ i √ i P ∥𝑔∥ ⩾ 3 𝑑 ⩽ P ∥𝑔∥ − E∥𝑔∥ ⩾ 2 𝑑 ⩽ 𝑒 −2𝑑 . Thus, conditioning on the above event we have with probability at least 1 − 𝑒 −2𝑑 that √ √ ∥𝜆𝑢ˆ 𝑢ˆ ⊤ ± 𝑔, 𝑢 ∗ 𝑢 ∗ (𝑢 ∗ )⊤ ∥𝐹 ⩽ 6 2 · 𝑑 · 𝛽. Dividing by 𝑔, 𝑢 ∗

and choosing the appropriate sign, we get that |𝜆| 𝑔, 𝑢 ∗

√

𝑢ˆ 𝑢ˆ ⊤ − 𝑢 ∗ (𝑢 ∗ )⊤

√

⩽ 6 2· 𝑑·𝛽· 𝐹

1 𝑔, 𝑢 ∗

Since ⟨𝑔, 𝑢 ∗ ⟩ is a Gaussian of variance ∥𝑢 ∗ ∥2 , Fact 3.24 implies that with probability at least 1 − 10−4 we have |⟨𝑔, 𝑢 ∗ ⟩| ⩾ 10−4 ∥𝑢 ∗ ∥. Combining this with the above using a union bound, with probability 1 − 𝑒 −2𝑑 − 10−4 we have |𝜆| 𝑔, 𝑢 ∗

√

𝑢ˆ 𝑢ˆ ⊤ − 𝑢 ∗ (𝑢 ∗ )⊤

√

⩽ 6 2· 𝑑·𝛽· 𝐹

1000 . ∥𝑢 ∗ ∥

Using Claim 5.4 that relates Frobenius closeness to closeness of vectors themselves we obtain √ 24000 𝑑 · 𝛽 ˜ ∥𝑢ˆ + 𝑢∥} ˜ ⩽ min{∥𝑢ˆ − 𝑢∥, , ∥𝑢 ∗ ∥3 29

where 𝑢˜ B ∥𝑢𝑢 ∗ ∥ . Now, since Σ ⪰ 12 𝐼 𝑑 , we get that ∗

∥𝑢 ∗ ∥3 = |𝜅3 (𝛾)| ·

Σ𝑤 ∥Σ1/2 𝑤∥

⩾ |𝜅3 (𝛾)| · 𝜆min (Σ1/2 )3 ⩾

Therefore we have

|𝜅 3 (𝛾)| √ . 2 2

√

˜ ∥𝑢ˆ + 𝑢∥} ˜ ≲ min{∥𝑢ˆ − 𝑢∥,

𝑑·𝛽 , |𝜅3 (𝛾)|

completing the proof.

□ So far we have shown how to recover a unit vector that is (approximately) parallel to Σ𝑤 using a random contraction on the sample third moment. To this end, we required a strong assumption on the finite-sample error to which we learn the third moment in the weaker injective tensor norm as opposed to the Frobenius norm. In particular, we required this error to be as small as inverse polynomial in the dimension (up to 𝜅2 , 𝜅3 factors) for the statement of the previous lemma. We will now show that we can indeed achieve such a high-accuracy estimate in the injective tensor norm using only 𝑂(𝑑 2 ) samples (up to 𝜅 2 , 𝜅3 factors). We start with the following claim that we will require. Claim 5.6. Let 𝜈 be the mean of 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Let 𝑣 be a fixed unit vector. Given 𝑛 i.i.d. samples {𝑥 𝑖 }𝑛𝑖=1 from 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 , for every 𝑡 > 0, we have that

" P

𝑛

1Õ ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 − ⟨𝑣, 𝑢 ∗ ⟩3 ⩾ 𝐶𝜅 2 (𝛾)−3/2 𝑛 𝑖=1

𝑡 3/2 𝑡 + 𝑛 𝑛

!# ⩽ 2 exp(−𝑡).

for an absolute constant 𝐶 > 0. Proof. The above claim follows by a direct application of Corollary 3.29 for the concentration of averages-of-cubes of sub-Gaussian random variables that we proved earlier. Since ⟨𝑣, 𝑥 𝑖 − 𝜈⟩ is sub-Gaussian with parameter 𝑂 1/ 𝜅2 (𝛾) (after preconditioning) and E ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 = ⟨𝑣, 𝑢 ∗ ⟩3 ,

we apply Corollary 3.29 with 𝑍 𝑖 = ⟨𝑣, 𝑥 𝑖 − 𝜈⟩ and 𝜎 = 𝑂 1/ 𝜅2 (𝛾) to obtain the result.

□

We remark that the above claim argues about concentration of zero-mean random variables. We will now account for the error incurred by finite sample approximations to 𝜈. More formally we show the following. Claim 5.7. Let 𝜈 be the mean of 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Let 𝑣 be a fixed unit vector. Let {𝑥 𝑖 }𝑛𝑖=1 be i.i.d samples Í from 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 and let 𝑥¯ = 𝑛1 𝑛𝑖=1 𝑥 𝑖 . Then, we have the following with probability at least 1 − 4𝑒 −10𝑑 𝑛

1Õ ¯ 3 − ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 ≲ 𝜅 2 (𝛾)−3/2 · ⟨𝑣, 𝑥 𝑖 − 𝑥⟩ 𝑛 𝑖=1

3/2 !

𝑑 𝑑 + 𝑛 𝑛

We defer the proof of Claim 5.7 to the Section A.3. Given the above concentration results, we will now present the main sample complexity result for this section. For 𝑛 = 𝑂(𝑑2 ) (up to 𝜅2 , 𝜅3 factors), the result below enables us to satisfy the requirements for the main result of this section Lemma 5.5. 30

Lemma 5.8. Let {𝑥 𝑖 }𝑛𝑖=1 be i.i.d. samples from 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Let 𝑥¯ = 𝑛1 𝑛𝑖=1 𝑥 𝑖 . Í b 3 = 1 𝑛 (𝑥 𝑖 − 𝑥) ¯ ⊗3 . Then we have with probability at least 1 − 6𝑒 −7𝑑 that the following holds 𝑀 𝑖=1 𝑛

b 3 − 𝑀3 ≲ 𝜅2 (𝛾)−3/2 sup 𝑣 ⊗3 , 𝑀 ∥𝑣∥=1

𝑑 3/2

𝑑 + 𝑛 𝑛

Let

! .

We give a proof sketch here and show that the above holds for any fixed 𝑣 with probability 1 − 6𝑒 −10𝑑 . The lemma then follows by a standard 𝜀-net argument over the unit sphere. We defer the full proof to Section A.3. Proof sketch. For a fixed vector 𝑣, taking 𝑡 = Θ(𝑑) in Claim 5.6, we have for an absolute constant 𝐾 1 > 0 that

" P

𝑛

1Õ ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 − ⟨𝑣, 𝑢 ∗ ⟩3 ⩾ 𝐾 1 · 𝜅 2 (𝛾)−3/2 𝑛

𝑖=1

𝑑 𝑑 3/2 + 𝑛 𝑛

!# ⩽ 2𝑒 −10𝑑 .

For the same fixed choice of 𝑣, we have for some other absolute constant 𝐾 2 > 0 from Claim 5.7 that " r 3/2 ! # 𝑛 𝑑 𝑑 1Õ −3/2 3 3 ¯ − ⟨𝑣, 𝑥 𝑖 − 𝜈⟩ ⩾ 𝐾 2 · 𝜅 2 (𝛾) · ⩽ 4𝑒 −10𝑑 . ⟨𝑣, 𝑥 𝑖 − 𝑥⟩ + P 𝑛 𝑛 𝑛 𝑖=1

Now since we have that

b 3 − 𝑀3 𝑣 ⊗3 , 𝑀

E ⩽

𝑛

𝑖=1

1Õ 1Õ ¯ 3 − ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 + ⟨𝑣, 𝑥 𝑖 − 𝑥⟩ ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 − ⟨𝑣, 𝑢 ∗ ⟩3 , 𝑛 𝑛

we can conclude that

b 3 − 𝑀3 𝑣 ⊗3 , 𝑀

r ⩽ 𝐾 · 𝜅2 (𝛾)−3/2

𝑑 𝑑3/2 + 𝑛 𝑛

with probability at least 1D− 6𝑒 −10𝑑 via aE union bound. So far, we have argued that for a fixed

b 3 − 𝑀3 is well concentrated. What remains is to show that the vector 𝑣, it is the case that 𝑣 ⊗3 , 𝑀

supremum of the quantity over the unit sphere can still be controlled. To this end, we construct an 𝜀-net, and complete the proof with a union bound argument over this net in Section A.3. □

5.2

Relative truncation parameter recovery

Our goal in the section will be to recover the relative truncation parameter 𝛾. From the previous section, we have a good estimate for the direction of Σ𝑤. In this section we will show how we can use this vector to recover the relative truncation parameter 𝛾. Recovering 𝛾 allows us to recover the parameters of the Gaussian directly from the moments. We first show how to get an estimate ˆ for skew(𝛾). Then, we show that this implies generally (meaning without any assumptions skew(𝛾) ˆ and skew(𝛾) are close) that also 𝛾ˆ is close to 𝛾. except that skew(𝛾) Lemma 5.9. Let 0 < 𝛽 < 𝑐, where 𝑐 is some sufficiently small absolute constant. Assume that min

𝑢∗ 𝑢∗ , 𝑢ˆ − ∗ 𝑢ˆ + ∗ ⩽ 𝛽, ∥𝑢 ∥ 2 ∥𝑢 ∥ 2

−1/2 b

−1/2

𝑀2 𝑀2

∥𝑀2

− 𝐼𝑑 ∥ ⩽ 𝛽

b 2 ≻ 0. Furthermore, let 𝑀 b 3 be a symmetric tensor satisfying for a unit vector 𝑢ˆ and a symmetric matrix 𝑀 D

b 3 − 𝑀3 ⩽ 𝛽. sup 𝑣 ⊗3 , 𝑀 ∥𝑣∥=1

Denoting by skew−1 the inverse function of skew, consider the estimator −1

𝛾ˆ B skew

−

b −1 𝑢ˆ 𝑢ˆ ⊤ 𝑀 2

3/2

⊗3

b 3 ⟩| . · |⟨𝑢ˆ , 𝑀

Then, we have that

ˆ − skew(𝛾)| ⩽ 𝑂 𝛽 · 𝜅 2 (𝛾)−3/2 |skew(𝛾)

as well as

b −1 𝑢ˆ − 𝑢˜ ⊤ 𝑀 −1 𝑢˜ ⩽ 𝑂(𝛽). 𝑢ˆ ⊤ 𝑀 2 2 At a high level, the above lemma say that we can learn the relative truncation parameter accurately given access to high-accuracy estimates for Σ𝑤, 𝑀2 and 𝑀3 by inverting the onedimensional skew function. We will require the following claim that bounds ∥𝑢 ∗ ∥. √ √1 |𝜅 3 (𝛾)|1/3 ⩽ ∥𝑢 ∗ ∥ ⩽ 2 · |skew(𝛾)|1/3 . Claim 5.10. For 𝑢 ∗ B 𝜅 3 (𝛾)1/3 ∥ΣΣ𝑤 1/2 𝑤∥ , we have 2

√

1/2

2 1/2 · Σ 𝑤 . By Lemma 4.6, we have √1 𝐼 ⪯ Σ1/2 ⪯ Proof. First, note that ∥ΣΣ𝑤 𝐼 and 𝑑 1/2 𝑤∥ = Σ ∥Σ1/2 𝑤∥ 𝜅2 (𝛾)1/2 𝑑 2 thus √ 1/2 𝑤 1 2 1/2 Σ ⩽ . √ ⩽ Σ 1/2 ∥Σ 𝑤∥ 𝜅2 (𝛾)1/2 2

This proves the claim by recalling that skew(𝛾) = 𝜅3 (𝛾)/𝜅2 (𝛾)3/2 .

□

Proof of Lemma 5.9. The proof of the lemma has two main parts: a population level analysis and a finite-sample error analysis. We further split the finite sample error analysis into two sub-parts. We will now begin with a population-level analysis that will show that knowing Σ𝑤, 𝑀2 and 𝑀3 will allow us to recover 𝛾. Recalling the definition of the population second central moment we have 𝑀2 = Σ −

1 − 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2

· (Σ𝑤) (Σ𝑤)⊤

This implies that when applying the linear transformation 𝑀2 on 𝑤, we get that 𝑀2 𝑤 = 𝜅2 (𝛾) · Σ𝑤 =⇒ 𝑤 = 𝜅2 (𝛾) · 𝑀2−1 Σ𝑤 Now by left-multiplying both sides by (Σ𝑤)⊤ and dividing by 𝜅 2 (𝛾) we obtain that (Σ𝑤)⊤ 𝑀2−1 (Σ𝑤) = 32

∥Σ1/2 𝑤∥2 . 𝜅2 (𝛾)

Σ𝑤 ˜ = ∥𝑢𝑢 ∗ ∥ = − ∥Σ𝑤∥ as the unit vector along the direction Recall that 𝑢 ∗ = 𝜅 3 (𝛾)1/3 ∥ΣΣ𝑤 1/2 𝑤∥ . We define 𝑢 ∗ of 𝑢 (note that 𝜅3 (𝛾) < 0 by Claim 3.8). Then we have ∗

𝑢˜ ⊤ 𝑀2−1 𝑢˜ =

𝜅3 (𝛾)2/3 ∥Σ1/2 𝑤∥2 = 𝜅2 (𝛾)∥Σ𝑤∥2 𝜅2 (𝛾)∥𝑢 ∗ ∥2

(5.1)

Furthermore recalling that the population third central moment is 𝑀3 = (𝑢 ∗ )⊗3 and taking its projection along 𝑢˜ we have that

Σ𝑤 Σ𝑤 ˜ = −𝜅3 (𝛾) , ⟨𝑀3 , 𝑢˜ ⟩ = ⟨𝑢 , 𝑢⟩ 1/2 ∥Σ𝑤∥ ∥Σ 𝑤∥ ⊗3

∗

3 = |𝜅3 (𝛾)| ·

∥Σ𝑤∥3 = ∥𝑢 ∗ ∥3 . 1/2 3 ∥Σ 𝑤∥

This naturally gives us the following identity that we will need for building our estimator: − 𝑢˜ ⊤ 𝑀2−1 𝑢˜

3/2

· ⟨𝑀3 , 𝑢˜ ⊗3 ⟩ =

𝜅3 (𝛾) 𝜅2 (𝛾)3/2

In the rest of the proof, we will carry out finite-sample error analysis since we only have access to the sample moments. We will first deal with the first term on the left hand side above.

Error analysis for 𝑢˜ ⊤ 𝑀2−1 𝑢˜

3/2

b −1 is symmetric, we get that Observe that, since 𝑀 2 b −1 𝑢ˆ = 𝑢˜ ⊤ 𝑀 b −1 𝑢˜ + (𝑢ˆ − 𝑢) b −1 (𝑢ˆ + 𝑢) ˜ ⊤𝑀 ˜ 𝑢ˆ ⊤ 𝑀 2 2 2 b −1 − 𝑀 −1 )𝑢˜ + (𝑢ˆ − 𝑢) b −1 (𝑢ˆ + 𝑢) ˜ ˜ ⊤𝑀 = 𝑢˜ ⊤ 𝑀2−1 𝑢˜ + 𝑢˜ ⊤ ( 𝑀 2 2 2 =

𝜅3 (𝛾)2/3 ⊤ b −1 −1 b −1 (𝑢ˆ + 𝑢) ˜ ˜ ⊤𝑀 ˜ + 𝑢 𝑀 − 𝑀 𝑢˜ + (𝑢ˆ − 𝑢) 2 2 2 ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾)

We will now bound the second and third term in the last equation above. For the second term, b −1 ⪯ 1 𝑀 −1 . Therefore we have b 2 ⪯ (1 + 𝛽)𝑀2 , it follows that 1 𝑀 −1 ⪯ 𝑀 since (1 − 𝛽)𝑀2 ⪯ 𝑀 1+𝛽 1−𝛽 2 2 2

b −1 ∥ = ∥ 𝑀 b −1 ( 𝑀 b 2 − 𝑀2 )𝑀 −1 ∥ ∥𝑀2−1 − 𝑀 2 2 2 1/2

−1/2 b

b −1 𝑀 (𝑀 = ∥𝑀 2 2 2

−1/2

𝑀2 𝑀2

−1/2

− 𝐼 𝑑 )𝑀2

∥

b −1 ∥ · ∥𝑀 1/2 ∥ · ∥𝑀 −1/2 ∥ ⩽ 𝛽 · ∥𝑀 2 2 2 (i)

⩽

4𝛽 ⩽ 8𝛽, 1−𝛽

as long as 𝛽 < 12 , where the inequality (i) follows from using our preconditioning guarantees proved in Lemma 4.6. Therefore

b −1 − 𝑀 −1 )𝑢˜ ⩽ 8𝛽. 𝑢˜ ⊤ ( 𝑀 2 2 Now for the third term we have that

b −1 (𝑢ˆ + 𝑢)| b −1 ∥ ⩽ 𝑂(𝛽) ˜ ⊤𝑀 ˜ ⩽ ∥𝑢ˆ − 𝑢∥ ˜ · ∥𝑢ˆ + 𝑢∥ ˜ · ∥𝑀 |(𝑢ˆ − 𝑢) 2 2 33

whenever 𝛽 < 1/2. Putting the two bounds together, we obtain,

b −1 𝑢ˆ − 𝑢˜ ⊤ 𝑀 −1 𝑢˜ = 𝑢ˆ ⊤ 𝑀 b −1 𝑢ˆ − 𝑢ˆ ⊤ 𝑀 2 2 2

𝜅3 (𝛾)2/3 ⩽ 𝑂(𝛽) ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾)

We are almost done as we have shown closeness of the quadratic forms. We will now show that raising the quadratic forms to a higher power of 3/2 still preserves closeness and conclude this part of the proof next. We will do this by an application of the mean value theorem from univariate calculus. To this end, we will try to obtain a bound on the range of values that the quadratic forms can take. Firstly, it is clear that

b −1 𝑢ˆ > 0, 𝑢ˆ ⊤ 𝑀 2

𝜅 3 (𝛾)2/3 > 0. ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾)

Now since ∥𝑢 ∗ ∥ ⩾ 𝜅3 (𝛾)1/3 · 𝜆min (Σ1/2 ) we have that 𝜅3 (𝛾)2/3 2 1 1 ⩽ ⩽ = ∗ 2 2 1/2 𝜅2 (𝛾) ∥𝑢 ∥ · 𝜅 2 (𝛾) 𝜅2 (𝛾)𝜆min (Σ ) 𝜅2 (𝛾)𝜆min (Σ)

(5.2)

where we used the fact that Σ ⪰ 12 𝐼 𝑑 . Applying the mean value theorem for the function 𝑓 (𝑥) = 𝑥 3/2 , 𝜅 (𝛾)2/3

b −1 𝑢ˆ and 𝑐 = ∗3 2 we know that there exists a 𝐵 between 𝑎 = 𝑢ˆ ⊤ 𝑀 such that the following holds 2 ∥𝑢 ∥ ·𝜅 (𝛾) 2

| 𝑓 (𝑎) − 𝑓 (𝑐)| = 𝑓 ′ (𝐵) · |𝑎 − 𝑐| =

3 √ · 𝐵 · |𝑎 − 𝑐|. 2

We have 𝐵 = 𝑂(1/𝜅2 (𝛾)) since 𝐵 ⩽ max

b −1 𝑢, 𝑢ˆ ⊤ 𝑀 2 ˆ

𝜅3 (𝛾)2/3 𝜅 3 (𝛾)2/3 𝜅 3 (𝛾)2/3 ⊤ b −1 ˆ ˆ ⩽ + 𝑢 𝑀 𝑢 − 2 ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾) ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾) ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾)

(ii) 1 2 ⩽ + 𝑂(𝛽) ⩽ 𝑂 . 𝜅2 (𝛾) 𝜅2 (𝛾) (i)

where (i) follows from inequality (5.2) and (ii) holds as long as 𝛽 is at most a sufficiently small constant. Therefore we have that

b −1 𝑢ˆ 𝑢ˆ ⊤ 𝑀 2

3/2

−

|𝜅3 (𝛾)| ∗ 3 ∥𝑢 ∥ · 𝜅 2 (𝛾)3/2

⩽ 𝑂(𝛽 · 𝜅2 (𝛾)−1/2 ),

concluding the first part of our error analysis.

Error analysis for ⟨𝑀3 , 𝑢˜ ⊗3 ⟩ ˜ ⩽ 𝛽 and analyze the case ∥𝑢ˆ + 𝑢∥ ˜ ⩽ 𝛽 afterwards. We We will first consider the case that ∥𝑢ˆ − 𝑢∥ have that

b 3 ⟩ = ⟨𝑢ˆ ⊗3 , 𝑀 b 3 − 𝑀3 ⟩ + ⟨𝑢ˆ ⊗3 , 𝑀3 ⟩ ⟨𝑢ˆ ⊗3 , 𝑀 2 3 ∗ 3 b ˜ 𝑢⟩ ˜ + 3 ⟨𝑢ˆ − 𝑢, ˜ 𝑢⟩ ˜ + ⟨𝑢ˆ − 𝑢, ˜ 𝑢⟩ ˜ = ⟨𝑢ˆ , 𝑀3 − 𝑀3 ⟩ + ∥𝑢 ∥ · 1 + 3 ⟨𝑢ˆ − 𝑢, . ⊗3

b 3 − 𝑀3 ⟩ ⩽ 𝛽. Thus, by applying Cauchy-Schwarz and By assumption, we have that sup∥𝑣∥=1 ⟨𝑣 ⊗3 , 𝑀 ˜ ⩽ 𝛽, we get that using ∥𝑢ˆ − 𝑢∥ b 3 ⟩ − ∥𝑢 ∗ ∥3 ⩽ 𝛽 + 𝑂(𝛽 · ∥𝑢 ∗ ∥3 ) ⩽ 𝑂(𝛽), ⟨𝑢ˆ ⊗3 , 𝑀 √ where we also used that ∥𝑢 ∗ ∥3 ⩽ 2 2|skew(𝛾)| ⩽ 𝑂(1) by Claims 3.18 and 5.10. Since ∥𝑢 ∗ ∥3 > 0, we even get that

b 3 ⟩ − ∥𝑢 ∗ ∥3 ⩽ 𝑂(𝛽). ⟨𝑢ˆ ⊗3 , 𝑀

(5.3)

˜ ⩽ 𝛽 by repeating the above argument for −𝑢ˆ instead of 𝑢. ˆ This also holds for the case ∥𝑢ˆ + 𝑢∥

Putting the pieces together We will now look at the total aggregate error that we incur. We begin by defining

b −1 𝑢ˆ 𝑆1 = 𝑢ˆ ⊤ 𝑀 2 and 𝜀1 = 𝑆1 −

3/2

b3⟩ , 𝑆2 = ⟨𝑢ˆ ⊗3 , 𝑀

|𝜅3 (𝛾)| ∥𝑢 ∗ ∥3 · 𝜅 2 (𝛾)3/2

, 𝜀2 = 𝑆2 − ∥𝑢 ∗ ∥3 .

By our computations earlier, we know that |𝜀1 | ⩽ 𝑂 𝛽 · 𝜅 2 (𝛾)−1/2 and |𝜀2 | ⩽ 𝑂 𝛽 . Now expanding 𝑆1 𝑆2 we have

𝑆1 𝑆2 = =

|𝜅 3 (𝛾)| ∥𝑢 ∗ ∥3 · 𝜅 2 (𝛾)3/2

+ 𝜀1

∥𝑢 ∗ ∥3 + 𝜀2

|𝜅3 (𝛾)| |𝜅3 (𝛾)| · ∥𝑢 ∗ ∥3 + 𝜀1 · ∥𝑢 ∗ ∥3 + 𝜀2 · ∗ 3 + 𝜀1 𝜀2 . ∗ 3 3/2 ∥𝑢 ∥ · 𝜅 2 (𝛾) ∥𝑢 ∥ · 𝜅 2 (𝛾)3/2

Now rearranging the above inequality gives us ˆ − skew(𝛾)| = 𝑆1 𝑆2 − |skew(𝛾)

|𝜅3 (𝛾)| 𝜅2 (𝛾)3/2

(5.4)

where the second inequality follows from our bounds on ∥𝑢 ∗ ∥ (cf. Claims 3.18 and 5.10) and the last inequality follows from |𝜅 2 (𝛾)| ⩽ 1 (cf. Claim 3.7) and 𝛽 ⩽ 1 by assumption. This completes the proof of Lemma 5.9. □

5.3

Relative truncation parameter recovery implies parameter recovery

In this section, we show that given an estimate for the relative truncation parameter as well as the Σ𝑤 and the first three moments, we can estimate the mean and covariance of the vector 𝑢˜ = − ∥Σ𝑤∥ (untruncated) Gaussian. 35

5.3.1

Mean estimation

The following lemma shows that we can recover the mean. Lemma 5.11. Suppose there is 𝛽 > 0 such that 𝛽 < 𝑐 · 𝜅 2 (𝛾)5/2 , for some sufficiently small absolute constant b 2 , 𝑢, ¯ 𝑀 ˆ 𝛾ˆ such that the following hold 𝑐 > 0, and there are 𝑥, 𝑥¯ − 𝜇 − 𝜅1 (𝛾)

Σ𝑤 ⩽𝛽 ∥Σ1/2 𝑤∥

b 2 − 𝑀2 Σ−1/2 ∥𝐹 ⩽ 𝛽 ∥Σ−1/2 𝑀

(5.6)

Σ𝑤 𝑢∗ = 𝑢ˆ + ⩽ 𝛽, ∗ ∥𝑢 ∥ ∥Σ𝑤∥

(5.7)

𝜅3 (𝛾)2/3 ⩽𝛽 ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾)

(5.8)

𝑢ˆ −

b −1 𝑢ˆ − 𝑢ˆ ⊤ 𝑀 2

ˆ − skew(𝛾)| ⩽ 𝑂 |skew(𝛾) Then the mean estimator

(5.5)

𝛽

𝜅 2 (𝛾)3/2

(5.9)

ˆ ⊤ −1 −1/2 𝜅1 ( 𝛾) b 𝑢ˆ 𝜇ˆ = 𝑥¯ + p 𝑢ˆ 𝑀 𝑢ˆ 2 ˆ 𝜅 2 ( 𝛾)

satisfies ∥𝜇ˆ − 𝜇∥ ⩽ 𝑂

𝛽

𝜅2 (𝛾)7/2

Proof. First, using assumption (5.5), we have that 𝜇ˆ − 𝜇 = 𝑥¯ − 𝜇 − 𝜅1 (𝛾)

⩽ 𝑥¯ − 𝜇 − 𝜅1 (𝛾) ⩽ 𝛽 + 𝜅1 (𝛾)

ˆ ⊤ −1 −1/2 𝜅 1 ( 𝛾) Σ𝑤 Σ𝑤 b 𝑢ˆ 𝑢ˆ + 𝜅 (𝛾) + 𝑢ˆ 𝑀 p 1 2 ∥Σ1/2 𝑤∥ ∥Σ1/2 𝑤∥ ˆ 𝜅2 ( 𝛾) ˆ ⊤ −1 −1/2 𝜅1 (𝛾) Σ𝑤 Σ𝑤 b 𝑢ˆ 𝜅 (𝛾) + + 𝑢ˆ 𝑀 𝑢ˆ p 1 2 ∥Σ1/2 𝑤∥ ∥Σ1/2 𝑤∥ ˆ 𝜅2 ( 𝛾)

ˆ ⊤ −1 −1/2 𝜅1 ( 𝛾) Σ𝑤 b 𝑢ˆ + 𝑢ˆ 𝑀 𝑢ˆ . p 2 ∥Σ1/2 𝑤∥ ˆ 𝜅2 ( 𝛾)

}

(∗)

We now want to bound (∗). We can furthermore split this up as follows (∗) ⩽ 𝜅1 (𝛾)

𝜅1 (𝛾) ⊤ −1 −1/2 Σ𝑤 b 𝑢ˆ 𝑢ˆ 𝑀 𝑢ˆ + p 2 ∥Σ1/2 𝑤∥ 𝜅2 (𝛾)

ˆ ⊤ −1 −1/2 𝜅1 (𝛾) ⊤ −1 −1/2 𝜅1 ( 𝛾) b 𝑢ˆ b 𝑢ˆ ˆ 𝑢ˆ 𝑀 𝑢 − 𝑢ˆ 𝑀 𝑢ˆ + p p 2 2 ˆ 𝜅2 (𝛾) 𝜅 2 ( 𝛾)

| ⩽ 𝑎1 + 𝜅1 (𝛾) |

}

𝑎1 B

∥Σ𝑤∥ Σ𝑤 + 𝜅 1 (𝛾) 1/2 𝑢ˆ 1/2 ∥Σ 𝑤∥ ∥Σ 𝑤∥

}

𝑎2 B

𝜅1 (𝛾) ⊤ −1 −1/2 ∥Σ𝑤∥ b 𝑢ˆ 𝑢ˆ . 𝑢ˆ 𝑀 + 𝜅1 (𝛾) 1/2 𝑢ˆ − p 2 ∥Σ 𝑤∥ 𝜅 2 (𝛾)

}

𝑎3 B

Putting these together, we get that 𝜇ˆ − 𝜇 ⩽ 𝛽 + 𝑎1 + 𝑎 2 + 𝑎 3 and it remains to bound 𝑎1 , 𝑎 2 and 𝑎 3 . We do so now. Bounding 𝑎1 . We have that

−1/2 ˆ 𝜅1 (𝛾) 𝜅1 ( 𝛾) b −1 𝑢ˆ ˆ . 𝑎1 ⩽ p · ∥𝑢∥ −p · 𝑢ˆ ⊤ 𝑀 2 ˆ 𝜅2 (𝛾) 𝜅2 (𝛾) 𝜅 (𝑡)

ˆ ⩽ 1. Thus, defining 𝜓(𝑡) = √ 1 We have that ∥𝑢∥

𝜅2 (𝑡)

𝑎1 ⩽

b −1 𝑢ˆ 𝑢ˆ ⊤ 𝑀 2

−1/2

we get that ˆ − 𝜓(𝛾) . · 𝜓( 𝛾)

By Claim 3.20, we have that ˆ − 𝜓(𝛾) ≲ |skew(𝛾) ˆ − skew(𝛾)| · max{𝜅 2 ( 𝛾) ˆ −2 , 𝜅2 (𝛾)−2 }. 𝜓( 𝛾) ˆ We can use assumption (5.9) to bound the first term, so it remains to give a lower bound on 𝜅 2 ( 𝛾). It follows from Claim 3.19 and assumption (5.9) that ˆ ≲ |skew( 𝛾) ˆ − skew(𝛾)| ⩽ |𝜅2 (𝛾) − 𝜅 2 ( 𝛾)|

𝛽 𝜅2 (𝛾)3/2

Moreover, by choosing a sufficiently small constant in our assumption 𝛽 < 𝑐 · 𝜅 2 (𝛾)5/2 , we have ˆ ⩾ 𝜅 2 (𝛾) − 21 𝜅2 (𝛾) = 21 𝜅2 (𝛾). Thus, we get that that 𝜅2 (𝛾) ˆ − 𝜓(𝛾) ≲ |skew(𝛾) ˆ − skew(𝛾)| · 𝜅 2 (𝛾)−2 ⩽ 𝜓( 𝛾)

𝛽 𝜅2 (𝛾)7/2

On the other hand, by Lemma 4.6, we have that 𝑀2 ⪯ 2𝐼 𝑑 , which together with assumption (5.6) implies

b −1 𝑢ˆ 𝑢ˆ ⊤ 𝑀 2

−1/2

−1/2 p 1/2 b −1 b2 ⩽ 𝜆min 𝑀 = 𝑀 ⩽ (1 + 𝛽)∥𝑀2 ∥ ⩽ 2, 2

as long as 𝛽 ⩽ 1. Together with the above this implies 𝑎1 ⩽ 𝑂

𝛽

𝜅2 (𝛾)7/2 ·

Bounding 𝑎 2 . For 𝑎2 , we have that 𝑎 2 ⩽ |𝜅 1 (𝛾)| ·

∥Σ𝑤∥ Σ𝑤 · + 𝑢ˆ . ∥Σ1/2 𝑤∥ ∥Σ𝑤∥ 37

∥Σ𝑤∥

We have that |𝜅1 (𝛾)| ⩽ 𝑂(𝜅2 (𝛾)−1/2 ) by Claim 3.9 and that ∥Σ1/2 𝑤∥ ⩽ 𝜅2 (𝛾)−1/2 by Lemma 4.6. Together with assumption (5.7), this implies 𝛽 𝑎2 ⩽ 𝑂 . 𝜅 2 (𝛾)

Bounding 𝑎 3 . For 𝑎3 , we have that 𝑎 3 ⩽ 𝜅1 (𝛾)

𝜅 1 (𝛾) ⊤ −1 −1/2 ∥Σ𝑤∥ b 𝑢ˆ ˆ − 𝑢ˆ 𝑀 · ∥𝑢∥ p 2 ∥Σ1/2 𝑤∥ 𝜅2 (𝛾)

|𝜅1 (𝛾)|

⩽p Now, consider 𝑓 (𝑥) =

√

𝜅2 (𝛾)

−1/2 ∥Σ𝑤∥ b −1 𝑢ˆ ˆ ⊤𝑀 𝑢 . − 2 ∥Σ1/2 𝑤∥

𝑥. For 𝑎, 𝑐 ⩾ 𝐵 > 0, we have that, by the mean value theorem, 1 𝑓 (𝑎) − 𝑓 (𝑐) ⩽ √ |𝑎 − 𝑐|. 2 𝐵 ∥Σ𝑤∥2

b −1 𝑢ˆ We want to apply this for 𝑎 = 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2 and 𝑐 = 𝑢ˆ ⊤ 𝑀 2

−1

. By Lemma 4.6, we have that

𝑎 ⩾ 21 𝜅 2 (𝛾). From assumption (5.8) and using also 𝑢 ∗ = 𝜅3 (𝛾)1/3 ∥ΣΣ𝑤 1/2 𝑤∥ , we furthermore have 𝑐 ⩾ 𝜅2 (𝛾)

as long as 𝛽 ⩽

∥Σ𝑤∥2 1 1 − 𝛽 ⩾ 𝜅2 (𝛾) − 𝛽 ⩾ 𝜅2 (𝛾), 1/2 2 2 4 ∥Σ 𝑤∥

𝜅2 (𝛾) 1 4 . Hence, we can choose 𝐵 = 4 𝜅 2 (𝛾). Thus, we get that

𝜅2 (𝛾)

−1/2 −1 ∥Σ𝑤∥ ∥Σ𝑤∥2 1 ⊤ b −1 b −1 𝑢ˆ ˆ ⊤𝑀 ˆ ˆ − 𝑢 ⩽ · 𝜅 (𝛾) − 𝑢 𝑀 𝑢 p 2 2 2 ∥Σ1/2 𝑤∥ ∥Σ1/2 𝑤∥2 𝜅2 (𝛾) ⩽p

𝛽 𝜅2 (𝛾)

by assumption (5.8). Together with the above as well as Claim 3.9, we get that 𝑎3 ⩽ 𝛽 ·

|𝜅1 (𝛾)| ⩽ 𝑂 𝛽 · 𝜅2 (𝛾)−3/2 . 𝜅2 (𝛾)

Completing the proof. Combining the bounds on 𝑎 1 , 𝑎 2 and 𝑎 3 , we get that 𝜇ˆ − 𝜇 ⩽ 𝛽 + 𝑂

𝛽 𝜅2 (𝛾)7/2

𝛽 𝛽 𝛽 +𝑂 +𝑂 ⩽ 𝑂 , 𝜅2 (𝛾) 𝜅2 (𝛾)3/2 𝜅2 (𝛾)7/2

where we used 𝜅2 (𝛾) ⩽ 1 and 𝜅 3 ⩽ 𝑂(1) by Claims 3.7 and 3.10 for the last inequality. 5.3.2

Covariance estimation

For covariance estimation, we need the following fact, whose proof we defer to Section A.2. Fact 5.12. For any 𝑎, 𝑏 ∈ R𝑑 we have that ∥𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ ∥𝐹 ⩽ ∥𝑎 + 𝑏∥ · ∥𝑎 − 𝑏∥. 38

□

Lemma 5.13. Suppose there is 𝛽 > 0 such that 𝛽 < 𝑐 · 𝜅 2 (𝛾)5/2 , for a sufficiently small absolute constant b 2 , 𝑢, ˆ 𝛾ˆ such that 𝑐 > 0, and 𝑀

b ∥Σ 𝑀2 − 𝑀2 Σ−1/2 ∥𝐹 ⩽ 𝛽 ∗ ∗ −1/2

min

𝑢ˆ −

𝑢 𝑢 , 𝑢ˆ + ∗ ∗ ∥𝑢 ∥ ∥𝑢 ∥

(5.10)

⩽ 𝛽,

(5.11)

𝜅 3 (𝛾)2/3 ⩽𝛽 ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾) 𝛽 ˆ − skew(𝛾)| ⩽ |skew(𝛾) . 𝜅2 (𝛾)3/2

b −1 𝑢ˆ − 𝑢ˆ ⊤ 𝑀 2

(5.12) (5.13)

Then the covariance estimator

b2 + b Σ=𝑀

1 −1 b −1 𝑢) − 1 (𝑢ˆ ⊤ 𝑀 · 𝑢ˆ 𝑢ˆ ⊤ 2 ˆ ˆ 𝜅2 ( 𝛾)

satisfies −1/2 b −1/2

∥Σ

ΣΣ

− 𝐼 𝑑 ∥𝐹 ⩽ 𝑂

𝛽

𝜅2 (𝛾)7/2

Proof. By the definition of our estimator, we get that −1/2 b −1/2

ΣΣ

−1/2 b

− 𝐼𝑑 = Σ

−1/2

𝑀2 Σ

1 −1 −1/2 b −1 𝑢) − 1 (𝑢ˆ ⊤ 𝑀 + Σ 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 − 𝐼 𝑑 . 2 ˆ ˆ 𝜅 2 ( 𝛾)

b 2 , 𝛾ˆ and 𝑢ˆ would be equal to 𝑀2 , 𝛾 and 𝑢, ˜ then this norm would be zero. The proof now If all 𝑀 b 2 by 𝑀2 , we get that consists of accounting for the errors in these estimates. First, substituting 𝑀 Σ

−1/2 b −1/2

ΣΣ

− 𝐼 𝑑 = 𝐴1 + Σ

−1/2

where

𝑀2 Σ

−1/2

1 −1 −1/2 b −1 𝑢) Σ − 1 (𝑢ˆ ⊤ 𝑀 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 − 𝐼 𝑑 , + 2 ˆ ˆ 𝜅 2 ( 𝛾)

b 2 − 𝑀2 )Σ−1/2 . 𝐴1 = Σ−1/2 ( 𝑀

Now, we can use the formula for 𝑀2 from Lemma 4.2 to get that Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑 = 𝐴1 + (1 − 𝜅2 (𝛾))

Σ1/2 𝑤𝑤 ⊤ Σ1/2 Σ1/2 𝑤

1 −1 −1/2 b −1 𝑢) − − 1 (𝑢ˆ ⊤ 𝑀 Σ 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 . 2 ˆ ˆ 𝜅2 (𝛾)

| We now want to show that (∗) ≈ (1 − 𝜅 2 (𝛾))

{z (∗)

Σ1/2 𝑤𝑤 ⊤ Σ1/2 Σ1/2 𝑤

}

b 2 , 𝛾ˆ and 𝑢ˆ by 𝑀2 , 𝛾 and 𝑢, ˜ then they would be equal. We will do Note that if we would replace 𝑀 so now and account for the error. Substituting 𝛾ˆ by 𝛾, we get that 1 −1 −1/2 b −1 𝑢) − 1 (𝑢ˆ ⊤ 𝑀 Σ 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 , (∗) = 𝐴2 + 2 ˆ 𝜅 2 (𝛾)

where

1 1 −1 −1/2 b −1 𝑢) 𝐴2 = − (𝑢ˆ ⊤ 𝑀 Σ 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 . 2 ˆ ˆ 𝜅2 ( 𝛾) 𝜅 2 (𝛾)

𝜅 (𝛾)2/3

b −1 𝑢ˆ by ∗3 2 Substituting 𝑢ˆ ⊤ 𝑀 , we get that 2 ∥𝑢 ∥ ·𝜅 (𝛾) 2

1 −1 (∗) = 𝐴2 + 𝐴3 + 𝜅 2 (𝛾)

= 𝐴2 + 𝐴3 + (1 − 𝜅2 (𝛾)) where

1 −1 𝐴3 = 𝜅2 (𝛾)

b −1 𝑢ˆ 𝑢ˆ ⊤ 𝑀 2

𝜅 3 (𝛾)2/3 ∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾)

−1

Σ−1/2 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2

∥𝑢 ∗ ∥2 −1/2 ⊤ −1/2 Σ 𝑢ˆ 𝑢ˆ Σ , 𝜅3 (𝛾)2/3

−1

−

∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾)

Σ−1/2 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 .

𝜅 3 (𝛾)2/3

∥𝑢 ∗ ∥

∥Σ𝑤∥

Σ𝑤 Finally, substituting 𝑢ˆ with 𝑢˜ = ∥Σ𝑤∥ and using that ∥Σ1/2 𝑤∥ = 𝜅 (𝛾)1/3 , we get that 3

(∗) = 𝐴2 + 𝐴3 + 𝐴4 + (1 − 𝜅2 (𝛾)) = 𝐴2 + 𝐴3 + 𝐴4 + (1 − 𝜅2 (𝛾)) where

∥𝑢 ∗ ∥2 Σ1/2 𝑤𝑤 ⊤ Σ1/2 ∥Σ𝑤∥2 𝜅 3 (𝛾)2/3 Σ1/2 𝑤𝑤 ⊤ Σ1/2 Σ1/2 𝑤

∥𝑢 ∗ ∥2 −1/2 ⊤ −1/2 Σ1/2 𝑤𝑤 ⊤ Σ1/2 𝐴4 = (1 − 𝜅 2 (𝛾)) Σ . 𝑢ˆ 𝑢ˆ Σ − 𝜅3 (𝛾)2/3 ∥Σ1/2 𝑤∥2

Combining all of the above we get that Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑 = 𝐴1 − 𝐴2 − 𝐴3 − 𝐴4 . Thus, in order to bound ∥Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑 ∥𝐹 , it is sufficient to bound the norms of 𝐴1 , 𝐴2 , 𝐴3 and 𝐴4 . We will do so now. Bounding ∥𝐴1 ∥𝐹 . For 𝐴1 , note that by (5.10) we have that ∥𝐴1 ∥𝐹 ⩽ 𝛽. Bounding ∥𝐴2 ∥𝐹 . For 𝐴2 , we can first bound ∥𝐴2 ∥𝐹 ⩽

1 1 −1 b −1 𝑢) − · (𝑢ˆ ⊤ 𝑀 Σ−1/2 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 𝐹 . 2 ˆ ˆ 𝜅2 ( 𝛾) 𝜅 2 (𝛾)

We bound these three terms separately. First, note that, by Lemma 4.6, Σ−1/2 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 𝐹 = Σ−1/2 𝑢ˆ

⩽ ∥Σ−1/2 ∥2 ⩽ 2.

(5.14)

For the middle term, we first want to bound −1 b −1 𝑢) (𝑢ˆ ⊤ 𝑀 − 2 ˆ

∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾) 𝜅3 (𝛾)2/3

−1 b −1 𝑢) = (𝑢ˆ ⊤ 𝑀 − 2 ˆ

where we used 𝑢 ∗ = 𝜅3 (𝛾)1/3 ∥ΣΣ𝑤 1/2 𝑤∥ (which implies assumption (5.12), −1 b −1 𝑢) (𝑢ˆ ⊤ 𝑀 − 2 ˆ

∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2

b −1 𝑢ˆ − = 𝑢ˆ ⊤ 𝑀 2

∥𝑢 ∗ ∥2 ·𝜅 2 (𝛾) 𝜅3 (𝛾)2/3

∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2 ∥Σ𝑤∥2 ·𝜅 2 (𝛾) ). ∥Σ1/2 𝑤∥2

We have that, using

2 ∥Σ1/2 𝑤∥2 ⊤ b −1 −1 ∥Σ𝑤∥ · 𝜅 2 (𝛾) ˆ ˆ · ( 𝑢 𝑀 𝑢) · 2 ∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2

⩽𝛽·

∥Σ𝑤∥2 · 𝜅 2 (𝛾)

b −1 𝑢ˆ · ∥Σ1/2 𝑤∥2 𝑢ˆ ⊤ 𝑀 2

b −1 𝑢, ˆ we can furthermore bound Using (5.12) again to lower bound 𝑢ˆ ⊤ 𝑀 2 ∥Σ𝑤∥2 · 𝜅 2 (𝛾) −1 b −1 𝑢) ˆ (𝑢ˆ ⊤ 𝑀 − 2 1/2 2 ∥Σ

𝑤∥

⩽𝛽· =𝛽·

By Lemma 4.6, we have that

∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2 ∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2

∥Σ1/2 𝑤∥2 · −𝛽 ∥Σ𝑤∥2 · 𝜅 2 (𝛾)

−1

∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2

∥Σ𝑤∥2

1 − ∥Σ1/2 𝑤∥2 · 𝜅 2 (𝛾) · 𝛽

∥Σ𝑤∥2 ·𝜅 2 (𝛾) ⩽ 2. Thus, assuming that 𝛽 < 21 , we get that ∥Σ1/2 𝑤∥2

−1 b −1 𝑢) − (𝑢ˆ ⊤ 𝑀 2 ˆ

∥𝑢 ∗ ∥2 · 𝜅 2 (𝛾) 𝜅3 (𝛾)2/3

Using the triangle inequality and

−1 b −1 𝑢) − = (𝑢ˆ ⊤ 𝑀 2 ˆ

∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2

⩽ 𝑂(𝛽).

(5.15)

∥Σ𝑤∥2 ·𝜅2 (𝛾) ⩽ 2 again, we furthermore get that ∥Σ1/2 𝑤∥2

−1 b −1 𝑢) (𝑢ˆ ⊤ 𝑀 ⩽ 2 ˆ

∥Σ𝑤∥2 · 𝜅 2 (𝛾) ∥Σ1/2 𝑤∥2

+ 𝑂(𝛽) ⩽ 𝑂(1).

𝛽 𝜅2 (𝛾)3/2

ˆ ⩾ 𝜅 2 (𝛾) − 12 𝜅2 (𝛾) = 21 𝜅2 (𝛾) and As before, by our assumption 𝛽 < 𝑐 · 𝜅 2 (𝛾)5/2 , we have that 𝜅2 (𝛾) thus 𝛽 1 ˆ ⩽𝑂 · |𝜅2 (𝛾) − 𝜅 2 ( 𝛾)| . ˆ 𝜅2 (𝛾) · 𝜅 2 ( 𝛾) 𝜅2 (𝛾)7/2 Combining the above, we get that ∥𝐴2 ∥𝐹 ⩽ 2 · 𝑂(1) · 𝑂

𝛽 𝜅2 (𝛾)7/2

=𝑂

𝛽 𝜅 2 (𝛾)7/2

Bounding ∥𝐴3 ∥𝐹 . For 𝐴3 , note that by (5.14) and (5.15) and using that 𝜅 2 (𝛾) ⩽ 1 (cf. Claim 3.7), we immediately get that ∥𝐴3 ∥𝐹 ⩽

−1 ∥𝑢 ∗ ∥2 · 𝜅 (𝛾) 1 2 b −1 𝑢ˆ − − 1 · 𝑢ˆ ⊤ 𝑀 · Σ−1/2 𝑢ˆ 𝑢ˆ ⊤ Σ−1/2 2 2/3 𝜅2 (𝛾) 𝜅3 (𝛾)

𝛽 1 ⩽ · 𝑂(𝛽) · 2 = 𝑂 . 𝜅2 (𝛾) 𝜅 2 (𝛾)

∥𝑢 ∗ ∥2

∥Σ𝑤∥2

Bounding ∥𝐴4 ∥𝐹 . For 𝐴4 , recalling that 𝜅 (𝛾)2/3 = ∥Σ1/2 𝑤∥2 and 𝜅 2 (𝛾) > 0, we get that 3

∥𝑢 ∗ ∥2 −1/2 ⊤ −1/2 Σ1/2 𝑤𝑤 ⊤ Σ1/2 Σ 𝑢ˆ 𝑢ˆ Σ − 𝜅3 (𝛾)2/3 ∥Σ1/2 𝑤∥2 𝐹

∥𝐴4 ∥𝐹 ⩽ |1 − 𝜅 2 (𝛾)| ·

∥Σ𝑤∥2 Σ1/2 𝑤𝑤 ⊤ Σ1/2 −1/2 ˆ 𝑢ˆ ⊤ Σ−1/2 − · Σ 𝑢 ∥Σ𝑤∥2 ∥Σ1/2 𝑤∥2 𝐹 ⊤Σ ∥Σ𝑤∥2 Σ𝑤𝑤 = 1/2 2 · Σ−1/2 𝑢ˆ 𝑢ˆ ⊤ − Σ−1/2 ∥Σ𝑤∥2 ∥Σ 𝑤∥ 𝐹

⩽

⩽ ∥Σ

∥Σ𝑤∥2 Σ𝑤 Σ𝑤 ∥ · 1/2 2 · 𝑢ˆ 𝑢ˆ ⊤ − ∥Σ𝑤∥ ∥Σ𝑤∥ ∥Σ 𝑤∥

−1/2 2

⊤ 𝐹

We now want to use assumption (5.11). Since 𝑢 ∗ ∝ −Σ𝑤, this assumption implies that min

Σ𝑤 Σ𝑤 𝑢ˆ − , 𝑢ˆ + ∥Σ𝑤∥ ∥Σ𝑤∥

⩽𝛽

Σ𝑤 Using Fact 5.12 on the unit vectors 𝑢ˆ and ∥Σ𝑤∥ to get that

Σ𝑤 Σ𝑤 𝑢ˆ 𝑢ˆ − ∥Σ𝑤∥ ∥Σ𝑤∥ ⊤

⊤

⩽ 2𝛽. 𝐹 ∥Σ𝑤∥2

Furthermore, using Lemma 4.6, we get that ∥Σ−1/2 ∥2 ⩽ 2 and ∥Σ1/2 𝑤∥2 ⩽ 𝜅21(𝛾) . Combining these with the above, we get that 𝛽 1 · 2𝛽 = 𝑂 . ∥𝐴4 ∥𝐹 ⩽ 2 · 𝜅2 (𝛾) 𝜅2 (𝛾)

Completing the proof. Combining the bounds on 𝐴1 , 𝐴2 , 𝐴3 and 𝐴4 , we get that Σ−1/2 b ΣΣ−1/2 − 𝐼 𝑑

𝐹

⩽ ∥𝐴1 ∥𝐹 + ∥𝐴2 ∥𝐹 + ∥𝐴3 ∥𝐹 + ∥𝐴4 ∥𝐹 𝛽 𝛽 𝛽 ⩽ 𝛽+𝑂 +𝑂 +𝑂 𝜅2 (𝛾) 𝜅2 (𝛾) 𝜅2 (𝛾)7/2 𝛽 ⩽𝑂 , 7/2 𝜅2 (𝛾)

where the last inequality follows because 𝜅2 (𝛾) ⩽ 1. This completes the proof.

5.4

□

Proof of Theorem 5.1

In this section, we prove our main result, Theorem 5.1. We start with combining the results from Sections 5.1 to 5.3 to give a sample complexity bound of Algorithm 2. Lemma 5.14. Consider a halfspace truncated Gaussian 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 , where Σ ≻ 0. Define 𝛾 = Then, given

𝑑2 𝑑2 𝑑2 𝑛=𝑂 + + 𝜅2 (𝛾)8 𝜅3 (𝛾)2 𝜅2 (𝛾)3 𝜅3 (𝛾)4 𝜅2 (𝛾)10 𝜅 3 (𝛾)2 𝜀2

𝜏−⟨𝑤,𝜇⟩

∥Σ1/2 𝑤 ∥

ˆ b samples, Algorithm 2 computes estimators 𝜇, Σ such that with probability at least 0.99

ˆ b 𝑑TV 𝒩 (𝜇, Σ), 𝒩 (𝜇, Σ) ⩽ 𝜀. Later, we will then bound this sample complexity in terms of 𝛼, as in Theorem 5.1. Finally, we will also argue about the runtime of Algorithm 2. Proof of Lemma 5.14. For any 𝛽 > 0, given 𝑛 = 𝑂

that, with high probability,

⊗3

b 3 − 𝑀3 sup 𝑣 , 𝑀 ∥𝑣∥=1

𝑑2 𝜅2 (𝛾)3 𝜅3 (𝛾)2 𝛽 2

samples, by Lemma 5.8, we have

𝛽 · |𝜅3 (𝛾)| ⩽ 𝑂(𝛽). ⩽𝑂 √ 𝑑

(5.16)

Now, we get, by Lemma 5.5, with high probability, min

𝑢∗ 𝑢∗ 𝑢ˆ − ∗ , 𝑢ˆ + ∗ ∥𝑢 ∥ ∥𝑢 ∥

⩽ 𝑂(𝛽).

(5.17)

b 2 computed from 𝑂(𝑑2 /(𝜅2 (𝛾)2 𝛽2 )) i.i.d. samples. From Next, consider the sample covariance p 𝑀 Claim 4.3 we know each sample is 1/ 𝜅2 (𝛾)-sub-Gaussian, hence as in Fact 3.26, we have, with high probability −1/2 b

∥𝑀2

−1/2

𝑀2 𝑀2

− 𝐼𝑑 ∥ ⩽ 𝛽

b 2 − 𝑀2 Σ−1/2 ∥𝐹 ⩽ 𝛽, ∥Σ−1/2 𝑀

and

(5.18)

1/2

where we also used that ∥Σ−1/2 𝑀2 ∥ ⩽ 12 by Lemma 4.6. Using equations (5.17), (5.18) and (5.16), we can apply Lemma 5.9 as long as 𝛽 < 𝑐1 (for some absolute constant 𝑐1 ). This shows that the estimator

b −1 𝑢ˆ 𝛾ˆ = skew−1 − 𝑢ˆ ⊤ 𝑀 2

3/2

satisfies ˆ − skew(𝛾)| ⩽ 𝑂 |skew( 𝛾)

b 3 ⟩| · |⟨𝑢 ⊗3 , 𝑀 𝛽

𝜅 2 (𝛾)3/2

(5.19)

From Lemma 5.9, we furthermore get the inequality

b −1 𝑢ˆ − 𝑢ˆ ⊤ 𝑀 2

∥Σ1/2 𝑤∥2 ⩽ 𝑂(𝛽) ∥Σ𝑤∥2 · 𝜅 2 (𝛾)

(5.20)

As long as 𝛽 < 𝑐2 · 𝜅 2 (𝛾)5/2 , using (5.18), (5.17), (5.20) and (5.19), we can apply Lemma 5.13 to get that the covariance estimator

b2 + b Σ=𝑀

1 −1 b −1 𝑢) − 1 (𝑢ˆ ⊤ 𝑀 · 𝑢ˆ 𝑢ˆ ⊤ 2 ˆ ˆ 𝜅2 ( 𝛾)

satisfies −1/2 b −1/2

∥Σ

ΣΣ

− 𝐼 𝑑 ∥𝐹 ⩽ 𝑂

𝛽 𝜅2 (𝛾)7/2

(5.21) −1/2

For mean estimation, we have that 𝑥 𝑖 is 2/𝜅 2 (𝛾)-sub-Gaussian since 𝑀2

𝑥 𝑖 is 1/ 𝜅2 (𝛾)-sub-

p √ 1/2 Gaussian by Claim 4.3 and ∥𝑀2 ∥ ⩽ 21 by Lemma 4.6. Thus, 𝑥¯ is ( 2/𝜅 2 (𝛾))/ 𝑛-sub-Gaussian (cf. 43

[Ver18, Proposition 2.7.1]). Given 𝑛 = 𝑂 𝑑/(𝜅2 (𝛾)𝛽 2 ) samples, we therefore have (see for example [Rig15, Theorem 1.19]), with high probability,

𝑥¯ − 𝜇 − 𝜅1 (𝛾)

Σ𝑤 ⩽ 𝛽. ∥Σ1/2 𝑤∥

(5.22)

To apply Lemma 5.11, it remains to argue that choosing the sign of 𝑢ˆ by computing the sign of b 3 ⟩ indeed ensures (5.7). From the proof of Lemma 5.9, we know that ⟨𝑢ˆ ⊗3 , 𝑀

b 3 ⟩ − ∥𝑢 ∗ ∥3 ⩽ 𝑂(𝛽) ˜ ⩽ 𝑂(𝛽) =⇒ ⟨𝑢ˆ ⊗3 , 𝑀 ∥𝑢ˆ − 𝑢∥

(5.23)

b 3 ⟩ + ∥𝑢 ∗ ∥3 ⩽ 𝑂(𝛽). ˜ ⩽ 𝑂(𝛽) =⇒ ⟨𝑢ˆ ⊗3 , 𝑀 ∥𝑢ˆ + 𝑢∥

(5.24)

If the term 𝑂(𝛽) on the right hand side is smaller than ∥𝑢 ∗ ∥3 , then only one of the two conclusions (5.23) or (5.24) can be true. By (5.17), one of the two must hold. Thus, exactly one of the two b 3 ⟩ is positive, then it must be conclusions (5.23) or (5.24) is true. Furthermore, if the sign of ⟨𝑢ˆ ⊗3 , 𝑀 (5.23) (and hence we want to pick 𝑢ˆ for (5.7)). If this sign is negative, it must be (5.24) (and we want ˆ Hence, if 𝛽 < 𝑐3 · |𝜅3 (𝛾)| (for a sufficiently small constant 𝑐3 ensuring that 𝑂(𝛽) ⩽ ∥𝑢 ∗ ∥3 to pick −𝑢). b 3 ⟩ < 0 indeed ensures that (after the potential by Claim 5.10), then flipping the sign of 𝑢ˆ if ⟨𝑢ˆ ⊗3 , 𝑀 sign flip) 𝑢∗ 𝑢ˆ − ∗ ⩽ 𝑂(𝛽). (5.25) ∥𝑢 ∥ Thus, as long as 𝛽 < 𝑐 4 · 𝜅2 (𝛾)5/2 , using (5.22), (5.18), (5.25), (5.20) and (5.19), we can apply Lemma 5.11 to get that the mean estimator ˆ ⊤ −1 −1/2 𝜅1 ( 𝛾) b 𝑢ˆ 𝜇ˆ = 𝑥¯ + p 𝑢ˆ 𝑀 𝑢ˆ 2 ˆ 𝜅 2 ( 𝛾) satisfies 𝜇ˆ − 𝜇 ⩽ 𝑂

𝛽

𝜅 2 (𝛾)7/2

Furthermore, by Lemma 4.6 we have that ∥Σ−1/2 ∥ ⩽ 𝑂(1) and thus the above also implies −1/2

(𝜇ˆ − 𝜇) ⩽ 𝑂

𝛽

𝜅2 (𝛾)7/2

(5.26)

Thus, summarized, for 𝛽 < 𝑐 · min{𝜅 2 (𝛾)5/2 , |𝜅3 (𝛾)|} for a sufficiently small constant 𝑐, we have that, with high probability,10

b ˆ Σ), 𝒩 (𝜇, Σ) ⩽ 𝑂 𝑑TV 𝒩 (𝜇,

𝛽

𝜅2 (𝛾)7/2

The sample complexity is 𝑑2 𝑑2 𝑑 𝑛 = 𝑂 max , , 3 2 2 2 2 𝜅2 (𝛾) 𝜅 3 (𝛾) 𝛽 𝜅 2 (𝛾) 𝛽 𝜅2 (𝛾)𝛽2

𝑑2 =𝑂 . 𝜅2 (𝛾)3 𝜅 3 (𝛾)2 𝛽2

Thus, setting 𝛽 = 𝑂 min{𝜅2 (𝛾)5/2 , |𝜅3 (𝛾)|, 𝜅2 (𝛾)7/2 · 𝜀} for a sufficiently small constant, we get the lemma. □

10 For Gaussian, closeness in parameters implies closeness in TV-distance, see e.g. [AAL23, Theorem 1.8].

The above proof is sufficient as long as 𝛼 is bounded away from 1 by a constant. As 𝛼 approaches 1, we have that 𝜅 3 (𝛾) → 0 and thus the sample complexity increases. To deal with this, we now give an alterative sample complexity bound that relies on the fact that if 𝛼 is close to 1, then the sample mean and covariance are good estimators and the correction terms in Algorithm 2 are small. Lemma 5.15. Consider a halfspace truncated Gaussian 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 , where Σ ≻ 0. Define 𝛾 =

𝜏−⟨𝑤,𝜇⟩

. ∥Σ1/2 𝑤 ∥ Assume that 𝛼 = Φ(𝛾) ⩾ 1 − 𝑐 for a sufficiently small constant 𝑐 > 0. Then, given 𝑛 = 𝑂 𝑑 2 /𝜀2 samples, ˆ b Algorithm 2 computes estimators 𝜇, Σ such that with probability at least 0.99

1 b ˆ Σ), 𝒩 (𝜇, Σ) ⩽ 𝑂 𝜀 + (1 − 𝛼) log 𝑑TV 𝒩 (𝜇, .

1−𝛼

Proof. The strategy of the proof is as follows: We want to argue that we are essentially outputting the sample mean and covariance and that these are good estimates. First, note that given 𝑛 = 𝑂(𝑑 2 /𝛽2 ) samples, the sample moments satisfy, with high probability 𝜇+

𝛽 𝜑(𝛾) Σ𝑤 − 𝑥¯ ⩽ √ · 1/2 Φ(𝛾) ∥Σ 𝑤∥ 𝑑

(analogous to (5.22))

b2 𝑀2 − 𝑀

(analogous to (5.18))

⩽𝛽

𝐹

b 3 − 𝑀3 ⩽ √𝛽 sup 𝑣 ⊗3 , 𝑀 𝑑 ∥𝑣∥=1 D

(Lemma 5.8)

Note that we used that for 𝛼 ⩾ 1 − 𝑐, we have that 𝜅2 (𝛾) ⩾ 𝜅 2 (Φ−1 (1 − 𝑐)) can be lower bounded by a constant by Claim 3.8. Covariance estimation. For covariance estimation, we want to argue that 𝑀2 is close to Σ, i.e. that the second central moment would be good estimate. Then, we will also argue that for 𝛼 close b2. to 1, we are essentially outputting 𝑀 First, note that, by the formula for 𝑀2 of Lemma 4.2, we have ∥Σ𝑤𝑤 ⊤ Σ∥𝐹 1 ⩽ 𝑂 (1 − 𝛼) log ∥𝑀2 − Σ∥𝐹 ⩽ (1 − 𝜅2 (𝛾)) · 1/2 2 1−𝛼 ∥Σ 𝑤∥

where we bounded 1 − 𝜅 2 (𝛾) by Claim 3.14 and used ∥Σ∥ ⩽ 2/𝜅 2 (Φ−1 (1 − 𝑐)) ⩽ 𝑂(1) for 𝛼 ⩾ 1 − 𝑐 b 2 ∥𝐹 ⩽ 𝛽 from above, we furthermore get by Claim 3.8 and Lemma 4.6. Combined with ∥𝑀2 − 𝑀 that 1 b 2 − Σ ⩽ 𝛽 + 𝑂 (1 − 𝛼) log 𝑀 . 1−𝛼 𝐹 Recall that the estimator of Algorithm 2 is

b2 + b Σ=𝑀

−1 1 b −1 𝑢ˆ − 1 · 𝑢ˆ ⊤ 𝑀 𝑢ˆ 𝑢ˆ ⊤ . 2 ˆ 𝜅 2 ( 𝛾)

It remains to argue that the correction term

1 ˆ −1 𝜅2 ( 𝛾)

b −1 𝑢ˆ · 𝑢ˆ ⊤ 𝑀 2

−1

𝑢ˆ 𝑢ˆ ⊤ is small. We have

b −1 ⩽ 𝑢ˆ ⊤ 𝑀 b −1 𝑢ˆ ⩽ 𝜆max 𝑀 b −1 ⩽ 𝑐2 . 𝑐1 ⩽ 𝜆min 𝑀 2 2 2 45

(5.27)

for appropriate constants 𝑐1 , 𝑐2 > 0, by Lemma 4.6, as long as 𝛽 is at most a sufficiently small b −1 ⪯ 𝑐2 · 𝑀 −1 ). constant (such that we have 2𝑐1 · 𝑀2−1 ⪯ 𝑀 2 2 2 ˆ in terms of 1 − 𝛼. For this, we want to show that 𝛾ˆ has Thus, it remains to lower bound 𝜅 2 (𝛾) to be large and use Claim 3.14. Since skew is monotonic by Lemma 3.17, if suffices to show that ˆ is large or equivalently that |skew(𝛾)| ˆ is small (since skew is always negative by Claims 3.7 skew(𝛾) and 3.8). We have that, using (5.27),

b −1 𝑢ˆ ˆ = 𝑢ˆ ⊤ 𝑀 |skew(𝛾)| 2

3/2 D

b3 · 𝑢ˆ ⊗3 , 𝑀

D ≲

b3 . 𝑢ˆ ⊗3 , 𝑀

Furthermore, we can bound the right hand side of the above as follows

b3 𝑢ˆ ⊗3 , 𝑀

D ⩽

b 3 − 𝑀3 𝑢ˆ ⊗3 , 𝑀

𝛽

𝑑

⩽ √ + ∥𝑢 ∗ ∥3 ⩽ √ + 𝑂(|skew(𝛾)|),

+ 𝑢ˆ ⊗3 , 𝑀3

b 3 − 𝑀3 , 𝑀3 = 𝑢 ∗⊗3 , as well as Claim 5.10 to bound ∥𝑢 ∗ ∥. where we used the guarantee on 𝑀 Combined with the above, this gives 𝛽 ˆ ⩽ 𝑂 max √ , |skew(𝛾)| |skew(𝛾)| 𝑑

By choosing 𝛽 to be smaller than a sufficiently small constant and 𝑐 sufficiently small, we can make ˆ ⩽ |skew(2)|, which implies 𝛾ˆ ⩾ 2 (since the right hand side arbitrarily small and get that |skew(𝛾)| ˆ skew is always negative and increasing, |skew| is decreasing). Using Claim 3.16 for both 𝛾 and 𝛾, we get that 𝛽 1 1 3/2 3/2 ˆ · log (1 − 𝛼) ⩽ 𝑂 max √ , (1 − 𝛼) · log (5.28) 1 − 𝛼ˆ 1−𝛼 𝑑 ˆ Since both 𝛼ˆ and 𝛼 are close to 1, if the maximum is the second term, then we also for 𝛼ˆ B Φ(𝛾). 11 have 1 1 ˆ · log (1 − 𝛼) ≲ (1 − 𝛼) · log . 1 − 𝛼ˆ 1−𝛼 If the maximum is the first term, using that log1/2 ˆ · log (1 − 𝛼)

1 1−𝛼ˆ

⩾ 1, we get that

𝛽 1 ≲√ . 1 − 𝛼ˆ 𝑑

Combining these two, we thus get 𝛽 1 1 ˆ · log (1 − 𝛼) ⩽ 𝑂 max √ , (1 − 𝛼) · log 1 − 𝛼ˆ 1−𝛼 𝑑

11 To formally prove this, define 𝑎 = log exp(− 𝑎ˆ ) 𝑎ˆ

1 1−𝛼

exp(− 𝑎ˆ ) 𝑎ˆ 3/2

⩽ 𝑂(1) and we want to , 𝑎ˆ = log 1−1𝛼ˆ and 𝑟 = 𝑎𝑎ˆ . We have exp(−𝑎)𝑎 3/2 1/2

show exp(−𝑎)𝑎 ⩽ 𝑂(1), for which it is sufficient to show 𝑎 1/2 ⩽ 𝑂(1). If 𝑟 ⩽ 1, then this is clear. Otherwise, since 𝛾ˆ ⩾ 2, 𝑎ˆ

1 we have 𝑎ˆ ⩾ log 1−Φ(2) ⩾ 1. Thus, by assumption, we have

exp(𝑟 − 1) · 𝑟 −3/2 ⩽ exp((𝑟 − 1) 𝑎ˆ ) · 𝑟 −3/2 =

exp(− 𝑎ˆ ) 𝑎ˆ 3/2 exp(−𝑎)𝑎 3/2

⩽ 𝑂(1).

Now, we have lim𝑟→∞ exp((𝑟 − 1)) · 𝑟 −3/2 = ∞ and thus exp((𝑟 − 1)) · 𝑟 −3/2 ⩽ 𝑂(1) can only be true if we also have 𝑟 ⩽ 𝑂(1).

Thus, we have that, using Claim 3.14 in the second inequality, 1 1 ˆ −1⩽ · (1 − 𝜅2 ( 𝛾)) ˆ 𝜅2 ( 𝛾) 𝜅2 (2) 1 ˆ · log ⩽ 𝑂 (1 − 𝛼) 1 − 𝛼ˆ 𝛽 1 ⩽ 𝑂 max √ , (1 − 𝛼) · log . 1−𝛼 𝑑

ˆ 2 = 1, This implies, noting that ∥𝑢ˆ 𝑢ˆ ⊤ ∥𝐹 = ∥𝑢∥

b2 b Σ−𝑀

𝐹

−1 𝛽 1 1 b −1 𝑢ˆ 𝑢ˆ 𝑢ˆ ⊤ ⩽ 𝑂 √ + (1 − 𝛼) · log − 1 · 𝑢ˆ ⊤ 𝑀 2 ˆ 1−𝛼 𝜅2 ( 𝛾) 𝑑 𝐹

b 2 − Σ∥𝐹 we get that the estimator from Algorithm 2 satisfies Combined with the bound on ∥ 𝑀 1 b Σ − Σ ⩽ 𝑂 𝛽 + (1 − 𝛼) · log 1−𝛼 𝐹

Finally, using Lemma 4.6, we get that Σ

−1/2 b −1/2

ΣΣ

− 𝐼𝑑

𝐹

⩽ Σ

−1/2 2

1 ⩽ 𝑂 𝛽 + (1 − 𝛼) · log 1−𝛼 𝐹

· b Σ−Σ

Mean estimation. For mean estimation, we want to similarly argue that the sample mean would be a good estimator and that we are outputting something close to the sample mean. First, recall that we have that the sample mean is close to the true mean, i.e., 𝜇+

𝜑(𝛾) Σ𝑤 · − 𝑥¯ ⩽ 𝛽. Φ(𝛾) ∥Σ1/2 𝑤∥

By picking 𝑐 sufficiently small, we can achieve 𝛾 ⩾ 2. Then, we furthermore get that 𝜑(𝛾) 2 1 Σ𝑤 ⩽ · 1/2 · Σ1/2 · 𝜑(𝛾) ⩽ 𝑂(𝜑(𝛾)), Φ(𝛾) ∥Σ 𝑤∥ Φ(2) where we again used Lemma 4.6 and the fact that 𝜅2 (𝛾) ⩾ 𝜅2 (2) to bound ∥Σ1/2 ∥. Using Fact 3.3 and Claim 3.15, we can bound √ 1 1 𝜑(𝛾) ⩽ 𝛾 + · (1 − Φ(𝛾)) ⩽ 2𝛾 · (1 − Φ(𝛾)) ⩽ 2 2 · (1 − Φ(𝛾)) · log1/2 . 𝛾 1 − Φ(𝛾)

Thus, we get that

𝜑(𝛾) Σ𝑤 1 · 1/2 ⩽ (1 − 𝛼) · log1/2 Φ(𝛾) ∥Σ 𝑤∥ 1 − Φ(𝛾)

and hence 1/2

𝜇 − 𝑥¯ ⩽ 𝛽 + (1 − 𝛼) · log

1 . 1 − Φ(𝛾)

Recall that the mean estimator of Algorithm 2 is ˆ 𝜅 1 ( 𝛾) −1/2 b −1 𝑢) ˆ 𝜇ˆ = 𝑥¯ + p (𝑢ˆ ⊤ 𝑀 𝑢. 2 ˆ ˆ 𝜅2 ( 𝛾) ˆ 𝜅 (𝛾)

It remains to argue that the correction term √ 1

ˆ 𝜅2 ( 𝛾)

using (5.27), 𝛾ˆ ⩾ 2 and Claim 3.8, as follows ˆ 𝜅1 ( 𝛾)

ˆ 𝜅 2 ( 𝛾)

−1/2 b −1 𝑢) (𝑢ˆ ⊤ 𝑀 𝑢ˆ ≲ p 2 ˆ

1 𝜅 2 (2)

b −1 𝑢) ˆ −1/2 𝑢ˆ is small. We can bound this, (𝑢ˆ ⊤ 𝑀 2

ˆ ≲ |𝜅 1 ( 𝛾)| ˆ = · |𝜅 1 ( 𝛾)|

ˆ ˆ 𝜑(𝛾) 𝜑(𝛾) ˆ ⩽ ≲ 𝜑(𝛾). ˆ Φ(𝛾) Φ(2)

As above for 𝛾, using Fact 3.3 and Claim 3.15, we can bound √

ˆ · log ˆ ⩽ 2 2 · (1 − Φ(𝛾)) 𝜑(𝛾)

1/2

√ 1 1 ˆ · log1/2 = 2 2 · (1 − 𝛼) . ˆ 1 − 𝛼ˆ 1 − Φ( 𝛾)

Using the same argument as in Footnote 11, (5.28) also implies ˆ · log (1 − 𝛼) Thus, it follows that

1/2

𝛽 1 1 ⩽ 𝑂 max √ , (1 − 𝛼) · log1/2 1 − 𝛼ˆ 1−𝛼 𝑑

𝛽 1 𝜇ˆ − 𝑥¯ ⩽ 𝑂 √ + (1 − 𝛼) · log1/2 1 − 𝛼 𝑑

¯ and using Lemma 4.6 again to bound ∥Σ−1/2 ∥, we get Combining this with the bound on ∥𝜇 − 𝑥∥ that 1 1/2 −1/2 Σ (𝜇ˆ − 𝜇) ⩽ 𝑂 𝛽 + (1 − 𝛼) · log . 1−𝛼 Conclusion. So far, we have shown that for 𝛽 smaller than a sufficiently small constant 𝑐 3 , we get that, using 𝑂(𝑑2 /𝛽 2 ) samples, the estimators in Algorithm 2 satisfy

1 b ˆ Σ), 𝒩 (𝜇, Σ) ⩽ 𝑂 𝛽 + (1 − 𝛼) log 𝑑TV 𝒩 (𝜇, .

1−𝛼

Note that without loss of generality we can assume that 𝜀 ⩽ 1, otherwise any output is good (the TV-distance is at most 1 regardless of the output). Finally, picking 𝛽 = min{𝑐 3 , 𝜀} proves the lemma. The claimed sample complexity follows since this 𝛽 satisfies 𝛽 ⩾ Ω(𝜀) (since 𝜀 ⩽ 1). □ Using Lemmas 5.14 and 5.15, we can now prove Theorem 5.1. Proof of Theorem 5.1. Correctness follows from Lemmas 5.14 and 5.15 if the number of samples used by the algorithm is large enough. To determine the sample complexity, we distinguish the threeo n 𝜀 cases (i) 𝛼 ⩽ 𝛼 1 , (ii) 𝛼 ∈ (𝛼 1 , 𝛼2 ) and (iii) 𝛼 ⩾ 𝛼 2 , where 𝛼1 = 1 − 𝑐 and 𝛼2 = max 1 − 𝑐, 1 − log(1/𝜀)

for 𝑐 as in Lemma 5.15. Again, we note that without loss of generality we can assume 𝜀 ⩽ 1, otherwise the guarantees hold trivially. Case (i): Sample complexity for 𝛼 ⩽ 𝛼1 . In this case, we want to use Lemma 5.14. By Claims 3.22 and 3.23 we get that 𝜅2 (𝛾) ⩾ Ω(log−1 (1/𝛼))

and 48

|𝜅3 (𝛾)| ⩾ Ω(log−3/2 (1/𝛼)).

Thus, the sample complexity in this case is

𝑑 𝑛 = 𝑂 2 · log14 (1/𝛼) . 𝜀

Case (ii): Sample complexity for 𝛼 ∈ (𝛼 1 , 𝛼2 ). In this case, we again use Lemma 5.14. We can 𝜀 bound 𝜅2 (𝛾) ⩾ Ω(1) by Claim 3.22. For 𝜅 3 , we have that 1 − 𝛼 ⩾ 1 − 𝛼 2 ⩾ 𝑐 ′ log(1/𝜀) . Thus, using Claim 3.23, we have that

3/2

|𝜅3 (𝛾)| ⩾ Ω (1 − 𝛼) · log

1 1−𝛼

⩾ Ω(𝜀).

Thus, the sample complexity in this case (again from Lemma 5.14) is 𝑛=𝑂

𝑑 . 𝜀4

Case (iii): Sample complexity for 𝛼 ⩾ 𝛼2 . In this case, we claim that we can use Lemma 5.15. The guarantee from this lemma is that the TV-distance is at most 1 𝑂 𝜀 + (1 − 𝛼) log 1−𝛼

and we need to show that this is at most 𝜀. Using that 𝑥 ↦→ (1 − 𝑥) · log(1/(1 − 𝑥)) is decreasing for 𝑥 close to 1, we get that the TV-distance is at most log(1/𝜀) 𝜀 𝑂 𝜀+ log 𝜀 log(1/𝜀)

⩽ 𝑂(𝜀).

The sample complexity by Lemma 5.15 is 𝑛 = 𝑂(𝑑2 /𝜀2 ). These three cases together prove correctness of Algorithm 2 given the sample complexity claimed in Theorem 5.1.12 Runtime. Finally, it remains to prove the runtime of the algorithm. We do so analogous to the arguments presented in Section 2. The runtime to compute the sample mean is 𝑂(𝑛 · 𝑑). The runtime to compute the sample covariance is 𝑂(𝑇(𝑛, 𝑑)). If we first do the contraction and then average over the samples, then also computing the random contraction 𝑅 of the third sample moment can be computed in time 𝑇(𝑛, 𝑑). Computing a spectral decomposition of this matrix can then be done in time 𝑂(𝑑 3 ), e.g. by computing a singular value decomposition. Computing the b 3 ⟩ can be done in time 𝑂(𝑛 · 𝑑) since sign of ⟨𝑢 ⊗3 , 𝑀 ⊗3

b3⟩ = ⟨𝑢 , 𝑀

𝑛 Õ

¯ 3. ⟨𝑢, 𝑥 𝑖 − 𝑥⟩

𝑖=1

Finally, computing the estimates can be done in time 𝑂(𝑑 2 ). Since 𝑛 ⩾ Ω(𝑑2 ), we have that 𝑇(𝑛, 𝑑) ⩾ Ω(𝑛 · 𝑑) ⩾ Ω(𝑑3 ) and thus the dominating term in the runtime is 𝑂(𝑇(𝑛, 𝑑)), which completes the proof. □ 12 The above argument shows that the TV-distance is 𝑂(𝜀). We can make the TV-distance 𝜀 instead of 𝑂(𝜀) by doing 𝜀 . the exact same argument for 𝜀′ = 𝑂(1)

Also note that we can simplify the sample complexity to be 𝑂((𝑑 2 /𝜀2 ) · polylog(1/𝛼)) for 𝛼 ⩽ 1 − 𝑐 (or in fact for 𝛼 being at most any constant smaller than 1) and 𝑂(𝑛 2 /𝜀4 ) for the remaining two cases. Thus, the proof above does indeed show correctness of the algorithm for the sample complexity as in Theorem 5.1.

Remark 5.16. In Lemmas 5.14 and 5.15 above we have been assuming that the skew−1 (·) function can be evaluated exactly and efficiently. In practice, we would instead only find an approximate inverse using binary search. We need to ensure that we get an estimate 𝛾ˆ such that ˆ − skew(𝛾)| ⩽ 𝜀 · poly(min{𝑂(1), 1/log(1/𝛼)}). |skew(𝛾)

e log(1/𝜀)) We can restrict the search to 𝛾ˆ ⩾ Ω( log 1/𝛼) (the lower bound on 𝛾). Furthermore, if 𝛾 ⩾ 𝑂( 𝜀 (this implies 𝛼 ⩾ 1− log 1/𝜀 ), then we showed in Lemma 5.15 and the proof of Theorem 5.1 that 𝛾ˆ = ∞ is a good p

e log(1/𝜀))]. enough estimate and thus we can restrict the binary search to the interval [Ω( log 1/𝛼), 𝑂( The derivative of skew is upper bounded by a constant and thus it is sufficient to get an estimate of the inverse up to 𝜀 · poly(min{𝑂(1), 1/log(1/𝛼)}). p

e log(1/𝜀))] up to the above accuracy, Hence, we need to do binary search on the interval [Ω( log 1/𝛼), 𝑂( which can be done in time log(1/𝜀) · poly(log log(1/𝜀), log log(1/𝛼)). We remark that this time is negligible with respect to the other computation.

Robustness

In this section, we show how to extend our result for halfspaces to the robust setting. We will present the results for the non-trivial truncation regime of 𝛼 bounded away from 1. Throughout this section as well we will assume Σ ≻ 0.

6.1

Preliminaries on robustness

We consider the following definition of robustness analogous to the well-studied strong contamination model in algorithmic statistics literature. We denote a distribution 𝒟 truncated to the set 𝑆 by 𝒟|𝑆 . Definition 6.1 (𝜂-corruption of 𝒟|𝑆 ). Samples 𝑋1 , . . . , 𝑋𝑛 are called an 𝜂-corruption of 𝒟|𝑆 if they are obtained as follows: 1. First, we draw 𝑌1 , . . . , 𝑌𝑛 ∼ 𝒟|𝑆 . 2. Then, an adversary picks 𝑋1 , . . . , 𝑋𝑛 ∈ R𝑑 such that |{𝑖 ∈ [𝑛] : 𝑋𝑖 ≠ 𝑌𝑖 }| ⩽ 𝜂𝑛. Observe that the adversary can remove all information about the truncation set. In particular the adversary is allowed to also pick 𝑋𝑖 such that 𝑋𝑖 ∈ R𝑑 \𝑆. We require the following definition of certifiably sub-Gaussian distributions, see e.g. [KSS18, Definition 1.1] or [DHPT25, Definition 1.3]. Definition 6.2 (Certifiably sub-Gaussian distribution). A distribution 𝒟 with mean 𝜇 is called 𝑠-certifiably sub-Gaussian if for all even 𝑚 ∈ N, there is a sum-of-squares proof of the following inequality in indeterminate 𝑣 √ 𝑚 E𝑋∼𝒟 𝑋 − 𝜇, 𝑣 ⩽ (𝐶𝑠 𝑚)𝑚 ∥𝑣∥𝑚 , where 𝐶 is a universal constant. 50

We also require the following two theorems that we will utilize to obtain robust estimates of the moments. Theorem 6.3 ([KSS18, Theorems 2.2 and 2.3]). Let 𝑘 ⩾ 6 be an even constant. Let 𝐶 > 0. There is a polynomial-time algorithm with the following guarantees: Let 𝒟 be a 𝐶-certifiably sub-Gaussian distribution on R𝑑 with mean 𝜇 and covariance Σ. Let 𝑀3,nc be the raw third moment of 𝒟. Let 𝜂 > 0 such that 𝐶 𝑘 · 𝜂1−2/𝑘 is at most a sufficiently small constant. Then given as input an 𝜂-corrupted sample of 𝒟 of size 𝑛 ⩾ (𝐶 + 𝑑)𝑂(𝑘) , the algorithm outputs estimates of the moments of 𝒟 with the following guarantees: 𝜇 − 𝜇ˆ ⩽ 𝑂(𝐶 𝑘)1/2 · 𝜂1−1/𝑘 · ∥Σ∥1/2 (1 − 𝑂(𝐶 𝑘)1/2 · 𝜂1−2/𝑘 )Σ ⪯ b Σ ⪯ (1 + 𝑂(𝐶 𝑘)1/2 · 𝜂1−2/𝑘 )Σ

b 3,nc , 𝑣 ⊗3 ∀𝑣 ∈ R𝑑 : 𝑀3,nc − 𝑀

⩽ 𝑂(𝐶 𝑘)3/2 · 𝜂1−3/𝑘 𝑀2,nc , 𝑣 ⊗2

where 𝑀2,nc corresponds to the raw second moment of 𝒟. Theorem 6.4 ([DHPT25, Theorem 1.6]). There exists a universal constant 𝐶 ′ > 0 such that any 𝑠-subGaussian distribution is 𝐶 ′ 𝑠-certifiably sub-Gaussian. Fact 6.5. The distribution 𝒟 B 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽𝜏} is 𝑂(∥Σ∥1/2 )–certifiably sub-Gaussian. We note that this follows directly by combining Corollary 4.4 and Theorem 6.4. As a direct consequence, we can apply Theorem 6.3 for estimating the truncated moments, even under adversarial corruption.

6.2

Learning parameters under halfspace truncation and robustness

As a first step we begin by obtaining a robust preconditioner for our setup. Lemma 6.6. Let 𝜂 > 0 be at most a sufficiently small constant. Given an 𝜂-corruption 𝑋1 , . . . , 𝑋𝑛 of e 2 such that 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽ 𝜏} , we can efficiently compute a matrix 𝑀

e 2 ⪯ (1 + 𝜁)𝑀2 (1 − 𝜁)𝑀2 ⪯ 𝑀 whenever ∥Σ∥1/2 𝑘 · 𝜂1−2/𝑘 ⩽ 𝑂(1) and 𝜁 = ∥Σ∥1/4 𝑘 1/2 𝜂1−2/𝑘 where 𝑘 is as in the above theorem of [KSS18]. The proof of Lemma 6.6 follows directly by an application of Theorem 6.3. Once we have access e 2 , if 𝜁 ⩽ min{𝜅2 (𝛾),1} , then as discussed in Claim 4.8 in Section 4.3, we can assume that to such an 𝑀 2 0.5𝐼 𝑑 ⪯ 𝑀2 ⪯ 2𝐼 𝑑 2 0.5𝐼 𝑑 ⪯ Σ ⪯ 𝐼𝑑 𝜅2 (𝛾) After transforming the samples we apply Theorem 6.3 (again) to obtain robust estimates for the second and third moment. Lemma 6.7. Let 𝛿 > 0 be an arbitrary constant. Let 0 < 𝜂 ⩽ 𝑂(1) for a sufficiently small constant. Given an 𝜂-corruption 𝑋1 , . . . , 𝑋𝑛 of 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽𝜏} (preconditioned as discussed above), where 𝑛 = poly(𝑑), we can efficiently compute estimates for the first non-central moment 𝜇ˆ trunc as well as the second and third b 2 and 𝑀 b 3 of 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽𝜏} with the following guarantees (central) moments 𝑀

𝜇trunc − 𝜇ˆ trunc ⩽ 𝑂 𝛿 log1/4 (1/𝛼) · 𝜂1−𝛿 51

1 − 𝑂 𝛿 log1/4 (1/𝛼) · 𝜂1−𝛿

b 2 ⪯ 1 + 𝑂 𝛿 log1/4 (1/𝛼) · 𝜂1−𝛿 𝑀2 ⪯ 𝑀

𝑀2

b 3 , 𝑣 ⊗3 ⩽ 𝑂 𝛿 max log(1/𝛼)1/4 , log(1/𝛼)7/8 · 𝜂1/2−𝛿 ∀𝑣 ∈ 𝒮 𝑑−1 : 𝑀3 − 𝑀

whenever 𝜂1−𝛿 ⩽ 𝑂 𝛿 (log−1/2 (1/𝛼)). Proof. We observe that for the estimate for 𝜇trunc and 𝑀2 it suffices to apply Theorem 6.3 (where 𝐶 is now at most 𝑂(𝜅2 (𝛾)−1/2 )) and pick 𝑘 such that 𝛿 ⩾ 3/𝑘 forsome constant 𝑘 to obtain the result. Note that, by Claim 3.22, we have that 𝜅2 (𝛾)−1 ⩽ 𝑂 log(1/𝛼) . For the estimate for the third central moments 𝑀3 , we cannot directly apply Theorem 6.3 since they only give an estimate for the non-central moments. Instead, we again use the estimate 𝜇ˆ trunc for the truncated mean 𝜇trunc . We note that this satisfies, as discussed before,

𝜇ˆ trunc − 𝜇trunc ⩽ 𝑂 𝛿 log1/4 (1/𝛼) · 𝜂1−𝛿 . Then, using new samples 𝑋𝑖 (such that they are independent of 𝜇ˆ trunc ), we apply the algorithm ′ from Theorem 6.3 to the samples 𝑋𝑖′ = 𝑋𝑖 − 𝜇ˆ trunc to get an estimate 𝑀3,nc of the third non-central

b ′ to estimate the third moment of the uncorrupted 𝑌𝑖′ = 𝑌𝑖 − 𝜇ˆ trunc 13 . We use this estimator 𝑀 3,nc central moment 𝑀3 of the uncorrupted 𝑌𝑖 . We have 𝑀3 = E[(𝑌𝑖 − 𝜇trunc )⊗3 ] = E[(𝑌𝑖 − 𝜇ˆ trunc + 𝜇ˆ trunc − 𝜇trunc )⊗3 ] = E[(𝑌𝑖 − 𝜇ˆ trunc )⊗3 + Sym((𝑌𝑖 − 𝜇ˆ trunc )⊗2 ⊗ (𝜇ˆ trunc − 𝜇trunc )) + Sym((𝑌𝑖 − 𝜇ˆ trunc ) ⊗ (𝜇ˆ trunc − 𝜇trunc )⊗2 ) + (𝜇ˆ trunc − 𝜇trunc )⊗3 ],

where we use Sym to denote all permutations (i.e., Sym(𝑎 ⊗2 ⊗ 𝑏) = 𝑎 ⊗ 𝑎 ⊗ 𝑏 + 𝑎 ⊗ 𝑏 ⊗ 𝑎 + 𝑏 ⊗ 𝑎 ⊗ 𝑎). ′ Now, Theorem 6.3 gives an estimate to 𝑀3,nc B E[(𝑌𝑖 − 𝜇ˆ trunc )⊗3 ] with the following guarantee

′ b ′ , 𝑣 ⊗3 𝑀3,nc −𝑀 3,nc

3/8

⩽ 𝑂 𝛿 log(1/𝛼)

·𝜂

1/2−𝛿

⊗2

· E[(𝑌𝑖 − 𝜇ˆ trunc ) ]

3/2

(6.1)

for any unit vector 𝑣. We can bound

E[(𝑌𝑖 − 𝜇ˆ trunc )⊗2 ] = E[(𝑌𝑖 − 𝜇trunc )⊗2 ] + (𝜇trunc − 𝜇ˆ trunc )⊗2 ⩽ E[(𝑌𝑖 − 𝜇trunc )⊗2 ] + (𝜇trunc − 𝜇ˆ trunc )⊗2 ⩽ 2 + 𝜇trunc − 𝜇ˆ trunc

(6.2)

where we used in the last step that the first term is just the second central moment of the (truncated) 𝑌𝑖 , which satisfies 𝑀2 ⪯ 2𝐼 𝑑 . We now get the following for the error of our estimate, where 𝑣 is again a unit vector,

b ′ , 𝑣 ⊗3 = 𝑀3 − 𝑀 ′ , 𝑣 ⊗3 + 𝑀 ′ − 𝑀 b ′ , 𝑣 ⊗3 . 𝑀3 − 𝑀 3,nc 3,nc 3,nc 3,nc

We can bound the second term by (6.1). For the first term, note that ′ 𝑀3 − 𝑀3,nc , 𝑣 ⊗3 = Sym(E[(𝑌𝑖 − 𝜇ˆ trunc )⊗2 ] ⊗ (𝜇ˆ trunc − 𝜇trunc )), 𝑣 ⊗3 13 Recall that 𝑌 corresponds to uncorrupted samples and 𝑋 corresponds to corrupted samples.

+ Sym(E[𝑌𝑖 − 𝜇ˆ trunc ] ⊗ (𝜇ˆ trunc − 𝜇trunc )⊗2 ), 𝑣 ⊗3 + (𝜇ˆ trunc − 𝜇trunc )⊗3 , 𝑣 ⊗3

⩽ 4 𝜇ˆ trunc − 𝜇trunc

+ 3 𝜇ˆ trunc − 𝜇trunc · E[(𝑌𝑖 − 𝜇ˆ trunc )⊗2 ]

⩽ 7 𝜇ˆ trunc − 𝜇trunc

+ 6 𝜇ˆ trunc − 𝜇trunc ,

where in the last step we used (6.2). Combining this with (6.1), we get

b ′ , 𝑣 ⊗3 ⩽ 𝑂 𝛿 log(1/𝛼)3/8 · 𝜂1/2−𝛿 · (2 + 𝜇trunc − 𝜇ˆ trunc ) 𝑀3 − 𝑀 3,nc 3

+ 7 𝜇ˆ trunc − 𝜇trunc

+ 6 𝜇ˆ trunc − 𝜇trunc

⩽ 𝑂 𝛿 max log(1/𝛼)1/4 , log(1/𝛼)7/8 · 𝜂1/2−𝛿 , where we used 𝜂 ⩽ 1 for the last step.

□

We now prove the following theorem.

Main robustness result Theorem 6.8 (Full statement of Theorem 1.2). Let 0 < 𝛿 < 12 be an arbitrary constant and let 𝛼 ⩽ 0.99. Define 𝐿 B max log1/4 (1/𝛼), log7/8 (1/𝛼)

and

√ 1 · 𝑑 · 𝐿 · 𝜂1/2−𝛿 . 𝛽 robust B max 1, |𝜅3 (𝛾)|

Let 𝑐 𝛿 be a sufficiently small constant depending only on 𝛿. Assume that 0<𝐿

2 1−2𝛿

2 1−2𝛿 © poly 𝜅2 (𝛾), 𝜅3 (𝛾) ª · 𝜂 ⩽ 𝑂𝛿 ®. √ 𝑑 « ¬

(6.3)

The polynomial in (6.3) is chosen sufficiently small so that 𝐿𝜂1−𝛿 ⩽ 𝑂 𝛿 (𝛽robust ),

𝛽robust ⩽ 𝑐 𝛿 · min 𝜅 2 (𝛾)5/2 , |𝜅3 (𝛾)|

and so that Lemma 6.7 applies. Furthermore, assume that

𝜂1−𝛿 ⩽ min 𝑂 𝛿 (log−1/2 (1/𝛼)), 𝑐 𝛿 · ∥Σ∥−1/2 , 𝑐 𝛿 · ∥Σ∥−1/4 · 𝜅 2 (𝛾) .

(6.4)

Given an 𝜂-corruption 𝑋1 , . . . , 𝑋𝑛 of 𝒩 (𝜇, Σ)|{𝑤 ⊤ 𝑥 ⩽𝜏} , where 𝑛 = poly(𝑑), with high probability, we can, in time poly(𝑑), compute estimators 𝜇ˆ and b Σ such that −1/2

(𝜇ˆ − 𝜇) ⩽ 𝑂 𝛿

1 · 𝛽 robust 𝜅 2 (𝛾)7/2

and −1/2 b −1/2

∥Σ

ΣΣ

− 𝐼 𝑑 ∥𝐹 ⩽ 𝑂 𝛿 53

1 · 𝛽 robust . 𝜅 2 (𝛾)7/2

Proof. We follow the proof of Theorem 5.1 but replace the sample moments with our robust estimates from Lemma 6.7. Note that we can do the preconditioning as discussed above by (6.4). Also note that we can apply Lemma 6.7 by (6.4). By the robust preconditioning step and affine invariance, it suffices to prove the theorem in the preconditioned coordinates, where 1 2 𝐼𝑑 ⪯ Σ ⪯ 𝐼𝑑 . 2 𝜅2 (𝛾)

1 𝐼 𝑑 ⪯ 𝑀2 ⪯ 2𝐼 𝑑 , 2

Covariance estimation. In order to apply Lemma 5.13 for the covariance estimation, we need to compute 𝑢ˆ and 𝛾ˆ and bound the following four quantities ((5.10), (5.11), (5.13) and (5.12) respectively):

b 2 − 𝑀2 Σ−1/2 ∥𝐹 , 1. ∥Σ−1/2 𝑀 𝑢∗ 𝑢∗ 2. min 𝑢ˆ − ∗ , 𝑢ˆ + ∗ ∥𝑢 ∥ ∥𝑢 ∥ ˆ − skew(𝛾)| , 3. |skew( 𝛾) 1/2 2 b −1 𝑢ˆ − ∥Σ 𝑤∥ 4. 𝑢ˆ 𝑇 𝑀 . 2 ∥Σ𝑤∥2 · 𝜅 2 (𝛾)

We remark that showing 3. is enough to show (5.13) by the proof of Lemma 5.9. Bounding 1. From Lemma 6.7, we immediately get √ b 2 − 𝑀2 Σ−1/2 ∥ b 2 − 𝑀2 Σ−1/2 ∥𝐹 ⩽ 𝑑 · ∥Σ−1/2 𝑀 ∥Σ−1/2 𝑀 √ ⩽ 𝑑 · 𝑂 𝛿 (𝜂1−𝛿 · 𝐿).

(6.5) (6.6)

The firstpinequality holds because for any matrix 𝐴 we have that the Frobenius norm satisfies ∥𝐴∥𝐹 ⩽ rank(𝐴) · ∥𝐴∥ and the last inequality follows from Lemma 6.7. As shown in [KSS18] the polynomial dependency on the dimension is necessary in general for certifiably sub-Gaussian distributions. Bounding 2. It suffices that we get a good estimate of 𝑀3 . Indeed, we are promised such a good estimate from Lemma 6.7. This directly implies that we can find a unit vector 𝑢ˆ ∈ R𝑑 such that √ 𝑢∗ 𝑢∗ 𝑑 1/2−𝛿 min 𝑢ˆ + ∗ , 𝑢ˆ − ∗ ⩽ 𝑂𝛿 ·𝐿·𝜂 (6.7) ∥𝑢 ∥ 2 ∥𝑢 ∥ 2 |𝜅3 (𝛾)| Bounding 3. From (6.3) we have that √ √ 𝑑 𝑑 1/2−𝛿 1−𝛿 1/2−𝛿 1/2−𝛿 max 𝑂 𝛿 ·𝐿·𝜂 , 𝑂 𝛿 (𝜂 · 𝐿), 𝑂 𝛿 (𝜂 · 𝐿) = 𝑂 𝛿 ·𝐿·𝜂 |𝜅3 (𝛾)| |𝜅3 (𝛾)|

⩽ 𝑐, where 𝑐 is a sufficiently small constant14. This enables us to use Lemma 5.9 and compute an √

estimate 𝛾ˆ with 𝛽 in that lemma set to 𝑂 𝛿

𝑑 · 𝐿 · 𝜂1/2−𝛿 |𝜅 3 (𝛾)|

ˆ − skew(𝛾)| ⩽ 𝑂 𝛿 |skew(𝛾)

√

. Therefore we have

𝑑 · 𝐿 · 𝜂1/2−𝛿 · 𝜅 2 (𝛾)−3/2 . |𝜅 3 (𝛾)|

14 Note that this is weaker than the condition that our theorem statement imposes on 𝜂.

(6.8)

Bounding 4. The second conclusion of Lemma 5.9, applied with √ 𝑑 1/2−𝛿 ·𝐿·𝜂 𝛽 = 𝑂𝛿 , |𝜅3 (𝛾)| gives

b −1 𝑢ˆ − 𝑢˜ ⊤ 𝑀 −1 𝑢˜ 𝑢ˆ ⊤ 𝑀 2 2

⩽ 𝑂𝛿

√

𝑑 · 𝐿 · 𝜂1/2−𝛿 . |𝜅3 (𝛾)|

Using (5.1), where 𝑢˜ = 𝑢 ∗ /∥𝑢 ∗ ∥ − Σ𝑤/∥Σ𝑤∥ 𝑢˜ ⊤ 𝑀2−1 𝑢˜ = Therefore

b −1 𝑢ˆ − 𝑢ˆ 𝑇 𝑀 2

𝜅3 (𝛾)2/3 ∥Σ1/2 𝑤∥2 = . 𝜅2 (𝛾)∥Σ𝑤∥2 ∥𝑢 ∗ ∥2 𝜅2 (𝛾)

√ ∥Σ1/2 𝑤∥2 𝑑 1/2−𝛿 ⩽ 𝑂𝛿 ·𝐿·𝜂 |𝜅3 (𝛾)| ∥Σ𝑤∥2 · 𝜅 2 (𝛾)

(6.9)

Conclusion for covariance estimation. Combining (6.5), (6.7), (6.8) and (6.9) we can apply Lemma 5.13 with √ 1 1/2−𝛿 · 𝑑·𝐿·𝜂 𝛽 = 𝑂 𝛿 max 1, |𝜅3 (𝛾)| Note that by (6.3) we have 𝛽 ≲ 𝜅2 (𝛾)5/2 . This gives us an estimator b Σ with error ∥Σ

−1/2 b −1/2

ΣΣ

− 𝐼 𝑑 ∥𝐹 ⩽ 𝑂 𝛿

√ 1 1 · max 1, · 𝑑 · 𝐿 · 𝜂1/2−𝛿 . |𝜅3 (𝛾)| 𝜅2 (𝛾)7/2

Mean estimation. For mean estimation, given Lemma 5.11, we only need to argue that we can additionally bound ((5.5) and (5.7)) 5. 𝑥¯ − 𝜇 − 𝜅1 (𝛾) 6. 𝑢ˆ −

Σ𝑤 ∥Σ1/2 𝑤∥

𝑢∗ ∥𝑢 ∗ ∥

Bounding 5. By Lemma 6.7, we are able to get an estimate 𝜇ˆ trunc for the mean of the truncated distribution 𝜇trunc = 𝜇 + 𝜅 1 (𝛾) ∥ΣΣ𝑤 1/2 𝑤∥ . This estimate satisfies 𝜇trunc − 𝜇ˆ trunc ⩽ 𝑂 𝛿 𝐿 · 𝜂1−𝛿 .

(6.10)

Thus we can take 𝑥¯ = 𝜇ˆ trunc and we are done. ∗ Bounding 6. Given that we already bounded 2., we only need to determine the sign of ∥𝑢𝑢 ∗ ∥ . We can do so using the third moment tensor. Let 𝑢˜ B

𝑢∗ . ∥𝑢 ∗ ∥

Since 𝑀3 = (𝑢 ∗ )⊗3 , we have ˜ 3 = ∥𝑢 ∗ ∥3 > 0. 𝑀3 , 𝑢˜ ⊗3 = (𝑢 ∗ )⊗3 , 𝑢˜ ⊗3 = ⟨𝑢 ∗ , 𝑢⟩ 55

Let 𝑎 ∈ {±1} be the sign such that ˜ ⩽ 𝑂𝛿 ∥𝑎 𝑢ˆ − 𝑢∥

√

𝑑 · 𝐿 · 𝜂1/2−𝛿 . |𝜅3 (𝛾)|

from (6.7). We can compute

E D E ⊗3 ⊗3 b b ˆ ˆ ˆ ⊗3 𝑀3 , (𝑎 𝑢) = 𝑀3 − 𝑀3 , (𝑎 𝑢) + 𝑀3 , (𝑎 𝑢) D E b 3 − 𝑀3 , (𝑎 𝑢) ˆ ⊗3 + 𝑀3 , (𝑎 𝑢) ˆ ⊗3 − 𝑢˜ ⊗3 + 𝑀3 , 𝑢˜ ⊗3 . = 𝑀

We bound the first term by Lemma 6.7:

b 3 − 𝑀3 , (𝑎 𝑢) ˆ ⊗3 𝑀

⩽ 𝑂 𝛿 𝐿 · 𝜂1/2−𝛿 .

˜ we get For the second term, using 𝑀3 = (𝑢 ∗ )⊗3 and the bound on ∥𝑎 𝑢ˆ − 𝑢∥, ˆ 3 − ⟨𝑢 ∗ , 𝑢⟩ ˜ 3 = ⟨𝑢 ∗ , 𝑎 𝑢⟩ √ 𝑑 1/2−𝛿 ∗ 3 ·𝐿·𝜂 , ⩽ 𝑂 𝛿 ∥𝑢 ∥ |𝜅3 (𝛾)|

ˆ ⊗3 − 𝑢˜ ⊗3 𝑀3 , (𝑎 𝑢)

where we used that 𝑎 𝑢ˆ and 𝑢˜ are unit vectors and 𝑂 𝛿

√

𝑑 · 𝐿 · 𝜂1/2−𝛿 |𝜅 3 (𝛾)|

⩽ 𝑂(1). Therefore,

E √ b 3 , (𝑎 𝑢) ˆ ⊗3 ⩾ ∥𝑢 ∗ ∥3 − 𝑂 𝛿 𝐿𝜂1/2−𝛿 − 𝑂 ∥𝑢 ∗ ∥3 ( 𝑑/|𝜅3 (𝛾)|)𝐿𝜂1/2−𝛿 . 𝑀

Hence, since by (6.3) and

∥𝑢 ∗ ∥3 ≳ |𝜅 3 (𝛾)|

in the precondition coordinates, we have

𝑂 𝛿 𝐿𝜂

1/2−𝛿

∗ 3

+ 𝑂 𝛿 ∥𝑢 ∥

√

𝑑 · 𝐿 · 𝜂1/2−𝛿 |𝜅3 (𝛾)|

1 ∗ 3 ∥𝑢 ∥ , 2

and we get

b 3 , (𝑎 𝑢) ˆ ⊗3 > 0. 𝑀

Since the tensor is order three,

b 3 , (−𝑎 𝑢) b 3 , (𝑎 𝑢) ˆ ⊗3 = − 𝑀 ˆ ⊗3 < 0. 𝑀

Thus, we can determine the correct sign for 𝑢ˆ by choosing the sign for which

b 3 , 𝑢ˆ ⊗3 > 0. 𝑀

Recall that we need to find a vector parallel to −Σ𝑤 and 𝑢 ∗ is a vector that points in the direction −Σ𝑤. After this sign choice, we have √ Σ𝑤 𝑑 1/2−𝛿 𝑢ˆ + ⩽ 𝑂𝛿 ·𝐿·𝜂 . ∥Σ𝑤∥ |𝜅 3 (𝛾)| 56

Conclusion for mean estimation. Combining (6.10), (6.5), (6.7), (6.9) and (6.8), we can apply Lemma 5.11 √ 1 1/2−𝛿 𝛽 = 𝑂 𝛿 max 1, · 𝑑·𝐿·𝜂 . |𝜅3 (𝛾)| Note that by (6.3) we have 𝛽 ≲ 𝜅2 (𝛾)5/2 . This gives us an estimator 𝜇ˆ with error Σ−1/2 (𝜇ˆ − 𝜇) ⩽ 𝑂( 𝜇ˆ − 𝜇 )

⩽ 𝑂𝛿

√ 1 1 · max 1, · 𝑑 · 𝐿 · 𝜂1/2−𝛿 . |𝜅3 (𝛾)| 𝜅2 (𝛾)7/2

□

Extension to intersections of two orthogonal halfspaces

In this section, we show how to extend our algorithm to truncations sets that are the intersection of two orthogonal halfspaces. Suppose 𝑑 ⩾ 2 and consider a random vector 𝑋 that follows a 𝑑-dimensional Gaussian 𝒩 (𝜇, 𝐼 𝑑 ) truncated to the intersection of 𝐻1 and 𝐻2 , where 𝐻1 = {𝑥 : ⟨𝑤1 , 𝑥⟩ ⩽ 𝜏1 } and 𝐻2 = {𝑥 : ⟨𝑤2 , 𝑥⟩ ⩽ 𝜏2 } are halfspaces with ∥𝑤1 ∥ = ∥𝑤 2 ∥ = 1 and 𝑤 1 ⊥ 𝑤2 . As before, we define 𝛼 > 0 to be the probability measure of 𝐻1 ∩ 𝐻2 under the underlying Gaussian. We will prove the following result. For simplicity, we assume that the underlying Gaussian has identity covariance and we only aim to estimate the mean. Theorem 7.1. There is an algorithm (Algorithm 3) that takes 𝑛 = 𝑂

𝑑 poly log 𝛼1 𝜀2

+ 𝑑𝜀4 i.i.d. samples

ˆ such that with probability from 𝒩 (𝜇, 𝐼 𝑑 ) truncated to 𝐻1 ∩𝐻2 as input and efficiently computes an estimate 𝜇, at least 0.99 it satisfies ˆ 𝐼 𝑑 ), 𝒩 (𝜇, 𝐼 𝑑 ) ⩽ 𝜀. 𝑑TV 𝒩 (𝜇, The runtime of the algorithm is 𝑂(𝑇(𝑛, 𝑑)), where 𝑇(𝑛, 𝑑) is the time needed to multiply a 𝑑 × 𝑛 matrix with its transpose. Remark 7.2. Before proving Theorem 7.1, we want to briefly discuss potential further extensions: 1. While we only prove our extension for mean estimation, it seems plausible that one can also do covariance estimation. In this case, we would want that 𝑤 1 and 𝑤2 are ‘Σ-orthogonal’, meaning that Σ𝑤1 and Σ𝑤 2 are orthogonal. One thing that is unclear however is whether and how we could do preconditioning in this case to avoid a dependence on the condition number (as preconditioning need not preserve orthogonality). 2. Finally, the correction terms in each direction are (almost) independent of each other due to orthogonality. This should enable us to generalize our estimator to an intersection of 𝑘 pairwise orthogonal halfspaces (for 𝑘 ⩽ 𝑑). The required sample and time complexity in this case would depend on the minimum

e 12 and the eigenvalue gap after the random contraction, which asymptotically could be as small as 𝑂 𝑘 sample complexity should increase by a factor that is polynomial in 𝑘. If there is non-trivial truncation in each direction, we would thus expect a sample complexity of 𝑂(𝑑2 /𝜀2 ) · polylog(1/𝛼) · poly(𝑘).

Algorithm 3: Algorithm for estimating the mean of an identity-covariance Gaussian truncated by two orthogonal halfspaces Input: Dimension 𝑑, sample size 𝑛, samples {𝑥 𝑖 }𝑛𝑖=1 , target accuracy 𝜀. Output: Estimated mean 𝜇ˆ ∈ R𝑑 . 1. Compute sample mean: 𝑥¯ = 𝑛1 𝑛𝑖=1 𝑥 𝑖 .; b 2 = 1 Í𝑛 (𝑥 𝑖 − 𝑥) ¯ ⊗2 .; 2. Compute sample second central moment: 𝑀 𝑖=1 𝑛

3. Draw a standard Gaussian vector: 𝑔 ∼ 𝒩 (0, 𝐼 𝑑 ).;

b3 = 1 4. Compute the random contraction: 𝑅 = (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) · 𝑀 𝑛

Í𝑛

¯ 𝑖=1 (𝑥 𝑖 − 𝑥)

⊗2 · ⟨𝑔, 𝑥

¯ 𝑖 − 𝑥⟩.

5. Compute 𝑤ˆ 1 and 𝑤ˆ 2 , the eigenvectors of 𝑅 corresponding to the two largest eigenvalue in absolute value (with unit norm).

b 3 > 0. 6. For ℓ ∈ {1, 2}, flip the sign of 𝑤ˆ ℓ if 𝑤ˆ ℓ⊗3 , 𝑀

b 2 𝑤ˆ ℓ . 𝑤ˆ ℓ⊤ 𝑀 7. For ℓ ∈ {1, 2}, estimate the relative truncation parameters 𝛾ℓ : 𝛾ˆℓ = 𝜅−1 2 8. Estimate the mean of the Gaussian: 𝜇ˆ = 𝑥¯ − 𝜅 1 ( 𝛾ˆ 1 ) · 𝑤ˆ 1 − 𝜅 1 ( 𝛾ˆ 2 ) · 𝑤ˆ 2 . ˆ 9. Return estimate 𝜇.

7.1

Moments

As for our main result, we first compute the first three moments of 𝒩 (𝜇, 𝐼 𝑑 )|𝐻1 ∩𝐻2 . The values 𝛾1 , 𝛾2 will be defined analogously to the relative truncation parameter in the main result. Lemma 7.3. For a random vector 𝑋 defined as above, there are real numbers 𝛾1 , 𝛾2 such that

E 𝑋 = 𝜇 + 𝜅1 (𝛾1 ) · 𝑤1 + 𝜅1 (𝛾2 ) · 𝑤2 𝑀2 B E(𝑋 − E 𝑋)⊗2 = 𝐼 𝑑 − (1 − 𝜅2 (𝛾1 )) · 𝑤 1 𝑤 1⊤ − (1 − 𝜅2 (𝛾2 )) · 𝑤 2 𝑤 2⊤ 𝑀3 B E(𝑋 − E 𝑋)⊗3 = 𝜅3 (𝛾1 ) · (𝑤1 )⊗3 + 𝜅 3 (𝛾2 ) · (𝑤 2 )⊗3 Moreover, we have 𝛼 = Φ(𝛾1 ) · Φ(𝛾2 ). Proof. For 𝑑 ⩾ 2, since 𝑤1 , 𝑤2 are perpendicular unit vectors, there exists an orthonormal matrix 𝑈 such that 𝑈 𝑒1 = 𝑤1 and 𝑈 𝑒2 = 𝑤2 . As before, we consider 𝑌 = 𝑈 ⊤ (𝑋 − 𝜇). This random variable follows a standard Gaussian 𝒩 (0, 𝐼𝑛 ) truncated as follows: 𝑤 1⊤ 𝑋 ⩽ 𝜏1

𝑤 2⊤ 𝑋 ⩽ 𝜏2

⇐⇒ ⇐⇒

(𝑈 ⊤ 𝑤 1 )⊤𝑌 ⩽ 𝜏1 − 𝑤 1⊤ 𝜇

(𝑈 ⊤ 𝑤 2 )⊤𝑌 ⩽ 𝜏2 − 𝑤 2⊤ 𝜇.

Since 𝑈 𝑒1 = 𝑤1 and 𝑈 𝑒2 = 𝑤2 , we get a simple characterization of 𝑌: Defining 𝛾1 = 𝜏1 − ⟨𝑤1 , 𝜇⟩ and 𝛾2 = 𝜏2 − ⟨𝑤2 , 𝜇⟩, 𝑌1 and 𝑌2 are truncated standard Gaussian with thresholds 𝛾1 and 𝛾2 , all other coordinates are standard Gaussian and all coordinates are mutually independent. Thus, as in Lemma 4.2, we get that

E 𝑌 = 𝜅1 (𝛾1 )𝑒1 + 𝜅1 (𝛾2 )𝑒2 , E(𝑌 − E 𝑌)⊗2 = 𝐼 𝑑 − (1 − 𝜅2 (𝛾1 ))𝑒1 𝑒1⊤ − (1 − 𝜅2 (𝛾2 ))𝑒2 𝑒2⊤ , 58

E(𝑌 − E 𝑌)⊗3 = 𝜅3 (𝛾1 )𝑒1⊗3 + 𝜅3 (𝛾2 )𝑒2⊗3 . Using this, we can compute the moments of 𝑋 as follows. First, the mean of 𝑋 is

E 𝑋 = 𝑈 E 𝑌 + 𝜇 = 𝜇 + 𝜅1 (𝛾1 ) · 𝑈 𝑒1 + 𝜅1 (𝛾2 ) · 𝑈 𝑒2 = 𝜇 + 𝜅1 (𝛾1 ) · 𝑤1 + 𝜅1 (𝛾2 ) · 𝑤2 . The covariance of 𝑋 is 𝑀2 = E(𝑋 − E 𝑋)⊗2 = 𝑈 ⊗2 E(𝑌 − E 𝑌)⊗2 = 𝑈 𝐼 𝑑 − (1 − 𝜅2 (𝛾1 )) · 𝑒1 𝑒1⊤ − (1 − 𝜅2 (𝛾2 )) · 𝑒2 𝑒2⊤ 𝑈 ⊤

= 𝐼 𝑑 − (1 − 𝜅2 (𝛾1 )) · 𝑤 1 𝑤 1⊤ − (1 − 𝜅2 (𝛾2 )) · 𝑤 2 𝑤2⊤ .

And the third central moment of 𝑋 equals 𝑀3 = E(𝑋 − E 𝑋)⊗3 = 𝑈 ⊗3 E(𝑌 − E 𝑌)⊗3 = 𝑈 ⊗3 𝜅3 (𝛾1 ) · 𝑒1⊗3 + 𝜅 3 (𝛾2 ) · 𝑒2⊗3

= 𝜅3 (𝛾1 ) · 𝑤 1⊗3 + 𝜅 3 (𝛾2 ) · 𝑤 2⊗3 . Finally, define 𝐻1′ = 𝑦 ∈ R𝑑 | 𝑦1 ⩽ 𝛾1 and 𝐻2′ = 𝑦 ∈ R𝑑 | 𝑦2 ⩽ 𝛾2 . Then, we have that

𝛼= = =

[1 [𝑋 ∈ 𝐻1 ∩ 𝐻2 ]]

[𝑌1 ⩽ 𝛾1 ] ·

𝑋∼𝒩 (𝜇,𝐼 𝑑 ) 𝑌∼𝒩 (0,𝐼 𝑑 ) 𝑌1 ∼𝒩 (0,1)

1 𝑌 ∈ 𝐻1′ ∩ 𝐻2′

𝑌2 ∼𝒩 (0,1)

[𝑌2 ⩽ 𝛾2 ]

= Φ(𝛾1 ) · Φ(𝛾2 ), which completes the proof.

□

Since we assume that the covariance of the underlying Gaussian is the identity, we don’t need to precondition the samples. Instead, we directly get that min{𝜅 2 (𝛾1 ), 𝜅2 (𝛾2 )} ⪯ 𝑀2 ⪯ 𝐼 𝑑 .

7.2

(7.1)

Relative truncation parameter estimation

By orthogonality between 𝑤1 , 𝑤2 , we could isolate them from the second central moment as long as 𝛾1 , 𝛾2 are constantly-separated. However, this is not true in general. It might be that 𝛾1 = 𝛾2 , which makes it information-theoretically impossible to identify 𝑤1 , 𝑤2 from 𝑀2 already when 𝑑 = 2. Therefore, as in the case of a single halfspace, we use the third central moment 𝑀3 . Note that with probability 1, we have that 𝜅3 (𝛾1 ) · ⟨𝑔, 𝑤 1 ⟩ and 𝜅 3 (𝛾2 ) · ⟨𝑔, 𝑤 2 ⟩ are distinct. Furthermore, we will be able to bound the distance between these eigenvalues using Fact 3.24. We will use the following theorem from matrix perturbation theory (see also [Bha97, Corollary III.2.6 and Theorem VII.3.2]). Theorem 7.4 ([DK70, Wey12]). Let 𝐴, 𝐸 ∈ R𝑑×𝑑 be symmetric matrices. Consider the eigenvalues 𝜆1 ⩾ 𝜆2 ⩾ . . . ⩾ 𝜆 𝑑 of 𝐴 with associated orthonormal eigenvectors 𝑣 𝑖 and the eigenvalues 𝜆˜ 1 ⩾ 𝜆˜ 2 ⩾ . . . 𝜆˜ 𝑑 and orthonormal eigenvectors 𝑣˜ 𝑖 of 𝐴˜ = 𝐴 + 𝐸. Define 𝛿 𝑖 = min 𝑗≠𝑖 |𝜆 𝑗 − 𝜆 𝑖 |. Then, if 𝛿 𝑖 > 0, we have min {∥𝑣˜ 𝑖 + 𝑣 𝑖 ∥2 , ∥𝑣˜ 𝑖 − 𝑣 𝑖 ∥2 } ⩽ 𝐶 for an absolute constant 𝐶 > 0. Furthermore, we have that |𝜆 𝑖 − 𝜆˜ 𝑖 | ⩽ ∥𝐸∥ . 59

∥𝐸∥ 𝛿𝑖

We now show the following lemma, which is analogous to Section 5.1. We note that after step 6 of Algorithm 3, the estimates 𝑤ˆ 1 and 𝑤ˆ 2 are exactly the ones from Lemma 7.5. Lemma 7.5. Let 𝛽 > 0. Given 𝑛 i.i.d. samples from 𝒩 𝜇, Σ truncated to 𝐻1 ∩ 𝐻2 , we can compute orthogonal unit vectors 𝑤ˆ 1 , 𝑤ˆ 2 such that the following hold for ℓ ∈ {1, 2}: If

𝑑2 𝑑2 , 𝑛=𝑂 2 + 𝛽 𝜅3 (𝛾ℓ )2 𝜅 3 (𝛾ℓ )4

then we have with high probability that ∥𝑤ˆ ℓ − 𝑤ℓ ∥ ⩽ 𝛽.

b3. Proof. We want to use Theorem 7.4 for the matrix 𝐴 = (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ )𝑀3 and 𝐴˜ = (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) 𝑀 ⊤ b Let the spectral decomposition of (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ) 𝑀3 be ⊤

b3 = (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ) 𝑀

𝑑 Õ

𝜆˜ 𝑗 𝑣˜ 𝑗 𝑣˜ ⊤ 𝑗 ,

𝑗=1

where 𝜆˜ 1 ⩾ 𝜆˜ 2 ⩾ · · · ⩾ 𝜆˜ 𝑑 . Since this matrix is symmetric, we can assume that the 𝑣˜ 𝑗 are an orthonormal basis of R𝑑 . To apply Theorem 7.4, we need to determine the eigenvalues and eigenvectors of (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ )𝑀3 . We have, since Σ = 𝐼 𝑑 , 𝑀3 = 𝜅3 (𝛾1 )𝑤1⊗3 + 𝜅 3 (𝛾2 )𝑤2⊗3 ,

(𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ )𝑀3 = 𝜅3 (𝛾1 ) · ⟨𝑔, 𝑤 1 ⟩ · 𝑤 1 𝑤 1⊤ + 𝜅 3 (𝛾2 ) · ⟨𝑔, 𝑤 2 ⟩ · 𝑤 2 𝑤2⊤ . Thus, the eigenvalues of (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ )𝑀3 are 𝜅 3 (𝛾1 ) · ⟨𝑔, 𝑤 1 ⟩, 𝜅3 (𝛾2 ) · ⟨𝑔, 𝑤 2 ⟩ and 0 with multiplicity 𝑛 − 2. Furthermore, the corresponding eigenvectors are 𝑤 1 and 𝑤2 (and an orthogonal basis of the complement). Let 𝑖1 , 𝑖2 ∈ [𝑑] be such that 𝑣 𝑖1 = 𝑤 1 and 𝑣 𝑖2 = 𝑤 2 . We will define our estimates as the two eigenvectors of 𝐴˜ corresponding to the largest eigenvalues in absolute value (up to a sign choice that we discuss later). This already implies that they are orthogonal. If we want one of our two estimates to be a good estimate for 𝑤ℓ , we need to ensure that even after the perturbation, the eigenvalue corresponding to 𝑤ℓ is one of the two largest in absolute value (i.e. it should be later than the perturbation of the 0 eigenvalues). We can ensure this as follows: The eigenvalues of 𝐴 that are 0 correspond to eigenvalues of 𝐴˜ of absolute value at most ˜ ˜ in absolute value, even after ∥𝐴 − 𝐴∥). Thus, if the eigenvalue for 𝑤ℓ is larger than 2 · ∥𝐴 − 𝐴∥) perturbation it will belong to the largest two (hence, one of our estimates will be close to 𝑤ℓ up ˜ ⩽ 1 · 𝜅 3 (𝛾ℓ ) · ⟨𝑔, 𝑤ℓ ⟩ , it suffices to argue that one of to sign). Hence, if we ensure that ∥𝐴 − 𝐴∥ 2 ∥𝑤ℓ ± 𝑣˜ 𝑖ℓ ∥ is small and how to recover the sign. In order to apply Theorem 7.4, we also need to compute the values of 𝛿 𝑖1 and 𝛿 𝑖2 . For this, note that ⟨𝑔, 𝜅 3 (𝛾1 ) · 𝑤1 − 𝜅 3 (𝛾2 ) · 𝑤2 ⟩ is distributed identical to the Gaussian 𝒩 (0, 𝜅 3 (𝛾1 )2 + 𝜅 3 (𝛾2 )2 ). Thus, by Gaussian anti-concentration (cf. Fact 3.24), we get that, with high probability, |𝜅3 (𝛾1 ) · ⟨𝑔, 𝑤 1 ⟩ − 𝜅 3 (𝛾2 ) · ⟨𝑔, 𝑤 2 ⟩| ⩾ Ω

𝜅3 (𝛾1 )2 + 𝜅 3 (𝛾2 )2

Similarly, we get |𝜅 3 (𝛾1 ) · ⟨𝑔, 𝑤 1 ⟩| ⩾ Ω (|𝜅 3 (𝛾1 )|)

and

|𝜅3 (𝛾2 ) · ⟨𝑔, 𝑤 2 ⟩| ⩾ Ω (|𝜅3 (𝛾2 )|) .

and

𝛿 𝑖2 ⩾ Ω (|𝜅3 (𝛾2 )|) .

Thus, we can lower bound 𝛿 𝑖1 ⩾ Ω (|𝜅 3 (𝛾1 )|)

b 3 − (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ )𝑀3 ∥. First, analogous to Lemma 5.8, we get It remains to bound ∥(𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ ) 𝑀 that, with high probability, D

⊗3

b 3 − 𝑀3 ≲ sup 𝑣 , 𝑀 ∥𝑣∥=1

𝑑 𝑑 3/2 + . 𝑛 𝑛

Note that we get a slightly better result here p since the samples 𝑥 𝑖 are in fact 1-sub-Gaussian in this case (because Σ = 𝐼 𝑑 ) and not only 𝑂(1/ 𝜅2 (𝛾))-sub-Gaussian as before. Now, observe that with high probability,

b 3 − 𝑀3 ) ⩽ ∥𝑔∥ sup 𝑣 ⊗3 , 𝑀 b 3 − 𝑀3 (𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ )( 𝑀

∥𝑣∥=1

√

⊗3

b 3 − 𝑀3 ⩽ 3 𝑑 · sup 𝑣 , 𝑀

∥𝑣∥=1

𝑑 𝑛

≲√ +

𝑑2 . 𝑛

Applying Theorem 7.4, we conclude that, for ℓ ∈ {1, 2}, min ∥𝑏 · 𝑣˜ 𝑖ℓ − 𝑤ℓ ∥ ≲ 𝜅 3 (𝛾ℓ )

𝑏∈{±1}

−1

𝑑2 𝑑 . · √ + 𝑛 𝑛

Let ℓ ∈ {1, 2}. It remains to argue that with the claimed sample complexity, we can make this error at most 𝛽 and that we can recover the sign. Note that for the claimed sample complexity we indeed have, using the above bound (and assuming 𝛽 ⩽ 𝑂(1)), that 1 · 𝜅3 (𝛾ℓ ) · ⟨𝑔, 𝑤ℓ ⟩ , 2 so 𝑣˜ 𝑖 will correspond to one of the two largest eigenvalues of 𝐴˜ up to sign. If

b 3 − 𝑀3 )∥ ⩽ ∥(𝐼 𝑑 ⊗ 𝐼 𝑑 ⊗ 𝑔 ⊤ )( 𝑀

ℓ

𝑑2 𝑛 = 𝑂 ′2 , 𝛽 · 𝜅 3 (𝛾ℓ )2

then we get that

min ∥𝑏 · 𝑣˜ 𝑖ℓ − 𝑤ℓ ∥ ⩽ 𝛽 ′ .

𝑏∈{±1}

b 3 ⟩. We have ⟨𝑤 ⊗3 , 𝑀3 ⟩ = 𝜅3 (𝛾ℓ ) < 0. To determine the sign, we again check the sign of ⟨𝑣˜ ⊗3 ,𝑀 𝑖ℓ ℓ Denote by 𝑏ℓ ∈ {±1} the correct sign for 𝑣˜ 𝑖ℓ (i.e. the one for which ∥𝑏ℓ 𝑣˜ 𝑖ℓ − 𝑤ℓ ∥ ⩽ 𝛽). Then, we have that D E D E b 3 ⩽ (𝑤ℓ − 𝑏ℓ 𝑣˜ 𝑖ℓ )⊗3 , 𝑀3 + 𝑏ℓ 𝑣˜ ⊗3 , 𝑀3 − 𝑀 b3 . 𝜅3 (𝛾ℓ ) − 𝑏ℓ 𝑣˜ ⊗3 , 𝑀 𝑖ℓ 𝑖ℓ We can bound the first term using 𝑀3 = 𝜅 3 (𝛾1 )𝑤1⊗3 + 𝜅 3 (𝛾2 )𝑤2⊗3 as follows, assuming 𝛽′ ⩽ 𝑂(1) (such that we can bound 𝛽′3 ⩽ 𝑂(𝛽 ′ )),

≲ (|𝜅3 (𝛾1 )| + |𝜅3 (𝛾2 )|) · ∥𝑤ℓ − 𝑏ℓ 𝑣˜ 𝑖ℓ ∥3 ≲ 𝛽′ .

(𝑤ℓ − 𝑏ℓ 𝑣˜ 𝑖ℓ )⊗3 , 𝑀3

For the second term, note that the sample complexity above implies15

b 𝑏ℓ 𝑣˜ ⊗3 𝑖ℓ , 𝑀 3 − 𝑀 3

≲ 𝛽′ .

𝛽′ 15 We could even get |𝜅 (𝛾 )| · √ , but the bounding this by 𝛽′ is sufficient. 3 ℓ 𝑑

Thus, we get that 𝜅 3 (𝛾ℓ ) −

b 𝑏ℓ 𝑣˜ ⊗3 𝑖ℓ , 𝑀 3

≲ 𝛽′ .

b 3 ⟩ < 0 and we can determine ,𝑀 If 𝛽 ′ ⩽ 𝑐 · 𝜅 3 (𝛾ℓ ) for a sufficiently small constant 𝑐, then ⟨𝑏ℓ 𝑣˜ ⊗3 𝑖ℓ b 3 ⟩. Summarized, for 𝛽′ ⩽ 𝑐 · 𝜅3 (𝛾ℓ ) (which also implies 𝛽′ ⩽ 𝑂(1) by the sign 𝑏ℓ from ⟨𝑣˜ ⊗3 ,𝑀 𝑖ℓ Claim 3.10), we can identify the sign 𝑏ℓ such that ∥𝑏ℓ 𝑣˜ 𝑖ℓ − 𝑤ℓ ∥ ⩽ 𝛽 ′ if we get 𝑑2 𝑛 = 𝑂 ′2 𝛽 · 𝜅 3 (𝛾ℓ )2

samples. We can now pick 𝛽′ = min{𝛽, 𝑐 · 𝜅3 (𝛾ℓ )} to get the result.

7.3

□

Proof of Theorem 7.1

b 2 𝑤ˆ ℓ , which yields an approxWith the learned normal vectors 𝑤ˆ 1 and 𝑤ˆ 2 , we can compute 𝑤ˆ ℓ⊤ 𝑀 imation to 𝜅 2 (𝛾ℓ ). By monotonicity of 𝜅2 (cf. Claim 3.8), we obtain estimation for the relative truncation parameters 𝛾ˆ 1 and 𝛾ˆ 2 , which allows us to compute an estimator for 𝜇. This allows us to prove Theorem 7.1. As before, we need to be careful if 𝛼ℓ is close to 1 as then 𝜅3 (𝛾ℓ ) → 0. Proof of Theorem 7.1. Recall from Lemma 7.3 that, for 𝑋 ∼ 𝒩 (𝜇, 𝐼 𝑑 )|𝐻1 ∩𝐻2 we have

E[𝑋] = 𝜇 + 𝜅1 (𝛾1 ) · 𝑤1 + 𝜅1 (𝛾2 ) · 𝑤2 . We thus get 𝜇ˆ − 𝜇 = 𝑥¯ − 𝜅 1 ( 𝛾ˆ 1 )𝑤ˆ 1 − 𝜅 1 ( 𝛾ˆ 2 )𝑤ˆ 2 − 𝜇 = ∥𝑥¯ − E[𝑋] + 𝜅 1 ( 𝛾ˆ 1 )𝑤ˆ 1 + 𝜅 1 ( 𝛾ˆ 2 )𝑤ˆ 2 − 𝜅 1 (𝛾1 )𝑤1 − 𝜅 1 (𝛾2 )𝑤2 ∥

⩽ ∥𝑥¯ − E[𝑋]∥ + ∥𝜅1 (𝛾ˆ 1 )𝑤ˆ 1 − 𝜅1 (𝛾1 )𝑤1 ∥ + ∥𝜅1 (𝛾ˆ 2 )𝑤ˆ 2 − 𝜅1 (𝛾2 )𝑤2 ∥ . We have that 𝑥 𝑖 is 1-sub-Gaussian and thus given 𝑛 = 𝑂(𝑑/𝛽 2 ) samples, as in Lemma 5.14, we have ∥𝑥¯ − E[𝑋]∥ ⩽ 𝛽. For the other two terms, we need to distinguish whether 𝛼ℓ is close to 1 or not. Both terms are the same, so we only show how to bound ∥𝜅 1 ( 𝛾ˆ 1 )𝑤ˆ 1 − 𝜅1 (𝛾1 )𝑤1 ∥. As before 1−𝛽 in Theorem 5.1, we distinguish two cases: If 𝛼1 ⩽ 𝛼 ∗ B max{1 − 𝑐, log(1/𝛽) } for a sufficiently small constant 𝑐 > 0, then we use Lemma 7.5 to get an estimate ∥𝑤ˆ ℓ − 𝑤ℓ ∥ ⩽ 𝛽 and use this to bound ∥𝜅1 ( 𝛾ˆ 1 )𝑤ˆ 1 − 𝜅 1 (𝛾1 )𝑤 1 ∥. If 𝛼 1 ⩾ 𝛼 ∗ , then we show that this norm is small by arguing that both 𝜅 1 (𝛾ˆ 1 ) and 𝜅 1 (𝛾) are small. Case (i): 𝛼1 ⩽ 𝛼∗ . Using 𝑑2 𝑑2 𝑛=𝑂 2 + 𝛽 𝜅3 (𝛾1 )2 𝜅3 (𝛾1 )4

b2 = 1 samples, the estimate 𝑤ˆ 1 from Lemma 7.5 satisfies ∥𝑤ˆ 1 − 𝑤1 ∥ ⩽ 𝛽. Let 𝑀 𝑛 sample covariance. We have

Í𝑛

¯ 𝑖=1 (𝑥 𝑖 − 𝑥)

b 2 𝑤ˆ 1 = 𝑤 ⊤ 𝑀 b 2 𝑤1 + (𝑤ˆ 1 − 𝑤1 )⊤ 𝑀 b 2 (𝑤ˆ 1 + 𝑤1 ) 𝑤ˆ 1⊤ 𝑀 1 b 2 − 𝑀2 )𝑤1 + (𝑤ˆ 1 − 𝑤1 )⊤ 𝑀 b 2 (𝑤ˆ 1 + 𝑤1 ) = 𝑤1⊤ 𝑀2 𝑤1 + 𝑤 1⊤ ( 𝑀 b 2 − 𝑀2 )𝑤1 + (𝑤ˆ 1 − 𝑤1 )⊤ 𝑀 b 2 (𝑤ˆ 1 + 𝑤1 ). = 𝜅2 (𝛾1 ) + 𝑤 1⊤ ( 𝑀 By Lemma 7.5, we have

b 2 (𝑤ˆ 1 + 𝑤1 ) ⩽ 2 𝑀 b 2 𝛽. (𝑤ˆ 1 − 𝑤 1 )⊤ 𝑀 62

2 be the

Furthermore, analogous to Claim 4.3, we get that the 𝑀2−1 𝑥 𝑖 are 𝑂(1/min{ 𝜅 2 (𝛾1 ), 𝜅2 (𝛾2 )})-subGaussian. Thus, using 𝑑 𝑛=𝑂 2 𝛽 · min{𝜅 2 (𝛾1 ), 𝜅2 (𝛾2 )}

b 2 ⩽ 𝛽. Since 𝑀2 ⪯ 𝐼 𝑑 , this also implies that ∥ 𝑀 b 2 ∥ ⩽ 2, samples, we get by Fact 3.26 that 𝑀2 − 𝑀 as long as 𝛽 ⩽ 1. Putting these together, we get that b 2 𝑤ˆ 1 − 𝜅2 (𝛾1 ) ⩽ 𝑂(𝛽). 𝑤ˆ 1⊤ 𝑀 b 2 𝑤ˆ 1 ) (this inverse function exists since 𝜅2 is monotonic by Claim 3.8). By Define 𝛾ˆ 1 = 𝜅−1 (𝑤ˆ 1⊤ 𝑀 2 Claim 3.21, we have |𝜅1 ( 𝛾ˆ 1 ) − 𝜅 1 (𝛾1 )| ≲ |𝜅2 ( 𝛾ˆ 1 ) − 𝜅 2 (𝛾1 )| · max{𝜅 2 (𝛾1 )−2 , 𝜅2 ( 𝛾ˆ 1 )−2 }. Now assume 𝛽 < 𝑐 ′ · 𝜅 2 (𝛾1 ) (for a sufficiently small constant 𝑐 ′ ). Then, we get 𝜅 2 ( 𝛾ˆ 1 ) ⩾ 21 𝜅2 (𝛾1 ) and thus |𝜅1 ( 𝛾ˆ 1 ) − 𝜅 1 (𝛾1 )| ≲ 𝛽 · 𝜅2 (𝛾1 )−2 . We thus have that ∥𝜅1 ( 𝛾ˆ 1 )𝑤ˆ 1 − 𝜅 1 (𝛾1 )𝑤 1 ∥ ⩽ ∥𝜅1 (𝛾ˆ 1 )𝑤ˆ 1 − 𝜅 1 (𝛾1 )𝑤ˆ 1 ∥ + ∥𝜅1 (𝛾1 )𝑤ˆ 1 − 𝜅 1 (𝛾1 )𝑤1 ∥ 𝛽 ⩽𝑂 + |𝜅 1 (𝛾1 )| · ∥𝑤ˆ 1 − 𝑤 1 ∥ 𝜅2 (𝛾1 )2 1 · 𝑂(𝛽), ⩽ 𝜅2 (𝛾1 )2

where we used Claim 3.9 for |𝜅 1 (𝛾1 )| ≲ 𝜅 2 (𝛾1 )−1/2 . This holds for all 𝛽 < 𝑐 ′ · 𝜅2 (𝛾1 ) and the sample complexity needed is 𝑑2 𝑑2 𝑑 𝑛=𝑂 2 + + 2 . 2 4 𝛽 𝜅3 (𝛾1 ) 𝛽 · min{𝜅 2 (𝛾1 ), 𝜅2 (𝛾2 )} 𝜅3 (𝛾1 )

Case (ii): 𝛼1 ⩾ 𝛼 ∗ . In this case, we want to argue that both |𝜅1 (𝛾1 )| and |𝜅1 ( 𝛾ˆ 1 )| are small. Note that by picking 𝑐 sufficiently small, we can achieve 𝛾1 ⩾ 2 and get |𝜅1 (𝛾1 )| ≲ 𝜑(𝛾1 )

⩽ 𝛾1 +

since Φ(𝛾1 ) ⩾ Φ(2) 1 · (1 − Φ(𝛾1 )) 𝛾1

by Fact 3.3

⩽ 2𝛾1 · (1 − Φ(𝛾1 )) √

⩽ 2 2 · (1 − Φ(𝛾1 )) · log1/2

since 𝛾1 ⩾ 1

1 1 − Φ(𝛾1 )

≲𝛽

by Claim 3.15 since 𝛼 1 = Φ(𝛾1 ) ⩾

1−𝛽 . log(1/𝛽)

Similarly, using Claim 3.14, we get that 1 − 𝜅2 (𝛾1 ) ≲ 𝛽. It remains to argue that |𝜅 1 (𝛾ˆ 1 )| is also at most 𝑂(𝛽). By Claim 3.11, it suffices to show that 𝜅2 ( 𝛾ˆ 1 ) ⩾ 1 − Ω(𝛽), as long as 𝛽 is at most a ˆ ⩾ 𝜅 2 (1), which implies 𝛾ˆ 1 ⩾ 1). Recall that sufficiently small constant (such that 𝜅2 ( 𝛾) 𝑀2 = 𝐼 𝑑 − (1 − 𝜅2 (𝛾1 )) · 𝑤 1 𝑤1⊤ − (1 − 𝜅2 (𝛾1 ) · 𝑤 2 𝑤2⊤ . 63

Thus, we have

𝑤ˆ 1⊤ 𝑀2 𝑤ˆ 1 = 𝜅2 (𝛾1 ) · ⟨𝑤 1 , 𝑤ˆ 1 ⟩2 + 𝜅 2 (𝛾2 ) · ⟨𝑤2 , 𝑤ˆ 1 ⟩2 +

1 − ⟨𝑤 1 , 𝑤ˆ 1 ⟩ − ⟨𝑤 2 , 𝑤ˆ 1 ⟩

Now, we have 𝜅2 (𝛾1 ) ⩾ 1 − Ω(𝛽). If 𝛼2 ⩾ 𝛼 ∗ , then we also have 𝜅2 (𝛾2 ) ⩾ 1 − Ω(𝛽) and thus 𝑤ˆ 1⊤ 𝑀2 𝑤ˆ 1 ⩾ 1 − Ω(𝛽). If 𝛼2 ⩽ 𝛼∗ , then we have ∥𝑤ˆ 2 − 𝑤 2 ∥ ⩽ 𝛽 and since 𝑤ˆ 1 and 𝑤ˆ 2 are orthogonal, we get 𝑤ˆ 1⊤ 𝑤 2 = 𝑤ˆ 1⊤ (𝑤2 − 𝑤ˆ 2 ) ⩽ 𝛽. Thus, we have 𝑤ˆ 1⊤ 𝑀2 𝑤ˆ 1 ⩾ (1 − Ω(𝛽)) ⟨𝑤1 , 𝑤ˆ 1 ⟩2 + (1 − ⟨𝑤 1 , 𝑤ˆ 1 ⟩2 − 𝛽 2 ) ⩾ 1 − Ω(𝛽) in this case as well (assuming 𝛽 ⩽ 1). As before, using 𝑑 𝑛=𝑂 2 𝛽 · min{𝜅 2 (𝛾1 ), 𝜅2 (𝛾2 )}

b 2 ∥ ⩽ 𝛽 and thus also 𝜅2 (𝛾ˆ 1 ) = 𝑤ˆ ⊤ 𝑀 b 2 𝑤ˆ 1 ⩾ 1 − Ω(𝛽). Hence, we have samples, we get ∥𝑀2 − 𝑀 1 |𝜅1 ( 𝛾ˆ 1 )| ⩽ 𝑂(𝛽) by Claim 3.11. Together, this shows that for 𝛽 at most a sufficiently small constant, we get that ∥𝜅1 ( 𝛾ˆ 1 )𝑤ˆ 1 − 𝜅 1 (𝛾1 )𝑤1 ∥ ⩽ 𝑂(𝛽). Sample complexity. Summarized, we showed that for 𝛽 < min{𝑐 ′ · 𝜅 2 (𝛾1 ), 𝑐 ′ · 𝜅 2 (𝛾2 ), 𝑐 ′′ } (for a sufficiently small constant 𝑐 ′′ ) we have 1 1 𝜇ˆ − 𝜇 ⩽ + · 𝑂(𝛽). 2 𝜅2 (𝛾1 ) 𝜅2 (𝛾2 )2

The sample complexity needed for this consists of different terms. We always have 𝑑 𝑂 2 𝛽 · min{𝜅 2 (𝛾1 ), 𝜅2 (𝛾2 )}

and for both ℓ ∈ {1, 2} we need to add 𝑑2 𝑑2 𝑂 2 + 𝛽 𝜅3 (𝛾ℓ )2 𝜅3 (𝛾ℓ )4

if 𝛼ℓ ⩽ 𝛼 ∗ . It remains to bound 𝜅 𝑖 (𝛾ℓ ). By Claim 3.22, we have 𝜅 2 (𝛾ℓ ) ⩾ Ω(min{1, log−1 (1/𝛼ℓ )}) ⩾ Ω(min{1, log−1 (1/𝛼)}). Similarly, if 𝛼ℓ ⩽ 𝛼 ∗ , then we can bound 𝜅3 (𝛾ℓ ) using Claim 3.23 as follows 𝜅3 (𝛾ℓ ) ⩾ Ω(min{log−3/2 (1/𝛼), 𝛽}). Putting these together, the sample complexity becomes 𝑑2 𝑑 𝑛 = 𝑂 2 poly log(1/𝛼) + 4 . 𝛽 𝛽

Note that we can again assume with loss of generality that 𝜀 ⩽ 1 (otherwise the conclusion is trivial). Thus, picking (for a sufficiently small constant 𝑐 ′′′ ) 𝛽 = min {𝑐 ′ · 𝜅 2 (𝛾1 ), 𝑐 ′ · 𝜅 2 (𝛾2 ), 𝑐 ′′ , 𝑐 ′′′ · min{𝜅 2 (𝛾1 ), 𝜅2 (𝛾2 )} · 𝜀} proves the theorem.16 Note that the runtime analysis of Algorithm 3 is exactly the same as the one for Algorithm 2 in the proof of Theorem 5.1. □ 16 Note that we do not need a poly log(1/𝛼) for the 𝑑 2 /𝜀4 term. The reason is the following: We only need this term if

𝛼ℓ is close to 1 (say larger than some constant). But in this case 𝜅2 (𝛾ℓ ) is a constant and thus 𝑂(𝑑2 /𝜀4 ) is sufficient.

Acknowledgments This research was supported by the Swiss National Science Foundation (SNSF), grant no. 10004947. We thank the anonymous reviewers for their helpful comments.

Deferred proofs from the main text

In this appendix we provide proofs of some claims that were deferred from the main text.

A.1

Cumulants of the one-dimensional truncated distribution

In this section we prove several facts from Section 3.2 on the first three cumulants of the onedimensional random variable 𝑍 𝛾 ∼ 𝒩 (0, 1)|(−∞,𝛾] . We first prove Fact 3.2 on the values of the cumulants that we restate below. Fact (Restatement of Fact 3.2). Let 𝛾 ∈ R. We have that 𝜅1 (𝛾) =

1 · Φ(𝛾)

∫ 𝛾

𝑡𝜑(𝑡)𝑑𝑡 = −

−∞

𝜑(𝛾) 𝜑(𝛾) 𝜅2 (𝛾) = 1 − 𝛾 − Φ(𝛾) Φ(𝛾)

𝜑(𝛾) , Φ(𝛾)

𝜑(𝛾) 𝜑(𝛾) 𝜑(𝛾) 𝜅3 (𝛾) = · 1 − 𝛾 2 − 3𝛾 · −2· Φ(𝛾) Φ(𝛾) Φ(𝛾)

Proof. Recall that the first three cumulants can be written in terms of the moments of 𝑍 𝛾 . In particular, recall from (3.1) that 𝜅1 (𝛾) = E 𝑍 𝛾 ,

𝜅2 (𝛾) = E

𝜅3 (𝛾) = E

𝑍𝛾 − E 𝑍𝛾

Based on this, we can now write 𝜅 𝑖 in terms of 𝜑(𝛾), Φ(𝛾). For 𝜅1 , observe that 1 𝜅1 (𝛾) = E[𝑍 𝛾 ] = · Φ(𝛾)

∫ 𝛾

𝑡𝜑(𝑡)d𝑡 = −

−∞

𝜑(𝛾) , Φ(𝛾)

(A.1)

d where for the last equality we used d𝑡 𝜑(𝑡) = −𝑡𝜑(𝑡). For 𝜅 2 , we first compute the second moment of 𝑍 𝛾 as follows:

𝑍 2𝛾

𝛾 1 = · 𝑡 2 𝜑(𝑡)d𝑡 Φ(𝛾) −∞ ∫ 𝛾 1 d = · 𝑡 · (−𝜑(𝑡))d𝑡 Φ(𝛾) −∞ d𝑡

∫

1 𝛾 = [−𝑡𝜑(𝑡)]−∞ + Φ(𝛾)

(i)

∫ 𝛾

−∞

𝜑(𝑡)d𝑡

= −𝛾

𝜑(𝛾) +1 Φ(𝛾)

where in equality (i) we used integration by parts. Combined with (A.1) this implies that 𝜅2 (𝛾) = E

𝑍 2𝛾

− E 𝑍𝛾

𝜑(𝛾) 𝜑(𝛾) =1−𝛾 − Φ(𝛾) Φ(𝛾)

(A.2)

Similarly, we can compute the third moment

𝑍 3𝛾

𝛾 1 𝑡 3 𝜑(𝑡)d𝑡 · = Φ(𝛾) −∞ ∫ 𝛾 1 d = 𝑡 2 · (−𝜑(𝑡))d𝑡 · d𝑡 Φ(𝛾) −∞

∫

1 𝛾 = [−𝑡 2 𝜑(𝑡)]−∞ + 2 Φ(𝛾) 𝜑(𝛾) 𝜑(𝛾) (ii) −2· , = −𝛾 2 · Φ(𝛾) Φ(𝛾)

(i)

∫ 𝛾

𝑡𝜑(𝑡)d𝑡

−∞

(A.3)

where in (i) we again used integration by parts and in (ii) we use (A.1). Thus, we can compute 𝜅 3 as follows 𝜅3 (𝛾) = E

𝑍𝛾 − E 𝑍𝛾

= E 𝑍 3𝛾 − 3 E 𝑍 2𝛾 · E 𝑍 𝛾 + 3 E Z𝛾 · E 𝑍 𝛾

− E 𝑍𝛾

= E 𝑍 3𝛾 − 3 E 𝑍 𝛾 · E 𝑍 2𝛾 − E 𝑍 𝛾

𝜑(𝛾) 𝜑(𝛾) 𝜑(𝛾) 𝜑(𝛾) 𝜑(𝛾) = −𝛾2 · −2· −3· − · 1−𝛾 − Φ(𝛾) Φ(𝛾) Φ(𝛾) Φ(𝛾) Φ(𝛾)

𝜑(𝛾) − − Φ(𝛾)

where in the last step we used (A.1), (A.2) and (A.3). After simplification we have that 𝜑(𝛾) 𝜑(𝛾) 𝜑(𝛾) · 1 − 𝛾 2 − 3𝛾 · −2· 𝜅3 (𝛾) = Φ(𝛾) Φ(𝛾) Φ(𝛾)

which completes the proof.

□

For several of the following proofs, we use the approximation of the inverse Mills ratio 𝜑(𝑡) ℎ(𝑡) = 1−Φ(𝑡) from Fact 3.3. We will in particular need the following bounds from Fact 3.3 that hold for all 𝑥 > 0: ℎ(𝑥) > 𝑟1 (𝑥) = 𝑥

(𝑘 = 0)

𝑥2 + 1 𝑥 𝑥(𝑥 2 + 3) ℎ(𝑥) > 𝑟3 (𝑥) = 𝑥2 + 2 4 𝑥 + 6𝑥 2 + 3 ℎ(𝑥) < 𝑟4 (𝑥) = 𝑥 3 + 5𝑥 ℎ(𝑥) < 𝑟2 (𝑥) =

(𝑘 = 1) (𝑘 = 2) (𝑘 = 3) 66

ℎ(𝑥) > 𝑟5 (𝑥) =

𝑥(𝑥 4 + 10𝑥 2 + 15) 𝑥 4 + 9𝑥 2 + 8

(𝑘 = 4).

Using these, we now prove the following two claims about 𝜅1 and 𝜅 2 . Claim (Restatement of Claim 3.6). The function 𝜅1 as defined in (3.1) is 1-Lipschitz on R. Claim (Restatement of Claim 3.7). For every 𝑡 ∈ R, we have that 0 < 𝜅 2 (𝑡) ⩽ 1. We first prove Claim 3.7 and then use it to prove Claim 3.6. Proof of Claim 3.7. First, note that 𝜅2 (𝛾) = E[(𝑍 𝛾 − E[𝑍 𝛾 ])2 ] > 0. To see 𝜅2 (𝑡) ⩽ 1, recall from (A.2) that 2 𝜑(𝑡) 𝜑(𝑡) 1 − 𝜅 2 (𝑡) = 𝑡 + . Φ(𝑡) Φ(𝑡) For 𝑡 ⩾ 0, both terms are non-negative and thus 1 − 𝜅2 (𝑡) ⩾ 0. For 𝑡 < 0, we get from Fact 3.3 that 𝜑(𝑡) 𝜑(−𝑡) = 1−Φ(−𝑡) ⩾ −𝑡 and thus Φ(𝑡) 𝜑(𝑡) 𝜑(𝑡) 𝜑(𝑡) 𝑡+ ⩾ (𝑡 − 𝑡) = 0 1 − 𝜅 2 (𝑡) = Φ(𝑡) Φ(𝑡) Φ(𝑡)

in this case as well.

□ 𝜑(𝑡)

Proof of Claim 3.6. Observe that, since 𝜅1 (𝑡) = − Φ(𝑡) by Fact 3.2, we get that 𝜑(𝑡) 𝜑(𝑡) d 𝜅1 (𝑡) = 𝑡 + d𝑡 Φ(𝑡) Φ(𝑡)

= 1 − 𝜅 2 (𝑡) ⩽ 1,

where we used again Fact 3.2 and that 𝜅2 (𝑡) ⩾ 0 by Claim 3.7.

□

Next, we prove the following claim that shows that the function 𝜅2 is monotonically increasing. d 𝜅2 (𝑡) = −𝜅3 (𝑡). Moreover, Claim (Restatement of Claim 3.8). For 𝜅2 and 𝜅3 as in (3.1), we have d𝑡 𝜅3 (𝑡) < 0 for every 𝑡 ∈ R and thus 𝜅2 (𝑡) is monotonically increasing. 𝜑(𝑡)

Proof of Claim 3.8. Recall from Fact 3.2 that 𝜅2 (𝑡) = 1 − 𝑡 Φ(𝑡) −

𝜑(𝑡) Φ(𝑡)

. Hence, we get that

𝜑(𝑡) 𝜑(𝑡) d d 𝜑(𝑡) d 𝜑(𝑡) 𝜅 2 (𝑡) = − −𝑡 −2 · d𝑡 d𝑡 Φ(𝑡) Φ(𝑡) Φ(𝑡) d𝑡 Φ(𝑡) 𝜑(𝑡) 𝜑(𝑡) d −𝜑(𝑡) =− + 𝑡+2 · d𝑡 Φ(𝑡) Φ(𝑡) Φ(𝑡)

𝜑(𝑡) 𝜑(𝑡) =− + 𝑡+2 Φ(𝑡) Φ(𝑡)

𝜑(𝑡) 𝜑(𝑡) 𝑡 + Φ(𝑡) Φ(𝑡)

𝜑(𝑡) 𝜑(𝑡) = (𝑡 − 1) + 3𝑡 Φ(𝑡) Φ(𝑡) 2

𝜑(𝑡) +2 Φ(𝑡)

𝜑(𝑡)

= −𝜅3 (𝑡)

d by Fact 3.2. Thus, d𝑡 𝜅 2 (𝑡) > 0 if and only if 𝑓 (𝑡) = 𝑡 2 − 1 + 3𝑡 · Φ(𝑡) + 2

𝜑(𝑡) Φ(𝑡)

latter by distinguishing the three cases (i) 1 ⩽ 𝑡, (ii) 0 ⩽ 𝑡 < 1 and (iii) 𝑡 < 0. 67

> 0. We show the

Case (i): 1 ⩽ 𝑡. In this case, 𝑡 2 − 1 ⩾ 0 and the other two terms are positive. Thus, if 𝑡 ⩾ 1, then 𝑓 (𝑡) > 0 holds. Case (ii): 0 ⩽ 𝑡 < 1. A direct computation shows 𝑓 (0) = 𝜋4 − 1 > 0. For 𝑡 ∈ (0, 1), d since 𝜑(𝑡) = d𝑡 Φ(𝑡) is decreasing, we can upper bound Φ(𝑡) by its tangent line at 𝑡 = 0, i.e., Φ(𝑡) ⩽ Therefore, we obtain

1 𝑡 +√ . 2 2𝜋

exp −𝑡 2 /2 𝜑(𝑡) ⩾ p >0 Φ(𝑡) 𝜋/2 + 𝑡

and thus

exp −𝑡 2 /2

𝑓 (𝑡) ⩾ 𝑡 2 − 1 + p

exp −𝑡 2 /2

· 3𝑡 + 2 p

𝜋/2 + 𝑡

! .

𝜋/2 + 𝑡

We show in Fact B.5 in Appendix B that this lower bound is monotonic in (0, 1). Thus, the lower bound attains its minimum at 𝑡 = 0 and we have for any 𝑡 ∈ (0, 1) that 𝑓 (𝑡) ⩾ 𝜋4 − 1 > 0. Case (iii): 𝑡 < 0. Finally, for 𝑡 < 0 consider the quadratic function 𝑞 𝑡 (𝑥) = 𝑡 2 − 1 + 3𝑡𝑥 + 2𝑥 2 . d We have d𝑥 𝑞 𝑡 (𝑥) = 3𝑡 + 4𝑥 and thus 𝑞 𝑡 (𝑥) is increasing in 𝑥 for 𝑥 > − 43 𝑡. Note that we have

𝑓 (𝑡) = 𝑞 𝑡

𝜑(𝑡) = 𝑞 𝑡 (ℎ(−𝑡)) Φ(𝑡)

for ℎ(−𝑡) the inverse Mills ratio as in as in Section 3.2 (note that −𝑡 > 0). By Fact 3.4, we have ℎ(−𝑡) >

1 √ 2 3 𝑡 + 8 − 3𝑡 > − 𝑡 4 4

and hence, we get that 𝑞 𝑡 (ℎ(−𝑡)) > 𝑞 𝑡

√ 1 4

𝑡 2 + 8 − 3𝑡

2 1 √ 2 1 √ 2 = 𝑡 − 1 + 3𝑡 · 𝑡 + 8 − 3𝑡 + 2 𝑡 + 8 − 3𝑡 4 4 √ √ 9 3 1 2 = 𝑡2 − 1 + 𝑡 𝑡2 + 8 − 𝑡2 + 𝑡 + 8 − 6𝑡 𝑡 2 + 8 + 9𝑡 2 = 0, 4 4 8

which shows that 𝑓 (𝑡) > 0 also holds for 𝑡 < 0. This completes the proof.

□ p

Next, we prove the following claim that gives a uniform upper bound on 𝜅1 (𝑡) 𝜅2 (𝑡).

Claim (Restatement of Claim 3.9). For every 𝑡 ∈ R, we have |𝜅1 (𝑡)| · 𝜅2 (𝑡) ⩽ 𝑂(1). 𝜑(𝑡)

𝜑(𝑡)

Proof. Recall from Fact 3.2 that 𝜅1 (𝑡) = − Φ(𝑡) and 𝜅2 (𝑡) = 1 − 𝑡 Φ(𝑡) −

𝜑(𝑡) Φ(𝑡)

. By Claim 3.7, we also

have that 0 < 𝜅 2 (𝑡) ⩽ 1. To show that |𝜅1 (𝑡)| · 𝜅2 (𝑡) is uniformly bounded, we distinguish the two cases (i) 𝑡 ⩾ −4 and (ii) 𝑡 < −4. 68

Case (i): 𝑡 ⩾ −4. In this case we have Φ(𝑡) ⩾ Φ(−4) > 0. Thus, we have, using that 𝜅 2 (𝑡) ⩽ 1 for all 𝑡, p 𝜑(𝑡) p 𝜑(𝑡) 1 · 𝜅2 (𝑡) ⩽ ⩽√ . |𝜅1 (𝑡)| · 𝜅2 (𝑡) = Φ(𝑡) Φ(−4) 2𝜋Φ(−4) Case (ii): 𝑡 < −4. For this case, we want to use Fact 3.3 to bound 𝜅1 (𝑡) and 𝜅2 (𝑡). First, for 𝜅 1 we have |𝜅1 (𝑡)| = ℎ(−𝑡) ⩽ 𝑟2 (−𝑡) = −𝑡 − 1/𝑡 ⩽ 2|𝑡| for 𝑡 < −4 < −1, where, as before, ℎ(−𝑡) is the inverse Mills ratio as in Section 3.2. For 𝜅 2 , consider the quadratic function 𝑞 𝑡 (𝑥) = −𝑥 2 − 𝑡𝑥 + 1. For 𝑡 < −4 < 0, we can write 𝜅2 (𝑡) = 1 − 𝑡 ℎ(−𝑡) − (ℎ(−𝑡))2 = 𝑞 𝑡 (ℎ(−𝑡)). d We have d𝑥 𝑞 𝑡 (𝑥) = −2𝑥 − 𝑡 and hence 𝑞 𝑡 (𝑥) is increasing in 𝑥 on the interval (− 2𝑡 , ∞). Now, using Fact 3.3, we get −𝑡 15 + 10𝑡 2 + 𝑡 4 𝑡 ℎ(−𝑡) ⩾ 𝑟5 (−𝑡) = ⩾− , 2 4 2 8 + 9𝑡 + 𝑡

where we used −𝑡 > 0 and 𝑡 2 − 1 ⩾ 0 for the last step. Hence, we have 𝑞 𝑡 (ℎ(−𝑡)) ⩽ 𝑞 𝑡 (𝑟5 (−𝑡)) and thus 𝑡 6 + 12𝑡 4 + 39𝑡 2 + 64 4𝑡 6 4 𝜅 2 (𝑡) = 𝑞 𝑡 (ℎ(−𝑡)) ⩽ 𝑞 𝑡 (𝑟5 (−𝑡)) = ⩽ = 2, 2 8 𝑡 𝑡 (𝑡 4 + 9𝑡 2 + 8) where we used 𝑡 < −4 to get 12𝑡 4 + 39𝑡 2 + 64 ⩽ 3𝑡 6 . Finally, combining the bounds on 𝜅1 (𝑡) and 𝜅 2 (𝑡), we get that, for all 𝑡 < −4,

|𝜅1 (𝑡)| · 𝜅2 (𝑡) ⩽ 2|𝑡| ·

2 = 4, |𝑡|

which completes the proof.

□

The following claim shows both 𝜅 3 (𝑡) and 𝜅1 (𝑡) · 𝜅 3 (𝑡) are also uniformly bounded. d Claim (Restatement of Claim 3.10). For every 𝑡 ∈ R, we have | d𝑡 𝜅2 (𝑡)| = |𝜅 3 (𝑡)| ⩽ 𝑂(1) and 0 < 𝜅1 (𝑡) · 𝜅 3 (𝑡) ⩽ 𝑂(1).

Proof. We first show that 𝜅3 (𝑡) is bounded on R. By Claim 3.7 we have 𝜅3 (𝑡) < 0 and thus, using Fact 3.2, we get ! 2 𝜑(𝑡) 𝜑(𝑡) 𝜑(𝑡) |𝜅3 (𝑡)| = · 2 + 3𝑡 + 𝑡2 − 1 . Φ(𝑡) Φ(𝑡) Φ(𝑡) We distinguish the two cases (i) 𝑡 > −1 and (ii) 𝑡 ⩽ −1. Case (i): 𝑡 > −1. Note that 𝜑(𝑡), 𝑡𝜑(𝑡) and 𝑡 2 𝜑(𝑡) are all uniformly bounded over 𝑡 ∈ R. For 1 1 𝑡 > −1 we furthermore have Φ(𝑡) ⩽ Φ(−1) ⩽ 𝑂(1). Putting these together, we get that for 𝑡 > −1 we have |𝜅3 (𝑡)| ⩽ 𝑂(1). Case (ii): 𝑡 ⩽ −1. For this case, we want to again use Fact 3.3. For 𝑡 < −1 < 0, we again use that 𝜑(𝑡) = ℎ(−𝑡). By Fact 3.3, we get that Φ(𝑡) 𝜑(𝑡) −𝑡 · (𝑡 2 + 3) 1 = 𝑟3 (−𝑡) < ℎ(−𝑡) = < 𝑟2 (−𝑡) = −𝑡 − . 2 𝑡 Φ(𝑡) 𝑡 +2 We thus get that |𝜅3 (𝑡)| = ℎ(−𝑡) · 2ℎ(−𝑡)2 + 3𝑡 · ℎ(−𝑡) + 𝑡 2 − 1 69

1 1 · 2 −𝑡 − ⩽ −𝑡 − 𝑡 𝑡

−𝑡 · (𝑡 2 + 3) + 3𝑡 · + 𝑡2 − 1 2 𝑡 +2

1 2(−𝑡 2 − 1)2 (𝑡 2 + 2) − 3𝑡 4 (𝑡 2 + 3) + 𝑡 2 (𝑡 2 − 1)(𝑡 2 + 2) = −𝑡 − · 𝑡 𝑡 2 (𝑡 2 + 2)

1 2𝑡 2 + 1 = −𝑡 − ·4· 2 2 𝑡 𝑡 (𝑡 + 2) 8 ⩽ −2𝑡 · 2 ⩽ 16, 𝑡

where we used −𝑡 − 1/𝑡 ⩽ −2𝑡 and 2𝑡 2 + 1 ⩽ 2(𝑡 2 + 2) for 𝑡 ⩽ −1. Hence, |𝜅3 (𝑡)| is uniformly bounded over 𝑡 ∈ R. For 𝜅1 (𝑡) · 𝜅 3 (𝑡), first note that by Fact 3.2 we 𝜑(𝑡) have 𝜅 1 (𝑡) = − Φ(𝑡) < 0 and thus 𝜅 1 (𝑡) · 𝜅3 (𝑡) > 0. To show a uniform upper bound, we distinguish the same two cases as above and use a similar strategy. Note that we can use the same bounds and 𝜑(𝑡) multiply them by Φ(𝑡) . If 𝑡 > −1, we have 𝜑(0) 𝜑(𝑡) ⩽ ⩽ 𝑂(1), Φ(𝑡) Φ(−1) and thus 𝜅 1 (𝑡) · 𝜅 3 (𝑡) ⩽ 𝑂(1) for these 𝑡. If 𝑡 ⩽ −1, again by Fact 3.3, we get 𝜑(𝑡) 1 = ℎ(−𝑡) ⩽ −𝑡 − ⩽ −2𝑡 ⩽ 2 𝑡 Φ(𝑡) and thus 𝜅 1 (𝑡) · 𝜅 3 (𝑡) ⩽ 32 for these 𝑡. Hence, also 𝜅 1 (𝑡) · 𝜅 3 (𝑡) is uniformly bounded, which completes the proof. □ We now prove how to lower bound 𝜅1 in terms of 1 − 𝜅 2 . Claim (Restatement of Claim 3.11). For 𝑡 ⩾ 1, we have |𝜅1 (𝑡)| ⩽ 1 − 𝜅2 (𝑡). Proof. Using Fact 3.2, we have 𝜑(𝑡) 𝜑(𝑡) 𝜑(𝑡) 𝜑(𝑡) ⩽𝑡· ⩽𝑡· + |𝜅1 (𝑡)| = Φ(𝑡) Φ(𝑡) Φ(𝑡) Φ(𝑡)

= 1 − 𝜅 2 (𝑡),

which completes the proof.

□

Next, we prove the following lower bound on 𝜅2 . Claim (Restatement of Claim 3.12). For 𝑡 ⩽ −3, we have 𝜅2 (𝑡) ⩾ 4𝑡12 . Proof. We use an analogous strategy as in the second part of the proof of Claim 3.9, where we showed an upper bound on 𝜅2 (𝑡) for 𝑡 < −4. Using Fact 3.3 for −𝑡, we get that, for 𝑡 ⩽ −3 < 0, −𝑡 = 𝑟1 (−𝑡) < ℎ(−𝑡) < 𝑟4 (−𝑡) = −

𝑡 4 + 6𝑡 2 + 3 . 𝑡(𝑡 2 + 5)

Now, consider again the quadratic function 𝑞 𝑡 (𝑥) = −𝑥 2 − 𝑡𝑥 + 1, which decreases on the interval (−𝑡/2, ∞). As before, we have 𝜅2 (𝑡) = 𝑞 𝑡 (ℎ(−𝑡)) and thus we get that 𝑡 4 + 6𝑡 2 + 3 𝜅 2 (𝑡) = 𝑞 𝑡 (ℎ(−𝑡)) ⩾ 𝑞 𝑡 (𝑟4 (−𝑡)) = − − 𝑡(𝑡 2 + 5)

𝑡 4 + 6𝑡 2 + 3 −𝑡 − + 1. 𝑡(𝑡 2 + 5)

Simplifying this expression, we can conclude that 𝜅2 (𝑡) ⩾

𝑡 4 + 4𝑡 2 − 9 𝑡4 1 ⩾ = 2, 6 6 4 2 4𝑡 4𝑡 𝑡 + 10𝑡 + 25𝑡

where we used 4𝑡 2 − 9 ⩾ 0 and 10𝑡 4 + 25𝑡 2 ⩽ 3𝑡 6 for 𝑡 ⩽ −3.

□

Similarly, we prove the following lower bound on 𝜅3 . 1 Claim (Restatement of Claim 3.13). For 𝑡 ⩽ −2, we have |𝜅 3 (𝑡)| ⩾ 50·|𝑡| 3.

Proof. We refine the argument from Claim 3.8, where we showed that 𝜅3 (𝑡) > 0. Claim 3.8, together with Fact 3.2, shows that

|𝜅3 (𝑡)| = −𝜅3 (𝑡) = ℎ(−𝑡) 𝑡 − 1 + 3𝑡 ℎ(−𝑡) + 2 (ℎ(−𝑡))

where ℎ(−𝑡) is again the inverse Mills ratio. From Fact 3.3 we have ℎ(−𝑡) > 𝑟5 (−𝑡) = −

𝑡(𝑡 4 + 10𝑡 2 + 15) ⩾ −𝑡. 𝑡 4 + 9𝑡 2 + 8

Consider the quadratic function 𝑞 𝑡 (𝑥) = 2𝑥 2 + 3𝑡𝑥 + 𝑡 2 − 1. We have |𝜅 3 (𝑡)| = ℎ(−𝑡)𝑞 𝑡 (ℎ(−𝑡)). As already argued in the proof of Claim 3.8, this function is increasing in 𝑥 for 𝑥 > − 34 𝑡. Thus, we get that 2𝑡 4 + 10𝑡 2 − 64 1 𝑞 𝑡 (ℎ(−𝑡)) ⩾ 𝑞 𝑡 (𝑟5 (−𝑡)) = 4 ⩾ , (𝑡 + 9𝑡 2 + 8)2 50𝑡 4 where we used 10𝑡 2 − 64 ⩾ − 32 𝑡 4 and 9𝑡 2 + 8 ⩽ 4𝑡 4 for 𝑡 ⩽ −2. Finally, again by Fact 3.3, we have that ℎ(−𝑡) ⩾ −𝑡 and thus |𝜅 3 (𝑡)| = ℎ(−𝑡)𝑞 𝑡 (ℎ(−𝑡)) ⩾ −𝑡 ·

1 1 = 50𝑡 4 50|𝑡|3

since 𝑡 < 0 and thus −𝑡 3 = |𝑡|3 .

□

Next, we prove the following lower bound on 𝜅2 (𝑡). Claim (Restatement of Claim 3.14). For 𝑡 ⩾ 1, we have 1 − 𝜅 2 (𝑡) ⩽ 16 · (1 − Φ(𝑡)) · log 1/(1 − Φ(𝑡)). Proof. Note that for 𝑡 ⩾ 1, we have Φ(𝑡) ⩾ Fact 3.2, 𝜑(𝑡) 𝜑(𝑡) 1 − 𝜅 2 (𝑡) = 𝑡 + Φ(𝑡) Φ(𝑡)

1 2 as well as 2𝜑(𝑡) ⩽

2 𝜋

⩽ 𝑡. It follows that, using

𝜑(𝑡) 𝜑(𝑡) = 𝑡+ ⩽ 2𝜑(𝑡) · 𝑡 + 2𝜑(𝑡) ⩽ 4𝑡 · 𝜑(𝑡). Φ(𝑡) Φ(𝑡)

𝜑(𝑡)

Applying Fact 3.3, we have 1−Φ(𝑡) ⩽ 𝑡 + 1𝑡 ⩽ 2𝑡 and thus 𝜑(𝑡) ⩽ 2𝑡(1 − Φ(𝑡)). We get that 1 − 𝜅 2 (𝑡) ⩽ 8𝑡 2 · (1 − Φ(𝑡)). 𝜑(𝑡)

Applying Fact 3.3 again, we have 1−Φ(𝑡) ⩾ 𝑡 and hence √ 2 2 1 𝑡 ⩾ ⩾ 𝑡 2𝜋 · 𝑒 𝑡 /2 ⩾ 𝑒 𝑡 /2 1 − Φ(𝑡) 𝜑(𝑡) 71

1 log ⩾ 𝑡 2 /2. 1 − Φ(𝑡)

=⇒

Combining this with the above, we get 1 1 − 𝜅 2 (𝑡) ⩽ 16 · (1 − Φ(𝑡)) · log , 1 − Φ(𝑡)

as claimed.

□

We now prove the following claim bounding log (1/(1 − Φ(𝑡))). 2

Claim (Restatement of Claim 3.15). For 𝑡 ⩾ 2, we have 𝑡2 ⩽ log (1/(1 − Φ(𝑡))) ⩽ 2𝑡 2 . Proof. Using Fact 3.3 we have √ 2 1 1 1 1 ⩽ · 𝑡+ · (𝑡 + 1) = 2𝜋𝑒 𝑡 /2 · (𝑡 + 1) , ⩽ 𝑡 1 − Φ(𝑡) 𝜑(𝑡) 𝜑(𝑡)

where we used 1𝑡 ⩽ 1 for 𝑡 ⩾ 2 ⩾ 1. Thus, we get that

√ 𝑡2 1 ⩽ + log (𝑡 + 1) + log 2𝜋 log 2 1 − Φ(𝑡) √ 𝑡2 ⩽ + 𝑡 + log 2𝜋 2 ⩽ 2𝑡 2 ,

where we used 𝑡 > 0 for log(𝑡 + 1) ⩽ 𝑡 and 𝑡 ⩾ 2 for 𝑡 + log already proved in Claim 3.14 for 𝑡 ⩾ 1.

√

2𝜋 ⩽ 23 𝑡 2 . The other direction was

□

Finally, we want to prove the following claim about the asymptotics of skew(𝑡). Claim (Restatement of Claim 3.16). There are absolute constants 𝑚, 𝑀 > 0 such that 𝑚⩽

|skew(𝑡)| (1 − Φ(𝑡)) log3/2 (1/(1 − Φ(𝑡)))

⩽𝑀

for all 𝑡 ⩾ 0. In order to prove this claim, we first prove the following upper bound on 𝜅3 (𝑡). Claim A.1 (Upper bound on 𝜅3 (𝑡)). For 𝑡 ⩾ 1 and 𝛿 = 1 − Φ(𝑡), we have that √ |𝜅3 (𝑡)| ⩽ 40 2 · 𝛿 · log3/2 (1/𝛿). Proof of Claim A.1. For 𝑡 ⩾ 1, we have as before (cf. Fact 3.2 and Claim 3.8) 𝜑(𝑡) 2 𝜑(𝑡) 𝜑(𝑡) 𝑡 − 1 + 3𝑡 · +2 |𝜅3 (𝑡)| = Φ(𝑡) Φ(𝑡) Φ(𝑡)

For 𝑡 ⩾ 1, we have Φ(𝑡) ⩾ Φ(0) ⩾ 21 and thus |𝜅 3 (𝑡)| ⩽ 2𝜑(𝑡) 𝑡 2 − 1 + 6𝑡 · 𝜑(𝑡) + 8𝜑(𝑡)2 72

1 1 ⩽ 2𝜑(𝑡) 𝑡 − 1 + 6𝑡 · √ + 8 · 2𝜋 2𝜋 2 ⩽ 10𝜑(𝑡)𝑡 ,

8 where we used in the last step that √6 𝑡 + 2𝜋 ⩽ 4𝑡 2 for 𝑡 ⩾ 1. Now, by Fact 3.3, we have that 2𝜋

𝜑(𝑡)

ℎ(𝑡) ⩽ 𝑡 + 1/𝑡 ⩽ 2𝑡 for 𝑡 ⩾ 1. Since ℎ(𝑡) = 1−Φ(𝑡) =

𝜑(𝑡) 𝛿 , it follows that

|𝜅 3 (𝑡)| ⩽ 20 · 𝛿 · 𝑡 3 . It remains to bound 𝑡 in terms of 𝛿. Again by Fact 3.3, we have ℎ(𝑡) ⩾ 𝑡 ⩾ 1 for 𝑡 ⩾ 1 and thus 𝑡2 2 1 𝛿 ⩽ 𝜑(𝑡) = √ 𝑒 𝑡 /2 ⩽ 𝑒 − 2 2𝜋

or equivalently 𝑡⩽

2 log(1/𝛿).

Combining this with the above we get that |𝜅3 (𝑡)| ⩽ 20 · 𝛿 · 2 log(1/𝛿)

3/2

√ = 40 2 · 𝛿 · log3/2 (1/𝛿),

which completes the proof.

□

Proof of Claim 3.16. First, from Claims 3.7 and 3.8 we know that 0 < 𝜅2 (𝑡) ⩽ 1 and 𝜅 3 (𝑡) < 0 for all 𝑡 ∈ R. Hence, we have in particular that 𝜅 3 (𝑡) < 0. 𝜅2 (𝑡)3/2

skew(𝑡) =

Define 𝛿 = 1 − Φ(𝑡). By Claim 3.23, we have that, for 𝑡 ⩾ 𝐶 for some constant 𝐶 (such that the minimum in Claim 3.23 is indeed the third term) 1 |𝜅3 (𝑡)| ⩾ √ · 𝛿 · log3/2 (1/𝛿). 4 2 Using 𝜅2 (𝑡) ⩽ 1, we get that

|skew(𝑡)| 𝛿 · log3/2 (1/𝛿)

⩾ √

4 2

|skew(𝑡)| On the interval 𝑡 ∈ [0, 𝐶], the function is continuous and strictly positive and thus there 𝛿·log3/2 (1/𝛿) ′ is 𝑚 > 0 such that

|skew(𝑡)|

𝛿 · log

for 𝑡 ∈ [0, 𝐶]. Defining 𝑚 = min 𝑚 ′ , √1 by Claim A.1 that

4 2

3/2

(1/𝛿)

⩾ 𝑚′

shows the lower bound. For the upper bound, we get

√ |𝜅3 (𝑡)| ⩽ 40 2 · 𝛿 · log3/2 (1/𝛿)

for 𝑡 ⩾ 1. Using Claim 3.8, we get that 0 < 𝜅2 (0) ⩽ 𝜅 2 (𝑡) for all 𝑡 ⩾ 0 and thus √ |skew(𝑡)| 40 2 ⩽ 𝜅2 (0)3/2 𝛿 · log3/2 (1/𝛿) |skew(𝑡)| is continuous for 𝑡 ∈ [0, 1] and thus has a uniform 𝛿·log3/2 (1/𝛿) n √ o 2 upper bound 𝑀 ′ . We get the claimed upper bound by defining 𝑀 = max 𝑀 ′ , 𝜅40(0)3/2 . □ 2

for 𝑡 ⩾ 1. As before, the function

A.2

Matrix and vector calculations

In this section, we prove several linear algebra facts that we deferred from Sections 4 and 5. We first show that a linear transformation of a Gaussian truncated by a halfspace is again a (different) Gaussian truncated by a (different) halfspace. Claim (Restatement of Claim 4.1). Let 𝐴 ∈ R𝑑×𝑑 be an invertible matrix and let 𝑏 ∈ R𝑑 . Consider the random variable 𝑥 ∼ 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Then, the random variable 𝑦 = 𝐴𝑥 + 𝑏 follows the distribution 𝒩 (𝐴𝜇 + 𝑏, 𝐴Σ𝐴⊤ )|⟨𝑤 ′ ,𝑦 ⟩⩽ 𝜏′ , where 𝑤′ =

𝐴−⊤ 𝑤 ∥𝐴−⊤ 𝑤∥

and

𝜏′ =

𝜏 + ⟨𝐴−⊤ 𝑤, 𝑏⟩ . ∥𝐴−⊤ 𝑤∥

Proof. We prove this claim by computing the density of 𝑦 explicitly. Note that the density of 𝑥 is

1 𝑓 (𝑥) ∝ exp − (𝑥 − 𝜇)⊤ Σ−1 (𝑥 − 𝜇) 1 𝑤 ⊤ 𝑥 ⩽ 𝜏 . 2

Hence, the density of 𝑦 is 1 𝑓 (𝐴−1 (𝑦 − 𝑏)) |det(𝐴)| 1 −1 ⊤ −1 −1 ∝ exp − (𝐴 (𝑦 − 𝑏) − 𝜇) Σ (𝐴 (𝑦 − 𝑏) − 𝜇) 1 𝑤 ⊤ 𝐴−1 (𝑦 − 𝑏) ⩽ 𝜏 2

𝑔(𝑦) =

1 = exp − (𝑦 − 𝑏 − 𝐴𝜇)⊤ (𝐴−⊤ Σ−1 𝐴−1 )(𝑦 − 𝑏 − 𝐴𝜇) 1 (𝐴−⊤ 𝑤)⊤ (𝑦 − 𝑏) ⩽ 𝜏 2

1 ⊤ = exp − (𝑦 − (𝐴𝜇 + 𝑏))⊤ (𝐴Σ𝐴⊤ )−1 (𝑦 − (𝐴𝜇 + 𝑏)) 1 (𝐴−⊤ 𝑤) 𝑦 ⩽ 𝜏 + 𝐴−⊤ 𝑤, 𝑏 2

Thus, 𝑔(𝑦) is the density of the Gaussian 𝒩 𝐴𝜇 + 𝑏, 𝐴Σ𝐴⊤ truncated to a halfspace with normal 𝐴−⊤ 𝑤. To conclude the lemma, we normalize the normal vector. Thus, the halfspace is

𝜏 + ⟨𝐴−⊤ 𝑤, 𝑏⟩ 𝐴−⊤ 𝑤 , 𝑦 ⩽ , 𝐻 = 𝑦: ∥𝐴−⊤ 𝑤∥ ∥𝐴−⊤ 𝑤∥

′

which completes the proof.

□ 1

Next, we show that the Frobenius norm of 𝐼 𝑑 − 𝐴− 2 𝐵𝐴− 2 is invariant under the transformation 𝐴 ↦→ 𝐿𝐴𝐿⊤ , 𝐵 ↦→ 𝐿𝐵𝐿⊤ for an invertible matrix 𝐿. Fact (Restatement of Fact 4.9). Let 𝐴, 𝐵 ∈ R𝑑×𝑑 be positive definite matrices. Let 𝐿 ∈ R𝑑×𝑑 be an invertible matrix. Then, we have that 1

𝐼 𝑑 − 𝐴− 2 𝐵𝐴− 2 1

𝐹

= 𝐼 𝑑 − (𝐿𝐴𝐿⊤ )− 2 𝐿𝐵𝐿⊤ (𝐿𝐴𝐿⊤ )− 2

𝐹

Proof. Let 𝑋 = 𝐿𝐴 2 and 𝑅 = (𝑋𝑋 ⊤ )− 2 𝑋. Note that 𝐿𝐴𝐿⊤ = 𝑋𝑋 ⊤ and observe that 𝑅𝑅⊤ = 𝑅⊤ 𝑅 = 𝐼 𝑑 , i.e., 𝑅 is orthonormal. The right-hand side of the identity can now be rewritten as 1

𝐼 𝑑 − (𝐿𝐴𝐿⊤ )− 2 𝐿𝐵𝐿⊤ (𝐿𝐴𝐿⊤ )− 2

𝐹

= 𝐼 𝑑 − (𝑋𝑋 ⊤ )− 2 𝑋𝐴− 2 𝐵𝐴− 2 𝑋 ⊤ (𝑋𝑋 ⊤ )− 2 74

𝐹

− 12

− 21

= 𝐼 𝑑 − 𝑅𝐴− 2 𝐵𝐴− 2 𝑅⊤

= 𝑅 𝐼𝑑 − 𝐴 1

𝐵𝐴

𝐹

𝑅⊤

. 𝐹

Let 𝑀 = 𝐼 𝑑 − 𝐴− 2 𝐵𝐴− 2 . Using that the trace is cyclic, we get that

𝑅𝑀𝑅⊤ 𝐹 = tr 𝑅𝑀𝑅⊤ 𝑅𝑀𝑅⊤

⊤

= tr 𝑀 𝑀 ⊤ = ∥𝑀∥2𝐹 ,

which completes the proof.

□

We now prove the following claim that shows that the best rank-1 approximation of a matrix 𝐴 that is close to some matrix 𝐵 is also a good approximation to 𝐵. Claim (Restatement of Claim 5.3). Let 𝐴, 𝐵 ∈ R𝑑×𝑑 be symmetric matrices with 𝐵 being an arbitrary rank-1 matrix. Consider the spectral decomposition of 𝐴, 𝐴=

𝑑 Õ

𝜆𝑖 𝑣 𝑖 𝑣 ⊤ 𝑖

𝑖=1

𝜆1 𝑣 1 𝑣 1⊤ − 𝐵 𝐹 ⩽ 2⟨𝜆1 𝑣 1 𝑣 1⊤ − 𝐵, 𝐴 − 𝐵⟩. 2

Proof. As 𝜆1 𝑣 1 𝑣 1⊤ is the best rank-1 approximation of 𝐴, we have 𝜆1 𝑣 1 𝑣 1⊤ − 𝐴 𝐹 ⩽ ∥𝐴 − 𝐵∥2𝐹 . We thus get that 2

𝜆1 𝑣 1 𝑣 1⊤ − 𝐵 𝐹 =

𝜆1 𝑣 1 𝑣1⊤ − 𝐴 + (𝐴 − 𝐵) 𝐹

= 𝜆1 𝑣 1 𝑣 1⊤ − 𝐴 𝐹 + 2⟨𝜆1 𝑣 1 𝑣 1⊤ − 𝐴, 𝐴 − 𝐵⟩ + ∥𝐴 − 𝐵∥2𝐹

⩽ 2 ∥𝐴 − 𝐵∥2𝐹 + ⟨𝜆1 𝑣1 𝑣1⊤ − 𝐴, 𝐴 − 𝐵⟩ = 2⟨𝜆1 𝑣1 𝑣 1⊤ − 𝐵, 𝐴 − 𝐵⟩, which completes the proof.

□

𝑏 𝑎 is close to ± ∥𝑏∥ . Next, we prove that if 𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ is small, then the vector ∥𝑎∥

Claim (Restatement of Claim 5.4). Let 𝑎, 𝑏 ∈ R𝑑 . Let 𝛽 > 0. Suppose that ∥𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ ∥𝐹 ⩽ 𝛽. Then, we 𝑎 𝑏 have that, for 𝑢 = ∥𝑎∥ and 𝑣 = ∥𝑏∥ , min{∥𝑢 + 𝑣∥2 , ∥𝑢 − 𝑣∥2 } ⩽ 8

𝛽2 ∥𝑎∥4

Proof. Note that min{∥𝑢 + 𝑣∥2 , ∥𝑢 − 𝑣∥2 } = 2 − 2|⟨𝑢, 𝑣⟩| ⩽ 2 and thus if 𝛽 ⩾ ∥𝑎∥2

∥𝑎∥2 2 , then the conclusion

trivially holds. Thus, from now on, we assume that 𝛽 < 2 . Since both the assumption and the conclusion are invariant under replacing 𝑏 by −𝑏, so we can without loss of generality assume that ⟨𝑎, 𝑏⟩ ⩾ 0. Let 𝜃 ∈ [0, 𝜋2 ] be the angle between 𝑎, 𝑏, or equivalently, ⟨𝑎, 𝑏⟩ = ∥𝑎∥∥𝑏∥ cos 𝜃. Then observe that 2

𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ 𝐹 = 𝑎𝑎 ⊤ 𝐹 + 𝑏𝑏 ⊤ 𝐹 − 2⟨𝑎𝑎 ⊤ , 𝑏𝑏 ⊤ ⟩ 75

= ∥𝑎∥4 + ∥𝑏∥4 − 2⟨𝑎, 𝑏⟩2

(A.4)

= ∥𝑎∥4 + ∥𝑏∥4 − 2∥𝑎∥2 ∥𝑏∥2 cos2 𝜃 = ∥𝑎∥2 − ∥𝑏∥2

+ 2∥𝑎∥2 ∥𝑏∥2 (1 − cos2 𝜃)

= ∥𝑎∥2 − ∥𝑏∥2

+ 2∥𝑎∥2 ∥𝑏∥2 sin2 𝜃

Both√terms are non-negative. Since the left-hand side is at most 𝛽 2 , we get that |∥𝑎∥2 − ∥𝑏∥2 | ⩽ 𝛽 and 2 · ∥𝑎∥∥𝑏∥ sin 𝜃 ⩽ 𝛽. From the first one we obtain ∥𝑎∥2 − ∥𝑏∥2 ⩽ 𝛽 <

∥𝑎∥2 2

=⇒

∥𝑎∥ ∥𝑏∥ ⩾ √ . 2

Using this lower bound on the second term, we get ∥𝑎∥2 sin 𝜃 ⩽

√

2 · ∥𝑎∥∥𝑏∥ sin 𝜃 ⩽ 𝛽

sin 𝜃 ⩽

=⇒

𝛽 . ∥𝑎∥2

Combined with 𝛽 < ∥𝑎∥2 /2, we have sin 𝜃 < 1/2, which implies 𝜃 < 𝜋6 . Finally, note that min{∥𝑢 + 𝑣∥2 , ∥𝑢 − 𝑣∥2 } = ∥𝑢 − 𝑣∥2 since we assumed ⟨𝑢, 𝑣⟩ ⩾ 0. Thus, we get that 2

min{∥𝑢 + 𝑣∥ , ∥𝑢 − 𝑣∥ } = ∥𝑢 − 𝑣∥ = 2 − 2 cos 𝜃 = 4 sin

𝛽2 𝜃 (i) ⩽ 2 sin2 𝜃 ⩽ 2 4 2 ∥𝑎∥

where in inequality (i) we used that for 0 ⩽ 𝜃 < 𝜋6 , sin 𝜃2 ⩽ √1 sin 𝜃.

□

Finally, we show how to relate 𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ to 𝑎 − 𝑏 and 𝑎 + 𝑏. Fact (Restatement of Fact 5.12). For any 𝑎, 𝑏 ∈ R𝑑 we have that ∥𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ ∥𝐹 ⩽ ∥𝑎 + 𝑏∥ · ∥𝑎 − 𝑏∥. Proof. Recall from (A.4) that 2

𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ 𝐹 = ∥𝑎∥4 + ∥𝑏∥4 − 2⟨𝑎, 𝑏⟩2 . The proof now follows by the following calculation: 2

𝑎𝑎 ⊤ − 𝑏𝑏 ⊤ 𝐹 = ∥𝑎∥4 + ∥𝑏∥4 − 4⟨𝑎, 𝑏⟩2 + 2⟨𝑎, 𝑏⟩2

⩽ ∥𝑎∥4 + ∥𝑏∥4 − 4⟨𝑎, 𝑏⟩2 + 2∥𝑎∥2 ∥𝑏∥2 2 2 2 2

= ∥𝑎∥ + ∥𝑏∥ + 2⟨𝑎, 𝑏⟩ · ∥𝑎∥ + ∥𝑏∥ − 2⟨𝑎, 𝑏⟩

= ∥𝑎 + 𝑏∥2 · ∥𝑎 − 𝑏∥2 .

□

A.3

Concentration results and error analysis for sample moments

In this section, we prove two claims that we deferred from Section 5 about the error due to the fact that we only have access to the sample moments. Claim (Restatement of Claim 5.7). Let 𝜈 be the mean of 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Let 𝑣 be a fixed unit vector. Í Let {𝑥 𝑖 }𝑛𝑖=1 be i.i.d samples from 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 and let 𝑥¯ = 𝑛1 𝑛𝑖=1 𝑥 𝑖 . Then, we have the following with probability at least 1 − 4𝑒 −10𝑑 𝑛

1Õ ¯ 3 − ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 ≲ 𝜅 2 (𝛾)−3/2 · ⟨𝑣, 𝑥 𝑖 − 𝑥⟩ 𝑛 𝑖=1

3/2 !

𝑑 𝑑 + 𝑛 𝑛

Proof. For every 𝑖 ∈ [𝑛], we have that ¯ 3 = (⟨𝑣, 𝑥 𝑖 − 𝜈⟩ + ⟨𝑣, 𝜈 − 𝑥⟩) ¯ 3 ⟨𝑣, 𝑥 𝑖 − 𝑥⟩ ¯ + 3 ⟨𝑣, 𝑥 𝑖 − 𝜈⟩ ⟨𝑣, 𝜈 − 𝑥⟩ ¯ 2 + ⟨𝑣, 𝜈 − 𝑥⟩ ¯ 3. = ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 + 3 ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 ⟨𝑣, 𝜈 − 𝑥⟩ Thus, summing over all 𝑖 ∈ [𝑛], we get that 𝑛

𝑛

1Õ 1Õ ¯ 3 − ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 ⩽ 3 |⟨𝑣, 𝑥¯ − 𝜈⟩| · ⟨𝑣, 𝑥 𝑖 − 𝑥⟩ ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 + 4 |⟨𝑣, 𝑥¯ − 𝜈⟩|3 , 𝑛 𝑛 𝑖=1

𝑖=1

Í𝑛

where we used that 𝑖=1 ⟨𝑣, 𝑥 𝑖 − 𝜈⟩ = ⟨𝑣, 𝑥¯ − 𝜈⟩. Thus, it remains to bound, with high probability, Í the two terms |⟨𝑣, 𝑥¯ − 𝜈⟩| and | 𝑛1 𝑛𝑖=1 ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 |. We do so separately. Bounding |⟨𝑣, 𝑥¯ − 𝜈⟩|. By 4.4, for every 𝑖 ∈ [𝑛], ⟨𝑣, 𝑥 𝑖 − 𝜈⟩ is sub-Gaussian with Corollary

parameter 𝜎 = ∥Σ1/2 ∥ = 𝑂 1/ 𝜅 2 (𝛾) , where we used Lemma 4.6 to bound ∥Σ∥. Thus, for any

fixed vector 𝑣, Hoeffding’s inequality (cf. [Ver18, Theorem 2.7.3]) implies that for every 𝑡 > 0 we have P [|⟨𝑣, 𝑥¯ − 𝜈⟩| ⩾ 𝑡] ⩽ 2 exp −Ω 𝑛𝑡 2 · 𝜅2 (𝛾) .

In particular, taking 𝑡 2 = 𝑂

𝑑 𝜅2 (𝛾)𝑛

, we get that with probability at least 1 − 2𝑒 −10𝑑 we have

s |⟨𝑣, 𝑥¯ − 𝜈⟩| ≲

𝑑 . 𝜅2 (𝛾)𝑛

Bounding | 𝑛1 𝑛𝑖=1 ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 |. For every 𝑖 ∈ [𝑛], since ⟨𝑣, 𝑥 𝑖 − 𝜈⟩ is 𝑂(1/𝜅2 (𝛾)1/2 )-sub-Gaussian, ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 is 𝑂(1/𝜅 2 (𝛾))-sub-exponential. Thus, for fixed 𝑣, the Bernstein inequality (cf. [Ver18, Theorem 2.9.1]) implies that for every 𝑡 > 0, we have

" P

𝑛

1Õ ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 − E⟨𝑣, 𝑥 1 − 𝜈⟩2 ⩾ 𝑡 ⩽ 2 exp −Ω min 𝑛𝑡 2 · 𝜅 2 (𝛾)2 , 𝑛𝑡 · 𝜅 2 (𝛾) . 𝑛 𝑖=1

Taking 𝑡 = 𝑂

1 · 𝜅2 (𝛾)

that

𝑑 𝑑 𝑛 + 𝑛

, it then follows that, with probability at least 1 − 2𝑒 −10𝑑 , we have

𝑛

1 1Õ ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 ≲ 𝑛 𝜅2 (𝛾)

𝑖=1

Since E⟨𝑣, 𝑥 1 − 𝜇⟩2 ⩽ 𝑂

1 𝜅2 (𝛾)

𝑑 𝑑 + + E⟨𝑣, 𝑥 1 − 𝜈⟩2 . 𝑛 𝑛

because of sub-Gaussianity, we have with probability at least

1 − 2𝑒 −10𝑑 that 𝑛

1Õ 1 ⟨𝑣, 𝑥 𝑖 − 𝜈⟩2 ≲ 𝑛 𝜅2 (𝛾) 𝑖=1

𝑑 𝑑 1 1 𝑑 + + . ≲ 1+ 𝑛 𝑛 𝑛 𝜅2 (𝛾) 𝜅2 (𝛾)

Taking a union bound over these two events, we get that, with probability 1 − 4𝑒 −10𝑑 , 𝑛

1Õ ¯ 3 − ⟨𝑣, 𝑥 𝑖 − 𝜈⟩3 ≲ 𝜅 2 (𝛾)−3/2 · ⟨𝑣, 𝑥 𝑖 − 𝑥⟩ 𝑛 𝑖=1

which completes the proof.

3/2 !

𝑑 𝑑 + 𝑛 𝑛

□ 77

We now prove Lemma 5.8. Recall that in Section 5 we already proved the statement for fixed 𝑣. So, it remains to bound the supremum over all unit vectors 𝑣. Lemma (Restatement of Lemma 5.8). Let {𝑥 𝑖 }𝑛𝑖=1 be i.i.d. samples from 𝒩 (𝜇, Σ)|⟨𝑤,𝑥⟩⩽ 𝜏 . Let 𝑥¯ = 1 Í𝑛 1 Í𝑛 b ¯ ⊗3 . Then we have with probability at least 1 − 6𝑒 −7𝑑 that the following 𝑖=1 𝑥 𝑖 . Let 𝑀3 = 𝑛 𝑖=1 (𝑥 𝑖 − 𝑥) 𝑛 holds ! r D E 3/2 𝑑 𝑑 b 3 − 𝑀3 ≲ 𝜅2 (𝛾)−3/2 sup 𝑣 ⊗3 , 𝑀 + . 𝑛 𝑛 ∥𝑣∥=1 Proof. In Section 5, we showed that for every fixed 𝑣 ∈ 𝒮 𝑑−1 we have

b 3 − 𝑀3 𝑣 ⊗3 , 𝑀

r ⩽ 𝐾 · 𝜅2 (𝛾)−3/2

𝑑 𝑑3/2 + 𝑛 𝑛

with probability 1 − 6𝑒 −10𝑑 , where 𝐾 > 0 is a universal constant (independent of 𝑣). Let

b 3 − 𝑀3 . 𝑇 := 𝑀 b 3 and 𝑀3 are symmetric order-3 tensors, 𝑇 is symmetric as well. Fix a 𝛿-net 𝒩𝛿 ⊆ 𝒮 𝑑−1 Since both 𝑀 of the unit sphere (for some 𝛿 to be chosen later). We will show that for any 𝑣 ∈ R𝑑 , |⟨𝑣 ⊗3 , 𝑇⟩| is bounded by sup𝑢∈𝒩𝛿 |⟨𝑢 ⊗3 , 𝑇⟩| and finish the proof by a union-bound argument over 𝒩𝛿 . For any 𝑣 ∈ 𝒮 𝑑−1 , choose 𝑣 ′ ∈ 𝒩𝛿 such that ∥𝑣 − 𝑣 ′ ∥ ⩽ 𝛿 and write Δ := 𝑣 − 𝑣 ′ . By definition, we have 𝑣 = Δ + 𝑣 ′ , and expanding (Δ + 𝑣 ′ )⊗3 gives (Δ + 𝑣 ′ )⊗3 = Δ⊗3 + Sym(Δ⊗2 ⊗ 𝑣 ′ ) + Sym(Δ ⊗ (𝑣 ′ )⊗2 ) + (𝑣 ′ )⊗3 , where we use Sym to denote all permutations of a tensor (so, for example, Sym(Δ⊗2 ⊗ 𝑣 ′ ) is equal to Δ ⊗ Δ ⊗ 𝑣 ′ + Δ ⊗ 𝑣 ′ ⊗ Δ + 𝑣 ′ ⊗ Δ ⊗ Δ). Taking inner product with 𝑇, and using the symmetry of 𝑇, we get ⟨𝑣 ⊗3 , 𝑇⟩ = ⟨(Δ + 𝑣 ′ )⊗3 , 𝑇⟩ = ⟨Δ⊗3 , 𝑇⟩ + 3 ⟨Δ⊗2 ⊗ 𝑣 ′ , 𝑇⟩ + 3 ⟨Δ ⊗ (𝑣 ′ )⊗2 , 𝑇⟩ + ⟨(𝑣 ′ )⊗3 , 𝑇⟩. Now, we bound the first three terms in terms of the injective tensor norm. Since 𝑇 is symmetric, sup

⟨𝑎 ⊗ 𝑏 ⊗ 𝑐, 𝑇⟩ = sup ⟨𝑢 ⊗3 , 𝑇⟩.

∥𝑎∥=∥𝑏∥=∥𝑐∥=1

∥𝑢∥=1

Therefore, for any 𝑎, 𝑏, 𝑐 ∈ R𝑑 , we have ⟨𝑎 ⊗ 𝑏 ⊗ 𝑐, 𝑇⟩ ⩽ ∥𝑎∥ ∥𝑏∥ ∥𝑐∥ sup ⟨𝑢 ⊗3 , 𝑇⟩. ∥𝑢∥=1

Applying this with (𝑎, 𝑏, 𝑐) = (Δ, Δ, Δ), (Δ, Δ, 𝑣 ′ ), and (Δ, 𝑣 ′ , 𝑣 ′ ), we obtain ⟨𝑣 ⊗3 , 𝑇⟩ ⩽ ∥Δ∥3 + 3∥Δ∥2 ∥𝑣 ′ ∥ + 3∥Δ∥∥𝑣 ′ ∥2 · sup ⟨𝑢 ⊗3 , 𝑇⟩ + ⟨(𝑣 ′ )⊗3 , 𝑇⟩

∥𝑢∥=1 3

⊗3

⩽ 𝛿 + 3𝛿 + 3𝛿 · sup ⟨𝑢 , 𝑇⟩ + ⟨(𝑣 ′ )⊗3 , 𝑇⟩ ∥𝑢∥=1

⩽ 𝛿 + 3𝛿 + 3𝛿 · sup ⟨𝑢 ⊗3 , 𝑇⟩ + sup ⟨𝑢 ⊗3 , 𝑇⟩, 𝑢∈𝒩𝛿

∥𝑢∥=1

where in the second step we used ∥𝑣 ′ ∥ = 1 (since 𝑣 ′ ∈ 𝒩𝛿 ⊆ 𝒮 𝑑−1 ) and ∥Δ∥ = ∥𝑣 − 𝑣 ′ ∥ ⩽ 𝛿. This holds for every 𝑣 ∈ 𝒮 𝑑−1 . Taking the supremum over 𝑣 ∈ 𝒮 𝑑−1 yields sup ⟨𝑣 ⊗3 , 𝑇⟩ ⩽ 𝛿3 + 3𝛿 2 + 3𝛿 · sup ⟨𝑣 ⊗3 , 𝑇⟩ + sup ⟨𝑢 ⊗3 , 𝑇⟩.

∥𝑣∥=1

𝑢∈𝒩𝛿

∥𝑣∥=1

Rearranging, we get that 1 − 3𝛿 − 3𝛿2 − 𝛿 3 · sup ⟨𝑣 ⊗3 , 𝑇⟩ ⩽ sup ⟨𝑢 ⊗3 , 𝑇⟩.

𝑢∈𝒩𝛿

∥𝑣∥=1

Now, we set 𝛿 = 41 . Then, we have that 1 − 3𝛿 − 3𝛿2 − 𝛿 3 = 1 − and thus sup ⟨𝑣 ⊗3 , 𝑇⟩ ⩽ ∥𝑣∥=1

3 3 1 3 − − = , 4 16 64 64

64 sup ⟨𝑢 ⊗3 , 𝑇⟩. 3 𝑢∈𝒩1/4

Moreover, there exists a 1/4-net 𝒩1/4 ⊆ 𝒮 𝑑−1 with cardinality |𝒩1/4 | ⩽ 9𝑑 [Ver18, Corollary 4.2.11]. Applying our fixed direction bound to each fixed 𝑢 ∈ 𝒩1/4 , and then taking a union bound, we get

b 3 − 𝑀3 ⟩ ⩾ 64𝐾 𝜅2 (𝛾)−3/2 P sup ⟨𝑣 ⊗3 , 𝑀 3 ∥𝑣∥=1

𝑑 𝑑 3/2 + 𝑛 𝑛

b 3 − 𝑀3 ⟩ ⩾ 𝐾𝜅2 (𝛾)−3/2 ⩽ P sup ⟨𝑢 ⊗3 , 𝑀 𝑢∈𝒩1/4

𝑑 𝑑 3/2 + 𝑛 𝑛

⩽ |𝒩1/4 | · 6𝑒 −10𝑑 ⩽ 9𝑑 · 6𝑒 −10𝑑 ⩽ 6𝑒 −7𝑑 . This proves the lemma.

□

Computations

In this appendix, we prove some of the results used for the proof of Lemma 3.17. Furthermore, we will give a proof of Claims 3.19 to 3.21. Finally, we provide a proof for a fact used in Appendix A. All of the proofs here are (partially) computer-assisted in that we provide Mathematica code to verify some properties of certain functions. Recall that for Lemma 3.17 we there used the function 𝑝 𝑡 (𝑥) = 2𝑡 3 − 6𝑡 + (𝑡 4 + 14𝑡 2 − 5)𝑥 + (2𝑡 3 + 20𝑡)𝑥 2 + (𝑡 2 + 8)𝑥 3 . with derivative

𝜕 𝑝 𝑡 (𝑥) = 𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 + 3(𝑡 2 + 8) · 𝑥 2 . 𝜕𝑥

Fact B.1. Define 𝑥ℓ (𝑡) = √𝑒

−𝑡 2 /2

𝜋/2+𝑡

. For any 𝑡 ⩾ 0, we have that

𝜕 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0 𝜕𝑥

and 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0. 79

Proof. This can be verified with the following Mathematica code: lb[t_] = Exp[-t ^2/2]/( Sqrt[Pi /2] + t); Reduce [t^4 + 14 t^2 - 5 + 4 (t^3 + 10 t) lb[t] + 3 (t^2 + 8) lb[t]^2 > 0] 3 Reduce [2 t^3 - 6 t + (t^4 + 14 t^2 - 5) lb[t] + (2 t^3 + 20 t) lb[t]^2 + (t^2 + 8) lb[ t]^3 > 0] 1 2

p𝜋

This shows that the two statements are true for 𝑡 ≠ − particular for 𝑡 ⩾ 0.

2 and 𝑡 > −

p𝜋

2 respectively and thus in

□

Fact B.2. Define

p 𝑥ℓ (𝑡) =

(𝜋 − 2)2 𝑡 2 + 2𝜋 − 2𝑡 𝜋

√ and

𝑥 𝑢 (𝑡) =

𝑡 2 + 2𝜋 − (𝜋 − 1)𝑡 . 𝜋

4 For − 10 < 𝑡 ⩽ 0, we have

𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 𝑢 (𝑡) + 3(𝑡 2 + 8) · 𝑥ℓ (𝑡)2 > 0 and 𝑝 𝑡 (𝑥ℓ (𝑡)) > 0. Proof. This can be verified with the following Mathematica code: xl[t_] = (Sqrt [(Pi - 2)^2 t^2 + 2 Pi] - 2 t)/Pi; xu[t_] = (Sqrt[t^2 + 2 Pi] - (Pi - 1) t)/Pi; 3 Reduce [t^4 + 14 t^2 - 5 + 4 (t^3 + 10 t) xu[t] + 3 (t^2 + 8) xl[t]^2 > 0] 4 N[%] 5 Reduce [2 t^3 - 6 t + (t^4 + 14 t^2 - 5) xl[t] + (2 t^3 + 20 t) xl[t]^2 + (t^2 + 8) xl[ t]^3 > 0] 6 N[%] 1 2

This shows that the statements are true for −1.83192 < 𝑡 < 4.5716 and −0.419026 < 𝑡 < 0.819969 4 respectively and thus in particular for − 10 < 𝑡 ⩽ 0. □ Fact B.3. Define 𝑟 𝑘+1 (𝑥) B 𝑥 +

1 𝑥 + 𝑥+ 2 3 .. .

𝑥+ 𝑥𝑘

4 Let 𝑥ℓ (𝑡) = 𝑟81 (−𝑡) and 𝑥 𝑢 (𝑡) = 𝑟20 (−𝑡). For 𝑡 ⩽ − 10 , we have 𝑥ℓ (𝑡) > 0,

𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 𝑢 (𝑡) + 3(𝑡 2 + 8) · 𝑥ℓ (𝑡)2 > 0

(B.1)

𝑝 𝑡 (𝑥ℓ (𝑡)) > 0.

(B.2)

and Proof. To prove this fact, we additionally define the following function 𝑥ℓ′ (𝑡) = 𝑟27 (−𝑡). We claim that 𝑥ℓ (𝑡) ⩾ 𝑥ℓ′ (𝑡) > 0 (which will also show 𝑥ℓ (𝑡) > 0) and thus, since 3(𝑡 2 + 8) > 0, it is sufficient to show 𝑡 4 + 14𝑡 2 − 5 + 4(𝑡 3 + 10𝑡) · 𝑥 𝑢 (𝑡) + 3(𝑡 2 + 8) · 𝑥ℓ′ (𝑡)2 . (B.3) in order to show (B.1). We can verify 𝑥ℓ (𝑡) ⩾ 𝑥ℓ′ (𝑡) > 0, (B.3) and (B.2) with the following Mathematica code: 80

r[k_Integer , x_] := Module [{ result = x, i}, For[i = k - 1, i >= 1, i--, result = x + i / result ]; result ] 2 Reduce [r[81, -t] >= r[27, -t] > 0] 3 Reduce [t^4 + 14 t^2 - 5 + 4 (t^3 + 10 t) r[20, -t] + 3 (t^2 + 8) r[27, -t]^2 > 0] 4 N[%] 5 Reduce [2 t^3 - 6 t + (t^4 + 14 t^2 - 5) r[81, -t] + (2 t^3 + 20 t) r[81, -t]^2 + (t^2 + 8) r[81, -t]^3 > 0] 6 N[%] 1

This shows that the three statements we need to verify are true for 𝑡 < 0, 𝑡 < −0.271834 and 4 𝑡 < −0.370331 respectively and thus in particular for 𝑡 < − 10 . □ 𝜑(𝑡) Fact B.4. Define 𝜅3 (𝑡) = Φ(𝑡) ·

𝜑(𝑡) 1 − 𝑡 2 − 3𝑡 · Φ(𝑡) − 2 ·

𝜑(𝑡) Φ(𝑡)

1 Then for −2 < 𝑡 < 2 we have |𝜅 3 (𝑡)| ⩾ 20 .

Proof. This can be verified with the following Mathematica code: phi[t_] = PDF[ NormalDistribution [0, 1], t]; pphi[t_] = CDF[ NormalDistribution [0, 1], t]; 3 kappa3 [t_] = 4 phi[t]/ pphi[t]*(1 - t^2 - 3 t*phi[t]/ pphi[t] - 2 (phi[t]/ pphi[t]) ^2); 5 Minimize [{- kappa3 [t], -2 <= t <= 2}, t]; 6 Reduce [ First [%] > 1/20] 1 2

This shows that the statement is true for −2 < 𝑡 < 2.

□

Now, we give a proof of Claims 3.18 to 3.21 that we restate below. Claim (Restatement of Claim 3.18). We have lim𝑡→−∞ skew(𝑡) = −2 and lim𝑡→+∞ skew(𝑡) = 0. Thus, for all 𝑡 ∈ R we have skew(𝑡) ∈ [−2, 0] and in particular |skew(𝑡)| ⩽ 2. Proof. We can compute these limits with the following Mathematica code: phi[t_] = PDF[ NormalDistribution [0, 1], t]; Phi[t_] = CDF[ NormalDistribution [0, 1], t]; 3 kappa2 [t_] = 1 - t phi[t]/ Phi[t] - (phi[t]/ Phi[t]) ^2; 4 kappa3 [t_] = 5 phi[t]/ Phi[t] (1 - t^2 - 3 t phi[t]/ Phi[t] - 2 (phi[t]/ Phi[t]) ^2); 6 skew[t_] = kappa3 [t]/ kappa2 [t ]^(3/2) ; 7 Limit [skew[t], t -> -\[ Infinity ]] 8 Limit [skew[t], t -> \[ Infinity ]] 1 2

This shows that lim𝑡→−∞ skew(𝑡) = −2 and lim𝑡→+∞ skew(𝑡) = 0. Together with the monotonicity of skew (cf. Lemma 3.17), this implies that skew(𝑡) ∈ [−2, 0] and in particular |skew(𝑡)| ⩽ 2 for all 𝑡 ∈ R. □ Claim (Restatement of Claim 3.19). There exists a constant 𝐶 > 0 such that for all 𝑡 ∈ R we have d d𝑡 skew(𝑡) ⩾ 𝐶. d | d𝑡 𝜅2 (𝑡)|

Moreover, |𝜅2 (𝑎) − 𝜅 2 (𝑏)| ⩽ 𝐶1 · |skew(𝑎) − skew(𝑏)| for every 𝑎, 𝑏 ∈ R. d

skew(𝑡)

Proof. We define the function 𝑔(𝑡) B d𝑡d

| d𝑡 𝜅2 (𝑡)|

. Note that by Lemma 3.17, we have 𝑔(𝑡) > 0 point-

wise for every 𝑡 ∈ R. Our goal is to show that also inf𝑡∈R 𝑔(𝑡) > 0. The Mathematica code below can be used to compute the two limits lim𝑡→∞ 𝑔(𝑡) and lim𝑡→−∞ 𝑔(𝑡). 81

phi[t_] = PDF[ NormalDistribution [0, 1], t]; Phi[t_] = CDF[ NormalDistribution [0, 1], t]; 3 kappa2 [t_] = 1 - t phi[t]/ Phi[t] - (phi[t]/ Phi[t]) ^2; 4 kappa3 [t_] = phi[t]/ Phi[t] (1 - t^2 - 3 t phi[t]/ Phi[t] - 2 (phi[t]/ Phi[t]) ^2); 5 skew[t_] = kappa3 [t]/ kappa2 [t ]^(3/2) ; 6 dskew [t_] = D[skew[t], t]; 7 dkappa2 [t_] = D[ kappa2 [t], t]; 8 Limit [ dskew[t]/ Abs[ dkappa2 [t]], t -> -\[ Infinity ]] 9 Limit [ dskew[t]/ Abs[ dkappa2 [t]], t -> \[ Infinity ]] 1 2

This shows that lim𝑡→∞ 𝑔(𝑡) = 6 and lim𝑡→−∞ 𝑔(𝑡) = ∞. Thus, there exist 𝑡− and 𝑡+ such that for all 𝑡 ∉ [𝑡− , 𝑡+ ] we have 𝑔(𝑡) ⩾ 5. Finally, since the function is continuous, the minimum of 𝑔 over the interval [𝑡− , 𝑡+ ] is achieved and thus there exists a constant 𝐶 ′ > 0 such that also 𝑔(𝑡) ⩾ 𝐶 ′ for all 𝑡 ∈ [𝑡− , 𝑡+ ]. Hence, we have that inf𝑡∈R 𝑔(𝑡) ⩾ min(𝐶 ′ , 5). Choosing 𝐶 = min(𝐶 ′ , 5) gives the first part of the claim. For the second part, note that for any 𝑎, 𝑏 ∈ R we have

∫ max(𝑎,𝑏)

|𝜅2 (𝑎) − 𝜅 2 (𝑏)| =

min(𝑎,𝑏)

∫ max(𝑎,𝑏) ⩽

d 𝜅2 (𝑠)d𝑠 d𝑡 d 𝜅2 (𝑠) d𝑠 d𝑡

min(𝑎,𝑏) ∫ max(𝑎,𝑏)

1 d · skew(𝑠)d𝑠 𝐶 min(𝑎,𝑏) d𝑡 1 = · skew(max(𝑎, 𝑏)) − skew(min(𝑎, 𝑏)) 𝐶 1 = · |skew(𝑎) − skew(𝑏)| . 𝐶

⩽

In the third step, we used the first part of the claim. And in the final step, we used that skew is monotonic by Lemma 3.17. This shows the second part of the claim. □ 𝜅 (𝑡)

1 Claim (Restatement of Claim 3.20). Define 𝜓(𝑡) B 𝜅 (𝑡) 1/2 . Then, there exists a constant 𝐶 > 0 such 2 that for all 𝑡 ∈ R we have d d𝑡 skew(𝑡) ⩾ 𝐶 · 𝜅2 (𝑡)2 . d | d𝑡 𝜓(𝑡)|

Moreover, 𝜓(𝑎) − 𝜓(𝑏) ⩽ 𝐶1 · |skew(𝑎) − skew(𝑏)| · max{𝜅 2 (𝑎)−2 , 𝜅2 (𝑏)−2 } for every 𝑎, 𝑏 ∈ R. Proof. We define the function 𝑔(𝑡) =

d d𝑡 skew(𝑡) d | d𝑡 𝜓(𝑡)|·𝜅 2 (𝑡)2

. As in the proof of Claim 3.19, we have by

Lemma 3.17 that 𝑔(𝑡) > 0 point-wise and we want to show that also the limits lim𝑡→∞ 𝑔(𝑡) and lim𝑡→−∞ 𝑔(𝑡) are strictly positive. The following Mathematica code shows that lim𝑡→∞ 𝑔(𝑡) = ∞ and lim𝑡→−∞ 𝑔(𝑡) = 6. phi[t_] = PDF[ NormalDistribution [0, 1], t]; Phi[t_] = CDF[ NormalDistribution [0, 1], t]; 3 kappa1 [t_] = -(phi[t]/ Phi[t]); 4 kappa2 [t_] = 1 - t phi[t]/ Phi[t] - (phi[t]/ Phi[t]) ^2; 5 kappa3 [t_] = 6 phi[t]/ Phi[t] (1 - t^2 - 3 t phi[t]/ Phi[t] - 2 (phi[t]/ Phi[t]) ^2); 7 skew[t_] = kappa3 [t]/ kappa2 [t ]^(3/2) ; 8 dskew [t_] = D[skew[t], t]; 9 psi[t_] = kappa1 [t]/ kappa2 [t ]^(1/2) ; 1 2

dpsi[t_] = D[psi[t], t]; Limit [ dskew[t]/ Abs[dpsi[t]* kappa2 [t]^2] , t -> -\[ Infinity ]] 12 Limit [ dskew[t]/ Abs[dpsi[t]* kappa2 [t]^2] , t -> \[ Infinity ]] 10 11

Thus, as in the proof of Claim 3.19, since 𝑔 is continuous, we get that there exists a constant 𝐶 > 0 such that 𝑔(𝑡) ⩾ 𝐶 for all 𝑡 ∈ R. For the second part, we argue analogous to the proof of Claim 3.19. Since 𝜅 2 is monotonically increasing by Claim 3.8, we have 𝜅 2 (𝑠)−2 ⩽ max{𝜅2 (𝑎)−2 , 𝜅2 (𝑏)−2 } for every 𝑠 between 𝑎 and 𝑏. Thus, we get that |𝜓(𝑎) − 𝜓(𝑏)| ⩽

⩽

∫ max(𝑎,𝑏)

d 𝜓(𝑠) d𝑠 d𝑡

min(𝑎,𝑏) ∫ max(𝑎,𝑏)

1 · 𝐶

min(𝑎,𝑏)

d skew(𝑠) · 𝜅 2 (𝑠)−2 d𝑠 d𝑡

max(𝑎,𝑏) d 1 skew(𝑠) · d𝑠 ⩽ · max{𝜅2 (𝑎)−2 , 𝜅2 (𝑏)−2 } 𝐶 min(𝑎,𝑏) d𝑡 1 ⩽ · max{𝜅2 (𝑎)−2 , 𝜅2 (𝑏)−2 } · |skew(𝑎) − skew(𝑏)| , 𝐶

∫

which completes the proof.

□

Claim (Restatement of Claim 3.21). There exists a constant 𝐶 > 0 such that for all 𝑡 ∈ R we have d d𝑡 𝜅 2 (𝑡) ⩾ 𝐶 · 𝜅2 (𝑡)2 . d | d𝑡 𝜅 1 (𝑡)|

Moreover, |𝜅1 (𝑎) − 𝜅 1 (𝑏)| ⩽ 𝐶1 · |𝜅 2 (𝑎) − 𝜅 2 (𝑏)| · max{𝜅 2 (𝑎)−2 , 𝜅2 (𝑏)−2 } for every 𝑎, 𝑏 ∈ R. Proof. We define the function 𝑔(𝑡) =

d d𝑡 𝜅 2 (𝑡)

d 𝜅1 (𝑡)|·𝜅2 (𝑡)2 | d𝑡

. Analogous as for Claims 3.19 and 3.20, we have

by Claim 3.8 that 𝑔(𝑡) > 0 point-wise and we want to show that also the two limits lim𝑡→∞ 𝑔(𝑡) and lim𝑡→−∞ 𝑔(𝑡) are strictly positive. With the following Mathematica code we get that lim𝑡→±∞ 𝑔(𝑡) = ∞. phi[t_] = PDF[ NormalDistribution [0, 1], t]; Phi[t_] = CDF[ NormalDistribution [0, 1], t]; 3 kappa1 [t_] = -(phi[t]/ Phi[t]); 4 kappa2 [t_] = 1 - t phi[t]/ Phi[t] - (phi[t]/ Phi[t]) ^2; 5 dkappa1 [t_] = D[ kappa1 [t], t]; 6 dkappa2 [t_] = D[ kappa2 [t], t]; 7 Limit [ dkappa2 [t]/ Abs[ dkappa1 [t]* kappa2 [t]^ 2], t -> -\[ Infinity ]] 8 Limit [ dkappa2 [t]/ Abs[ dkappa1 [t]* kappa2 [t]^2] , t -> \[ Infinity ]] 1 2

As for the proof of Claim 3.20, this implies that 𝑔(𝑡) ⩾ 𝐶 for all 𝑡 ∈ R for some constant 𝐶 > 0. Furthermore, also as in Claim 3.20, we get that max(𝑎,𝑏) 1 d |𝜅1 (𝑎) − 𝜅 1 (𝑏)| ⩽ 𝜅2 (𝑠) · 𝜅 2 (𝑠)−2 d𝑠 𝐶 min(𝑎,𝑏) d𝑡 1 ⩽ · max{𝜅2 (𝑎)−2 , 𝜅2 (𝑏)−2 } · |𝜅2 (𝑎) − 𝜅2 (𝑏)| , 𝐶

∫

which completes the proof.

□

Finally, we prove the following fact that we used in Appendix A. 83

Fact B.5. The function exp −𝑡 2 /2

𝑔(𝑡) = 𝑡 2 − 1 + p

exp −𝑡 2 /2

· 3𝑡 + 2 p

𝜋/2 + 𝑡

is monotonically increasing on the interval (0, 1). d Proof. This can be verified by the following Mathematica code, which confirms that d𝑡 𝑔(𝑡) > 0 for all 𝑡 ∈ (0, 1).

g[t_] = t^2 - 1 + Exp[-t ^2/2]/( Sqrt[Pi /2] + t)*(3 t + 2 Exp[-t ^2/2]/( Sqrt[Pi /2] + t)); 3 dg[t_] = D[g[t], t]; 4 Reduce [{ dg[t] > 0, 0 < t < 1}] 1 2

□

Determining 𝑤

In this appendix, we show that for a vector 𝑢 ∝ Σ𝑤, we can compute a vector proportional to 𝑤 by computing 𝑀2−1 𝑢. This is not needed for our algorithm, but we include it for completeness (instead, this would be needed if one wants to estimate the survival set as they do in [LMZ24]). Note that if Σ is isotropic (or more generally, 𝑤 is an eigenvector of Σ), then we can learn the direction of 𝑤 from the third central moment. For arbitrary 𝑤 and Σ, we get the following (where we assume Σ ≻ 0; otherwise we cannot invert 𝑀2 ). Claim C.1. Assume Σ ≻ 0. Then the second central moment 𝑀2 of the truncated distribution is invertible e B 𝑀2−1 𝑢 ∝ 𝑤. and for 𝑢 ∝ Σ𝑤 we have 𝑤 Proof. From Lemma 4.2 we know Σ𝑤

𝑀2 = Σ − (1 − 𝜅2 (𝛾)) so we get

Σ1/2 𝑤

!⊤

Σ𝑤 Σ1/2 𝑤

𝑀2 𝑤 = Σ𝑤 − (1 − 𝜅 2 (𝛾))Σ𝑤 = 𝜅2 (𝛾)Σ𝑤, 2

where we used that 𝑤 ⊤ Σ𝑤 = Σ1/2 𝑤 . We now show that 𝑀2 is invertible. Let {𝑣 𝑖 }𝑑−1 𝑖=1 be an orthonormal basis for the orthogonal complement of Σ1/2 𝑤 with respect to R𝑑 . Recall from Claim 3.7 that we have 𝜅2 (𝑡) ∈ (0, 1]. It follows that 𝐼 𝑑 − (1 − 𝜅2 (𝛾))

𝑑−1 Õ

𝑣 𝑗 𝑣 𝑇𝑗 +

𝑗=1

= 𝜅2 (𝛾) ·

Σ1/2 𝑤

Σ1/2 𝑤 Σ1/2 𝑤 Σ1/2 𝑤

Σ1/2 𝑤 Σ1/2 𝑤

Σ1/2 𝑤

!⊤

Σ1/2 𝑤

!⊤ − (1 − 𝜅2 (𝛾))

Σ1/2 𝑤 Σ1/2 𝑤 Σ1/2 𝑤

!⊤ +

𝑑−1 Õ 𝑗=1

𝑣 𝑗 𝑣 𝑇𝑗

Σ1/2 𝑤 Σ1/2 𝑤

!⊤

⪰ 𝜅2 (𝛾)

Σ1/2 𝑤

!⊤ +

𝑑−1 Õ 𝑗=1

𝑣 𝑗 𝑣 𝑇𝑗 ® = 𝜅2 (𝛾) · 𝐼 𝑑

ª ¬

where the last inequality follows since 𝜅2 (𝛾) ⩽ 1. This implies that, using 𝜅 2 (𝛾) > 0, Σ1/2 𝑤

𝑀2 = Σ1/2 𝐼 𝑑 − (1 − 𝜅2 (𝛾))

Σ1/2 𝑤

Σ1/2 𝑤 Σ1/2 𝑤

!⊤! Σ1/2

⪰ Σ1/2 𝜅 2 (𝛾)𝐼 𝑑 Σ1/2 = 𝜅2 (𝛾)Σ ≻ 0. Thus 𝑀2 is invertible and we obtain 𝑤 = 𝜅2 (𝛾)𝑀2−1 Σ𝑤. Hence, if 𝑢 ∝ Σ𝑤, then we get

e = 𝑀2−1 𝑢 ∝ 𝑀2−1 Σ𝑤 ∝ 𝑤, 𝑤

which completes the proof.

□

References [AAL23]

Jamil Arbas, Hassan Ashtiani, and Christopher Liaw, Polynomial time and private learning of unbounded gaussian mixture models, Proceedings of the 40th International Conference on Machine Learning, 2023, pp. 1018–1040. 44

[ADVW+ 25] Josh Alman, Ran Duan, Virginia Vassilevska Williams, Yinzhan Xu, Zixuan Xu, and Renfei Zhou, More asymmetry yields faster matrix multiplication, Proceedings of the 2025 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), SIAM, Philadelphia, PA, 2025, pp. 2005–2039. MR 4863478 2 [BCV14]

Aditya Bhaskara, Moses Charikar, and Aravindan Vĳayaraghavan, Uniqueness of tensor decompositions with applications to polynomial identifiability, Conference on Learning Theory, PMLR, 2014, pp. 742–778. 5

[Ber60]

Daniel Bernoulli, Essai d’une nouvelle analyse de la mortalite causee par la petite verole, et des avantages de l’inoculation pour la prevenir, Histoire de l’Acad., Roy. Sci. (Paris) avec Mem (1760), 1–45. 1

[Bha97]

Rajendra Bhatia, Matrix analysis, Graduate Texts in Mathematics, vol. 169, SpringerVerlag, New York, 1997. 59

[Bir42]

Z. W. Birnbaum, An inequality for Mill’s ratio, Ann. Math. Statistics 13 (1942), 245–246. MR 6640 14

[BMA26]

Mathias Barreto, Olivier Marchal, and Julyan Arbel, Optimal sub-Gaussian variance proxy for truncated Gaussian and exponential random variables, Statist. Probab. Lett. 228 (2026), Paper No. 110555, 9. MR 4963412 23

[Boy59]

A. V. Boyd, Inequalities for Mills’ ratio, Rep. Statist. Appl. Res. Un. Japan. Sci. Engrs. 6 (1959), 44–46. MR 118856 14 85

[BRV13]

Mikhail Belkin, Luis Rademacher, and James Voss, Blind signal separation in the presence of gaussian noise, Conference on Learning Theory, PMLR, 2013, pp. 270–287. 5

[BS37]

Chester Ittner Bliss and W. L. Stevens, The calculation of the time-mortality curve, Annals of Applied Biology 24 (1937), no. 4, 815–852. 1

[BS16]

Boaz Barak and David Steurer, Proofs, beliefs, and algorithms through the lens of sum-ofsquares, Course notes: http://www.sumofsquares.org/public/index.html 1 (2016), 3. 5

[CGWY25]

Clement Louis Canonne, Themis Gouleakis, Yuhao Wang, and Qiping Yang, Gaussian mean testing under truncation, Proceedings of The 28th International Conference on Artificial Intelligence and Statistics (Yingzhen Li, Stephan Mandt, Shipra Agrawal, and Emtiyaz Khan, eds.), Proceedings of Machine Learning Research, vol. 258, PMLR, 03–05 May 2025, pp. 4879–4887. 5

[CHLZ12]

Bilian Chen, Simai He, Zhening Li, and Shuzhong Zhang, Maximum block improvement and polynomial optimization, SIAM J. Optim. 22 (2012), no. 1, 87–107. MR 2902686 12

[Coh91]

A. Clifford Cohen, Truncated and censored samples, Statistics: Textbooks and Monographs, vol. 119, Marcel Dekker, Inc., New York, 1991, Theory and applications. MR 1153408 1

[CZZ10]

T. Tony Cai, Cun-Hui Zhang, and Harrison H. Zhou, Optimal rates of convergence for covariance matrix estimation, Ann. Statist. 38 (2010), no. 4, 2118–2144. MR 2676885 1, 2

[DGTZ18]

Constantinos Daskalakis, Themis Gouleakis, Christos Tzamos, and Manolis Zampetakis, Efficient statistics, in high dimensions, from truncated samples, 59th Annual IEEE Symposium on Foundations of Computer Science—FOCS 2018, IEEE Computer Soc., Los Alamitos, CA, 2018, pp. 639–649. MR 3899629 1, 3, 10

[DHPT25]

Ilias Diakonikolas, Samuel B. Hopkins, Ankit Pensia, and Stefan Tiegel, SoS certifiability of subgaussian distributions and its algorithmic applications, STOC’25—Proceedings of the 57th Annual ACM Symposium on Theory of Computing, ACM, New York, 2025, pp. 1689–1700. MR 4928554 50, 51

[DK70]

Chandler Davis and W. M. Kahan, The rotation of eigenvectors by a perturbation. III, SIAM J. Numer. Anal. 7 (1970), 1–46. MR 264450 59

[DK23]

Ilias Diakonikolas and Daniel M Kane, Algorithmic high-dimensional robust statistics, Cambridge university press, 2023. 4, 5

[DKK+ 19]

Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Ankur Moitra, and Alistair Stewart, Robust estimators in high-dimensions without the computational intractability, SIAM J. Comput. 48 (2019), no. 2, 742–864. MR 3945261 5, 6

[DKPZ24]

Ilias Diakonikolas, Daniel M Kane, Thanasis Pittas, and Nikos Zarifis, Statistical query lower bounds for learning truncated gaussians, The Thirty Seventh Annual Conference on Learning Theory, PMLR, 2024, pp. 1336–1363. 1

[DLNS24]

Anindya De, Huan Li, Shivam Nadimpalli, and Rocco A Servedio, Detecting lowdegree truncation, Proceedings of the 56th Annual ACM Symposium on Theory of Computing, 2024, pp. 1027–1038. 5 86

[DNS23]

Anindya De, Shivam Nadimpalli, and Rocco A Servedio, Testing convex truncation, Proceedings of the 2023 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), SIAM, 2023, pp. 4050–4082. 5

[Fis31]

Ronald Aylmer Fisher, Properties and applications of Hh functions, Introduction to British A.A.S. Math. Tables 1 (1931), xxvi–xxxv. 1

[Fri13]

Shmuel Friedland, Best rank one approximation of real symmetric tensors can be chosen symmetric, Front. Math. China 8 (2013), no. 1, 19–40. MR 3010470 12

[Gal98]

Francis Galton, An examination into the registered speeds of american trotting horses, with remarks on their value as hereditary data, Proceedings of the Royal Society of London 62 (1898), no. 379-387, 310–315. 1

[GU14]

Armengol Gasull and Frederic Utzet, Approximating Mills ratio, J. Math. Anal. Appl. 420 (2014), no. 2, 1832–1853. MR 3240110 14

[Har70]

Richard A Harshman, Foundations of the parafac procedure: Models and conditions for an “explanatory” multi-modal factor analysis, UCLA working papers in phonetics 16 (1970), no. 1, 84. 11

[HL13]

Christopher J. Hillar and Lek-Heng Lim, Most tensor problems are NP-hard, J. ACM 60 (2013), no. 6, Art. 45, 39. MR 3144915 28

[HRVF04]

Philippe Huber, Elvezio Ronchetti, and Maria-Pia Victoria-Feser, Estimation of generalized linear latent variable models, Journal of the Royal Statistical Society Series B: Statistical Methodology 66 (2004), no. 4, 893–908. 5

[Hub92]

Peter J Huber, Robust estimation of a location parameter, Breakthroughs in statistics: Methodology and distribution, Springer, 1992, pp. 492–518. 5

[KC22]

Arun Kumar Kuchibhotla and Abhishek Chakrabortty, Moving beyond sub-Gaussianity in high-dimensional statistics: applications in covariance estimation and linear regression, Inf. Inference 11 (2022), no. 4, 1389–1456. MR 4526326 20

[KKT25]

Konstantinos Karatapanis, Vasilis Kontonis, and Christos Tzamos, Oracle efficient truncated statistics, The Thirteenth International Conference on Learning Representations, 2025. 2, 3

[KSS18]

Pravesh K. Kothari, Jacob Steinhardt, and David Steurer, Robust moment estimation and improved clustering via sum of squares, STOC’18—Proceedings of the 50th Annual ACM SIGACT Symposium on Theory of Computing, ACM, New York, 2018, pp. 1035–1046. MR 3826315 4, 5, 50, 51, 54

[KTZ19]

Vasilis Kontonis, Christos Tzamos, and Manolis Zampetakis, Efficient truncated statistics with unknown truncation, 2019 IEEE 60th Annual Symposium on Foundations of Computer Science, IEEE Comput. Soc. Press, Los Alamitos, CA, 2019, pp. 1578–1595. MR 4228243 1, 2, 3

[Lee14]

Alice Lee, Table of the gaussian “tail" functions; when the “tail" is larger than the body, Biometrika 10 (1914), no. 2/3, 208–214. 1

[LL22]

Allen Liu and Jerry Li, Clustering mixtures with almost optimal separation in polynomial time, Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, 2022, pp. 1248–1261. 5

[LMZ24]

Jane H. Lee, Anay Mehrotra, and Manolis Zampetakis, Efficient statistics with unknown truncation, polynomial time algorithms, beyond Gaussians, 2024 IEEE 65th Annual Symposium on Foundations of Computer Science—FOCS 2024, IEEE Computer Soc., Los Alamitos, CA, 2024, pp. 988–1006. MR 4849262 1, 3, 4, 5, 10, 12, 13, 84

[LMZ26]

, Smoothed analysis of learning from positive samples, Proceedings of the 58th Annual ACM Symposium on Theory of Computing (New York, NY, USA), STOC ’26, Association for Computing Machinery, 2026, p. 42–53. 5

[LRA93]

S. E. Leurgans, R. T. Ross, and R. B. Abel, A decomposition for three-way arrays, SIAM J. Matrix Anal. Appl. 14 (1993), no. 4, 1064–1083. MR 1238921 11

[LRV16]

Kevin A Lai, Anup B Rao, and Santosh Vempala, Agnostic estimation of mean and covariance, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS), IEEE, 2016, pp. 665–674. 5

[MV10]

Ankur Moitra and Gregory Valiant, Settling the polynomial learnability of mixtures of gaussians, 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, IEEE, 2010, pp. 93–102. 5

[Pea94]

Karl Pearson, Contributions to the mathematical theory of evolution, Philosophical Transactions of the Royal Society of London. A 185 (1894), 71–110. 5

[Pea02]

Karl Pearson, On the systematic fitting of curves to observations and measurements, Biometrika 1 (1902), no. 3, 265–303. 1

[PL08]

Karl Pearson and Alice Lee, On the generalised probable error in multiple normal correlation, Biometrika 6 (1908), no. 1, 59–68. 1

[Rig15]

Philippe Rigollet, 18.s997: High dimensional statistics, (Lecture Notes), Cambridge, MA, USA: MIT Open-CourseWare (2015). 44

[Sam53]

M. R. Sampford, Some inequalities on Mill’s ratio and related functions, Ann. Math. Statistics 24 (1953), 130–132. MR 54890 14

[Ver18]

Roman Vershynin, High-dimensional probability: An introduction with applications in data science, vol. 47, Cambridge university press, 2018. 19, 44, 77, 79

[vH16]

Ramon van Handel, Probability in high dimension, Lecture notes for APC 550, Princeton University, December 2016, Version: December 21, 2016. 19

[Wey12]

Hermann Weyl, Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung), Math. Ann. 71 (1912), no. 4, 441–479. MR 1511670 59

[ZZ25]

Manolis Zampetakis and Felix Zhou, Private statistical estimation via truncation, Advances in Neural Information Processing Systems (D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, eds.), vol. 38, Curran Associates, Inc., 2025, pp. 90015–90049. 5

Record · ID 310805 · SHA-256 9318b7fa8efe024a
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.