arXiv CS open access

Gradient Descent on Two ReLU Neurons: Global Landscape and Bifurcation Dynamics

Binghua Li et al. · arxiv_cs

arXiv CS · Papers

deep-learning machine-learning neural-networks

Understanding how gradient descent learns features remains a central challenge in neural network theory. We study this question in a minimal multi-index setting that already exhibits rich multi-phase gradient dynamics: a well-specified two-neuron ReLU teacher-student model under isotropic Gaussian i

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

arXiv:2610.42118v1 [cs.LG] 30 Oct 2026

Gradient Descent on Two ReLU Neurons: Global Landscape and Bifurcation Dynamics Binghua Li* University of California, Berkeley

Mengzhe Li* École Normale Supérieure Paris-Saclay

binghuali@berkeley.edu

mengzhe.li_mengzhe@ens-paris-saclay.fr

Denny Wu New York University & Flatiron Institute

Tianhao Wang University of California, San Diego

dwu@flatironinstitute.org

tianhaowang@ucsd.edu

Abstract Understanding how gradient descent learns features remains a central challenge in neural network theory. We study this question in a minimal multi-index setting that already exhibits rich multi-phase gradient dynamics: a well-specified two-neuron ReLU teacher-student model under isotropic Gaussian inputs, trained by population gradient flow from small initialization. We show that the dynamics are organized by two directions: the “easy” bisector direction, which carries the leading signal, and the “hard” splitting direction, which governs specialization. To characterize the loss structure and learning behavior, we analyze the population squared-loss landscape and show that every critical point with both student weights nonzero is either a global minimum or a saddle. We then track the gradient-flow trajectory, showing that student neurons first collapse toward a bisector saddle before escaping and specializing to teacher neurons. Our results provide both a landscape and dynamics account of the multi-phase symmetry-breaking behavior that arises in a simple multi-index model and under standard algorithmic and architectural choices.

Introduction

A central promise of gradient-based training of neural networks is feature learning: rather than fitting the target function on a fixed feature space, neural networks can extract features by aligning their parameters with the latent structure of the data or target function. A recent line of theoretical works has made this intuition precise by analyzing teacher-student models under Gaussian inputs, where the target function is a multi-index model (Abbe et al., 2022; Damian et al., 2022; Ba et al., 2022) which depends on a small number of input directions (e.g., a shallow neural network with Od (1) neurons). In these settings, it has been demonstrated that gradient descent training of a shallow “student” neural network discovers the relevant subspace, and achieves a sample complexity superior to fixed features methods (or neural networks in the “lazy” regime (Chizat et al., 2019)). However, most existing theory becomes tractable only after imposing simplifying assumptions that deviate from standard algorithmic and architectural choices. Examples include (i) near-orthogonal teacher neurons (Li et al., 2020; Ren and Lee, 2025), (ii) local convergence conditions (Zhou et al., 2021; Zhu et al., 2025), (iii) activation functions with high information exponent (Oko et al., 2024; Şimşek et al., 2025; Ren et al., 2025), and (iv) modified algorithms such as loss preprocessing ∗

Equal contribution.

(Damian et al., 2022; Zhang et al., 2026) and spherical/Stiefel gradients (Ben Arous et al., 2021, 2026; Abbe et al., 2023). While these assumptions enable clean analyses, they do not capture the complex multi-phase learning dynamics present in standard ReLU networks (Boursier et al., 2022; Wang and Ma, 2023; Min et al., 2025). A notable multi-index example exhibiting such nontrivial dynamics is the “well-specified” setting in which both the student and teacher are two-layer neural networks with matching activation functions and more than one neuron: for commonly used nonlinearities such as ReLU, the optimization trajectory exhibits symmetry-breaking and bifurcation behavior, with neurons initially moving toward an “average” direction before specializing to different teacher directions. While this plateau-specialization phenomenon has appeared in the statistical physics literature (Saad and Solla, 1995; Goldt et al., 2020), it is not captured by recent analyses that rely on the above simplifications. To our knowledge, a rigorous and quantitative understanding of the loss landscape and multi-phase training dynamics remains incomplete and largely open.

1.1

Our contributions

We study arguably the simplest multi-index model setting that exhibits this bifurcation phenomenon: learning a two-neuron ReLU network under isotropic Gaussian inputs via population gradient flow, starting from small initialization. Specifically, the teacher model is f ∗ (x) = ReLU(⟨v1 , x⟩) + ReLU(⟨v2 , x⟩),

x ∼ N (0, Id ),

(1.1)

where ReLU(z) = max{z, 0}, ∥v1 ∥2 = ∥v2 ∥2 = 1 and ∠(v1 , v2 ) = α ∈ (0, π2 ], and the student has the same architecture with trainable weights w1 , w2 . v1 − v2 (leap-2) As we will see, the geometry of this model is orv1 ganized by two directions in the teacher plane: the bisector (v1 + v2 ), which carries the dominant signal, and the difference (v1 − v2 ), which governs specialization. In the language of leap complexity for Gaussian multi-index models (Abbe et al., 2023; Bietti et al., 2025), the target function (1.1) has leap exponent 2, 0 v1 + v 2 where the bisector and difference vectors specify the ws (leap-1) “easy” leap-1 direction and the “hard” leap-2 direction, respectively. This suggests a two-stage picture for gradient-based learning: the student neurons are first attracted toward the leap-1 bisector, and only Phase 1 Phase 2 (collapse) (split) later separate toward the individual teacher directions v2 along the leap-2 direction. Our goal is to make this student w1 student w2 teacher vi saddle ws collapse-then-splitting picture quantitative by studying both the population squared loss landscape and Figure 1: Illustration of bifurcation dynamics. the gradient-flow dynamics from small initialization. Specifically, • Landscape analysis. Section 3 studies the loss geometry and the critical-point structure for both the correlation loss and the MSE loss. We reduce the stationarity conditions to a three-dimensional problem and isolate the remaining in-plane and out-of-plane critical families. We establish that every critical point of the population loss with both student weights nonzero is either a global minimum or a saddle, so none of these critical points is a spurious local minimum. This resolves a two-neuron conjecture from Tian (2017) in the acute angle (α ≤ π/2) setting. • Small-initialization dynamics. Section 4 studies the gradient-flow trajectory from small initialization and characterizes the symmetry breaking and specialization behavior — see Figure 1. 2

We show that for sufficiently small initialization, both student neurons stay close to the bisector for fixed finite time and approach the symmetric saddle. We then analyze escape from that saddle, including global convergence to a minimum for typical initializations, as well as a quantitative escape-time bound that is logarithmic in the ambient dimension d with a structured initialization.

1.2

Further related work

Learning multi-index models. Recent works have shown shallow neural networks can efficiently learn low-dimensional latent structure via gradient descent (Abbe et al., 2022; Damian et al., 2022; Ba et al., 2022; Bietti et al., 2022; Glasgow, 2024; Dandi et al., 2024). Specifically, the sample complexity has been shown to be superior to that of kernel methods and to be governed by the information exponent (Dudeja and Hsu, 2018; Ben Arous et al., 2021) for single-index models and the leap exponent (Abbe et al., 2023; Bietti et al., 2025) for multi-index models. Our target function (1.1) is an example of a leap-2 multi-index model, but, crucially, it also contains a leap-1 subspace, which complicates the SGD dynamics. In fact, most prior works studying gradient-based feature learning either directly assume link functions with information exponent 2 (Li et al., 2020; Martin et al., 2024; Ren and Lee, 2025; Ben Arous et al., 2025) or higher (Oko et al., 2024; Şimşek et al., 2025; Ren et al., 2025), or remove the low-leap components through algorithmic modifications such as deflation / projection (Ge et al., 2021; Abbe et al., 2023) and loss preprocessing (Damian et al., 2022; Zhang et al., 2026). Consequently, these analyses do not fully characterize the learning dynamics of two ReLU neurons with standard gradient-based training. Training dynamics of ReLU networks. Restricting to shallow neural networks with ReLU activations, prior works have studied landscape properties such as the existence of spurious minima (Safran and Shamir, 2018; Safran et al., 2021); the concurrent work of Huang et al. (2026) gives a summary-statistics characterization of the population minima and proposes an ansatz to simplify the fixed-point equations. For gradient-based training dynamics, while the single-neuron case is relatively well-understood (Tian, 2017; Soltanolkotabi, 2017; Yehudai and Shamir, 2020; Xu and Du, 2023), when the target network contains more than one neuron, existing dynamical analyses either assume local convergence, thereby bypassing the bifurcation behavior (Zhou et al., 2021; Zhu et al., 2025), or require exponential width to invoke mean-field global convergence (Chizat, 2022; Akiyama and Suzuki, 2021). The two-phase trajectory illustrated in Figure 1 also relates to the “plateau” phenomenon observed in the statistical mechanics literature (Biehl et al., 1996; Fukumizu and Amari, 2000). Specifically, the specialization phenomenon has been investigated in Goldt et al. (2020); Oostwal et al. (2021) using the ODE analysis originated from Saad and Solla (1995); Riegler and Biehl (1995). However, to our knowledge, a complete understanding of the landscape and rigorous characterization of the learning dynamics have not yet been established even in the two-neuron setting (1.1) studied here.

Preliminary

Notation. For a nonzero vector w ∈ Rd , we denote its direction by w̄ := w/∥w∥2 , and write orthogonal projection onto the subspace perpendicular to w as Pw⊥ := I − w̄w̄⊤ ∈ Rd×d . The angle between two nonzero vectors u, v ∈ Rd is ∠(u, v) := arccos(⟨u, v⟩/(∥u∥2 ∥v∥2 )). For vectors a1 , . . . , ak , span{a1 , . . . , ak } denotes the linear subspace generated by them. We use ϕ(z) := ReLU(z) = max{0, z} for the activation function.

2.1

Problem setup

We use the teacher f ∗ defined in (1.1). The student has trainable weights w1 , w2 ∈ Rd and output (2.1)

f (x) = ϕ(⟨w1 , x⟩) + ϕ(⟨w2 , x⟩).

The student is trained by gradient flow on the population squared loss under isotropic Gaussian inputs: L(w1 , w2 ) :=

1 Ex∼N (0,Id ) (f (x) − f ∗ (x))2 , 2

dwi = −∇wi L(w1 , w2 ), dt

i ∈ {1, 2},

(2.2)

starting from an arbitrary nonzero initialization w(0) = (w1 (0), w2 (0)) with w1 (0) ̸= 0 and w2 (0) ̸= 0. We define its initialization scale by rinit :=

1 ∥w1 (0)∥2 + ∥w2 (0)∥2 . 2

(2.3)

√ For independent Gaussian initialization wi (0) ∼ N (0, σ 2 Id ), i = 1, 2, both norms are of order σ d with high probability. Our main focus is the small-initialization regime, which for the Gaussian initialization above corresponds to taking σ sufficiently small. We refer to span{v1 , v2 } as the teacher plane. A student weight is in-plane if it lies in the teacher plane and out-of-plane if it has a nonzero component orthogonal to that plane. Within the teacher plane, we highlight two important directions: the bisector and the splitting direction, v̄ :=

v1 + v 2 , ∥v1 + v2 ∥2

v∆ :=

v1 − v 2 . ∥v1 − v2 ∥2

(2.4)

The pair (v̄, v∆ ) forms an orthonormal basis of the teacher plane. We define the initial planar splitting amplitude by ainit := ⟨w1 (0) − w2 (0), v∆ ⟩ . (2.5) For C ≥ 1, we say that the initialization is C-balanced if max ∥wi (0)∥2 ≤ C min ∥wi (0)∥2 .

(2.6)

i, j ∈ {1, 2},

(2.7)

i=1,2

We use the angle shorthand θij := ∠(wi , vj ),

γ := ∠(w1 , w2 ),

where θij is the angle between student neuron i and teacher neuron j, and γ is the angle between the two student directions. As we will see, in the small-initialization regime, the dynamics follow the two-stage collapsethen-splitting picture from the introduction. As illustrated in Figure 1, both student directions first collapse toward the bisector direction v̄ and then split along the direction v∆ , eventually specializing toward the individual teacher directions v1 and v2 . The next subsection explains the teacher-side hierarchy behind this behavior.

2.2

Leap complexity of the teacher

We use the standard Hermite-support notion of leap complexity for Gaussian multi-index models (Abbe et al., 2023; Bietti et al., 2025); here only the following two-coordinate version is needed. Let 4

{hk }k≥0 denote the normalized probabilists’ Hermite polynomials. Suppose U, W are independent standard Gaussians and g ∈ L2 (N (0, I2 )). Its Hermite expansion in L2 (N (0, I2 )) is X g(U, W ) = gbm,n hm (U )hn (W ). (2.8) m,n≥0

Along the ordering U then W , a leap is the first Hermite degree at which a new coordinate appears. Thus the leap to U is the smallest m > 0 with gbm,0 ̸= 0; once U is included, the leap to W is the smallest n > 0 for which some gbm,n is nonzero. For the two-ReLU teacher network defined in (1.1), let z1 := ⟨v̄, x⟩ and z2 := ⟨v∆ , x⟩ be the projections of the input onto the bisector and splitting directions defined in (2.4). Because the teacher lies in span{v̄, v∆ }, f ∗ (x) depends on x only through these two coordinates, and hence we can write f ∗ (x) = g(z1 , z2 ). Also, since (z1 , z2 ) ∼ N (0, I2 ), we may Hermite expand g with respect to coordinates (z1 , z2 ). The following proposition shows that although the nonlinearity ϕ has information/leap exponent 1, the sum of two ReLU neurons is a leap-2 function. Proposition 2.1 (Leap complexity of teacher). The two-ReLU teacher network f ∗ in (1.1) has leap exponent 2, where the bisector coordinate z1 has leap 1 and the splitting coordinate z2 has leap 2. The proof is in Appendix A.4. This leap structure gives useful intuition for the two-stage dynamics shown in Figure 1. Specifically, the leap at each subspace determines the gradient magnitude and hence the order of learning: the leap-1 bisector v̄ gives a linear signal visible at random initialization, while the leap-2 splitting direction v∆ appears in a smaller quadratic signal. Consequently, near small initialization the student–student interaction is negligible, and each student neuron is driven by its correlation with the teacher. At a random initialization in high dimension, each student direction is nearly orthogonal to the teacher plane, so the degree-one part of this correlation, which points toward v̄, dominates. We show in Theorem 3.1 that, over unit-norm student weights, the full correlation term is minimized at the bisector, which results in a collapse toward v̄ before the higher-order signal along v∆ drives splitting and specialization. The ensuing sections will make this intuition precise through landscape and gradient-flow analysis.

Landscape analysis

We begin with a landscape analysis, with the goal of classifying all stationary points of the population squared loss with both student weights nonzero as global minima or saddles. We first consider the correlation loss landscape, which admits a simple characterization but cannot drive neuron specialization, and then analyze the more challenging squared loss setting.

3.1

Warm-up: landscape of the correlation loss

We first work with a more general setting where the teacher is a width-k ReLU network, f ∗ (x) =

k X

ϕ(⟨vi , x⟩),

(3.1)

k ≥ 2.

i=1

P d The student has m ReLU neurons, f (x) = m j=1 ϕ(⟨wj , x⟩), with trainable weights w1 , . . . , wm ∈ R . The setup in Section 2 corresponds to the special case k = m = 2. By the positive homogeneity of ReLU, the population squared loss expands as X m m X 1 1 ∗ 2 ∗ 2 ∗ E[(f (x) − f (x)) ] = E[f (x) ] − ∥wj ∥2 · E[f (x)ϕ(⟨w̄j , x⟩)] + O ∥wi ∥2 ∥wj ∥2 . 2 2 j=1

i,j=1

The first term is constant in the student weights, while the last term is second order in their norms. Thus, for small student weights, the first-order decrease of the loss is governed by the correlations between the teacher output f ∗ (x) and the normalized student activations ϕ(⟨w̄j , x⟩). We refer to this leading L2 correlation term as the correlation loss, Lcorr (w1 , . . . , wm ) := −

m X

Ex∼N (0,Id ) f ∗ (x)ϕ(⟨wj , x⟩) .

(3.2)

j=1

For the width-two result below, let α⋆ ∈ (π/2, π) be the unique solution of tan

α⋆ α⋆ =π− . 2 2

(3.3)

Numerically, α⋆ ≈ 127.521476◦ . While many existing works on gradient-based feature learning employ a correlation-loss approximation to simplify the SGD dynamics (Abbe et al., 2023; Damian et al., 2023; Lee et al., 2024), the following theorem shows that the collapsed configuration at the normalized average teacher direction is the unique minimizer on the unit sphere, and hence cannot recover the two teacher neurons. Theorem 3.1. Consider the teacher model f ∗ in (3.1). Suppose that v1 , . . . , vk are unit vectors satisfying ∠(vi , vj ) = α for all i = ̸ j, and that either 0 < α ≤ π/2, or k = 2 and 0 < α ≤ α⋆ . Pk Pk Let v̄ = ( i=1 vi )/∥ i=1 vi ∥2 . Then, for every student width m, (v̄, . . . , v̄) is the unique global minimum of Lcorr among all unit-norm student weights. The proof is in Appendix B. Remark 3.2. For k = 2, the threshold α⋆ is sharp: above it, the bisector is no longer a local minimizer of Lcorr in (3.2) (Lemma B.5). For fixed k > 2 and d ≥ k, (v̄, . . . , v̄) remains the unique global minimizer when cos α is negative and sufficiently close to zero. This follows from a perturbation argument at α = π/2. There, for a single student, v̄ is a strict minimizer of Lcorr on the unit sphere, with positive definite Hessian on the sphere, and Lcorr is bounded away from its minimum outside any neighborhood of v̄. Both properties persist under small changes of α. The size of this range of α may depend on k and d, and we do not determine how far it extends. The following two observations connect the theorem to the training dynamics. • This result aligns with the first stage of the dynamics observed in Figure 1: near small initialization, the squared loss is dominated by the correlation term, which is minimized when all student directions equal the averaged teacher direction v̄. In the two-neuron setting of (1.1), this averaged direction is exactly the bisector, explaining why the student neurons initially move toward this direction. This result is close in spirit to Şimşek et al. (2025), which studies correlation-loss gradient in the high-information-exponent setting, whereas our ReLU setting has information exponent 1. • On the other hand, since the correlation loss Lcorr ignores the student-student interaction terms when ∥wj ∥2 becomes non-negligible, it cannot explain why collapsed students eventually repel and specialize. To capture this effect, we have to characterize the full squared-loss landscape.

3.2

Landscape of the squared loss

Now we turn to the squared loss (2.2). To simplify the analysis, we write the first-order condition in angular variables. Recall the angle shorthand from (2.7): θij = ∠(wi , vj ) and γ = ∠(w1 , w2 ). 6

Lemma 3.3 (Characterization of stationary points). Define a1 := sin θ11 + sin θ12 − π∥w1 ∥2 − sin γ∥w2 ∥2 and a2 := sin θ21 + sin θ22 − π∥w2 ∥2 − sin γ∥w1 ∥2 . Then for w1 , w2 ̸= 0, (w1 , w2 ) is a stationary point of the squared loss L in (2.2) if and only if (π − θ11 )v1 + (π − θ12 )v2 + a1 w̄1 − (π − γ)∥w2 ∥2 w̄2 = 0, (π − θ21 )v1 + (π − θ22 )v2 + a2 w̄2 − (π − γ)∥w1 ∥2 w̄1 = 0.

(3.4)

The proof is in Appendix C.1.1. With the above first-order condition in hand, we classify all stationary points with w1 ̸= 0 and w2 ̸= 0, up to permutation of student neurons. Additional notation. Before stating the main landscape theorem, we introduce some quantities that will be used to describe the stationary points. Recall that α = ∠(v1 , v2 ) ∈ (0, π/2] is the angle between the teacher neurons. Define 2 α α α 2 α α α r+ := π− cos + sin , r− := sin − cos . (3.5) π 2 2 2 π 2 2 2 For 0 < κ < π/2, define α , θα (κ) = arccos cos κ · cos 2 2 sin θα (κ) rα (κ) = , sin(2κ) + 2κ α Fα (κ) = (π − θα (κ)) cos − (π − 2κ)rα (κ) cos κ. 2

(3.6) (3.7) (3.8)

Lemma 3.4 (Unique out-of-plane root). For every α ∈ (0, π), the equation Fα (κ) = 0 has exactly one solution in (0, π/2), which we denote by κα . The proof is in Appendix C.1.2. Now we are ready to state the main landscape theorem for the squared loss. Theorem 3.5 (Landscape of the squared loss). Let L be the population squared loss of the two-neuron ReLU model in (2.2), and let (w1 , w2 ) be a stationary point of L with w1 ̸= 0 and w2 ̸= 0. Then, after possibly swapping w1 and w2 , exactly one of the following holds. 1. Global minimum: w1 = v1 and w2 = v2 ; this is a global minimum. 2. In-plane saddles: w1 , w2 ∈ span{v1 , v2 } satisfy one of the following three forms:   for some s1 , s2 > 0 with s1 + s2 = r+ ; w1 = s1 v̄, w2 = s2 v̄, w1 = −s1 v̄, w2 = −s2 v̄, for some s1 , s2 > 0 with s1 + s2 = r− ;   w1 = r+ v̄, w2 = −r− v̄.

(3.9)

Every such point has a strictly negative Hessian direction. 3. Out-of-plane saddles: necessarily d ≥ 3, and for some unit vector ξ ⊥ span{v1 , v2 }, w1 = rα (κα ) (cos κα · v̄ + sin κα · ξ) ,

w2 = rα (κα ) (cos κα · v̄ − sin κα · ξ) .

(3.10)

Every such point has a strictly negative Hessian direction. Consequently, the squared loss L has no spurious local minima on {(w1 , w2 ) : w1 ̸= 0, w2 ̸= 0}. 7

To the best of our knowledge, this is the first complete classification of all stationary points with both student weights nonzero (under Euclidean geometry) in a well-specified shallow ReLU teacher–student model with more than one teacher neuron. We make the following remark on the significance of this landscape result. Remark 3.6. Tian (2017) conjectured that the unconstrained population squared loss for k = 2 ReLU teacher-student under isotropic Gaussian inputs has no spurious local minima. Tian’s own analysis was conditional on a numerically-verified inequality (Conjecture 1 of Tian, 2017) and resolved only the case where both student vectors lie inside or outside Cone(v1 , v2 ). Wu et al. (2018) made progress by proving the absence of spurious local minima for orthogonal teachers on the unit-norm manifold. Our Theorem 3.5 settles this open problem in the acute angle setting: it provides an unconditional, complete classification of every critical point of the unconstrained squared loss with both student weights nonzero for general α ∈ (0, π/2], showing that each is either the global minimum or a saddle. We prove Theorem 3.5 in three steps. The first is a dimension reduction: any solution (w1 , w2 ) of (3.4) is contained in the teacher plane together with at most one orthogonal direction. The next two steps analyze the resulting three-dimensional problem and the planar cases that remain. 3.2.1

Step 1: reduce to three dimensions

The dimension reduction is a simple consequence of the characterization in Lemma 3.3. It is useful to decompose each student weight into the part lying in the teacher plane span{v1 , v2 } and the part orthogonal to that plane. For a solution of (3.4), the two orthogonal parts cannot vary independently: the system (3.4) forces them to be linearly dependent, as summarized in the following proposition. ∥

Proposition 3.7. Let w1 , w2 ̸= 0 satisfy (3.4). For i = 1, 2, decompose wi = wi + wi⊥ , where ∥ wi ∈ span{v1 , v2 } and wi⊥ ⊥ span{v1 , v2 }. Then w1⊥ and w2⊥ are linearly dependent. Consequently, dim span{v1 , v2 , w1 , w2 } ≤ 3. The proof is in Appendix C.1.3. Consequently, for any stationary point (w1 , w2 ), the vectors w1 , w2 are contained in the teacher plane together with at most one orthogonal direction. 3.2.2

Step 2: reduce the nonplanar case to two dimensions

Given the previous reduction, there are two possible types of stationary points: either both w1 and w2 lie in the teacher plane, or at least one of them lies outside the teacher plane. We next show that for the latter case, the weights lie in the plane spanned by the bisector and an orthogonal direction. Lemma 3.8 (Three-dimensional symmetry reduction). Let w1 , w2 = ̸ 0 satisfy (3.4). If at least one of w1 , w2 lies outside the teacher plane, then w1 , w2 ∈ span{v̄, ξ} for some unit vector ξ ⊥ span{v1 , v2 }. The proof is in Appendix C.2.2. Geometrically, Lemma 3.8 says that for a stationary point, if one weight is not contained in the teacher plane, both weights have zero component along the splitting direction v∆ . Thus the three-dimensional problem reduces to two planar problems, one in span{v1 , v2 } and one in span{v̄, ξ}. 3.2.3

Step 3: solve the 2D cases

Out-of-plane solutions. We first consider solutions in span{v̄, ξ} that do not lie in the teacher plane. By Lemma C.4, such a solution is symmetric with respect to the teacher plane: after possibly replacing ξ by −ξ, the two student weights have the same component in v̄ and opposite components in ξ. This corresponds to the out-of-plane saddle form in (3.10). The stationarity condition therefore reduces to a scalar equation for the angle away from the teacher plane. 8

Lemma 3.9 (Out-of-plane stationary points). Let α ∈ (0, π/2], suppose d ≥ 3, and let ξ be any unit vector perpendicular to span{v1 , v2 }. For any nonzero w1 , w2 ∈ span{v̄, ξ} such that ⟨w1 , ξ⟩ ̸= 0 or ⟨w2 , ξ⟩ = ̸ 0, the pair (w1 , w2 ) is a stationary point of the squared loss (2.2) if and only if, after possibly swapping the students or replacing ξ by −ξ, it has the form (3.10). Furthermore, every such stationary point has a strictly negative Hessian direction and thus is a saddle. The proof is in Appendix C.3. For d = 2, the out-of-plane branch is vacuous. For d ≥ 3, the out-of-plane stationary points form a family of saddles indexed by the unit vectors in span{v1 , v2 }⊥ . In-plane solutions. We now turn to the other planar case where w1 , w2 ∈ span{v1 , v2 }. This case contains the global minima where the student weights match the teacher weights up to permutation; the only other stationary points are the three bisector-supported families listed below. Lemma 3.10 (In-plane stationary points). Let r+ and r− be as defined in (3.5). Let w1 , w2 ̸= 0 satisfy w1 , w2 ∈ span{v1 , v2 }. Then (w1 , w2 ) is a stationary point of the squared loss (2.2) if and only if, after possibly swapping w1 and w2 , either (w1 , w2 ) is a global minimum with w1 = v1 , w2 = v2 or (w1 , w2 ) is a saddle satisfying one of the three forms in (3.9). The in-plane classification is proved in Appendix C.4. Combining the above results yields the complete classification of stationary points in Theorem 3.5. The symmetric point w1 = w2 = (r+ /2)v̄ in the positive-bisector saddle family is the saddle shown in Figure 1. It remains to analyze how gradient flow moves through this landscape from small initialization.

Training dynamics

Next we study the gradient-flow trajectory from small initialization. Our analysis divides into two phases: Phase 1 shows that student neurons rapidly align with the bisector v̄ and then their norms grow, and Phase 2 studies what happens near the saddle. We then present global convergence guarantees and escape-time estimates. For concise notation, we write rs = r+ /2 where r+ is defined in Theorem 3.5, ws = rs v̄, and wsaddle := (ws , ws ) for the symmetric positive-bisector saddle.

4.1

Phase 1: alignment with the bisector

We first study how the gradient flow approaches the saddle point wsaddle . This involves two subphases: The direction dynamics drives both student neurons toward v̄, and then the radial dynamics grows their norms to the saddle radius. Subphase 1: Directional alignment. By direct calculation, the student directions along gradient flow admit the following evolution: dw̄i 1 = Pw̄⊥i (π − θi1 )v1 + (π − θi2 )v2 − (π − γ)w3−i , | {z } | {z } dt 2π∥wi ∥2 teacher force

i = 1, 2.

student interaction

The projected teacher force is the directional part of the teacher-student correlation term analyzed in Theorem 3.1. Note that this term dominates the student interaction when ∥w3−i ∥2 is very small. Thus the teacher force controls the angular motion while the weights are small. Moreover, the projected teacher force points toward the bisector by Lemma E.3: v̄, Pw̄⊥i (π − θi1 )v1 + (π − θi2 )v2 ≥ 0, 9

with equality only at w̄i = v̄ or w̄i = −v̄. Thus, except at these two directions, the teacher force increases the alignment of the student neurons with v̄. The factor 1/∥wi ∥2 makes this aligning very fast near small initialization, while the norm barely changes. Subphase 2: Radial growth toward the saddle. Subphase 1 brings both student directions rapidly close to v̄ while their norms are still small. The subsequent radial-growth stage takes only a fixed amount of time. Over this interval, the direction dynamics can amplify the angular errors by only a bounded factor. Therefore, for sufficiently small initialization, the alignment established in Subphase 1 persists until the trajectory reaches the saddle neighborhood. To understand the radial dynamics in the second subphase, suppose first that wi = ri v̄ exactly. The norm equations then reduce to 1 ṙi = rs − (r1 + r2 ), 2

i = 1, 2.

Since r1 + r2 is initially small, both derivatives are positive and the two norms grow. Moreover, d (r1 + r2 ) = r+ − (r1 + r2 ), dt

d (r1 − r2 ) = 0. dt

Hence the total radius is attracted to r+ = 2rs , while the radial imbalance remains fixed. Small, balanced initialization makes this imbalance small, so both norms grow nearly together toward rs . The alignment and balance estimates show that the full trajectory follows this reduced dynamics up to small errors and therefore enters a neighborhood of wsaddle . Combining the two subphases gives the following finite-time approach result. Lemma 4.1 (Phase 1: finite-time approach to the bisector saddle). Fix d ≥ 2 and unit teacher weights v1 , v2 ∈ Rd with α := ∠(v1 , v2 ) ∈ (0, π/2]. Consider the two-student population gradient flow (2.2) under isotropic Gaussian inputs, and write w(t) = (w1 (t), w2 (t)). Fix an absolute constant C ≥ 1 and an angular margin η ∈ (0, π/2). For every target radius ρ > 0, there are t0 = t0 (α, ρ) < ∞ and r∗ = r∗ (α, C, η, ρ) > 0 such that the following holds. Recall the initialization scale rinit from (2.3) and the balance condition (2.6). If 0 < rinit ≤ r∗ , the initialization is C-balanced, and ∠(w̄i (0), −v̄) ≥ η for i = 1, 2, then the gradient-flow solution exists on [0, t0 ], both student weights remain nonzero on this interval, and there is a time tin ≤ t0 such that ∥w(tin ) − wsaddle ∥2 < ρ. The proof is in Appendix E.3. The dependence on C and η enters the allowed initialization scale; the time bound is independent of these two parameters. For Gaussian initialization, the following corollary separates the small-scale condition from the dimension-dependent probability of balance and angular separation. Corollary 4.2 (Phase 1 under Gaussian initialization). In the setting of Lemma 4.1, fix C > 1, iid

2 I ), η ∈ (0, π/2), ρ > 0, and d √ let t0 , r∗ be the constants in that lemma. Suppose w1 (0), w2 (0) ∼ N (0, σ −cd where 0 < σ ≤ r∗ /(2 d). There exists c = c(C, η) > 0 such that, with probability at least 1 − 6e , the gradient-flow solution exists on [0, t0 ], both student weights remain nonzero on this interval, and

∥w(tin ) − wsaddle ∥2 < ρ

for some tin ∈ [0, t0 ].

The proof is in Appendix E.3. The next subsection studies how gradient flow escapes this saddle and separates the student neurons. 10

4.2

Phase 2: escape from the bisector saddle

Before turning to the local escape mechanism, we establish a qualitative global convergence result for Lebesgue-almost every initialization, with no restriction on its scale or structure. Theorem 4.3 (Unrestricted almost-everywhere global convergence). Fix d ≥ 2 and unit teachers with α = ∠(v1 , v2 ) ∈ (0, π/2]. For Lebesgue-almost every initialization (w1 (0), w2 (0)) ∈ (Rd )2 , the population gradient-flow trajectory (2.2) is defined for every t ≥ 0, both student weights remain nonzero at every finite time, and (w1 (t), w2 (t)) −→ (v1 , v2 )

(v2 , v1 ).

The proof is in Appendix D.2. Note that while Theorem 4.3 establishes global convergence, it does not quantify the convergence rate or the escape behavior. The following subsections characterize how the trajectory escapes the saddle under additional assumptions. 4.2.1

Escape directions near the bisector saddle

Near wsaddle , Taylor expansion of gradient flow gives d w(t) − wsaddle = −∇2 L(wsaddle ) w(t) − wsaddle + o ∥w(t) − wsaddle ∥2 . dt

(4.1)

Thus negative Hessian directions are pushed away from the saddle, positive directions shrink, and the kernel remains flat. Recall the landscape theorem Theorem 3.5 identifies the positive-bisector saddle family; here we focus on wsaddle = rs (v̄, v̄). Complete the orthonormal pair (v̄, v∆ ) from (2.4) to an orthonormal basis v̄, v∆ , u3 , . . . , ud of Rd . The Hessian directions at this saddle are as follows. Lemma 4.4 (Hessian eigenspaces at the bisector saddle). In the setting of Theorem 3.5, the Hessian of the loss at the bisector saddle wsaddle has the following eigenspace structure: • The negative eigenspace of ∇2 L(wsaddle ) is span{(v∆ , −v∆ ), (uj , −uj ) : 3 ≤ j ≤ d}. Moreover, (v∆ , −v∆ ) has eigenvalue − 2 sin(α/2) , while each (uj , −uj ) has eigenvalue − sin(α/2) πrs πrs . • The positive eigenspace of ∇2 L(wsaddle ) is span{(v̄, v̄), (v∆ , v∆ ), (uj , uj ) : 3 ≤ j ≤ d}. • The kernel of ∇2 L(wsaddle ) is span{(v̄, −v̄)}. The proof is in Appendix H.1. We observe that the planar antisymmetric mode (v∆ , −v∆ ) is the most unstable direction: its eigenvalue is twice as negative as the out-of-plane antisymmetric eigenvalues. Thus, if a trajectory enters a sufficiently small saddle neighborhood with homogeneous coordinates in the Hessian eigenspace, the planar split would grow fastest along (v∆ , −v∆ ) and dominate the escape. This gives the geometric intuition for the observed in-plane splitting in Figure 1. Write w− (t) for the projection of w(t) − wsaddle onto the negative eigenspace in the lemma, and let ϵ be the smallest value of ∥w− (t)∥2 over the gradient flow trajectory. Near the saddle, keeping only the linear term in the Taylor expansion (4.1) gives the approximation d ∥w− (t)∥22 ≈ 2⟨w− (t), −∇2 L(wsaddle )w− (t)⟩ ≍ ∥w− (t)∥22 . dt Thus, after ∥w− (t)∥2 reaches its minimum, it will grow exponentially. This suggests that the escape time from the saddle is on the order of log(1/ϵ) (see a rigorous version in Theorem 4.8). However, this calculation does not fully describe the escape path. In particular, it leaves two questions open: 11

1. Which negative direction carries the escape? The component w− (t) includes both the planar split (v∆ , −v∆ ) and the out-of-plane splits (uj , −uj ), j ≥ 3. 2. How large is ϵ? Can this quantity be decided by properties of the initialization? The next subsection answers the first question in a deterministic way: a condition on the signs and sizes of the initial planar and out-of-plane coordinates keeps the trajectory away from the out-of-plane saddles. 4.2.2

Exclusion of out-of-plane saddles

Recall the orthonormal basis {v̄, v∆ , u3 , . . . , ud } extending the teacher-plane directions. For each student weight i ∈ {1, 2}, we decompose wi (t) = zi,1 (t)v̄ + zi,2 (t)v∆ +

d X

zi,j (t)uj .

j=3

In words, zi,1 is the bisector coordinate, zi,2 is the planar splitting coordinate, and zi,j , j ≥ 3, are the out-of-plane coordinates. Assumption 4.5. Fix C⊥ > 0. The planar splitting coordinates have opposite signs, z1,2 (0)z2,2 (0) < 0, and every out-of-plane coordinate is dominated by the corresponding planar splitting coordinate: |zi,j (0)| ≤ C⊥ |zi,2 (0)| for every i ∈ {1, 2} and 3 ≤ j ≤ d. Notice that only the split inside the teacher plane receives an additive force from the teacher directions. The out-of-plane coordinates have no such source: each is multiplied by a scalar and coupled to the corresponding coordinate of the other student. Hence the conditions of Assumption 4.5 continue to hold along the flow, so the trajectory stays away from saddles with zero planar split and nonzero out-of-plane split. The theorem below makes this comparison precise. Since the loss is differentiable only where both student weights are nonzero, the solution is defined up to the first time a weight reaches zero; we write [0, Tmax ) for this maximal interval, with Tmax = ∞ if no weight reaches zero. By Theorem 4.3, Tmax = ∞ for Lebesgue-almost every initialization. Theorem 4.6 (Separation from out-of-plane saddles). Under Assumption 4.5, |z1,j (t) − z2,j (t)| ≤ C⊥ |z1,2 (t) − z2,2 (t)| for every j ≥ 3 and 0 ≤ t < Tmax . Let S⊥ denote the out-of-plane saddle family in Theorem 3.5. Then there exists a constant c > 0 (depending on α, C⊥ , and d) such that ∥w(t) − w∥ b 2 ≥ c for every w b ∈ S⊥ and 0 ≤ t < Tmax . Remark 4.7. The theorem first gives the comparison estimate |z1,j (t) − z2,j (t)| ≤ C⊥ |z1,2 (t) − z2,2 (t)|,

j ≥ 3, 0 ≤ t < Tmax .

This estimate keeps the trajectory away from the out-of-plane saddle family, whose saddles have zero planar split and nonzero out-of-plane split. The opposite signs of the planar splitting coordinates also rule out convergence to the in-plane saddle families; see Lemma F.3. Combining this deterministic saddle exclusion with the zero-weight avoidance and trajectory-convergence results used in Theorem 4.3 gives global convergence for Lebesgue-almost every initialization satisfying Assumption 4.5, without invoking the stable-manifold saddle-avoidance step; see Corollary F.4. For independent Gaussian initialization wi (0) ∼ N (0, σ 2 Id ), i = 1, 2, the coordinates z1,2 (0) and z2,2 (0) are independent centered Gaussians, so the sign condition of Assumption 4.5 holds with probability 1/2. Since z1,2 (0) and z2,2 (0) are almost surely nonzero, the domination condition holds almost surely with C⊥ = max{|zi,j (0)|/|zi,2 (0)| : i = 1, 2, 3 ≤ j ≤ d} when d ≥ 3; when d = 2 there are no out-of-plane coordinates and any C⊥ > 0 works. Thus the theorem applies with probability 1/2, with a constant C⊥ that depends on the initialization. 12

4.2.3

Logarithmic escape under structured initialization

Phase 1 explains how small initializations reach a neighborhood of the bisector saddle. The results above then describe the subsequent dynamics qualitatively: almost every trajectory converges to a global minimum, while under Assumption 4.5 the out-of-plane coordinates remain controlled by the planar splitting coordinate, ruling out convergence to out-of-plane saddles. Neither result, however, quantifies how long symmetry breaking takes. We now study this local time scale under a structured initialization condition, which holds with constant probability at small random initialization. We use the coordinates zi,j of Section 4.2.2 and write µα := 2 sin(α/2)/(πrs ) for the planar escape rate from Lemma 4.4. For ε > 0, let 2 X d X 2 2 Tε := inf t ≥ 0 : zi,j (t) ≥ ε i=1 j=2

be the first time the total squared distance of the student weights from the bisector axis reaches ε2 . The next theorem shows that Tε is governed by the initial planar splitting and, up to a bounded error, depends logarithmically on ainit and rinit . Theorem 4.8 (Logarithmic escape governed by the initial planar splitting). Fix α ∈ (0, π/2] and an absolute constant C > 1. For every sufficiently small ε > 0, there are constants c ∈ (0, 1) and K < ∞, depending only on α, C, ε, such that the following holds for every d ≥ 2. Consider the gradient flow in (2.2) from a C-balanced initialization (see (2.6)) satisfying ! √ √ √ d rinit d 1 + log + log d ≤ c. rinit d ≤ c, rinit Assume also that rinit √ , C d Then Tε is finite and satisfies ainit ≥

z1,1 (0), z2,1 (0) > 0,

z1,2 (0)z2,2 (0) < 0.

µα rinit 1 log Tε − ≤ K. µα ainit

The proof is in Appendix H.2. Remark 4.9 (Dimension dependence and i.i.d. initialization). When the initial planar splitting is of √ the size of a single coordinate, that is, ainit ≍ rinit / d, the theorem gives 1 1 Tε = µ−1 + log d + O(1). α − 1 log rinit 2µα For example, if rinit = d−γ for a fixed γ > 1/2, then the two upper bounds on rinit in the theorem hold for all sufficiently large d, and Tε = Θ(log d). Independent Gaussian initialization is a natural example. If wi (0) ∼ N (0, σ 2 Id ) independently for i = 1, 2, then with constant probability √ the initialization satisfies the other conditions of Theorem 4.8, namely C-balance, ainit ≥ rinit /(C d) and the sign conditions, and √ rinit rinit ≍ σ d, ainit ≍ σ ≍ √ . d On this event, for 0 < σ ≤ 1, the two upper bounds on rinit hold once σd 1 + log(1/σ) + log d is sufficiently small, and the theorem gives 1 1 + log d + Oα,C,ε (1). Tε = µ−1 α − 1 log σ 2 If σ = d−β for a fixed β > 1, then these bounds hold for all sufficiently large d, and Tε = Θ(log d). 13

Empirical validation. The calculation above shows that a planar splitting of the size of a single coordinate contributes an explicit logarithmic term in d; for Gaussian initialization at fixed σ, the formula suggests a slope of 1/2 with respect to log d. The random-initialization experiments in Figure 2 support this prediction: across the tested values of α, Tε grows approximately linearly with log d, while Tglobal − Tε remains nearly constant. This suggests a two-stage picture in which the ambient dimension primarily affects the time needed to break the bisector symmetry, whereas the subsequent descent to a global minimizer occurs on an essentially dimension-independent time scale.

Figure 2: Escape time Tε and global-convergence time Tglobal vs. ambient dimension d, one panel per α,

for random initialization at fixed σ = 10−4 . Each point is the mean over 5 seeds, and error bars are ±1 s.d. squared distance of the two student weights from the bisector axis, P2 The Pd time Tε2 is when the total 2 z (t) , first reaches ε (ε = 0.1); Tglobal is when the student state first reaches distance 0.05 from i,j i=1 j=2 the global minimizer set. Dashed lines are log-d fits; fitted slopes ∈ [0.48, 0.55] match the 12 log d prediction above, and the gap Tglobal − Tε shown in the panel titles is almost d-independent.

Conclusion

We studied population gradient flow for a two-neuron ReLU teacher-student model, a minimal multi-index setting that already shows staged high-dimensional feature learning. The loss has no spurious local minima with both student weights nonzero, but it contains a bisector saddle that organizes training. From small initialization, gradient flow first learns the easy bisector direction, then escapes along the harder splitting direction. The logarithmic escape time identifies where dimension enters: in the leap-2 symmetry-breaking phase, rather than in the final descent after specialization begins. Several directions remain open, including: (i) obtaining an end-to-end escapetime analysis from random initialization with high probability; (ii) extending the landscape and dynamical characterization to other teacher-student settings and high-leap multi-index functions.

Acknowledgements The authors thank Margalit Glasgow, Jason D. Lee, and Theodor Misiakiewicz for discussions and feedback. ChatGPT and Claude were used to polish the writing, to check and reorganize proofs, and to produce figures from the numerical experiments.

References Abbe, E., Boix-Adserà, E. and Misiakiewicz, T. (2022). The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks. In Conference on Learning Theory. PMLR. Abbe, E., Boix-Adserà, E. and Misiakiewicz, T. (2023). SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics. In The Thirty Sixth Annual Conference on Learning Theory. PMLR. Akiyama, S. and Suzuki, T. (2021). On learnability via gradient method for two-layer ReLU neural networks in teacher-student setting. In International Conference on Machine Learning. Ba, J., Erdogdu, M. A., Suzuki, T., Wang, Z., Wu, D. and Yang, G. (2022). High-dimensional asymptotics of feature learning: How one gradient step improves the representation. Advances in Neural Information Processing Systems 35 37932–37946. Bah, B., Rauhut, H., Terstiege, U. and Westdickenberg, M. (2022). Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers. Information and Inference: A Journal of the IMA 11 307–353. Ben Arous, G., Erdogdu, M. A., Vural, N. M. and Wu, D. (2025). Learning quadratic neural networks in high dimensions: SGD dynamics and scaling laws. Advances in Neural Information Processing Systems 38 162874–162961. Ben Arous, G., Gerbelot, C. and Piccolo, V. (2026). Stochastic gradient descent in high dimensions for multi-spiked tensor PCA. Communications on Pure and Applied Mathematics 79 2291–2369. Ben Arous, G., Gheissari, R. and Jagannath, A. (2021). Online stochastic gradient descent on non-convex losses from high-dimensional inference. Journal of Machine Learning Research 22 1–51. Biehl, M., Riegler, P. and Wöhler, C. (1996). Transient dynamics of on-line learning in two-layered neural networks. Journal of Physics A: Mathematical and General 29 4769–4780. Bietti, A., Bruna, J. and Pillaud-Vivien, L. (2025). On learning Gaussian multi-index models with gradient flow part I: General properties and two-timescale learning. Communications on Pure and Applied Mathematics 78 2354–2435. Bietti, A., Bruna, J., Sanford, C. and Song, M. J. (2022). Learning single-index models with shallow neural networks. Advances in neural information processing systems 35 9768–9783. Bolte, J., Daniilidis, A. and Lewis, A. (2007). The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM Journal on Optimization 17 1205–1223. Boursier, E., Pillaud-Vivien, L. and Flammarion, N. (2022). Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs. Advances in Neural Information Processing Systems 35 20105–20118. Chizat, L. (2022). Sparse optimization on measures with over-parameterized gradient descent. Mathematical Programming 194 487–532. 15

Chizat, L., Oyallon, E. and Bach, F. (2019). On lazy training in differentiable programming. Advances in neural information processing systems 32. Damian, A., Lee, J. and Soltanolkotabi, M. (2022). Neural networks can learn representations with gradient descent. In Conference on Learning Theory. PMLR. Damian, A., Nichani, E., Ge, R. and Lee, J. D. (2023). Smoothing the landscape boosts the signal for SGD: Optimal sample complexity for learning single index models. Advances in Neural Information Processing Systems 36 752–784. Dandi, Y., Krzakala, F., Loureiro, B., Pesce, L. and Stephan, L. (2024). How two-layer neural networks learn, one (giant) step at a time. Journal of Machine Learning Research 25 1–65. Dudeja, R. and Hsu, D. (2018). Learning single-index models in Gaussian space. In Conference On Learning Theory. PMLR. Fukumizu, K. and Amari, S.-i. (2000). Local minima and plateaus in hierarchical structures of multilayer perceptrons. Neural networks 13 317–327. Ge, R., Ren, Y., Wang, X. and Zhou, M. (2021). Understanding deflation process in overparametrized tensor decomposition. Advances in Neural Information Processing Systems 34 1299–1311. Glasgow, M. (2024). SGD finds then tunes features in two-layer neural networks with near-optimal sample complexity: A case study in the XOR problem. In The Twelfth International Conference on Learning Representations. Goldt, S., Advani, M. S., Saxe, A. M., Krzakala, F. and Zdeborová, L. (2020). Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setup. Journal of Statistical Mechanics: Theory and Experiment 124010. Hirsch, M. W., Pugh, C. C. and Shub, M. (1977). Invariant Manifolds, vol. 583 of Lecture Notes in Mathematics. Springer. Huang, J., Loureiro, B. and Sarao Mannelli, S. (2026). Sharp description of local minima in the loss landscape of high-dimensional two-layer ReLU neural networks. In Proceedings of the 43rd International Conference on Machine Learning. Lee, J. D., Oko, K., Suzuki, T. and Wu, D. (2024). Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit. Advances in Neural Information Processing Systems 37 58716–58756. Li, Y., Ma, T. and Zhang, H. R. (2020). Learning over-parametrized two-layer neural networks beyond NTK. In Conference on learning theory. PMLR. Martin, S., Bach, F. and Biroli, G. (2024). On the impact of overparameterization on the training of a shallow neural network in high dimensions. In International Conference on Artificial Intelligence and Statistics. PMLR. Min, H., Zhu, Z. and Vidal, R. (2025). Neural collapse under gradient flow on shallow ReLU networks for orthogonally separable data. Advances in Neural Information Processing Systems 38 105551–105590. 16

Oko, K., Song, Y., Suzuki, T. and Wu, D. (2024). Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinations. In The Thirty Seventh Annual Conference on Learning Theory. PMLR. Oostwal, E., Straat, M. and Biehl, M. (2021). Hidden unit specialization in layered neural networks: ReLU vs. sigmoidal activation. Physica A: Statistical Mechanics and its Applications 564 125517. Ren, Y. and Lee, J. D. (2025). Learning orthogonal multi-index models: A fine-grained information exponent analysis. Advances in Neural Information Processing Systems 38 183843–183894. Ren, Y., Nichani, E., Wu, D. and Lee, J. D. (2025). Emergence and scaling laws in SGD learning of shallow neural networks. Advances in Neural Information Processing Systems 38 42829–42911. Riegler, P. and Biehl, M. (1995). On-line backpropagation in two-layered neural networks. Journal of Physics A: Mathematical and General 28 L507–L513. Saad, D. and Solla, S. A. (1995). On-line learning in soft committee machines. Physical Review E 52 4225–4243. Safran, I. and Shamir, O. (2018). Spurious local minima are common in two-layer ReLU neural networks. In International conference on machine learning. PMLR. Safran, I. M., Yehudai, G. and Shamir, O. (2021). The effects of mild over-parameterization on the optimization landscape of shallow ReLU neural networks. In Conference on Learning Theory. PMLR. Şimşek, B., Bendjeddou, A. and Hsu, D. (2025). Learning Gaussian multi-index models with gradient flow: Time complexity and directional convergence. In Proceedings of The 28th International Conference on Artificial Intelligence and Statistics, vol. 258 of Proceedings of Machine Learning Research. PMLR. Soltanolkotabi, M. (2017). Learning ReLUs via gradient descent. Advances in neural information processing systems 30. Sumner, J. S., Jagers, A. A., Vowe, M. and Anglesio, J. (1991). Inequalities involving trigonometric functions. The American Mathematical Monthly 98 264–267. Tian, Y. (2017). An analytical formula of population gradient for two-layered ReLU network and its applications in convergence and critical point analysis. In International Conference on Machine Learning. Wang, M. and Ma, C. (2023). Understanding multi-phase optimization dynamics and rich nonlinear behaviors of ReLU networks. Advances in Neural Information Processing Systems 36 35654–35747. Wu, C., Luo, J. and Lee, J. D. (2018). No spurious local minima in a two hidden unit ReLU network. In International Conference on Learning Representations, Workshop Track. Xu, W. and Du, S. (2023). Over-parameterization exponentially slows down gradient descent for learning a single neuron. In The Thirty Sixth Annual Conference on Learning Theory. PMLR. Yehudai, G. and Shamir, O. (2020). Learning a single neuron with gradient methods. In Conference on Learning Theory. PMLR. 17

Zhang, B., Wang, Z., Fu, H. and Lee, J. D. (2026). Neural networks learn generic multi-index models near information-theoretic limit. In The Fourteenth International Conference on Learning Representations. Zhou, M., Ge, R. and Jin, C. (2021). A local convergence theory for mildly over-parameterized two-layer neural network. In Conference on Learning Theory. PMLR. Zhu, Z., Liu, F. and Cevher, V. (2025). How gradient descent balances features: A dynamical analysis for two-layer neural networks. In The Thirteenth International Conference on Learning Representations.

Contents 1 Introduction 1.1 Our contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Further related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 2 3

2 Preliminary 2.1 Problem setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Leap complexity of the teacher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 4 4

3 Landscape analysis 3.1 Warm-up: landscape of the correlation loss . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Landscape of the squared loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Step 1: reduce to three dimensions . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Step 2: reduce the nonplanar case to two dimensions . . . . . . . . . . . . . . 3.2.3 Step 3: solve the 2D cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 5 6 8 8 8

4 Training dynamics 4.1 Phase 1: alignment with the bisector . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Phase 2: escape from the bisector saddle . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Escape directions near the bisector saddle . . . . . . . . . . . . . . . . . . . . 4.2.2 Exclusion of out-of-plane saddles . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Logarithmic escape under structured initialization . . . . . . . . . . . . . . .

9 9 11 11 12 13

5 Conclusion

A Technical lemmas 21 A.1 Gaussian ReLU kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 A.2 Closed-form loss and gradient flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 A.3 Hessian of the population loss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 A.4 Proof of Proposition 2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 B Correlation-loss landscape 27 B.1 Reduction to one student in span{v1 , . . . , vk } . . . . . . . . . . . . . . . . . . . . . . 27 B.2 The case α ≤ π/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 B.3 The case k = 2 and α > π/2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 B.4 Proof of Theorem 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 C Squared-loss landscape 33 C.1 Stationarity and dimension reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 C.1.1 Stationary equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 C.1.2 Uniqueness of the out-of-plane root . . . . . . . . . . . . . . . . . . . . . . . . 33 C.1.3 Reduction to three dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . 35 C.2 Three-dimensional symmetry reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 35 C.2.1 The teacher force . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 C.2.2 Elimination of the splitting coordinate . . . . . . . . . . . . . . . . . . . . . . 36 C.3 Out-of-plane stationary points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 C.3.1 A scalar inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 C.3.2 Mirror symmetry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 19

C.3.3 Exact scalar characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . C.3.4 Negative curvature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.4 In-plane stationary points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.4.1 Coordinates in the teacher plane . . . . . . . . . . . . . . . . . . . . . . . . . C.4.2 Stationary points above the angle threshold . . . . . . . . . . . . . . . . . . . C.4.3 Classification up to the angle threshold . . . . . . . . . . . . . . . . . . . . . . C.5 Completion of the squared-loss landscape theorem . . . . . . . . . . . . . . . . . . .

43 44 44 44 46 47 52

D Global convergence and boundary avoidance 53 D.1 Almost-everywhere avoidance of zero weights . . . . . . . . . . . . . . . . . . . . . . 53 D.1.1 Polar coordinates near {w1 = 0} . . . . . . . . . . . . . . . . . . . . . . . . . 53 D.1.2 Equilibria of X with r1 = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 D.1.3 The equilibria (0, −v1 , v2 ) and (0, −v2 , v1 ) . . . . . . . . . . . . . . . . . . . . 56 D.1.4 Avoidance of ∂Ω . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 D.2 Saddle avoidance and global convergence . . . . . . . . . . . . . . . . . . . . . . . . . 63 E Approach to the bisector saddle 64 E.1 Norm bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 E.2 Alignment with the bisector and balance of the norms . . . . . . . . . . . . . . . . . 65 E.3 Proofs of Lemma 4.1 and Corollary 4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . 68 F Exclusion of saddles and global convergence

G Approach to the bisector saddle under structured initialization 72 G.1 Coordinates and estimates near the bisector . . . . . . . . . . . . . . . . . . . . . . . 72 G.2 Alignment with the bisector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 G.3 Growth of the bisector coordinates up to rs /2 . . . . . . . . . . . . . . . . . . . . . . 78 G.4 Entry into a neighborhood of the bisector saddle . . . . . . . . . . . . . . . . . . . . 79 H Logarithmic escape from the bisector saddle 82 H.1 Hessian at the bisector saddle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 H.2 Proof of Theorem 4.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

Technical lemmas

This appendix collects the lemmas used in the landscape and training-dynamics proofs. Throughout this appendix, unless stated otherwise, v1 , v2 ∈ Rd are unit teacher directions with α = ∠(v1 , v2 ) ∈ (0, π/2], and the angle notation θij and γ follows the convention in (2.7). Whenever directions or angles of student weights are used, the corresponding weights are assumed to be nonzero.

A.1

Gaussian ReLU kernel

We first recall the Gaussian ReLU kernel identity from Safran and Shamir (2018). For nonzero u, v ∈ Rd , let θu,v := ∠(u, v). Then Ex∼N (0,Id ) ϕ(⟨u, x⟩)ϕ(⟨v, x⟩) = ∥u∥2 ∥v∥2 l(θu,v ), (A.1a) l(θ) :=

(π − θ) cos θ + sin θ , 2π

θ ∈ [0, π].

(A.1b)

Lemma A.1 (Regularity of L and continuation of gradient flow). The Gaussian ReLU kernel in (A.1a) is jointly C 2 in its two nonzero vector arguments, including at aligned and antipodal directions. Consequently, the population loss L is C 2 on the open set where both student weights are nonzero. On this set, gradient flow has a unique local solution, and any solution that stays in a compact subset of this set on [0, T ), with T < ∞, extends beyond T . Proof of Lemma A.1. Write U = (Rd \ {0})2 ,

K(u, v) = ∥u∥2 ∥v∥2 l(∠(u, v)).

For (u, v) ∈ U, write c = ⟨ū, v̄⟩,

z = ū − cv̄,

τ = ∥z∥2 =

p 1 − c2 .

When |c| < 1, K(u, v) = ∥u∥2 ∥v∥2 l(arccos c) is smooth. Near c = ±1, the sign of c is fixed and ( arcsin τ, c > 0, θ = ∠(u, v) = π − arcsin τ, c < 0, ( √ c/2 + F (τ ), c > 0, τ − 1 − τ 2 arcsin τ l(θ) = F (τ ) = . 2π F (τ ), c < 0, The function F is odd and analytic near zero, with τ 3 3τ 5 + + O(τ 7 ), 6 40 p τ2 τ4 1 − τ2 = 1 − − + O(τ 6 ), 2 8 τ3 τ5 F (τ ) = + + O(τ 7 ) = τ 3 ψ(τ 2 ), 6π 15π arcsin τ = τ +

where ψ is analytic near zero and ψ(0) = 1/(6π). For H(z) = ∥z∥32 and z ̸= 0, ∇H(z) = 3∥z∥2 z, ∇2 H(z) = 3∥z∥2 Id +

3zz ⊤ , ∥∇2 H(z)∥op ≤ 6∥z∥2 . ∥z∥2 21

At zero,

∥∇H(z)∥2 = 3∥z∥2 −→ 0. ∥z∥2

|H(z) − H(0)| = ∥z∥22 −→ 0, ∥z∥2

Thus ∇H(0) = 0 and ∇2 H(0) = 0, and ∇2 H(z) → ∇2 H(0) as z → 0. Hence H ∈ C 2 (Rd ), and F (∥z∥2 ) = H(z)ψ(∥z∥22 ) is C 2 near 0. Since c, z, ∥u∥2 , and ∥v∥2 depend smoothly on (u, v) ∈ U, the two endpoint formulas give K ∈ C 2 (U). Since L(w1 , w2 ) =

i,j=1

X 1 X 1 X K(wi , wj ) − K(vi , vj ), K(wi , vj ) + 2 2

it follows that L ∈ C 2 (U) and −∇L ∈ C 1 (U). Local existence and uniqueness follow from the Picard–Lindelöf theorem. For continuation, suppose that w([0, T )) ⊂ Q for some T < ∞ and some compact set Q ⊂ U , and let BQ = maxw∈Q ∥∇L(w)∥2 < ∞. For 0 ≤ s < t < T , Z t ∥w(t) − w(s)∥2 ≤

∥∇L(w(a))∥2 da ≤ BQ (t − s). s

Therefore the limit wT = limt↑T w(t) exists and lies in Q ⊂ U. The local solution starting at wT extends w beyond T . ■ Lemma A.2 (Monotonicity and bounds for l). The function l defined in (A.1b) is strictly decreasing on [0, π] and satisfies 0 ≤ l(θ) ≤ 1/2 for every θ ∈ [0, π]. Proof of Lemma A.2. Differentiating gives l′ (θ) = −

(π − θ) sin θ , 2π

(A.2)

which is negative for θ ∈ (0, π). Hence l is strictly decreasing on [0, π]. The endpoint values l(0) = 1/2 and l(π) = 0 give the claimed bounds. ■ Lemma A.3 (Bounds on l(θ1 ) + l(θ2 )). Let u ∈ Rd be nonzero and write θj := ∠(u, vj ) for j = 1, 2. Then α α 2l π − ≤ l(θ1 ) + l(θ2 ) ≤ 2l . 2 2 Moreover, equality in the upper bound holds if and only if θ1 = θ2 = α/2. Proof of Lemma A.3. We begin with the lower bound. Set ϑj := π − θj = ∠(−ū, vj ),

j = 1, 2.

The triangle inequality for angles gives ϑ1 + ϑ2 ≥ α. Write q(x) := sin x − x cos x,

l(π − x) =

q(x) , 2π

Since q ′ (x) = x sin x ≥ 0, the function q is increasing. Let a := min{ϑ1 , α}, 22

b := α − a.

x ∈ [0, π].

Then a, b ∈ [0, α], a + b = α, a ≤ ϑ1 , and b ≤ ϑ2 . Indeed, if ϑ1 < α, then b = α − ϑ1 ≤ ϑ2 ; otherwise b = 0. Therefore, q(ϑ1 ) + q(ϑ2 ) ≥ q(a) + q(b). (A.3) Moreover,

q ′′ (x) = sin x + x cos x ≥ 0,

x ∈ [0, α],

because α ≤ π/2. Thus q is convex on [0, α], and Jensen’s inequality yields α a+b = 2q q(a) + q(b) ≥ 2q . 2 2

(A.4)

Combining (A.3) and (A.4) gives α l(θ1 ) + l(θ2 ) = l(π − ϑ1 ) + l(π − ϑ2 ) ≥ 2l π − . 2

(A.5)

We now prove the upper bound and identify its equality case. Case 1: θ1 , θ2 ∈ [0, α]. Define l1 on [0, α] by l1 (x) := l(x) + l(α − x). The triangle inequality gives θ2 ≥ α − θ1 . Since l is decreasing, l(θ1 ) + l(θ2 ) ≤ l(θ1 ) + l(α − θ1 ) = l1 (θ1 ).

(A.6)

Differentiating (A.2) gives 2πl1′′ (x) = sin x − (π − x) cos x + sin(α − x) − (π − α + x) cos(α − x) α α α α α α α cos cos x − −2 x− sin sin x − . = 2 sin − π − 2 2 2 2 2 2 2

(A.7)

Because 0 < α/2 ≤ π/4,

α 3π α ≤1< ≤π− . 2 4 2 Hence α α α sin − π − cos < 0. 2 2 2 For x ∈ [0, α], |x − α/2| ≤ α/2 ≤ π/4, so α α α cos x − > 0, x− sin x − ≥ 0. 2 2 2 tan

It follows from (A.7) that l1′′ (x) < 0 on [0, α]. The symmetry l1 (x) = l1 (α − x) and strict concavity show that l1 has a unique maximum at x = α/2. Therefore, α α l1 (x) ≤ l1 = 2l , x ∈ [0, α], (A.8) 2 2 with equality only at x = α/2. Combining (A.6) and (A.8) yields α l(θ1 ) + l(θ2 ) ≤ 2l . 2 If equality holds, (A.8) forces θ1 = α/2. Equality in (A.6), together with the strict monotonicity of l, then forces θ2 = α − θ1 = α/2. Conversely, θ1 = θ2 = α/2 attains equality. 23

Case 2: Exactly one of θ1 , θ2 lies in (α, π]. By symmetry, assume θ1 ∈ (α, π] and θ2 ∈ [0, α]. The reverse triangle inequality gives θ1 − θ2 ≤ α, so θ2 ≥ θ1 − α. Monotonicity of l then gives α l(θ1 ) + l(θ2 ) ≤ l(θ1 ) + l(θ1 − α) ≤ l(α) + l(0) ≤ 2l . 2 The last inequality is (A.8) evaluated at x = 0. Since α/2 > 0, the equality condition in (A.8) shows that this inequality is strict. Case 3: θ1 , θ2 ∈ (α, π]. Monotonicity gives l(θ1 ) + l(θ2 ) ≤ 2l(α) ≤ 2l

α 2

The second inequality is strict because l is strictly decreasing and α > 0. Thus the upper bound holds in all three cases, with equality if and only if θ1 = θ2 = α/2. Together with the lower bound (A.5), this completes the proof. ■

A.2

Closed-form loss and gradient flow

The identities in this subsection hold for the wider range α = ∠(v1 , v2 ) ∈ (0, π). We recall the angles θij and γ from (2.7), the kernel profile l from (A.1b), and the shorthand θu,v = ∠(u, v) from Appendix A.1. For nonzero u, v ∈ Rd , the derivative of the Gaussian ReLU kernel with respect to its first argument is ∇u Ex∼N (0,Id ) ϕ(⟨u, x⟩)ϕ(⟨v, x⟩) = Ex∼N (0,Id ) 1(u⊤ x ≥ 0)ϕ(⟨v, x⟩)x =

∥v∥2 sin θu,v π − θu,v ū + v. 2π 2π

(A.9)

The Lipschitz continuity of ReLU and Gaussian integrability justify differentiating under the expectation. The hyperplane {x : u⊤ x = 0} has Gaussian measure zero, so using either > or ≥ in the indicator gives the same expectation. Expanding the squared residual and applying (A.1a) to each student–student, teacher–teacher, and student–teacher pair gives, for nonzero w1 , w2 , h 2 i 1 L(w1 , w2 ) = Ex∼N (0,Id ) ϕ(⟨w1 , x⟩) + ϕ(⟨w2 , x⟩) − ϕ(⟨v1 , x⟩) − ϕ(⟨v2 , x⟩) 2 2 2 X X 1 1 ∥wi ∥2 l(θij ) + l(α) + . (A.10) = ∥w1 ∥22 + ∥w2 ∥22 + ∥w1 ∥2 ∥w2 ∥2 l(γ) − 4 2 i=1

j=1

Since l is bounded by Lemma A.2, the right-hand side of (A.10) extends continuously to all w1 , w2 ∈ Rd if every term carrying a factor ∥wi ∥2 with wi = 0 is set to zero. We use (A.10) with this convention whenever one or both student weights vanish. On the domain w1 , w2 ̸= 0, applying (A.9) to the student–teacher and student–student terms in the gradient flow (2.2) gives 1 sin γ ∥w2 ∥2 dw1 1 π−γ = (π − θ11 )v1 + (π − θ12 )v2 + (sin θ11 + sin θ12 )w̄1 − + w1 − w2 , dt 2π 2 2π ∥w1 ∥2 2π (A.11) 1 π−γ 1 sin γ ∥w1 ∥2 dw2 = (π − θ21 )v1 + (π − θ22 )v2 + (sin θ21 + sin θ22 )w̄2 − w1 − + w2 . dt 2π 2π 2 2π ∥w2 ∥2 (A.12) 24

Lemma A.4 (Direction and norm equations). Let w(t) = (w1 (t), w2 (t)) solve the gradient flow (2.2). At each time t for which w1 (t), w2 (t) ̸= 0, the following identities hold: dw̄1 1 (I − w̄1 w̄1⊤ ) (π − θ11 )v1 + (π − θ12 )v2 − (π − γ)w2 , = dt 2π∥w1 ∥2 dw̄2 1 (I − w̄2 w̄2⊤ ) (π − θ21 )v1 + (π − θ22 )v2 − (π − γ)w1 , = dt 2π∥w2 ∥2 d∥w1 ∥2 1 sin θ11 + sin θ12 + (π − θ11 ) cos θ11 + (π − θ12 ) cos θ12 = dt 2π 1 sin γ + (π − γ) cos γ − ∥w1 ∥2 − ∥w2 ∥2 , 2 2π d∥w2 ∥2 1 sin θ21 + sin θ22 + (π − θ21 ) cos θ21 + (π − θ22 ) cos θ22 = dt 2π sin γ + (π − γ) cos γ 1 − ∥w1 ∥2 − ∥w2 ∥2 . 2π 2

(A.13) (A.14)

(A.15)

(A.16)

Proof of Lemma A.4. Project (A.11) and (A.12) onto the orthogonal complement of each w̄i for the direction equations and onto each w̄i for the norm equations. ■ Lemma A.5 (Bounds on the radial velocity). Let w(t) = (w1 (t), w2 (t)) solve the gradient flow (2.2). At each time t for which w1 (t), w2 (t) ̸= 0, r− ∥w1 ∥2 + ∥w2 ∥2 d∥wi ∥2 ∥wi ∥2 − ≤ ≤1− , 2 2 dt 2

i = 1, 2,

where r− is defined in (3.5). Proof of Lemma A.5. By (A.1b), the norm equation (A.15) reads d∥w1 ∥2 1 = l(θ11 ) + l(θ12 ) − ∥w1 ∥2 − l(γ)∥w2 ∥2 . dt 2 By Lemma A.3 and (3.5), l(θ11 )+l(θ12 ) ≥ 2l(π−α/2) = r− /2, and Lemma A.2 gives l(θ11 )+l(θ12 ) ≤ 1 and 0 ≤ l(γ) ≤ 1/2. These bounds give the claim for i = 1; the case i = 2 follows in the same way from (A.16). ■

A.3

Hessian of the population loss

As in Appendix A.2, we consider α ∈ (0, π). We recall θw,v = ∠(w, v) and write nv,w := v̄ − cos θw,v w̄,

n̄v,w :=

nv,w ∥nv,w ∥2

for nonzero w, v ∈ Rd with θw,v ∈ (0, π). Lemma A.6 (Hessian of the population loss). For nonzero w1 , w2 , the Hessian of the population loss L(w1 , w2 ) has diagonal blocks ∂2L 1 = Id + Q1 (wi , w3−i ) − Q1 (wi , v1 ) − Q1 (wi , v2 ), 2 2 ∂wi and off-diagonal blocks ∂2L = Q2 (wi , w3−i ), ∂wi ∂w3−i 25

i = 1, 2,

where sin θw,v ∥v∥2 Id − w̄w̄⊤ + n̄v,w n̄⊤ v,w , 2π∥w∥2 1 Q2 (w, v) = (π − θw,v )Id + n̄v,w w̄⊤ + n̄w,v v̄ ⊤ . 2π At collinear pairs, Q1 and Q2 are defined by continuous extension: Q1 (w, v) = 0 when w̄ = ±v̄, while Q2 (w, v) = 21 Id when w̄ = v̄ and Q2 (w, v) = 0 when w̄ = −v̄. Q1 (w, v) =

Proof of Lemma A.6. This is the block Hessian formula from Theorem 5 of Safran and Shamir (2018), specialized to the two-student, two-teacher loss (2.2). ■

A.4

Proof of Proposition 2.1

We use the normalized probabilists’ Hermite polynomials from Section 2.2, (−1)k 2 dk 2 hk (x) := √ ex /2 k e−x /2 , dx k!

k ≥ 0.

Proof of Proposition 2.1. Let a := cos(α/2) and b := sin(α/2). Since ∥v1 ∥2 = ∥v2 ∥2 = 1, the directions in (2.4) satisfy v1 = av̄ + bv∆ , v2 = av̄ − bv∆ . With z1 = ⟨v̄, x⟩ and z2 = ⟨v∆ , x⟩ as in Section 2.2, f ∗ (x) = g(z1 , z2 ) := ϕ(az1 + bz2 ) + ϕ(az1 − bz2 ). We expand g as in (2.8) with (U, W ) = (z1 , z2 ), and write ϕbk := EG∼N (0,1) [ϕ(G)hk (G)]. Since a2 + b2 = 1, the Hermite addition formula r X k! m hk (az1 ± bz2 ) = a (±b)n hm (z1 )hn (z2 ) m! n! m+n=k

and the orthonormality of {hm (z1 )hn (z2 )}m,n≥0 give r E[ϕ(az1 ± bz2 )hm (z1 )hn (z2 )] = ϕbm+n Therefore

r gbm,n = ϕbm+n

(m + n)! m a (±b)n . m! n!

(m + n)! m n a b 1 + (−1)n . m! n!

In particular, gbm,1 = 0 for every m ≥ 0. √ It remains to compute gb1,0 and gb0,2 . Since h1 (x) = x and h2 (x) = (x2 − 1)/ 2, 1 ϕb1 = E[G2 1(G > 0)] = , 2

1 1 ϕb2 = √ E[(G3 − G) 1(G > 0)] = √ . 2 π 2

Thus

b2 gb0,2 = 2ϕb2 b2 = √ ̸= 0, π where a > 0 and b > 0 because α ∈ (0, π/2]. Hence, along the ordering z1 then z2 , the leap to z1 is 1, and the leap to z2 is 2 because gbm,1 = 0 for every m while gb0,2 ̸= 0. The leap exponent is therefore 2. ■ gb1,0 = 2ϕb1 a = a ̸= 0,

Remark A.7 (Unequal output weights). The argument above uses that the two teacher neurons have equal output weights. If instead c1 ̸= c2 ,

y = c1 ϕ(⟨v1 , x⟩) + c2 ϕ(⟨v2 , x⟩),

then y = c1 ϕ(az1 + bz2 ) + c2 ϕ(az1 − bz2 ), and Stein’s lemma gives b E[z2 y] = (c1 − c2 ) ̸= 0. 2 The splitting coordinate then appears at degree 1, so the leap-2 structure is specific to the symmetric teacher considered here.

Correlation-loss landscape

B.1

Reduction to one student in span{v1 , . . . , vk }

The correlation loss (3.2) is separable across student neurons, so it is enough to analyze one student on the unit sphere. For a single student w ∈ Sd−1 , (A.1a) gives, with l as in (A.1b), Lcorr (w) = −

k X

l(∠(w, vi )).

i=1

We also write g(t) = l(arccos t) for t ∈ [−1, 1], so that Lcorr (w) = − √ t(π − arccos t) + 1 − t2 . g(t) = 2π

i=1 g(⟨w, vi ⟩) and

(B.1)

A direct computation gives π − arccos t , 2π

1 > 0 (t ∈ (−1, 1)). (B.2) 2π 1 − t2 P P We next compute ⟨v̄, vi ⟩, where v̄ = i vi /∥ i vi ∥2 is the direction in Theorem 3.1. Since ⟨vi , vj ⟩ = cos α for i ̸= j, g ′ (t) =

k X i=1

g ′′ (t) =

√

= k + k(k − 1) cos α = k(1 + (k − 1) cos α),

vi 2

vi ,

k X

= 1 + (k − 1) cos α.

j=1

Therefore the angle ρ := ∠(v̄, vi ) is the same for every i, with r 1 + (k − 1) cos α cos ρ = . k

(B.3)

Under the setting of Theorem 3.1, we have 1 + (k − 1) cos α > 0: this is immediate when α ≤ π/2, and for k = 2 it follows from α < π. Since also α > 0, we have ρ ∈ (0, π/2). Moreover, Lcorr (v̄) = −kg(cos ρ). Lemma B.1 (Minimizers lie in span{v1 , . . . , vk }). Under the setting of Theorem 3.1, every minimizer of Lcorr on Sd−1 lies in span{v1 , . . . , vk }.

Proof of Lemma B.1. P Let w∗ be a minimizer of Lcorr on the unit sphere. At a unit vector w, the gradient of (3.2) is − ki=1 g ′ (⟨w, vi ⟩)vi plus a multiple of w, so the Lagrange multiplier condition gives, for some λ ∈ R, k X g ′ (⟨w∗ , vi ⟩)vi = λw∗ . i=1

The left-hand side lies ̸ 0. If λ = 0, taking the Pk in span{v1 , . . . , vk }, so it suffices to show that λ = inner product with j=1 vj gives 0=

k X

X k k X g (⟨w∗ , vi ⟩) vi , vj = (1 + (k − 1) cos α) g ′ (⟨w∗ , vi ⟩). ′

i=1

j=1

i=1

Because 1 + (k − 1) cos α > 0 and g ′ ≥ 0, this forces g ′ (⟨w∗ , vi ⟩) = 0 for every i. Since g ′ (t) = 0 only at t = −1, we get w∗ = −vi for all i, which contradicts α > 0. Hence λ ̸= 0 and w∗ ∈ span{v1 , . . . , vk }. ■ It remains to compare Lcorr (w) with Lcorr (v̄) for unit vectors w ∈ span{v1 , . . . , vk }. We do this for 0 < α ≤ π/2 in Lemmas B.3 and B.4, and for k = 2 and π/2 < α < π in Lemma B.5.

B.2

The case α ≤ π/2

For 0 < α ≤ π/2 and general k, we treat ρ ≥ π/4 and ρ ≤ π/4 separately. The first case relies on the following scalar inequality for g. Lemma B.2. Let ρ ∈ [π/4, π/2), and let g be as in (B.1). Then g(y) ≤ g(cos ρ) +

π−ρ 2 (y − cos2 ρ), 4π cos ρ

y ∈ [−1, 1],

with equality only at y = cos ρ. Proof of Lemma B.2. Write y = cos θ with θ ∈ [0, π]. Since g(cos θ) = l(θ), where l is defined in (A.1b), the inequality is equivalent to Rρ (θ) ≥ 0, where Rρ (θ) := l(ρ) +

π−ρ (cos2 θ − cos2 ρ) − l(θ). 4π cos ρ

Note that Rρ (ρ) = 0. By (A.2), Rρ′ (θ) =

sin θ Hρ (θ), 2π

Hρ (θ) := (π − θ) −

π−ρ cos θ. cos ρ

We show that Hρ < 0 on (0, ρ) and Hρ > 0 on (ρ, π). On [0, ρ], we have Hρ (ρ) = 0 and Hρ (0) = π − (π − ρ)/ cos ρ, which is negative if and only √ if π cos ρ < π − ρ. The function ρ 7→ π − ρ − π cos ρ is positive at ρ = π/4, where it equals 3π/4 − π/ 2, √ and its derivative −1 + π sin ρ ≥ −1 + π/ 2 is positive on [π/4, π/2). Hence Hρ (0) < 0. Moreover, Hρ′ (θ) = −1 +

π−ρ sin θ, cos ρ

Hρ′′ (θ) =

π−ρ cos θ, cos ρ

so Hρ is convex on [0, π/2]. Since Hρ (0) < 0 = Hρ (ρ), convexity gives Hρ < 0 on (0, ρ). 28

On (ρ, π/2], we use Hρ′ (ρ) = −1 + (π − ρ) tan ρ. The function ρ 7→ (π − ρ) tan ρ equals 3π/4 > 1 at ρ = π/4 and is increasing on [π/4, π/2), since (π − ρ) − sin ρ cos ρ d (π − ρ) tan ρ = >0 dρ cos2 ρ because π − ρ > π/2 > sin ρ cos ρ. Hence Hρ′ (ρ) > 0. By convexity, Hρ′ is nondecreasing on [0, π/2], so Hρ′ > 0 on [ρ, π/2] and Hρ (θ) > Hρ (ρ) = 0 for ρ < θ ≤ π/2. On [π/2, π), we have cos θ ≤ 0 and π − θ > 0, so Hρ (θ) > 0. Since sin θ > 0 on (0, π), the function Rρ is strictly decreasing on [0, ρ] and strictly increasing on [ρ, π]. Thus Rρ (θ) > Rρ (ρ) = 0 for θ ̸= ρ. ■ The case ρ ≥ π/4 now follows from Lemma B.2. Lemma B.3. Suppose that 0 < α ≤ π/2 and ρ ≥ π/4, with ρ as in (B.3). Then Lcorr (w) ≥ Lcorr (v̄) for every unit vector w ∈ span{v1 , . . . , vk }, with equality only at w = v̄. Proof of Lemma B.3. Applying Lemma B.2 to each ⟨w, vi ⟩ and summing over i gives Lcorr (w) − Lcorr (v̄) = kg(cos ρ) −

X i

π−ρ g(⟨w, vi ⟩) ≥ 4π cos ρ

! X k cos2 ρ − ⟨w, vi ⟩2

with only P if ⟨w, vi ⟩ = cos ρ for everyPi. The coefficient (π −ρ)/(4π cos ρ) is positive. Moreover, P equality 2 ⊤ ⊤ ⊤ 2 i ⟨w, vi ⟩ = w ( i vi vi )w. The matrix i vi vi has eigenvalue 1 + (k − 1) cos α = k cos ρ in the direction v̄ and eigenvalue 1 − cos α on the orthogonalPcomplement of v̄ inside span{v1 , . . . , vk }. Since cos α ≥ 0, we have 1 − cos α ≤ k cos2 ρ, and hence i ⟨w, vi ⟩2 ≤ k cos2 ρ. Therefore Lcorr (w) ≥ Lcorr (v̄). If Lcorr (w) = Lcorr (v̄), the inequality in the display is an equality, so ⟨w, vi ⟩ = cos ρ = ⟨v̄, vi ⟩ for all i. The Gram matrix (⟨vi , vj ⟩)i,j = (1 − cos α)I + cos α 11⊤ is positive definite because 0 ≤ cos α < 1, and both w and v̄ lie in span{v1 , . . . , vk }. Hence w = v̄. ■ When ρ ≤ π/4, we use a different argument: we write w in terms of its angle φ from v̄ and bound Lcorr (w) separately for φ ≥ ρ and φ ≤ ρ. Lemma B.4. Suppose that 0 < α ≤ π/2 and ρ ≤ π/4, with ρ as in (B.3). Then Lcorr (w) ≥ Lcorr (v̄) for every unit vector w ∈ span{v1 , . . . , vk }, with equality only at w = v̄. Proof of Lemma B.4. Let ṽi = (vi −cos ρ v̄)/ sin ρ be the unit vector in the direction of the component of vi orthogonal to v̄. These vectors satisfy vi = cos ρ v̄ + sin ρ ṽi ,

ṽi ⊥ v̄,

∥ṽi ∥2 = 1,

ṽi = 0,

We write

⟨ṽi , ṽj ⟩ = −

1 k−1

(i ̸= j). (B.4)

w = cos φ v̄ + sin φ u, where φ ∈ [0, π], u ⊥ v̄, ∥u∥2 = 1, and u ∈ span{ṽ1 , . . . , ṽk }. When sin φ = 0, the vector u is arbitrary, and we fix any such u. Let ti = ⟨u, ṽi ⟩ and zi = ⟨w, vi ⟩ = cos ρ cos φ + ti sin ρ sin φ. 29

With a slight abuse of notation, we write Lcorr (φ, u) := Lcorr (cos φ v̄ + sin φ u) = −

k X

g(zi ),

i=1

where g is defined in (B.1). In particular, Lcorr (0, u) = Lcorr (v̄). We note that the numbers ti satisfy X

ti = 0,

Indeed,

k , k−1

t2i =

−1 ≤ ti ≤ 1.

i ṽi ⟩ = 0, and |ti | ≤ 1 by Cauchy–Schwarz. For every j, (B.4) gives X 1 X k k ti ṽi , ṽj = tj − ti = tj = u, ṽj . k−1 k−1 k−1

i ti = ⟨u,

i̸=j

P P k Both i ti ṽi and u lie in span{ṽ1 , . . . , ṽk }, so i ti ṽi = k−1 u. Taking the inner product with u P 2 gives i ti = k/(k − 1). We first show that Lcorr (φ, u) is nondecreasing in φ on [ρ, π]. We have X X ∂ Lcorr (φ, u) = cos ρ sin φ g ′ (zi ) − sin ρ cos φ ti g ′ (zi ). ∂φ i

Since

i ti = 0, the identity

′ ′ i<j (ti − tj ) g (zi ) − g (zj )

ti g ′ (zi ) =

(B.5)

′ i ti g (zi ) −

i ti

′ i g (zi )

1X (ti − tj ) g ′ (zi ) − g ′ (zj ) ≥ 0, k

gives (B.6)

i<j

because g ′ is increasing by (B.2) and zi is nondecreasing in ti (recall sin ρ > 0 and sin φ ≥ 0). If φ ≥ π/2, then cos φ ≤ 0, so both terms in (B.5) are nonnegative, by g ′ ≥ 0 and (B.6). If ρ ≤ φ < π/2, then cos φ > 0 and tan φ ≥ tan ρ, so cot ρ tan φ ≥ 1 ≥ ti and X ∂ Lcorr (φ, u) = sin ρ cos φ (cot ρ tan φ − ti ) g ′ (zi ) ≥ 0. ∂φ i

Hence Lcorr (φ, u) is nondecreasing in φ on [ρ, π], and Lcorr (φ, u) ≥ Lcorr (ρ, u),

φ ≥ ρ.

It remains to consider 0 ≤ φ ≤ ρ. We first take 0 ≤ φ < ρ; the bound (B.7) obtained below then extends to φ = ρ by continuity. Since ρ ≤ π/4, we have ρ + φ < π/2, and for every t ∈ [−1, 1], 0 ≤ cos(ρ + φ) ≤ cos ρ cos φ + t sin ρ sin φ ≤ cos(ρ − φ) < 1. Hence the function hφ (t) = g(cos ρ cos φ + t sin ρ sin φ) is C 3 on [−1, 1]. Differentiating (B.2) once more gives g ′′′ (s) = s/(2π(1 − s2 )3/2 ), which is nonnegative for s ∈ [0, 1). By the chain rule, 3 ′′′ h′′′ φ (t) = (sin ρ sin φ) g (cos ρ cos φ + t sin ρ sin φ) ≥ 0,

t ∈ [−1, 1].

Let s0 = −1/(k − 1) ∈ [−1, 0), and let Qφ be the quadratic polynomial Qφ (t) = hφ (s0 ) + h′φ (s0 )(t − s0 ) +

hφ (1) − hφ (s0 ) − h′φ (s0 )(1 − s0 ) (t − s0 )2 , (1 − s0 )2 30

which satisfies Qφ (1) = hφ (1), Qφ (s0 ) = hφ (s0 ), and Q′φ (s0 ) = h′φ (s0 ). For t ∈ [−1, 1], the Hermite interpolation remainder with a double node at s0 and a simple node at 1 gives h′′′ φ (ξ) (t − s0 )2 (t − 1) 6 ′′′ for P some ξ ∈P[−1, 1]. Since hφ (ξ) ≥ 0 and t − 1 ≤ 0, we have hφ ≤ Qφ on [−1, 1], and hence i hφ (ti ) ≤ i Qφ (ti ). P P P Since Qφ is quadratic, i Qφ (ti ) depends on the ti only through i ti = 0 and i t2i = k/(k − 1). The k numbers 1, s0 , . . . , s0 have the same two sums, because 1 + (k − 1)s0 = 0 and 1 + (k − 1)s20 = k/(k − 1). Therefore X −Lcorr (φ, u) = hφ (ti ) ≤ Qφ (1) + (k − 1)Qφ (s0 ) = hφ (1) + (k − 1)hφ (s0 ) = −Lcorr (φ, ṽ1 ), (B.7) hφ (t) − Qφ (t) =

where the last equality holds because u = ṽ1 gives t1 = 1 and tj = s0 for j ̸= 1, by (B.4). Thus, for each φ ∈ [0, ρ], Lcorr (φ, ·) is minimized at u = ṽ1 . For u = ṽ1 , we have z1 = cos(ρ − φ), and zj = z2 for every j ≥ 2, where sin ρ sin φ , Lcorr (φ, ṽ1 ) = −g cos(ρ − φ) − (k − 1)g(z2 ). (B.8) z2 = cos ρ cos φ − k−1 We now show that Lcorr (φ, ṽ1 ) > Lcorr (v̄) for 0 < φ ≤ ρ. Since 0 ≤ φ ≤ ρ ≤ π/4, (B.9)

0 ≤ cos(ρ + φ) ≤ z2 ≤ cos ρ.

Since the arccosine is decreasing, (B.9) gives ρ ≤ arccos z2 ≤ ρ + φ. We write δ(φ) = arccos z2 − ρ, so that 0 ≤ δ(φ) ≤ φ. By (B.2), π−ρ+φ π − ρ − δ(φ) g ′ cos(ρ − φ) = , g ′ (z2 ) = . 2π 2π Differentiating (B.8) therefore gives 1 h ∂ Lcorr (φ, ṽ1 ) = − (π − ρ + φ) sin(ρ − φ) ∂φ 2π sin ρ cos φ i + (k − 1) π − ρ − δ(φ) cos ρ sin φ + k−1 h i 1 = cos ρ sin φ [k(π − ρ) + φ − (k − 1)δ(φ)] − sin ρ cos φ φ + δ(φ) . (B.10) 2π The bracket in (B.10) is an affine function of δ(φ) with slope −(k − 1) cos ρ sin φ − sin ρ cos φ < 0. Since δ(φ) ≤ φ, replacing δ(φ) by φ gives ∂ 1 Lcorr (φ, ṽ1 ) ≥ [cos ρ sin φ [k(π − ρ) − (k − 2)φ] − 2φ sin ρ cos φ] . ∂φ 2π

(B.11)

For 0 < φ ≤ ρ, dividing by cos ρ sin φ > 0 shows that the right-hand side of (B.11) is positive whenever 2 tan ρ · φ cot φ < k(π − ρ) − (k − 2)φ. This inequality holds: since φ cot φ ≤ 1 and ρ ≤ π/4, the left-hand side is at most 2 tan ρ ≤ 2, while the right-hand side is at least k(π − ρ) − (k − 2)ρ = kπ − 2(k − 1)ρ ≥ (k + 1)π/2 ≥ 3π/2 > 2. Hence Lcorr (φ, ṽ1 ) is strictly increasing on [0, ρ], and Lcorr (φ, ṽ1 ) > Lcorr (0, ṽ1 ) = Lcorr (v̄),

0 < φ ≤ ρ.

Combining the two ranges with (B.7), Lcorr (φ, u) ≥ Lcorr (φ, ṽ1 ) > Lcorr (v̄) for 0 < φ ≤ ρ, and Lcorr (φ, u) ≥ Lcorr (ρ, u) ≥ Lcorr (ρ, ṽ1 ) > Lcorr (v̄) for φ ≥ ρ. Thus Lcorr (w) ≥ Lcorr (v̄), with equality only when φ = 0, that is, w = v̄. ■ 31

B.3

The case k = 2 and α > π/2

For k = 2, we have ρ = α/2. An obtuse α therefore gives cos α < 0, which is outside the range of Lemma B.3. In this case span{v1 , v2 } = span{v̄, v∆ }, and we compare Lcorr along its unit circle directly. Lemma B.5. Let k = 2 and π/2 < α < π, and let α⋆ be as in (3.3). (a) If α ≤ α⋆ , then v̄ is the unique minimizer of Lcorr on the unit sphere. (b) If α > α⋆ , then v̄ is not a local minimizer of Lcorr on the unit sphere. Proof of Lemma B.5. Recall from (2.4) that v̄ and v∆ form an orthonormal basis of the teacher plane, with v1 = cos(α/2) v̄ + sin(α/2) v∆ , v2 = cos(α/2) v̄ − sin(α/2) v∆ . As in the proof of Lemma B.4, we write Lcorr (φ, v∆ ) = Lcorr (cos φ v̄ + sin φ v∆ ). For 0 ≤ φ ≤ π/2, with l as in (A.1b), Lcorr (φ, v∆ ) = −l(|α/2 − φ|) − l(α/2 + φ). For 0 < φ < α/2, direct differentiation gives ∂ cos(α/2) sin φ Lcorr (φ, v∆ ) = [(π − α/2) − tan(α/2) · φ cot φ] . ∂φ π

(B.12)

We first prove (a). Since arccos(−t) = π − arccos t, the formula (B.1) gives g(t) − g(−t) = t/2 for t ∈ [−1, 1]. Consequently, 1 Lcorr (−w) − Lcorr (w) = ⟨w, v1 + v2 ⟩ = cos(α/2) ⟨w, v̄⟩. 2 By Lemma B.1, every minimizer lies in span{v̄, v∆ }. By the identity above, a minimizer cannot have ⟨w, v̄⟩ < 0, and reflection across v̄ leaves Lcorr unchanged while changing the sign of ⟨w, v∆ ⟩. It therefore suffices to show that Lcorr (φ, v∆ ) > Lcorr (v̄) for 0 < φ ≤ π/2. The definition (3.3) of α⋆ , together with the monotonicity of x 7→ tan x + x, gives tan(α/2) ≤ π − α/2. Since φ cot φ < 1, the derivative (B.12) is positive for 0 < φ < α/2. For α/2 < φ ≤ π/2, ∂ Lcorr (φ, v∆ ) = −l′ (φ − α/2) − l′ (φ + α/2) > 0 ∂φ by (A.2), since 0 < φ + α/2 < π. The one-sided derivatives agree at φ = α/2 because l′ (0) = 0. Hence Lcorr (φ, v∆ ) > Lcorr (v̄) for every φ ∈ (0, π/2]. We now prove (b). Since α > α⋆ , we have tan(α/2) > π − α/2. Differentiating (B.12) at φ = 0 gives ∂2 (π − α/2) cos(α/2) − sin(α/2) Lcorr (φ, v∆ ) = < 0, 2 ∂φ π φ=0 while the first derivative vanishes at φ = 0. Hence Lcorr (φ, v∆ ) < Lcorr (v̄) for small φ > 0, so v̄ is not a local minimizer of Lcorr . ■

B.4

Proof of Theorem 3.1

Proof of Theorem 3.1. Since Lcorr is continuous on the compact unit sphere, it has a minimizer, and by Lemma B.1 every minimizer lies in span{v1 , . . . , vk }. If 0 < α ≤ π/2, then Lemma B.3 (when ρ ≥ π/4) or Lemma B.4 (when ρ ≤ π/4) shows that v̄ is the only unit vector in span{v1 , . . . , vk } attaining the minimum of Lcorr . If k = 2 and π/2 < α ≤ α⋆ , the same conclusion is part (a) of Lemma B.5. In either case, v̄ is the unique minimizer of Lcorr on the unit sphere. For unit-norm w1 , . . . , w m , m X Lcorr (w1 , . . . , wm ) = Lcorr (wj ) ≥ mLcorr (v̄), j=1

with equality if and only if wj = v̄ for every j. Thus (v̄, . . . , v̄) is the unique global minimum.

Squared-loss landscape

C.1

Stationarity and dimension reduction

■

The proofs below use the condition α ≤ α⋆ in the form sin(α/2) ≤ (π − α/2) cos(α/2). The following lemma shows that the two are equivalent. Lemma C.1 (Equivalent angle threshold). The equation tan(α/2) = π − α/2 has exactly one solution in (π/2, π), namely α⋆ in (3.3). Moreover, for every α ∈ (0, π), the following conditions are equivalent: (a) α ≤ α⋆ ; (b) tan(α/2) ≤ π − α/2; (c) sin(α/2) ≤ (π − α/2) cos(α/2). Proof of Lemma C.1. The function α 7→ tan(α/2) + α/2 − π is strictly increasing on (0, π), since its derivative is sec2 (α/2) + 1 /2 > 0. It equals 1 − 3π/4 < 0 at α = π/2 and tends to +∞ as α ↑ π. Thus it has a unique root α⋆ ∈ (π/2, π) and is nonpositive exactly on (0, α⋆ ], so (a) and (b) are equivalent. Since cos(α/2) > 0, multiplying (b) by cos(α/2) shows that (b) and (c) are equivalent. ■ C.1.1

Stationary equations

We first derive the stationarity equations (3.4) from the gradient-flow equations (A.11)–(A.12). dw1 dw2 i Proof of Lemma 3.3. Since dw dt = −∇wi L, the pair (w1 , w2 ) is stationary if and only if dt = dt = 0 1 in (A.11)–(A.12). Setting dw dt = 0 in (A.11), multiplying by 2π, and using wi = ∥wi ∥2 w̄i gives

(π − θ11 )v1 + (π − θ12 )v2 + a1 w̄1 − (π − γ)∥w2 ∥2 w̄2 = 0, 2 which is the first identity in (3.4). Setting dw dt = 0 in (A.12) similarly yields the second identity. These steps can be reversed, so (3.4) holds if and only if (w1 , w2 ) is stationary. ■

C.1.2

Uniqueness of the out-of-plane root

The out-of-plane saddles in Theorem 3.5 are determined by a root of Fα in (0, π/2), so we next show that this root exists and is unique. Proof of Lemma 3.4. Fix α ∈ (0, π). By (3.6), θα (κ) = arccos cos(α/2) cos κ , so for 0 < κ < π/2 we have θα (κ) < π/2 and 0 < θα′ (κ) = cos(α/2) sin κ/ sin θα (κ) < 1, where the last inequality follows

from sin2 θα (κ) − cos2 (α/2) sin2 κ = sin2 (α/2) > 0. Using sin(2κ) = 2 sin κ cos κ, we first rewrite (3.7) as 2 sin θα (κ) sin θα (κ) rα (κ) = = . sin(2κ) + 2κ κ + sin κ cos κ Substituting this expression into (3.8) yields (π − 2κ) sin θα (κ) cos κ Fα (κ) = π − θα (κ) cos(α/2) − . κ + sin κ cos κ The fraction in the second term is strictly positive because 0 < κ < π/2 and 0 < θα (κ) < π/2. Dividing by this fraction shows that Fα (κ) = 0 if and only if Qα (κ) = 1, where cos(α/2) π − θα (κ) (κ + sin κ cos κ) Qα (κ) = (π − 2κ) sin θα (κ) cos κ π − θα (κ) κ + sin κ cos κ sin κ (C.1) = cos(α/2) π − 2κ sin κ cos κ sin θα (κ) π − θα (κ) 2κ sin κ = cos(α/2) 1+ . π − 2κ sin(2κ) sin θα (κ) We can show that each of the three positive factors after cos(α/2) in the last line is strictly increasing in κ. For the first factor, 2 π − θα (κ) − (π − 2κ)θα′ (κ) d π − θα (κ) 2κ = > > 0. 2 dκ π − 2κ (π − 2κ) (π − 2κ)2 Here π − 2κ > 0, θα′ (κ) < 1, and θα (κ) < π/2 give 2 π − θα (κ) − (π − 2κ)θα′ (κ) > 2 π − θα (κ) − (π − 2κ) = π − 2θα (κ) + 2κ > 2κ. For the second factor, 0 < 2κ < π implies t sin t > 0 for 0 < t < 2κ, so integration by parts gives Z 2κ 0< t sin t dt = (−t cos t + sin t)|2κ 0 = sin(2κ) − 2κ cos(2κ). 0

Therefore

2 sin(2κ) − 2κ cos(2κ) = > 0. sin2 (2κ) −1/2 For the third factor, the identity sin κ/ sin θα (κ) = 1 + sin2 (α/2) cot2 κ and sin2 (α/2) > 0 give strict monotonicity. Hence Qα in (C.1) is continuous and strictly increasing on (0, π/2). Finally, as κ ↓ 0, we have θα (κ) → α/2, so the third factor tends to 0 while the first two stay bounded. As κ ↑ π/2, we have θα (κ) → π/2, so the first two factors tend to +∞ while the third tends to 1. Hence lim Qα (κ) = 0, lim Qα (κ) = +∞. d dκ

2κ 1+ sin(2κ)

κ↓0

κ↑π/2

The intermediate value theorem and strict monotonicity therefore give exactly one solution of Qα (κ) = 1, and thus exactly one root of Fα in (0, π/2). ■

C.1.3

Reduction to three dimensions

The teacher terms in (3.4) lie in the teacher plane, so the components of (3.4) orthogonal to this plane involve only the students. These components give the dimension reduction in Proposition 3.7. Proof of Proposition 3.7. Projecting the two equations in (3.4) onto the orthogonal complement of span{v1 , v2 } gives a1 w̄1⊥ − (π − γ)∥w2 ∥2 w̄2⊥ = 0 and a2 w̄2⊥ − (π − γ)∥w1 ∥2 w̄1⊥ = 0. If γ < π, then (π − γ)∥w2 ∥2 > 0, so the first identity expresses w̄2⊥ as a multiple of w̄1⊥ , possibly zero. If γ = π, then w̄2 = −w̄1 , so w̄2⊥ = −w̄1⊥ . In either case w̄1⊥ and w̄2⊥ are linearly dependent, and hence so are w1⊥ and w2⊥ . Thus w1 and w2 lie in the span of the teacher plane and at most one further direction, which gives dim span{v1 , v2 , w1 , w2 } ≤ 3. ■

C.2

Three-dimensional symmetry reduction

C.2.1

The teacher force

The angle between v̄ and each teacher is α/2, so v2 = cos(α/2) v̄ − sin(α/2) v∆ .

v1 = cos(α/2) v̄ + sin(α/2) v∆ ,

For a unit vector x, we define its teacher force by T (x) = π − ∠(x, v1 ) v1 + π − ∠(x, v2 ) v2 . The teacher terms in the ith equation of (3.4) are T (w̄i ). The proof of Lemma 3.8 relies on the following comparison between the directions of T (x) and x. Lemma C.2 (Direction of the teacher force). Assume 0 < α ≤ α⋆ , and let x ∈ Rd satisfy ∥x∥2 = 1. Write p = ⟨x, v̄⟩, q = ⟨x, v∆ ⟩, and xP = pv̄ + qv∆ . Then ⟨T (x), v̄⟩ > 0 and sgn⟨T (x), v∆ ⟩ = sgn q. If p > 0 and q ̸= 0, the teacher force lies strictly closer to the bisector than xP : 0<

⟨T (x), v∆ ⟩ q < ⟨T (x), v̄⟩ p

q ⟨T (x), v∆ ⟩ < <0 p ⟨T (x), v̄⟩

if q > 0,

if q < 0.

Proof of Lemma C.2. We write H(t) = π − arccos t for −1 ≤ t ≤ 1. This function is strictly increasing and nonnegative, and it vanishes only at t = −1. Since x and vj are unit vectors, π − ∠(x, vj ) = H(⟨x, vj ⟩) for j = 1, 2, and the decomposition of v1 , v2 in the basis v̄, v∆ gives ⟨x, v1 ⟩ = p cos(α/2) + q sin(α/2),

⟨x, v2 ⟩ = p cos(α/2) − q sin(α/2).

Substituting into the definition of T (x) and taking inner products with v̄ and v∆ yields ⟨T (x), v̄⟩ = cos(α/2) H(p cos(α/2) + q sin(α/2)) + H(p cos(α/2) − q sin(α/2)) , ⟨T (x), v∆ ⟩ = sin(α/2) H(p cos(α/2) + q sin(α/2)) − H(p cos(α/2) − q sin(α/2)) . Since cos(α/2) > 0 and H ≥ 0, the first quantity is nonnegative. It vanishes only if ⟨x, v1 ⟩ = ⟨x, v2 ⟩ = −1, that is, only if x = −v1 = −v2 , which is impossible because v1 ̸= v2 . Hence ⟨T (x), v̄⟩ > 0. In the second quantity, the two arguments of H differ by 2q sin(α/2), and sin(α/2) > 0, so the strict monotonicity of H shows that ⟨T (x), v∆ ⟩ has the sign of q. It remains to prove the bounds on the ratio when p > 0 and q ̸= 0. By reflection across the v̄-axis, we may assume q > 0. The lower bound then follows from the sign of ⟨T (x), v∆ ⟩, and since ⟨T (x), v̄⟩ > 0 and p > 0, the upper bound is equivalent to q⟨T (x), v̄⟩ − p⟨T (x), v∆ ⟩ > 0. We write 35

xP in polar form, xP = ρ(cos φ v̄ + sin φ v∆ ) with 0 < ρ ≤ 1 and 0 < φ < π/2, so that p = ρ cos φ and q = ρ sin φ. Then p cos(α/2) + q sin(α/2) = ρ cos(φ − α/2),

p cos(α/2) − q sin(α/2) = ρ cos(φ + α/2).

Substituting these identities into the formulas for ⟨T (x), v̄⟩ and ⟨T (x), v∆ ⟩ and using the angleaddition formulas, we obtain h i q⟨T (x), v̄⟩ − p⟨T (x), v∆ ⟩ = cos(α/2) H ρ cos(φ − α/2) + H ρ cos(φ + α/2) sin φ ρ h i − sin(α/2) H ρ cos(φ − α/2) − H ρ cos(φ + α/2) cos φ = H ρ cos(φ − α/2) (cos(α/2) sin φ − sin(α/2) cos φ) + H ρ cos(φ + α/2) (cos(α/2) sin φ + sin(α/2) cos φ) = H ρ cos(φ − α/2) sin(φ − α/2) + H ρ cos(φ + α/2) sin(φ + α/2). If φ ≥ α/2, both terms on the right are nonnegative and the second is positive, so the expression is positive. Suppose now that 0 < φ < α/2, and set x0 = α/2 − φ and y0 = α/2 + φ, so that 0 < x0 < y0 < α < π. Since sin(φ − α/2) = − sin x0 and sin(φ + α/2) = sin y0 , the expression is positive exactly when H(ρ cos x0 ) sin x0 < H(ρ cos y0 ) sin y0 . (C.2) √ To compare the two sides, we write R(ρ) = H(ρ cos x0 )/H(ρ cos y0 ) and K(u) = u/ 1 − u2 H(u) . For −1 < u < 1, √ H(u) − u 1 − u2 ′ K (u) = . H(u)2 (1 − u2 )3/2 The numerator is positive: for u < 0 both of its terms are positive, and for u ≥ 0 we have √ H(u) ≥ π/2 > 1/2 ≥ u 1 − u2 . Thus K is strictly increasing. Since cos x0 > cos y0 , it follows that d K(ρ cos x0 ) − K(ρ cos y0 ) log R(ρ) = > 0, dρ ρ and therefore R(ρ) ≤ R(1) =

π − x0 . π − y0

(C.3)

It therefore suffices to show that (π − x0 ) sin x0 < (π − y0 ) sin y0 . Using x0 = α/2 − φ and y0 = α/2 + φ, we have (π − y0 ) sin y0 − (π − x0 ) sin x0 = 2 (π − α/2) cos(α/2) sin φ − φ sin(α/2) cos φ . By Lemma C.1, (π − α/2) cos(α/2) ≥ sin(α/2), so the right-hand side is at least 2 sin(α/2)(sin φ − φ cos φ), which is positive because tan φ > φ. Combined with (C.3), this gives (C.2), and hence the upper bound for q > 0. The case q < 0 follows by reflection. ■ C.2.2

Elimination of the splitting coordinate

Proof of Lemma 3.8. By Proposition 3.7, there exists a unit vector ξ ⊥ span{v̄, v∆ } such that w̄1 , w̄2 ∈ span{v̄, v∆ , ξ}. We write w̄i = pi v̄ + qi v∆ + ηi ξ and Ti = T (w̄i ) for i = 1, 2. Since one of the students lies outside the teacher plane, at least one ηi is nonzero. By Lemma C.2, ⟨Ti , v̄⟩ > 0,

sgn⟨Ti , v∆ ⟩ = sgn qi . 36

(C.4)

Each Ti lies in the teacher plane, so it has no ξ-component. We first show that 0 < γ < π. If γ = 0, then w̄1 = w̄2 , and the first equation in (3.4) becomes T1 + (a1 − π∥w2 ∥2 )w̄1 = 0. Its ξ-coordinate shows that the coefficient of w̄1 vanishes, so T1 = 0, which contradicts (C.4). If γ = π, then w̄2 = −w̄1 , both ηi are nonzero, and the equations become Ti + ai w̄i = 0. Their ξ-coordinates give ai = 0, and hence again Ti = 0. Thus (C.5)

0 < γ < π.

We next show that η1 η2 ̸= 0. If, for instance, η1 = 0, then η2 = ̸ 0, and the ξ-coordinate of the first equation in (3.4) gives (π − γ)∥w2 ∥2 η2 = 0, which contradicts (C.5). The case η2 = 0 is the same. Let t = η2 /η1 . The ξ-coordinates of the two equations in (3.4) give a1 = (π − γ)∥w2 ∥2 t and a2 = (π − γ)∥w1 ∥2 /t, and substituting these values back into (3.4) yields w̄2 = tw̄1 +

1 T1 , (π − γ)∥w2 ∥2

T2 = −

∥w1 ∥2 T1 . t∥w2 ∥2

(C.6)

Since T1 and T2 both have positive v̄-coordinate, the coefficient in the second identity is positive. Hence t < 0, and T2 is a positive multiple of T1 . By (C.4), the numbers q1 and q2 are then either both zero or both nonzero with the same sign. It remains to rule out the second possibility. Suppose that q1 and q2 are nonzero with the same sign. By reflection across the v̄-axis, we may assume q1 , q2 > 0, so that ⟨T1 , v∆ ⟩ > 0. Taking the v̄and v∆ -coordinates of the first identity in (C.6) gives p2 = tp1 +

⟨T1 , v̄⟩ , (π − γ)∥w2 ∥2

q2 = tq1 +

⟨T1 , v∆ ⟩ . (π − γ)∥w2 ∥2

(C.7)

If p1 ≤ 0, the first identity gives p2 > 0, because t < 0 and ⟨T1 , v̄⟩ > 0. After exchanging the students if necessary, we may therefore assume p1 > 0. Then Lemma C.2, applied to w̄1 , gives ⟨T1 , v∆ ⟩/⟨T1 , v̄⟩ < q1 /p1 , that is, q1 ⟨T1 , v̄⟩ − p1 ⟨T1 , v∆ ⟩ > 0. Substituting (C.7), we obtain ⟨T1 , v∆ ⟩ ⟨T1 , v̄⟩ q2 ⟨T1 , v̄⟩ − p2 ⟨T1 , v∆ ⟩ = ⟨T1 , v̄⟩ tq1 + − ⟨T1 , v∆ ⟩ tp1 + (π − γ)∥w2 ∥2 (π − γ)∥w2 ∥2 = t(q1 ⟨T1 , v̄⟩ − p1 ⟨T1 , v∆ ⟩) < 0, (C.8) where the last inequality uses t < 0. Since ⟨T1 , v̄⟩, ⟨T1 , v∆ ⟩, and q2 are positive, this inequality forces p2 > 0. We may then apply Lemma C.2 to w̄2 . Since T2 is a positive multiple of T1 , it gives ⟨T1 , v∆ ⟩ ⟨T2 , v∆ ⟩ q2 = < , ⟨T1 , v̄⟩ ⟨T2 , v̄⟩ p2 that is, q2 ⟨T1 , v̄⟩ − p2 ⟨T1 , v∆ ⟩ > 0. This contradicts (C.8). Hence q1 = q2 = 0, and w1 , w2 ∈ span{v̄, ξ}. ■

C.3

Out-of-plane stationary points

By Lemma 3.8, a stationary point with a student outside the teacher plane lies in span{v̄, ξ}. For such points, we first show that the two students are mirror images across the teacher plane, then reduce stationarity to the equation Fα (κ) = 0, and finally find a direction of negative curvature. Together these steps prove Lemma 3.9. 37

C.3.1

A scalar inequality

In the proof of Lemma C.4 below, an out-of-plane stationary point that is not symmetric across the teacher plane leads to two scalar identities. The following inequality shows that these identities are incompatible. Lemma C.3 (A scalar inequality). Let 0 < t < m < π/2, and put X =t+

2(π − 2m) t − cos2 m tan t , 2m − sin 2m

Y =t+

2(π − 2m) t + sin2 m cot t . 2m + sin 2m

If λ > 0 satisfies λX = π/2 + arctan(λ tan t/ tan2 m), then λX > Y /λ. Proof of Lemma C.3. We fix m and write a = m − sin m cos m, b = m + sin m cos m, P (t) = t − cos2 m tan t, and Q(t) = t + sin2 m cot t. Then a, b > 0, X = t + (π − 2m)P (t)/a, and Y = t + (π − 2m)Q(t)/b. We have X > 0 and Y > 0, since P (t) > t − cos2 t tan t = t − sin t cos t > 0 and Q(t) > 0. We first collect three bounds on P and Q. Since P ′ (t) = 1 − cos2 m sec2 t > 0 and Q′ (t) = 1 − sin2 m csc2 t < 0 on 0 < t < m, and since P (m) = a and Q(m) = b, we have 0 < P (t) < a and Q(t) > b. The product P (t)Q(t) is also increasing on (0, m), because tan2 m − tan2 t d P (t) tan2 m − tan2 t tan t − t + t tan2 t P (t)Q(t) = Q(t) − = > 0. 2 2 dt 1 + tan m tan t 1 + tan2 m tan2 t Hence P (t)Q(t) < P (m)Q(m) = ab. Finally, the function t(P (t)/a + Q(t)/b) equals 2m at t = m, and we claim that it is increasing on (0, m), so that t(P (t)/a + Q(t)/b) < 2m. Its derivative is d P (t) Q(t) 2t − cos2 m(tan t + t sec2 t) 2t + sin2 m(cot t − t csc2 t) t + = + . (C.9) dt a b a b For the first numerator in (C.9), the inequalities 0 < t < m < π/2 give cos2 m < cos2 t. Since tan t + t sec2 t > 0, it follows that 2t − cos2 m(tan t + t sec2 t) > 2t − cos2 t(tan t + t sec2 t) = t − sin t cos t > 0. For the second numerator, the inequality t − sin t cos t > 0 gives cot t − t csc2 t = (sin t cos t − t)/ sin2 t < 0. Since 0 < sin2 m < 1, multiplying this negative quantity by sin2 m makes it strictly larger. Therefore 2t + sin2 m(cot t − t csc2 t) > 2t + cot t − t csc2 t =

sin t cos t − t cos 2t > 0. sin2 t

For the last inequality, the numerator vanishes at t = 0 and its derivative is 2t sin 2t > 0 on (0, π/2). Both numerators in (C.9) are therefore positive, which proves the claim. Expanding XY and using P (t)Q(t) < ab and t(P (t)/a + Q(t)/b) < 2m, we obtain P (t) Q(t) 2 2 (π − m) − XY = (π − m) − t + (π − 2m) t + (π − 2m) a b P (t) Q(t) P (t)Q(t) 2 2 2 = m − t + (π − 2m) 2m − t + + (π − 2m) 1 − a b ab > 0. 38

√ Therefore XY < π − m. We also need the bound

t(aQ(t) − bP (t)) π < , ab − P (t)Q(t) 2

(C.10)

in which the denominator is positive and so is the numerator, since aQ(t) − bP (t) > ab − bP (t) = b(a − P (t)) > 0. Because b = m + 12 sin 2m < π/2, it is enough to show that the left-hand side is less than b, that is, b(ab − P (t)Q(t)) − t(aQ(t) − bP (t)) > 0. A direct expansion gives b(ab − P (t)Q(t)) − t(aQ(t) − bP (t)) = bm2 − at2 − 2m sin2 m t cot t. At t = m, the right-hand side of (C.11) is zero. Its derivative with respect to t is t − sin t cos t − a . 2t m sin2 m t sin2 t

(C.11)

(C.12)

For 0 < u < π/2, the function u 7→ (u − sin u cos u)/(u sin2 u) is strictly increasing, since " # d u − sin u cos u cos u tan u sin u 2 = + −2 >0 du u u u sin2 u sin3 u by Wilker’s inequality (Sumner et al., 1991). Since t < m, the bracketed factor m sin2 m (t − m cos m sin t cos t)/(t sin2 t) − a in (C.12) is strictly less than m sin2 m m−sin − a = 0. Thus the m sin2 m derivative in (C.12) is negative on (0, m). Since the right-hand side of (C.11) vanishes at t = m, it is positive for 0 < t < m, which proves (C.10). The conclusion of the lemma will follow from the inequality r ! √ tan2 m X XY + arctan < π. (C.13) tan t Y √ √ If XY ≤ √ π/2, this holds because the arctangent√is less than π/2. We may therefore assume XY > π/2. Since XY < π−m, we then have m < π− XY < π/2. The tangent function is strictly convex √ on (0, π/2), so tan u > tan m+sec2 m (u−m) for every u ∈ (0, π/2) with u = ̸ m. Taking u = π− XY √ √ and using sec2 m = 1 + tan2 m gives tan(π − XY ) > tan m + (1 + tan2 m)(π − XY − m). Hence (C.13) follows once we show r √ tan2 m X ≤ tan m + (1 + tan2 m)(π − XY − m). (C.14) tan t Y p p √ Multiplying this inequality by cos2 m and using XY = Y X/Y , we can rewrite it as X/Y (Y + sin2 m cot t) ≤ π − m + sin m cos m. Since Y + sin2 m cot t equals t + sin2 m cot t times (π − m + sin m cos m)/(m + sin m cos m), this inequality is equivalent to 2 X t + sin2 m cot t ≤ Y (m + sin m cos m)2 . (C.15) √ To prove (C.15) when XY > π/2, we fix t and m and set, for µ ≥ π − 2m, Xµ = t + µP (t)/a and Yµ = t + µQ(t)/b. At µ = π − 2m, these are the original quantities X and Y , and (C.15) reads Yπ−2m b2 − Xπ−2m Q(t)2 ≥ 0. The expression Yµ b2 − Xµ Q(t)2 is affine in µ, with slope Q(t)(ab − P (t)Q(t)) d Yµ b2 − Xµ Q(t)2 = > 0. dµ a 39

Suppose that (C.15) fails. Then this affine expression is negative at µ = π − 2m, so it vanishes at exactly one µ∗ > π − 2m, where Xµ∗ Q(t)2 = Yµ∗ b2 . Solving this equation gives µ∗ = ta Q(t)2 − 2 b / Q(t)(ab − P (t)Q(t)) , and therefore b tb aQ(t) − bP (t) π Yµ = + µ∗ = t < Q(t) ∗ Q(t) ab − P (t)Q(t) 2 √ p by (C.10). Since Xµ and Yµ increase with µ, this gives XY < Xµ∗ Yµ∗ < π/2, contrary to our assumption. Hence p (C.15) holds, and √ so does (C.14). Together √ with the tangent-line bound, it gives (tan2 m/ tan t) X/Y < tan(π − XY ), and since 0 < π − XY < π/2, taking arctangents proves (C.13). We now derive the lemma from (C.13). Since arctan r + arctan(1/r) = π/2 for r > 0, the inequality (C.13) can be written as r ! √ π Y tan t XY < + arctan , 2 2 tan m X p Xµ∗ Yµ∗ =

and dividing by

Y /X gives π 2 + arctan

tan t tan2 m

p Y /X

q Y X

> X.

(C.16)

For fixed t and m, the function r 7→ π/2 + arctan(r tan t/ tan2 m) /r on r > 0 is strictly decreasing, because its derivative is r tan t/ tan2 m − π2 + arctan(r tan t/ tan2 m) 1+(r tan t/ tan2 m)2 < 0. r2 By the p hypothesis of the lemma, this function ptakes the value X at r = λ, while √ by (C.16) its value at r = Y /X is larger than X. Hence λ > Y /X, and consequently λX > XY > Y /λ. ■ C.3.2

Mirror symmetry

We now use Lemma C.3 to show that the two students are mirror images across the teacher plane. Lemma C.4 (Out-of-plane symmetry). Let ξ be any unit vector orthogonal to span{v1 , v2 }. Assume w1 , w2 ̸= 0 satisfy (3.4), lie in span{v̄, ξ}, and do not both lie in the teacher plane. Then, after possibly replacing ξ by −ξ, there exist r > 0 and κ ∈ (0, π/2) such that w2 = r(cos κ v̄ − sin κ ξ).

w1 = r(cos κ v̄ + sin κ ξ),

Proof of Lemma C.4. We use coordinates with respect to the orthonormal basis v̄, v∆ , ξ, so that v1 = (cos(α/2), sin(α/2), 0) and v2 = (cos(α/2), − sin(α/2), 0). Since w1 , w2 ∈ span{v̄, ξ}, we can write w̄i = (xi , 0, zi ). Then ⟨w̄i , v1 ⟩ = ⟨w̄i , v2 ⟩, so each student makes the same angle with both teachers; we denote this angle by θi . The system (3.4) becomes (π−θi )(v1 +v2 )+ 2 sin θi −π∥wi ∥2 −sin γ ∥wj ∥2 w̄i −(π−γ)∥wj ∥2 w̄j = 0, {i, j} = {1, 2}. (C.17) We first show that z1 z2 ̸= 0. By hypothesis, z1 and z2 are not both zero. If z1 = 0 and z2 ̸= 0, then w̄1 and w̄2 are neither parallel nor antiparallel, so 0 < γ < π, and the third coordinate of (C.17) for i = 1 gives (π − γ)∥w2 ∥2 z2 = 0, a contradiction. The case z2 = 0 is the same. 40

We next show that 0 < γ < π. If γ were 0 or π, then w̄2 = ±w̄1 , and (C.17) for i = 1 would express (π − θ1 )(v1 + v2 ), a nonzero multiple of v̄, as a multiple of w̄1 . This is impossible because z1 ̸= 0. To remove the term in w̄i from the ith equation, we take its inner product with (zi , 0, −xi ), which is orthogonal to w̄i . This gives 2 cos(α/2)(π − θ1 )z1 + (π − γ)∥w2 ∥2 (x1 z2 − z1 x2 ) = 0, 2 cos(α/2)(π − θ2 )z2 − (π − γ)∥w1 ∥2 (x1 z2 − z1 x2 ) = 0.

(C.18)

The coefficients 2 cos(α/2)(π − θi ) and (π − γ)∥wj ∥2 are positive. The first identity therefore gives sgn(x1 z2 − z1 x2 ) = − sgn z1 , and the second gives sgn(x1 z2 − z1 x2 ) = sgn z2 , so z1 and z2 have opposite signs. Replacing ξ by −ξ if necessary, we may assume z1 > 0 > z2 and write w̄1 = (cos p, 0, sin p) and w̄2 = (cos q, 0, − sin q) with p, q ∈ (0, π). Then x1 z2 − z1 x2 = − sin(p + q), which is negative by the first identity in (C.18). Hence p + q < π and γ = p + q, and (C.18) becomes (π − p − q) sin(p + q)∥w2 ∥2 = 2 cos(α/2)(π − θ1 ) sin p, (π − p − q) sin(p + q)∥w1 ∥2 = 2 cos(α/2)(π − θ2 ) sin q.

(C.19)

The teacher angles satisfy cos θ1 = cos(α/2) cos p and cos θ2 = cos(α/2) cos q. Taking the inner product of the ith equation in (C.17) with w̄i gives π∥w1 ∥2 + sin(p + q) + (π − p − q) cos(p + q) ∥w2 ∥2 = 2 sin θ1 + (π − θ1 ) cos θ1 , sin(p + q) + (π − p − q) cos(p + q) ∥w1 ∥2 + π∥w2 ∥2 = 2 sin θ2 + (π − θ2 ) cos θ2 . (C.20) Eliminating ∥w1 ∥2 , ∥w2 ∥2 from (C.20) and (C.19) gives (π − p − q) sin(p + q) sin θ1 + (π − θ1 ) cos θ1 = sin(p + q) + (π − p − q) cos(p + q) cos(α/2)(π − θ1 ) sin p + π cos(α/2)(π − θ2 ) sin q, (C.21) (π − p − q) sin(p + q) sin θ2 + (π − θ2 ) cos θ2

= π cos(α/2)(π − θ1 ) sin p + sin(p + q) + (π − p − q) cos(p + q) cos(α/2)(π − θ2 ) sin q. (C.22) We now show that (C.21) and (C.22) force p = q. Suppose instead that p ̸= q; by symmetry we may assume p > q, so that p−q p+q π 0< < < . (C.23) 2 2 2 The formula (3.6) for θα (κ) makes sense for every κ ∈ (0, π), and θ1 = θα (p), θ2 = θα (q). Direct differentiation gives cos(α/2) sin κ θα′ (κ) = p > 0, 1 − cos2 (α/2) cos2 κ

1 − θα′ (κ)2 =

sin2 (α/2) > 0, 1 − cos2 (α/2) cos2 κ

so 0 < θα′ (κ) < 1 on (0, π). By the mean value theorem, there exists ζ ∈ (q, p) such that θ1 − θ2 = θα (p) − θα (q) = θα′ (ζ)(p − q). Since 0 < θα′ (ζ) < 1 and p > q, it follows that 0<

θ1 − θ 2 p−q < . 2 2 41

(C.24)

Moreover, cos θ1 + cos θ2 = cos(α/2)(cos p + cos q) > 0 because p + q < π, so (θ1 + θ2 )/2 < π/2. Adding and subtracting the identities cos θ1 = cos(α/2) cos p and cos θ2 = cos(α/2) cos q, we obtain θ1 + θ 2 θ1 − θ 2 p+q p−q cos cos = cos(α/2) cos cos , 2 2 2 2 (C.25) θ1 + θ 2 θ1 − θ 2 p+q p−q sin sin = cos(α/2) sin sin . 2 2 2 2 Hence θ1 + θ 2 θ1 − θ 2 p+q p−q tan tan = tan tan . 2 2 2 2 Since 0 < (θ1 + θ2 )/2 < π/2, this gives ! 2 tan θ1 −θ π θ1 + θ 2 2 = + arctan π− . (C.26) p−q 2 2 tan p+q tan 2 2 We now subtract (C.22) from (C.21) and write p, q, θ1 , θ2 in terms of the half-sums (p + q)/2, (θ1 + θ2 )/2 and the half-differences (p − q)/2, (θ1 − θ2 )/2. This gives h θ1 + θ 2 θ1 − θ 2 θ1 + θ 2 θ1 + θ 2 θ1 − θ 2 (π − p − q) sin(p + q) 2 cos sin −2 π− sin sin 2 2 2 2 2 θ1 − θ 2 i θ1 + θ 2 cos − (θ1 − θ2 ) cos 2 2 h θ1 + θ 2 p+q p−q = cos(α/2) sin(p + q) + (π − p − q) cos(p + q) − π 2 π − cos sin 2 2 2 p+q p − qi − (θ1 − θ2 ) sin cos . 2 2 Using (C.25) in this identity and collecting the terms containing π − (θ1 + θ2 )/2, we get θ1 + θ 2 p+q p−q π− p + q − sin(p + q) cos tan 2 2 2 θ1 − θ 2 p+q θ1 − θ 2 θ1 − θ 2 2 p+q = sin p + q − sin(p + q) + 2(π − p − q) − cos tan . 2 2 2 2 2 Therefore p+q p − q θ1 − θ 2 2(π − p − q) θ1 − θ 2 θ1 − θ 2 θ1 + θ 2 2 p+q π− = tan cot + − cos tan . 2 2 2 2 p + q − sin(p + q) 2 2 2 (C.27) Adding (C.21) and (C.22) instead, we obtain in the same way h θ1 + θ 2 θ1 − θ 2 θ1 + θ 2 θ1 + θ 2 θ1 − θ 2 (π − p − q) sin(p + q) 2 sin cos +2 π− cos cos 2 2 2 2 2 i θ1 − θ 2 θ1 + θ 2 + (θ1 − θ2 ) sin sin 2 2 h θ1 + θ 2 p+q p−q = cos(α/2) sin(p + q) + (π − p − q) cos(p + q) + π 2 π − sin cos 2 2 2 i p+q p−q − (θ1 − θ2 ) cos sin . 2 2 42

Using (C.25) again and collecting π − (θ1 + θ2 )/2, we obtain θ1 + θ 2 p+q p−q π− p + q + sin(p + q) sin cot 2 2 2 θ1 − θ 2 p+q θ1 − θ 2 θ1 − θ 2 2 p+q . = cos p + q + sin(p + q) + 2(π − p − q) + sin cot 2 2 2 2 2 Thus

θ1 + θ 2 p+q p − q θ1 − θ 2 2(π − p − q) π− = cot tan + 2 2 2 2 p + q + sin(p + q) θ1 − θ 2 θ1 − θ 2 2 p+q × . + sin cot 2 2 2

(C.28)

We now apply Lemma C.3 with t = (θ1 − θ2 )/2, m = (p + q)/2, and λ = tan p+q tan p−q 2 2 . By (C.23) and (C.24), we have 0 < t < m < π/2. In the notation of the lemma, (C.27) says that π − (θ1 + θ2 )/2 = λX, and (C.28) says that π − (θ1 + θ2 )/2 = Y /λ. Combined with (C.26), the first identity shows that λ satisfies the hypothesis of the lemma, which then gives λX > Y /λ. This contradicts λX = π − (θ1 + θ2 )/2 = Y /λ. Hence p = q. Thus w̄1 = (cos p, 0, sin p) and w̄2 = (cos p, 0, − sin p) with 0 < p < π/2, so that θ1 = θ2 and γ = 2p. It remains to show that ∥w1 ∥2 = ∥w2 ∥2 . Dividing the third coordinates of (C.17) by sin p gives 0 = 2 sin θ1 − π∥w1 ∥2 + (π − γ) − sin γ ∥w2 ∥2 , 0 = 2 sin θ1 − π∥w2 ∥2 + (π − γ) − sin γ ∥w1 ∥2 . Subtracting these two equations yields (2π − γ − sin γ)(∥w1 ∥2 − ∥w2 ∥2 ) = 0, and the coefficient is positive because 0 < γ < π. Hence ∥w1 ∥2 = ∥w2 ∥2 , and the lemma follows with r = ∥w1 ∥2 and κ = p. ■ C.3.3

Exact scalar characterization

By Lemma C.4, an out-of-plane stationary point is determined by a common norm r and an angle κ from v̄. It remains to find the pairs (r, κ) that give stationary points. We recall θα , rα , and Fα from (3.6)–(3.8), and that κα is the unique root of Fα in (0, π/2) by Lemma 3.4. Proof of the characterization in Lemma 3.9. By Lemma C.4, after possibly swapping the students or replacing ξ by −ξ, every pair in question has the form w2 = r(cos κ v̄ − sin κ ξ)

w1 = r(cos κ v̄ + sin κ ξ),

(C.29)

for some r > 0 and 0 < κ < π/2, and every such pair lies in span{v̄, ξ} outside the teacher plane. For the pair (C.29), all four teacher–student angles equal θα (κ) by (3.6), and the angle between the students is γ = 2κ. We write R(r, κ) = 2 sin θα (κ) − r sin(2κ) + 2κ , Gα (r, κ) = π − θα (κ) cos(α/2) − (π − 2κ)r cos κ. Substituting (C.29) into (3.4) reduces its two vector equations to 2Gα (r, κ) + cos κ R(r, κ) v̄ + sin κ R(r, κ)ξ = 0, 2Gα (r, κ) + cos κ R(r, κ) v̄ − sin κ R(r, κ)ξ = 0. 43

Since sin κ > 0, the pair is stationary if and only if R(r, κ) = 0,

Gα (r, κ) = 0.

Since sin(2κ) + 2κ > 0, the first equation means r = 2 sin θα (κ)/(sin(2κ) + 2κ) = rα (κ). With this value of r, the second equation reads 0 = Gα (rα (κ), κ) = π − θα (κ) cos(α/2) − (π − 2κ)rα (κ) cos κ = Fα (κ). By Lemma 3.4, the pair is therefore stationary if and only if κ = κα and r = rα (κα ). C.3.4

■

Negative curvature

To finish the proof of Lemma 3.9, we move the two students toward different teachers, along the direction (v∆ , −v∆ ), and show that the loss has negative curvature in this direction. Proof of the strict-saddle claim in Lemma 3.9. Fix a stationary point (C.29) with κ = κα and r = rα (κα ), and consider the perturbation (v∆ , −v∆ ). Since v∆ is orthogonal to w̄1 and w̄2 , the blocks in Lemma A.6 give, for i = 1, 2, sin(2κ) , 2π sin θα (κ) sin2 (α/2) ⊤ v∆ Q1 (wi , vj )v∆ = 1+ , 2πr sin2 θα (κ) π − 2κ ⊤ v∆ Q2 (w1 , w2 )v∆ = . 2π

⊤ v∆ Q1 (wi , w3−i )v∆ =

j ∈ {1, 2},

Consequently, (v∆ , −v∆ )⊤ ∇2 L(w1 , w2 )(v∆ , −v∆ ) sin2 (α/2) π − 2κ sin(2κ) 2 sin θα (κ) − 1+ − . =1+ 2 π πr π sin θα (κ) Since r = rα (κ), (3.7) gives 2 sin θα (κ)/r = sin(2κ) + 2κ, and therefore (v∆ , −v∆ )⊤ ∇2 L(w1 , w2 )(v∆ , −v∆ ) = −

sin(2κ) + 2κ sin2 (α/2) < 0. π sin2 θα (κ)

Thus every point of the out-of-plane family has a strictly negative Hessian direction.

C.4

■

In-plane stationary points

In the teacher plane, each student is described by its norm and its angle from v̄, and stationarity reduces to four scalar equations. Solving them for 0 < α ≤ α⋆ proves Lemma 3.10. C.4.1

Coordinates in the teacher plane

For φ ∈ [−π, π), write e(φ) = cos φ v̄ + sin φ v∆ . Its derivative is the positively oriented unit tangent e′ (φ) = − sin φ v̄ + cos φ v∆ . For φ, ϑ ∈ R, define their circular distance by dS1 (φ, ϑ) := min |φ − ϑ + 2πk| = arccos cos(φ − ϑ) ∈ [0, π]. k∈Z

With l as in (A.1b), define P(φ) = 2π l dS1 (φ, α/2) + l dS1 (φ, −α/2) , P ′ (φ) = − π − dS1 (φ, α/2) sin(φ − α/2) − π − dS1 (φ, −α/2) sin(φ + α/2). The function P is 2π-periodic, even, and continuously differentiable. In these coordinates, (3.4) splits into a radial and a tangential equation for each student. Lemma C.5 (Planar stationarity equations). Let 0 < α < π, let w1 , w2 ∈ span{v1 , v2 } be nonzero, and write wi = ∥wi ∥2 e(φi ), φi ∈ [−π, π), i = 1, 2, and let γ = dS1 (φ1 , φ2 ). Then (w1 , w2 ) is stationary if and only if π∥w1 ∥2 + 2πl(γ)∥w2 ∥2 = P(φ1 ),

(C.30)

π∥w2 ∥2 + 2πl(γ)∥w1 ∥2 = P(φ2 ),

(C.31)

′

(C.32)

′

(C.33)

P (φ1 ) = (π − γ)∥w2 ∥2 sin(φ2 − φ1 ), P (φ2 ) = (π − γ)∥w1 ∥2 sin(φ1 − φ2 ).

If in addition 0 < α ≤ α⋆ , then P ′ (φ) < 0 for 0 < φ < π and P ′ (φ) > 0 for −π < φ < 0. In particular, the zeros of P ′ are exactly the integer multiples of π. Proof of Lemma C.5. The closed-form loss (A.10) gives 2πL =

π (∥w1 ∥22 + ∥w2 ∥22 ) + 2π∥w1 ∥2 ∥w2 ∥2 l(γ) − ∥w1 ∥2 P(φ1 ) − ∥w2 ∥2 P(φ2 ) + constant. 2

Differentiating with respect to ∥w1 ∥2 and ∥w2 ∥2 gives (C.30) and (C.31). For the tangential equations, we write Bi = (π − θi1 )v1 + (π − θi2 )v2 . The direction equations (A.13)–(A.14) state that (I − e(φi )e(φi )⊤ ) Bi − (π − γ)w3−i = 0. Since e′ (φi ) ⊥ e(φi ), taking the inner product with e′ (φi ) removes the projection and gives ⟨e′ (φi ), Bi ⟩ = (π − γ)⟨e′ (φi ), w3−i ⟩. To identify the left-hand side with P ′ (φi ), let a = ±α/2. If φ ̸≡ a and φ ̸≡ a + π modulo 2π, then cos dS1 (φ, a) = cos(φ − a), and differentiating gives sin dS1 (φ, a) ∂φ dS1 (φ, a) = sin(φ − a). Since 2πl′ (t) = −(π − t) sin t by (A.2) and ⟨e′ (φ), e(a)⟩ = − sin(φ − a), the chain rule gives ∂ ∂ 2πl dS1 (φ, a) = 2πl′ dS1 (φ, a) dS1 (φ, a) = − π − dS1 (φ, a) sin(φ − a) ∂φ ∂φ ′ = π − dS1 (φ, a) ⟨e (φ), e(a)⟩. With v1 = e(α/2), v2 = e(−α/2), θi1 = dS1 (φi , α/2), and θi2 = dS1 (φi , −α/2), it follows that P ′ (φi ) = (π − θi1 )⟨e′ (φi ), v1 ⟩ + (π − θi2 )⟨e′ (φi ), v2 ⟩ = ⟨e′ (φi ), Bi ⟩. Since ⟨e′ (φi ), e(φ3−i )⟩ = sin(φ3−i − φi ), this gives (C.32)–(C.33). The computation above assumes that e(φi ) ̸= ±v1 , ±v2 , because φ 7→ dS1 (φ, a) is not differentiable at φ ≡ a and φ ≡ a + π. The composition φ 7→ l(dS1 (φ, a)) is nevertheless differentiable at these points. Near φ ≡ a the circular distance is |φ − a|, and near φ ≡ a + π it is π − |φ − a − π|. Since the formula (A.1b) for l extends smoothly to R and l′ (0) = l′ (π) = 0 by (A.2), both one-sided derivatives vanish. Hence the formula for P ′ (φi ) holds for every φi , and the tangential equations remain valid also when γ = 0 or γ = π. Conversely, the gradient with respect to each student lies in the teacher 45

plane, and the four equations say that its radial and tangential components vanish. Hence the four equations are equivalent to stationarity. It remains to determine the sign of P ′ when 0 < α ≤ α⋆ . Since P is even, P ′ is odd, so it suffices to consider 0 < φ < π. For 0 ≤ φ ≤ α/2, direct expansion gives P ′ (φ) = 2 sin φ φ cot φ sin(α/2) − (π − α/2) cos(α/2) . For 0 < φ ≤ α/2, this is negative, because φ cot φ < 1 and sin(α/2) ≤ (π − α/2) cos(α/2) by Lemma C.1. For α/2 ≤ φ ≤ π − α/2, P ′ (φ) = 2 (α/2) sin(α/2) cos φ − (π − φ) cos(α/2) sin φ . If φ ≥ π/2, both terms are nonpositive and the second is negative. If α/2 ≤ φ < π/2, then (π − φ) tan φ > (α/2) tan(α/2) because π − φ > α/2 and tan φ ≥ tan(α/2); multiplying by cos(α/2) cos φ > 0 shows again that P ′ (φ) < 0. Finally, for π − α/2 ≤ φ ≤ π, we put x = π − φ ∈ [0, α/2] and find P ′ (φ) = −2 (α/2) cos(α/2) sin x + x sin(α/2) cos x , which is negative for x > 0 and vanishes at x = 0, that is, at φ = π. Together with P ′ (0) = 0 and the oddness of P ′ , this proves the claims about the sign and the zeros of P ′ . ■ C.4.2

Stationary points above the angle threshold

The assumption α ≤ α⋆ in the sign condition cannot be dropped. Above the threshold, P ′ vanishes inside (0, α/2), which gives the following stationary points. Proposition C.6 (Additional stationary families above the threshold). Suppose α ∈ (α⋆ , π). There is a unique φ0 ∈ (0, α/2) satisfying (C.34) For σ = ±1 and every s1 , s2 > 0 with s1 + s2 = P(φ0 )/π, the pair (w1 , w2 ) = s1 e(σφ0 ), s2 e(σφ0 ) is stationary. These families are not among the configurations listed in Lemma 3.10. φ0 cot φ0 = (π − α/2) cot(α/2).

Proof of Proposition C.6. The function φ 7→ φ cot φ is strictly decreasing on (0, α/2), since its derivative (sin φ cos φ − φ)/ sin2 φ is negative. Moreover, φ cot φ → 1 as φ ↓ 0, and (α/2) cot(α/2) < (π − α/2) cot(α/2) < 1. Here the first inequality uses α < π, and the second is equivalent to tan(α/2) > π − α/2, which holds by Lemma C.1 because α > α⋆ . Thus (C.34) has a unique root φ0 ∈ (0, α/2). The distances from φ0 to the two teachers, α/2 − φ0 and α/2 + φ0 , lie in (0, π). Since l is strictly decreasing on [0, π] by (A.2) and l(π) = 0, it follows that P(φ0 ) > 0, so there are positive s1 , s2 with the stated sum. By the formula for P ′ on [0, α/2] in the proof of Lemma C.5, P ′ (φ) = 2 sin φ sin(α/2) (φ cot φ − (π − α/2) cot(α/2)) for 0 < φ < α/2, so P ′ (φ0 ) = 0. For the pair in the proposition, γ = 0, so the tangential equations reduce to P ′ (φ0 ) = 0 and both radial equations reduce to π(s1 + s2 ) = P(φ0 ). By Lemma C.5, the pair is stationary. Since 0 < φ0 < α/2, these points are not parallel to ±v̄ and differ from (v1 , v2 ), so they are not among the configurations in Lemma 3.10. ■

C.4.3

Classification up to the angle threshold

We now return to 0 < α ≤ α⋆ , which we assume for the rest of this subsection. The points 0, ±α/2, and ±(π − α/2) split (−π, π) into the open intervals J2 = (−π, −π + α/2), J1 = (−π + α/2, −α/2), J0 = (−α/2, 0), I0 = (0, α/2), I1 = (α/2, π − α/2), I2 = (π − α/2, π). The only symmetries of the loss that we use are the exchange of the two students and the reflection φ 7→ −φ across v̄; the map φ 7→ φ + π is not a symmetry of the loss. Proof of Lemma 3.10. As in Lemma C.5, we write wi = ∥wi ∥2 e(φi ) with φi ∈ [−π, π). Both r+ and r− in (3.5) are positive; for r− , this uses tan(α/2) > α/2. We first treat collinear directions. If γ = 0, the two directions coincide, and the tangential equations (C.32)–(C.33) give P ′ (φ1 ) = 0, so by Lemma C.5 the common direction is v̄ or −v̄. Since 2πl(0) = π and P(0) = 2 (π − α/2) cos(α/2) + sin(α/2) = πr+ , P(π) = 2 sin(α/2) − (α/2) cos(α/2) = πr− , the radial equations (C.30)–(C.31) give ∥w1 ∥2 +∥w2 ∥2 = r+ in the first case and ∥w1 ∥2 +∥w2 ∥2 = r− in the second. If γ = π, the right-hand sides of the tangential equations vanish, so P ′ (φ1 ) = P ′ (φ2 ) = 0 and the two directions are v̄ and −v̄. After exchanging the students, (φ1 , φ2 ) = (0, −π), and since l(π) = 0, the radial equations give ∥w1 ∥2 = r+ and ∥w2 ∥2 = r− . This determines all stationary points with collinear directions. Noncollinear directions. Assume now that 0 < γ < π. Multiplying the two tangential equations gives P ′ (φ1 )P ′ (φ2 ) = −(π − γ)2 ∥w1 ∥2 ∥w2 ∥2 sin2 (φ2 − φ1 ) < 0. By Lemma C.5, one direction therefore lies in (−π, 0) and the other in (0, π). After relabeling, we write φ− = φ1 ∈ (−π, 0) and φ+ = φ2 ∈ (0, π). Since P ′ (φ− ) > 0, the tangential equation (C.32) gives (π − γ)∥w2 ∥2 sin(φ+ − φ− ) > 0, so sin(φ+ − φ− ) > 0. As 0 < φ+ − φ− < 2π, it follows that 0 < φ+ − φ− < π.

(C.35)

Up to the exchange of the students and the reflection φ 7→ −φ, (C.35) leaves four cases for the intervals containing φ− and φ+ : case 1 2 J0 × I1 3 J2 × I0 4

(φ− , φ+ ) J0 × I0 , or, by reflection, J1 × I0 , or, by reflection, J0 × I2 , J1 × I1 .

The products J1 × I2 , J2 × I1 , and J2 × I2 are excluded by (C.35), since on them φ+ − φ− ≥ π. Directions at the endpoints ±α/2 and ±(π − α/2) are included in the adjacent cases. We now show that these cases contain no stationary point other than (w1 , w2 ) = (v1 , v2 ), which lies on the boundary of Case 1.

Case 1: φ− ∈ J0 and φ+ ∈ I0 . We write φ+ = x and φ− = −y with 0 < x, y ≤ α/2, so that γ = x + y, and set p = x − y. Then |p| ≤ γ ≤ α < π. (C.36) On [0, α/2], P(φ) = 2 (π − α/2) cos(α/2) + sin(α/2) cos φ + φ sin(α/2) sin φ , and P ′ (φ) is given by the first formula in the proof of Lemma C.5. Here (φ1 , φ2 ) = (−y, x). Since P is even and (π − γ) sin γ > 0, the tangential equations give ∥w1 ∥2 = −

P ′ (x) , (π − γ) sin γ

∥w2 ∥2 = −

P ′ (y) , (π − γ) sin γ

and the radial equations read π∥w2 ∥2 + 2πl(γ)∥w1 ∥2 = P(x) and π∥w1 ∥2 + 2πl(γ)∥w2 ∥2 = P(y). Substituting the norms and multiplying by (π − γ) sin γ, we obtain −πP ′ (y) − 2πl(γ)P ′ (x) = (π − γ) sin γ P(x), −πP ′ (x) − 2πl(γ)P ′ (y) = (π − γ) sin γ P(y). After division by 2, both equations are linear and homogeneous in the nonzero vector (π − α/2) cos(α/2), sin(α/2) , with coefficient matrix M=

γ sin y + sin γ sin x −γ(π − x) cos y − sin γ(π − y) cos x . γ sin x + sin γ sin y −γ(π − y) cos x − sin γ(π − x) cos y

Hence det M = 0. Expanding the determinant and using x + y = γ and x − y = p, we find det M = (γ 2 − sin2 γ) (π − x) sin x cos y − (π − y) sin y cos x γ 2 − sin2 γ = (2π − γ) sin p − p sin γ . 2 Since γ > sin γ > 0, it follows that (2π − γ) sin p = p sin γ. If p = ̸ 0, this says that sin |p|/|p| = sin γ/(2π − γ). But |p| ≤ γ < π by (C.36), and sin t/t is strictly decreasing on (0, π), so sin |p|/|p| ≥ sin γ/γ > sin γ/(2π − γ), a contradiction. Hence p = 0. Thus x = y = γ/2, and either of the two equations above becomes γ γ γ (γ + sin γ) (π − α/2) cos(α/2) sin − sin(α/2) π − cos = 0. 2 2 2 Since γ + sin γ > 0, it follows that (π − γ/2) cot(γ/2) = (π − α/2) cot(α/2). The function γ 7→ (π − γ/2) cot(γ/2) is strictly decreasing on (0, π), because its derivative is π − γ2 + sin γ2 cos γ2 − < 0. 2 sin2 γ2 Hence γ = α and (φ− , φ+ ) = (−α/2, α/2). Since −P ′ (α/2) = (π − α) sin α, the formulas for the norms give ∥w1 ∥2 = ∥w2 ∥2 = 1. Thus the only stationary point in Case 1, including its boundary, is (v1 , v2 ), up to the exchange of the students.

Case 2: φ− ∈ J0 and φ+ ∈ I1 . By the reflection φ 7→ −φ, we may assume that φ− ∈ J0 and φ+ ∈ I1 , including the endpoints except φ+ = π − α/2, which is treated in Case 3. We write φ1 = φ− = α/2 − b0 and φ2 = φ+ = α/2 + a0 , and set s = a0 + b0 and ρ = α − b0 . Then a0 ≥ 0, α/2 < b0 ≤ α, s < π by (C.35), and a0 + α < π because φ+ < π − α/2. First suppose that a0 > 0 and ρ > 0. The teacher–student angles are θ11 = b0 , θ12 = ρ, θ21 = a0 , and θ22 = a0 + α. Substituting them into (C.30)–(C.33), using the tangential equations to eliminate the radii, and collecting the coefficients of sin b0 , sin a0 , sin ρ, and sin(a0 + α), we obtain 0 = −a0 (2π − s) sin b0 + s(2π − a0 − α) sin ρ + sin s ρ sin(a0 + α) − b0 sin a0 , 0 = −b0 (2π − s) sin a0 + sρ sin(a0 + α) + sin s (2π − a0 − α) sin ρ − a0 sin b0 . We write U = a0 sin b0 , V = b0 sin a0 , M = (2π − a0 − α) sin ρ, and N = ρ sin(a0 + α), which are all positive. Since 2π − a0 − α = 2π − s − ρ and a0 + α = s + ρ, these equations can be written as (2π − s)U + sin s V = sM + sin s N,

(C.37)

sin s U + (2π − s)V = sin s M + sN.

(C.38)

and adding and subtracting them gives (2π − s + sin s)(U + V ) = (s + sin s)(M + N ),

(C.39)

(2π − s − sin s)(U − V ) = (s − sin s)(M − N ).

(C.40)

Since 0 < ρ < s + ρ < π, we have 2π − s − ρ > s + ρ, and since sin t/t is strictly decreasing on (0, π), M > (s + ρ) sin ρ > ρ sin(s + ρ) = N . The coefficients in (C.40) are positive, so U > V , that is, sin b0 /b0 > sin a0 /a0 , and hence a0 > b0 . Together with b0 > α/2 and s + ρ = a0 + α < π, this gives ns o s 0 < ρ = α − b0 < b0 < a0 , b0 < , ρ < min ,π − s . (C.41) 2 2 To compare U − V with M − N , we fix s ∈ (0, π) and introduce Rs (x) =

(s − x) sin x − x sin(s − x) , (s − x) sin x + x sin(s − x)

Qs (x) =

(2π − s − x) sin x − x sin(s + x) , (2π − s − x) sin x + x sin(s + x)

for 0 < x < s/2 and 0 < x < min{s/2, π − s}, respectively. Both denominators are positive, the second because 0 < s+x < π. By (C.41), (U −V )/(U +V ) = Rs (b0 ) and (M −N )/(M +N ) = Qs (ρ), so dividing (C.40) by (C.39) gives Rs (b0 ) = Λs Qs (ρ),

Λs =

s − sin s 2π − s + sin s . s + sin s 2π − s − sin s

(C.42)

As x ↓ 0, Rs (x) and Qs (x) tend to Rs (0) = (s − sin s)/(s + sin s) and Qs (0) = (2π − s − sin s)/(2π − s + sin s), so Λs = Rs (0)/Qs (0). We next show that s−x Rs (0), 0 < x < s/2, (C.43) Rs (x) < s s−x Qs (x) ≥ Qs (0), 0 < x < min{s/2, π − s}. (C.44) s For the first bound, we write A(t) = log(t/ sin t), which is positive for 0 < t < π, and ψ(t) = (t − sin t)/ t(t + sin t) . Dividing the numerator and the denominator of Rs (x) by sin x sin(s − x) gives A(s − x) − A(x) A(s − x) Rs (x) = tanh < tanh = (s − x)ψ(s − x). 2 2 49

To see that ψ is nondecreasing on (0, π), we compute ψ ′ (t) =

sin2 t + 2t sin t − t2 (1 + 2 cos t) . t2 (t + sin t)2

The numerator vanishes at t = 0 and has derivative 2 t2 sin t − (1 + cos t)(t − sin t) ≥ 0, because Rt t − sin t = 0 (1 − cos u) du ≤ t(1 − cos t) implies (1 + cos t)(t − sin t) ≤ t sin2 t ≤ t2 sin t. Hence Rs (x) < (s − x)ψ(s) = s−x s Rs (0), which is (C.43). For the second bound, we write B(t) = log (2π − t)/ sin t for 0 < t < π. In the same way, Qs (x) = tanh (B(s + x) − A(x))/2 , and we first show that B(s + x) − A(x) ≥

s−x B(s). s

(C.45)

With A(0) = 0, the function Hs (x) = B(s + x) − A(x) − s−x s B(s) vanishes at x = 0, and B(s) 1 1 Hs′ (x) = − − cot x − cot(s + x) + . s x 2π − s − x The function t 7→ 1/t − cot t is increasing on (0, π), since its derivative 1/ sin2 t − 1/t2 is positive, and t 7→ cot t + 1/(2π − t) is decreasing, since its derivative −1/ sin2 t + 1/(2π − t)2 is negative. Because x < s/2 and s + x > s, B(s) 2 s 1 ′ Hs (x) ≥ − − cot − cot s + . s s 2 2π − s Since cot(s/2) = 1/ sin s + cot s, the right-hand side is nonnegative as soon as s s B(s) ≥ 2 − + . sin s 2π − s

(C.46)

Applying log z ≥ 2(z − 1)/(z + 1) for z > 1 with z = (2π − s)/ sin s, we see that (C.46) follows from s (2π − s)2 − sin2 s ≥ 4(2π − s) sin2 s. (C.47) √ If 0 < s ≤ 1, then sin s ≤ s and (2π − s)/s ≥ 2π − 1 > 2 + 5, which implies (2π − s)2 − s2 ≥ 4(2π − s)s and hence (C.47). If 1 ≤ s < π, then sin2 s ≤ 1, and it is enough to show that χ(s) = s (2π − s)2 − 1 − 4(2π − s) is positive. Since χ′′ (s) = 6s − 8π < 0 on [1, π], this follows from χ(1) = 4(π 2 − 3π + 1) > 0 and χ(π) = π(π 2 − 5) > 0. Thus (C.46) holds, so Hs′ ≥ 0, and (C.45) follows from Hs (0) = 0. Since tanh is increasing, and concave on [0, ∞) with tanh 0 = 0, s−x B(s) s−x s − x B(s) Qs (x) ≥ tanh ≥ tanh = Qs (0), s 2 s 2 s which is (C.44). Since 0 < ρ < b0 by (C.41), the two bounds give Rs (b0 ) <

s − b0 s−ρ s−ρ Rs (0) < Rs (0) = Λs Qs (0) ≤ Λs Qs (ρ), s s s

which contradicts (C.42). Hence there is no stationary point with a0 > 0 and ρ > 0. It remains to treat the endpoints. If ρ = 0 and a0 > 0, then M = N = 0, and (C.37)–(C.38) say that (U, V ) lies in the kernel of the matrix with diagonal entries 2π − s and off-diagonal entries sin s. This matrix is invertible because 2π − s > sin s, so U = V = 0, which contradicts a0 > 0. If a0 = 0, then φ+ = α/2 lies in the closure of Case 1, where the only stationary point is (v1 , v2 ). Thus Case 2 contributes no further stationary point. 50

Case 3: φ− ∈ J2 and φ+ ∈ I0 . By reflection, we may assume that φ− ∈ J2 and φ+ ∈ I0 , and we write φ+ = p and φ− = −π + q with 0 ≤ p, q ≤ α/2. Then φ+ − φ− = π − (q − p), so (C.35) gives p < q, and γ = π − δ with δ = q − p ∈ (0, α/2]. Substituting these directions into (C.30)–(C.33) and eliminating the two radii, we obtain 0 = sin δ Ap + δBp,q − πCq , where

(C.48)

Ap = 2p sin(α/2) cos p − 2(π − α/2) cos(α/2) sin p, Bp,q = 2(π − α/2) cos(α/2) sin q − 2p sin(α/2) cos q + 2 sin(α/2) cos p sin δ, Cq = 2q sin(α/2) cos q + α cos(α/2) sin q.

By Lemma C.1, sin(α/2) ≤ (π − α/2) cos(α/2). Since tan p > p for 0 < p < π/2, it follows that Ap = 2p cos p sin(α/2) − (π − α/2) cos(α/2) tan p/p < 0 for p > 0, while A0 = 0. Thus Ap ≤ 0. Next, the middle term of Bp,q is nonpositive. Dropping it and using δ ≤ q ≤ α/2, cos p ≤ 1, and sin δ ≤ sin q < π cos q, where the last inequality holds because tan q ≤ tan(α/2) ≤ π − α/2 < π, we obtain δBp,q ≤ 2δ(π − α/2) cos(α/2) sin q + 2δ sin(α/2) cos p sin δ < πα cos(α/2) sin q + 2πq sin(α/2) cos q = πCq . Hence the right-hand side of (C.48) is negative, a contradiction. Thus Case 3 contains no stationary point. Case 4: φ− ∈ J1 and φ+ ∈ I1 . We write φ+ = α/2 + a0 and φ− = −α/2 − b0 with a0 , b0 ≥ 0, so that γ = α + a0 + b0 < π and the teacher–student angles are θ11 = b0 + α, θ12 = b0 , θ21 = a0 , and θ22 = a0 + α. Substituting them into the equations of Lemma C.5 and eliminating the two radii, we obtain 0 = − sin γ (γ − a0 ) sin a0 + (γ − a0 − α) sin(a0 + α) − (2π − γ) a0 sin(b0 + α) + (a0 + α) sin b0 . Both terms on the right are nonpositive, and the second is negative when a0 , b0 > 0. Hence there is no stationary point with a0 , b0 > 0. If exactly one of a0 and b0 vanishes, the point lies in the closure of Case 2, and if both vanish, it is (v1 , v2 ). Together with the collinear directions, Cases 1–4 show that every stationary point is one of the configurations listed in Lemma 3.10. Stationarity and curvature of the listed points. Conversely, every listed configuration is stationary. The pair (v1 , v2 ) reproduces the teacher, so the loss, which is nonnegative, vanishes there; it is therefore a global minimum. For two students along v̄, we have γ = 0 and P ′ (0) = 0, and since 2πl(0) = π and P(0) = πr+ , the equations of Lemma C.5 hold exactly when ∥w1 ∥2 +∥w2 ∥2 = r+ . The same computation for two students along −v̄ gives ∥w1 ∥2 + ∥w2 ∥2 = r− . For the pair (r+ v̄, −r− v̄), we have P ′ (0) = P ′ (π) = 0 and l(π) = 0, and the radial equations hold with the norms r+ and r− . It remains to find a direction of negative curvature at each of these saddles. If both students are parallel to v̄, or both are parallel to −v̄, the Hessian in Lemma A.6, restricted to perturbations (t1 v∆ , t2 v∆ ), is ! 2 sin(α/2) 1 1 2 − π∥w1 ∥2 2 Hv∆ = 2 sin(α/2) , 1 1 2 2 − π∥w2 ∥2 51

The direction (1, −1), which moves the two students apart along v∆ , gives 2 sin(α/2) 1 1 ⊤ (1, −1)Hv∆ (1, −1) = − + < 0. π ∥w1 ∥2 ∥w2 ∥2 At (r+ v̄, −r− v̄), the off-diagonal Hessian block vanishes by the continuous extension in Lemma A.6, and the perturbation (0, v∆ ) of the second student gives 1 2 sin(α/2) sin(α/2) + (α/2) cos(α/2) < 0. =− − 2 πr− 2 sin(α/2) − (α/2) cos(α/2) Thus every listed configuration other than (v1 , v2 ) has a strictly negative Hessian direction.

C.5

■

Completion of the squared-loss landscape theorem

The statements of Proposition 3.7 and Lemmas 3.8 to 3.10 assume 0 < α ≤ π/2, but their proofs in the preceding subsections use only 0 < α ≤ α⋆ , with α⋆ as in (3.3). These results therefore hold for every α ∈ (0, α⋆ ], and they give the following extension of Theorem 3.5. Theorem C.7 (Squared-loss landscape up to the angle threshold). Let 0 < α ≤ α⋆ , and let (w1 , w2 ) with w1 , w2 ̸= 0 be a stationary point of the squared loss (2.2). After possibly swapping the students, exactly one of the following holds: (1) w1 = v1 and w2 = v2 . This is a global minimum. (2) The pair belongs to one of the three in-plane families w1 = s1 v̄,

w2 = s2 v̄,

s1 , s2 > 0,

s1 + s2 = r+ ,

w1 = −s1 v̄,

w2 = −s2 v̄,

s1 , s2 > 0,

w1 = r+ v̄,

w2 = −r− v̄.

s1 + s2 = r− ,

Every such point has a strictly negative Hessian direction. (3) Necessarily d ≥ 3, and there is a unit vector ξ ⊥ span{v1 , v2 } such that w1 = rα (κα )(cos κα v̄ + sin κα ξ), w2 = rα (κα )(cos κα v̄ − sin κα ξ). Every such point has a strictly negative Hessian direction. Consequently, every local minimum of the squared loss with w1 ̸= 0 and w2 ̸= 0 is a global minimum. Proof of Theorem C.7. By Lemma 3.3, the pair satisfies (3.4), so by Proposition 3.7 the vectors v1 , v2 , w1 , w2 span a space of dimension at most three. If w1 , w2 ∈ span{v1 , v2 }, Lemma 3.10 shows that the pair is (v1 , v2 ) or belongs to one of the three families in (2), and that each point of these families has a strictly negative Hessian direction. Otherwise, d ≥ 3, and there is a unit vector ξ ⊥ span{v1 , v2 } such that w1 , w2 ∈ span{v̄, v∆ , ξ}. By Lemma 3.8, w1 , w2 ∈ span{v̄, ξ}, and Lemma 3.9 shows that the pair has the form in (3) and has a strictly negative Hessian direction. Since this pair does not lie in span{v1 , v2 }, the three cases are disjoint. Conversely, the same lemmas show that every configuration in (1)–(3) is stationary. The pair (v1 , v2 ) has zero loss, and every other stationary point has a strictly negative Hessian direction. Since a local minimum with w1 ̸= 0 and w2 ̸= 0 is stationary, it is a global minimum. ■ Proof of Theorem 3.5. Since π/2 < α⋆ , the theorem is the restriction of Theorem C.7 to 0 < α ≤ π/2. ■ 52

Global convergence and boundary avoidance

For the proof of Theorem 4.3, we fix d ≥ 2 and unit teachers with α = ∠(v1 , v2 ) ∈ (0, π/2], and we write Ω = {(w1 , w2 ) ∈ (Rd )2 : w1 ̸= 0, w2 ̸= 0}, its boundary ∂Ω = {(w1 , w2 ) : w1 = 0 or w2 = 0}, G = {(v1 , v2 ), (v2 , v1 )} and ri = ∥wi ∥2 . A set is null if it has Lebesgue measure zero; on a manifold, this is understood in local coordinates. For the student output f in (2.1), the cross term 2E[ϕ(⟨w1 , x⟩)ϕ(⟨w2 , x⟩)] in E[f (x)2 ] is nonnegative, and E[ϕ(⟨wi , x⟩)2 ] = ri2 /2 by (A.1a). Since f 2 ≤ 2(f ∗ )2 + 2(f − f ∗ )2 , we obtain, for all w1 , w2 ∈ Rd , 1 2 r1 + r22 ≤ E[f (x)2 ] ≤ 2E[f ∗ (x)2 ] + 4L(w1 , w2 ). (D.1) 2 Since L does not increase along gradient flow, every trajectory stays in a bounded set determined by its initial loss.

D.1

Almost-everywhere avoidance of zero weights

For w(0) ∈ Ω, let [0, T ) be the maximal interval on which the solution of gradient flow exists in Ω; it is well defined by Lemma A.1, and T ∈ (0, ∞] depends on w(0). A trajectory may leave Ω in finite time, and it may approach ∂Ω as t → ∞. We show that both happen only for a null set of initializations. The first lemma shows that the two weights cannot tend to zero together. Lemma D.1 (Lower bound on ∥w1 ∥2 + ∥w2 ∥2 ). For every w(0) ∈ Ω and 0 ≤ t < T , r1 (t) + r2 (t) ≥ r− + r1 (0) + r2 (0) − r− e−t ≥ min{r1 (0) + r2 (0), r− }, where r− is defined in (3.5). Proof of Lemma D.1. By Lemma A.5, ṙi ≥ (r− − r1 − r2 )/2 for i = 1, 2. The sum therefore satisfies d dt (r1 + r2 ) ≥ r− − (r1 + r2 ), and Grönwall’s inequality gives the first inequality. The second holds because the middle expression is a convex combination of r1 (0) + r2 (0) and r− . ■ D.1.1

Polar coordinates near {w1 = 0}

By symmetry it suffices to study trajectories along which w1 tends to zero. We use the coordinates (r1 , w̄1 , w2 ), where w1 = r1 w̄1 , and set Ψw2 (w̄1 ) = l(θ11 ) + l(θ12 ) − r2 l(γ), with θ11 , θ12 , γ as in (2.7). Substituting w1 = r1 w̄1 into (A.10), with its convention at w1 = 0, gives L(r1 w̄1 , w2 ) = L(0, w2 ) − r1 Ψw2 (w̄1 ) +

r12 . 4

(D.2)

For a nonzero vector b, (A.2) shows that the spherical gradient of u 7→ l(∠(u, b)) at w̄1 is ∇Sd−1 l(∠(·, b)) (w̄1 ) = p ∠(w̄1 , b) Pw̄⊥1 b̄,

p(θ) =

π−θ . 2π

(D.3)

With this formula, (A.15) and (A.13) become the first two equations below, and differentiating (D.2) in w2 gives the third: r1 ṙ1 = Ψw2 (w̄1 ) − , 2 1 (D.4) w̄˙ 1 = ∇Sd−1 Ψw2 (w̄1 ), r1 ẇ2 = Fb (w2 ) + r1 ∇w2 Ψw2 (w̄1 ), 53

where Fb (w2 ) = −∇w2 L(0, w2 ); thus ẇ2 = Fb (w2 ) is the gradient flow of L restricted toR {w1 = 0}. t The factor 1/r1 in the second equation is removed by the rescaled time τ (t) = 0 dt′ /r1 (t′ ). With ′ = d/dτ , the triple (r1 , w̄1 , w2 ) solves (r1 , w̄1 , w2 )′ = X(r1 , w̄1 , w2 ), where the vector field X is r1 times the right-hand side of (D.4); its components are r1 (Ψw2 (w̄1 ) − r1 /2), ∇Sd−1 Ψw2 (w̄1 ) and r1 (Fb (w2 ) + r1 ∇w2 Ψw2 (w̄1 )). These formulas make sense for every r1 ∈ R and every w̄1 ∈ Sd−1 . By Lemma A.1, the kernel in (A.1a) is C 2 in its two nonzero arguments, so Ψw2 (w̄1 ) and L(0, w2 ) are C 2 in (w̄1 , w2 ) for w2 ̸= 0. Hence X is a C 1 vector field on R × Sd−1 × (Rd \ {0}), and X(0, w̄1 , w2 ) = (0, ∇Sd−1 Ψw2 (w̄1 ), 0). The next lemma describes a trajectory whose first weight tends to zero. Lemma D.2 (Behavior as Rw1 tends to zero). Let w(0) ∈ Ω, and suppose that r1 (t) → 0 as t ↑ T . If ∞ T = ∞, suppose also that 0 ∥ẇ(t)∥2 dt < ∞. Then w2 (t) converges to some w2,∗ = ̸ 0, τ (t) → ∞, and Ψw2 (t) (w̄1 (t)) converges to some ψ∗ ≤ 0, with ψ∗ = 0 if T = ∞. Every limit point u∗ of w̄1 (t) as t ↑ T satisfies ∇Sd−1 Ψw2,∗ (u∗ ) = 0 and Ψw2,∗ (u∗ ) = ψ∗ . Proof of Lemma D.2. By (D.1) and Lemma D.1, there are 0 < c0 ≤ C0 and t0 < T such that r1 (t) ≤ C0 and c0 ≤ r2 (t) ≤ C0 for t0 ≤ t < T . On the compact set {|r1 | ≤ C0 , c0 ≤ ∥w2 ∥2 ≤ C0 }, the field X, its derivative, and the right-hand sides of the first and third equations in (D.4) are bounded. If T < ∞, then ẇ2 and ṙ1 are bounded on [t0 , T ). Hence w2 (t) converges, and r1 (t) ≤ C(T − t) 0 for some C, so that τ (t) − τ (t0 ) ≥ C −1 log TT−t −t → ∞. If T = ∞, then w2 (t) converges because its path has finite length, and τ (t) − τ (t0 ) ≥ (t − t0 )/C0 → ∞. In both cases ∥w2,∗ ∥2 ≥ c0 , and R∞ RT ′ τ (t0 ) ∥w2 (τ )∥2 dτ = t0 ∥ẇ2 (t)∥2 dt < ∞. Let ψ(τ ) = Ψw2 (τ ) (w̄1 (τ )). Since w̄1′ = ∇Sd−1 Ψw2 (w̄1 ), we have ψ ′ = ∥∇Sd−1 Ψw2 (w̄1 )∥22 + Dw2 Ψw2 (w̄1 )[w2′ ]. The last term is integrable, because Dw2 Ψ is bounded on the compact set above and w2′ is integrable. Since ψ is bounded, ψ Rminus the integral of this term is bounded and ∞ nondecreasing. Hence ψ has a limit ψ∗ , and ∥∇Sd−1 Ψw2 (w̄1 )∥22 dτ < ∞. The derivative of 2 ′ τ 7→ ∥∇Sd−1 Ψw2 (w̄1 )∥2 is bounded, because w̄1 and w2′ are bounded and ∇Sd−1 Ψ is C 1 . Barbalat’s lemma therefore gives ∇Sd−1 Ψw2 (w̄1 ) → 0, and continuity gives the two conditions at every limit point u∗ . Finally, ṙ1 = Ψw2 (w̄1 ) − r1 /2 by (D.4). If ψ∗ > 0, then ṙ1 ≥ ψ∗ /4 for t close to T , which contradicts r1 (t) → 0. If T = ∞ and ψ∗ < 0, then ṙ1 ≤ ψ∗ /2 for all large t, so r1 reaches zero in finite time, which contradicts T = ∞. ■ D.1.2

Equilibria of X with r1 = 0

Let Z0 be the set of points (0, w̄1 , w2 ) with w2 ̸= 0, ∇Sd−1 Ψw2 (w̄1 ) = 0 and Ψw2 (w̄1 ) ≤ 0. These points are equilibria of X, and by Lemma D.2, for a trajectory as in that lemma, the limit points of (r1 , w̄1 , w2 ) lie in Z0 . We show that the divergence of X is positive on Z0 except at two points, so that trajectories staying near the other points form a null set. Lemma D.3 (A sine inequality at equilibria of X). Let (0, w̄1 , w2 ) ∈ Z0 . Then r2 sin γ ≤ sin θ11 + sin θ12 , with equality if and only if Ψw2 (w̄1 ) = 0 and (w̄1 , w2 ) = (−v1 , v2 )

(w̄1 , w2 ) = (−v2 , v1 ).

(D.5)

Proof of Lemma D.3. For θ ∈ (0, π), define Q by Q(p(θ)) = l(θ)/ sin θ, with p as in (D.3). By (A.1b), Q(p(θ))/p(θ) = cot θ + 1/(π − θ). The function l(θ)/ sin θ has derivative −((π − θ) + sin θ cos θ)/(2π sin2 θ), which is negative because 12 | sin 2θ| = 12 | sin 2(π − θ)| < π − θ, and second 54

derivative (1 + (π − θ) cot θ)/(π sin2 θ) = 2l(θ)/ sin3 θ, which is positive because l > 0 on [0, π) by Lemma A.2. The derivative of cot θ + 1/(π − θ) is 1/(π − θ)2 − 1/ sin2 θ, which is negative because sin θ = sin(π − θ) < π − θ. Since p is affine and decreasing, Q is strictly increasing and strictly convex on (0, 1/2), and Q(p)/p is strictly increasing there. By (D.3), the condition ∇Sd−1 Ψw2 (w̄1 ) = 0 reads p(θ11 )Pw̄⊥1 v1 + p(θ12 )Pw̄⊥1 v2 = r2 p(γ)Pw̄⊥1 w̄2 ,

(D.6)

and the norm of the right-hand side is r2 p(γ) sin γ. Since Q is defined only on (0, 1/2), we first treat the case in which w̄1 is aligned with a teacher, say θ11 = 0, so that θ12 = α. Taking norms in (D.6) gives p(α) sin α = r2 p(γ) sin γ, so γ ∈ (0, π). Since Ψw2 (w̄1 ) ≤ 0 and l(0) = 1/2, 1 Q(p(γ)) + l(α) ≤ r2 l(γ) = p(α) sin α , 2 p(γ)

l(α) = p(α) sin α

Q(p(α)) . p(α)

Hence Q(p(γ))/p(γ) > Q(p(α))/p(α), so p(γ) > p(α) and r2 sin γ = p(α) sin α/p(γ) < sin α = sin θ11 + sin θ12 . We may therefore assume that θ11 , θ12 ̸= 0. Let S = sin θ11 + sin θ12 . Then S > 0, since S = 0 would force θ11 = θ12 = π and hence v1 = v2 . A teacher with θ1j = π contributes nothing to Ψw2 (w̄1 ) or to (D.6), because l(π) = p(π) = 0, and we omit it from the sums below. If the left-hand side of (D.6) vanishes, so does the right-hand side, and γ ∈ {0, π}. The case γ = π is impossible, because then Ψw2 (w̄1 ) = l(θ11 ) + l(θ12 ) > 0. Hence γ = 0 and r2 sin γ = 0 < S. Otherwise, let p̂ be the norm of the left-hand side of (D.6) divided by S. The triangle inequality gives 0 < p̂ ≤ (p(θ11 ) sin θ11 + p(θ12 ) sin θ12 )/S < 1/2, where the last inequality holds because p(θ1j ) < 1/2 for θ1j ̸= 0. Taking norms in (D.6) gives r2 p(γ) sin γ = S p̂ > 0, so γ ∈ (0, π). By Jensen’s inequality for the convex function Q with weights sin θ1j /S, and then by the monotonicity of Q, 2 2 X X sin θ1j sin θ1j l(θ11 ) + l(θ12 ) = S Q(p(θ1j )) ≥ SQ p(θ1j ) ≥ SQ(p̂). (D.7) S S j=1

j=1

On the other hand, Ψw2 (w̄1 ) ≤ 0 gives l(θ11 ) + l(θ12 ) ≤ r2 l(γ) = r2 p(γ) sin γ

Q(p(γ)) Q(p(γ)) = S p̂ . p(γ) p(γ)

Combining the two bounds, Q(p̂)/p̂ ≤ Q(p(γ))/p(γ). Hence p̂ ≤ p(γ), and r2 sin γ = S p̂/p(γ) ≤ S, which proves the inequality. It remains to identify the case of equality. If r2 sin γ = S, then p̂ = p(γ), so equality holds in (D.7), in the triangle inequality, and in Ψw2 (w̄1 ) ≤ 0. If both sin θ11 and sin θ12 were positive, strict convexity of Q would give θ11 = θ12 , and equality in the triangle inequality would make Pw̄⊥1 v1 and Pw̄⊥1 v2 positive multiples of each other. These projections have the same norm, and ⟨w̄1 , v1 ⟩ = ⟨w̄1 , v2 ⟩, so v1 = v2 , a contradiction. Hence one sine vanishes, say sin θ11 = 0; since θ11 ̸= 0, we have θ11 = π and w̄1 = −v1 . Then S = sin θ12 > 0 and p̂ = p(θ12 ), so p(γ) = p(θ12 ) gives γ = θ12 , and r2 sin γ = S gives r2 = 1. Now (D.6) gives Pw̄⊥1 v2 = Pw̄⊥1 w̄2 , and ⟨w̄1 , v2 ⟩ = cos θ12 = cos γ = ⟨w̄1 , w̄2 ⟩, so w2 = v2 . The case sin θ12 = 0 gives (w̄1 , w2 ) = (−v2 , v1 ). Conversely, Ψv2 (u) = l(∠(u, v1 )) for every u ∈ Sd−1 , because the two terms involving v2 cancel. This function is nonnegative and vanishes at u = −v1 , so both Ψv2 (−v1 ) and its spherical gradient vanish there. At (w̄1 , w2 ) = (−v1 , v2 ), we have r2 sin γ = sin(π − α) = sin θ11 + sin θ12 . The point (−v2 , v1 ) is symmetric. ■ 55

Lemma D.4 (Positive divergence of X). Let Z be the set Z0 without the two points with (w̄1 , w2 ) as in (D.5). Then div X > 0 on Z, where the divergence and the volume on R × Sd−1 × (Rd \ {0}) are those of the product metric. Consequently, every compact K ⊂ Z has a relatively compact open neighborhood U such that the set T (U ) of points of U whose forward orbit under X stays in U is null. Proof of Lemma D.4. At r1 = 0, the derivative of the first component of X with respect to r1 is Ψw2 (w̄1 ), the divergence of the second component on the sphere is tr HessSd−1 Ψw2 (w̄1 ), and the third component has the factor r1 , so its divergence in w2 vanishes. To compute the trace, let b be a unit vector, u ∈ Sd−1 and θ = ∠(u, b). By (A.2), the first and second derivatives of l(arccos(·)) at cos θ are (π − θ)/(2π) and 1/(2π sin θ). Hence the spherical Hessian of u 7→ l(∠(u, b)) (π−θ) cos θ ⊥ (π−θ) cos θ 1 θ θ ⊥ ⊤ ⊥ is 2π sin Pu , and its trace is sin = d sin θ Pu bb Pu − 2π 2π − (d − 1) 2π 2π − (d − 1)l(θ). This identity extends to θ ∈ {0, π} by continuity, since Ψ is C 2 . Applying it to the three terms of Ψw2 , we obtain d div X(0, w̄1 , w2 ) = (D.8) sin θ11 + sin θ12 − r2 sin γ − (d − 2)Ψw2 (w̄1 ). 2π On Z, the last term is nonnegative because d ≥ 2 and Ψw2 (w̄1 ) ≤ 0, and the first term is positive by Lemma D.3, since the points of equality are excluded. Hence div X > 0 on Z. Let K ⊂ Z be compact. By continuity, there are a relatively compact open neighborhood U of K and c > 0 such that div X ≥ c on U . Orbits that stay in U exist for all τ ≥ 0, because U is relatively compact. For each integer N ≥ 1, the set of points of U whose orbit stays in U on [0, N ] is open, by continuous dependence on initial conditions, and T (U ) is the intersection of these sets. Hence T (U ) is Borel. Let Φτ be the flow of X. For τ > 0, Φτ maps into U , and R τ T (U ) injectively ′ by Liouville’s formula its Jacobian determinant on T (U ) is exp 0 (div X) ◦ Φτ ′ dτ ≥ ecτ . Hence vol(U ) ≥ ecτ vol(T (U )), and letting τ → ∞ gives vol(T (U )) = 0. ■ We now turn this lemma into a statement about the original variables. Lemma D.5. There are null sets B1 , B2 , . . . ⊂ Ω with the following property. Let w be as in Lemma D.2, and suppose that no limit point of (w̄1 (t), w2 (t)) as t ↑ T is one of the two points in (D.5). Then there is n such that w(t) ∈ Bn for all t sufficiently close to T . Proof of Lemma D.5. For n ≥ 1, let Kn be the set of points (0, w̄1 , w2 ) ∈ Z with 1/n ≤ ∥w2 ∥2 ≤ n whose distance from the two excluded points is at least 1/n. Each Kn is closed and bounded, hence compact. Let Un be the neighborhood of Kn given by Lemma D.4. The map (r1 , w̄1 , w2 ) 7→ (r1 w̄1 , w2 ) is smooth between manifolds of dimension 2d, so it maps the null set T (Un ) onto a null set; let Bn be its intersection with Ω. Let w be as in the lemma. By Lemma D.2, the set of limit points of (r1 , w̄1 , w2 ) as τ → ∞ is a compact subset of Z. It has positive distance from the two excluded points, so it lies in some Kn . Since (r1 , w̄1 , w2 ) stays bounded, it approaches this limit set as τ → ∞, and hence lies in Un for all large τ . It therefore lies in T (Un ) for all large τ , and w(t) ∈ Bn for all t close to T . ■ D.1.3

The equilibria (0, −v1 , v2 ) and (0, −v2 , v1 )

At the two points in (D.5), both terms in (D.8) vanish, so Lemma D.4 does not apply. We treat (w̄1 , w2 ) = (−v1 , v2 ); exchanging v1 and v2 gives the other point. We show that a trajectory reaching this point in finite time does so at explicit rates, and that, after a blow-up of the variables by these rates, it lies on a manifold of dimension less than 2d. We write w̄1 = exp−v1 z with z ∈ v1⊥ , where exp−v1 is the exponential map of the sphere at −v1 , whose tangent space there is v1⊥ , and we let G(z) be the matrix of the round metric in these 56

normal coordinates. Then G(0) = I, the first derivatives of G vanish at z = 0, and G(z)z = z by the Gauss lemma. In rescaled time, the angular equation reads z ′ = G(z)−1 ∇z Ψw2 (exp−v1 z), and therefore ⟨z, z ′ ⟩ = ⟨z, ∇z Ψw2 (exp−v1 z)⟩. We write K(u, v) = ∥u∥2 ∥v∥2 l(∠(u, v)) for the kernel in (A.1a), so that Ψw2 (w̄1 ) = K(w̄1 , v1 ) + K(w̄1 , v2 ) − K(w̄1 , w2 ). Since ∠(exp−v1 z, v1 ) = π − ∥z∥2 , the first term equals l(π − ∥z∥2 ) = (sin ∥z∥2 − ∥z∥2 cos ∥z∥2 )/(2π) = ∥z∥32 a0 (∥z∥22 ), and its gradient is ∥z∥2 z a1 (∥z∥22 ), where a0 and a1 are real analytic with a0 (0) = 1/(6π) and a1 (0) = 1/(2π). By (A.9), G0 := ∇w2 K(−v1 , w2 ) w2 =v2 =

sin α v2 − αv1 , 2π

Fb (v2 ) =

sin α v2 + (π − α)v1 , 2π

and a direct computation gives G⊤ 0 Fb (v2 ) = −να , where να =

α(π − α) − sin2 α − (π − 2α) sin α cos α . 4π 2

The numerator vanishes at α = 0 and has derivative 2(π − 2α) sin2 α, which is positive on (0, π/2); hence να > 0 for α ∈ (0, π/2]. Since Fb (v2 ) ̸= 0, the flow ẇ2 = Fb (w2 ) passes through v2 with nonzero velocity, and we first straighten its orbit near v2 . Lemma D.6 (Coordinates near v2 ). There are a cubic polynomial curve Γ with Γ(0) = v2 , Γ′ (0) = −Fb (v2 ), Γ′′ (0) = DFb (v2 )Fb (v2 ) and Γ′ (s) = −Fb (Γ(s)) + O(s3 ) as s ↓ 0, and a linear isometry J : Rd−1 → Fb (v2 )⊥ , such that χ(s, y) = Γ(s) + Jy is a real-analytic diffeomorphism from a neighborhood of (0, 0) onto a neighborhood of v2 . Writing w2 = χ(s, y) along the trajectory, the third equation in (D.4) reads ṡ = −1 + Rs (s, y) + r1 Bs (s, y, z),

ẏ = Ry (s, y) + r1 By (s, y, z),

(D.9)

where B = (Bs , By ) is real analytic near the origin, R = (Rs , Ry ) is C 1 , and, for some C, δ > 0, ∥R(s, y)∥2 ≤ C(s3 + ∥y∥2 ) whenever 0 ≤ s ≤ δ and ∥y∥2 ≤ δ. Moreover, for every Λ < ∞, after b on a neighborhood of {0} × {ŷ : ∥ŷ∥2 ≤ Λ} such that decreasing δ, there is a real-analytic function R 3 3 b R(s, s ŷ) = s R(s, ŷ) for 0 ≤ s ≤ δ and ∥ŷ∥2 ≤ Λ. Proof of Lemma D.6. For w2 = v2 + sω with s > 0 and Pv⊥2 ω ̸= 0, (A.9) gives ! ! s∥Pv⊥2 ω∥2 v2 1 s∥Pv⊥2 ω∥2 ∇w2 K(w2 , v2 ) = + w2 − arctan v2 . 2 2π 1 + s⟨v2 , ω⟩ ∥w2 ∥22 ⊥ The right-hand side is real analytic in (s, ω) near (0, −Fb (v2 )), because Pv⊥2 Fb (v2 ) = π−α 2π Pv2 v1 ̸= 0. Replacing ∇w2 K(w2 , v2 ) by it in Fb (w2 ) = −w2 /2 + ∇w2 K(w2 , v1 ) + ∇w2 K(w2 , v2 ) defines a realanalytic function Fe(s, ω) near (0, −Fb (v2 )) with Fe(s, ω) = Fb (v2 +sω) for s > 0 and Fe(0, ω) = Fb (v2 ) for all ω. In particular, Dω Fe(0, ·) and Dω2 Fe(0, ·) vanish, and ∂s Fe(0, ω) = DFb (v2 )ω, since Fb is C 1 by Lemma A.1. Define 1 11 2 e ω1 = − ∂s Fe(0, −Fb (v2 )), ω2 = − ∂s F (0, −Fb (v2 )) + Dω ∂s Fe(0, −Fb (v2 ))[ω1 ] , 2 3 2

and Γ(s) = v2 + s ω(s) with ω(s) = −Fb (v2 ) + sω1 + s2 ω2 . Expanding Fe(s, ω(s)) in s, and using that Dω Fe(0, ·) and Dω2 Fe(0, ·) vanish, we find that the coefficients of 1, s and s2 in Fe(s, ω(s)) + Γ′ (s) vanish. This proves Γ′ (s) = −Fb (Γ(s)) + O(s3 ), and Γ′′ (0) = 2ω1 = DFb (v2 )Fb (v2 ). 57

Fix a linear isometry J : Rd−1 → Fb (v2 )⊥ . The matrix [Γ′ (s) J] is invertible for small s, because Γ′ (0) is orthogonal to the range of J. By the inverse function theorem, χ is a real-analytic diffeomorphism near (0, 0). Define R(s, y) = [Γ′ (s) J]−1 Fb (χ(s, y)) + Γ′ (s) , B(s, y, z) = [Γ′ (s) J]−1 ∇w2 Ψw2 (exp−v1 z) w2 =χ(s,y) . Since ẇ2 = Γ′ (s)ṡ + J ẏ, the third equation in (D.4) becomes (D.9). The function R is C 1 because Fb is. The function B is real analytic, because ∇w2 Ψw2 (w̄1 ) = −∇w2 K(w̄1 , w2 ) and the angle between exp−v1 z and χ(s, y) stays near π − α ∈ (0, π), where K is analytic. By the property of Γ, R(s, 0) = O(s3 ), and R is Lipschitz in y; this gives the bound on R. Finally, for bounded ŷ we have χ(s, s3 ŷ) = v2 + s ω(s) + s2 J ŷ . Since Dω Fe(0, ·) and Dω2 Fe(0, ·) vanish, the coefficients of 1, s and s2 in Fe(s, ω(s) + s2 J ŷ) + Γ′ (s) do not depend on ŷ, and so they b ŷ), which vanish. Dividing this analytic function by s3 and multiplying by [Γ′ (s) J]−1 defines R(s, b ŷ) for s > 0. satisfies R(s, s3 ŷ) = s3 R(s, ■ Along a trajectory that reaches (0, −v1 , v2 ) at time T , we will see in Lemma D.8 that Ψw2 (w̄1 ) behaves like −να (T − t) and r1 like να (T − t)2 /2, that s is close to T − t, and that ∥z∥2 and ∥y∥2 are at most of order (T − t)1/2 and (T − t)3 . We therefore blow up the variables by these powers. When α = π/2, the constant term in the angular equation of the blow-up vanishes and ∥z∥2 is at most of order T − t, so this case uses different powers. The next lemma gives the expansions and the equations in the blow-up variables. Lemma D.7 (Blow-up near (0, −v1 , v2 )). Let h(z) = Dw2 K(exp−v1 z, w2 )[Fb (v2 )] w2 =v2 and β0 = sin α 1 By (0, 0, 0). Then ∇h(0) = − (π−2α) (v2 −cos α v1 ), and if α = π/2, then ∇2 h(0) = 16 I + 4π1 2 v2 v2⊤ , 4π 2 where I is the identity on v1⊥ . For bounded ŷ and (z, s) near (0, 0),

Ψχ(s,s3 ŷ) (exp−v1 z) = l(π − ∥z∥2 ) + s g(z, s, ŷ),

(D.10)

where g is real analytic, g(z, 0, ŷ) = h(z), and, if α = π/2, ∂s ∇z g(0, 0, ŷ) = v2 /(16π 2 ). Fix Λ < ∞, and let ∥ẑ∥2 , |r̂|, ∥ŷ∥2 ≤ Λ. Suppose first that 0 < α < π/2. If we set s = ζ 2 , z = ζ ẑ, r1 = ζ 4 r̂ and y = ζ 6 ŷ with ζ > 0, and change time by dσ = ζ dτ , the rescaled-time equations for (s, z, r1 , y) become dζ = − 12 ζ 2 r̂ + ζ 6 E1 , dσ dr̂ = ζ r̂(2r̂ − να ) + ζ 2 E3 , dσ

dẑ = q1 (ẑ) + ζE2 , dσ dŷ = ζ(r̂2 β0 + 3r̂ŷ) + ζ 2 E4 , dσ

(D.11)

where q1 (ẑ) = ∥ẑ∥2 ẑ/(2π) + ∇h(0). If α = π/2, we set instead z = sẑ, r1 = s2 r̂ and y = s3 ŷ with s > 0, and dσ = s dτ ; this gives ds = −sr̂ + s3 E1 , dσ dr̂ = r̂(2r̂ − να ) + sE3 , dσ

dẑ = q2 (ẑ, r̂) + sE2 , dσ dŷ = r̂2 β0 + 3r̂ŷ + sE4 , dσ

(D.12)

where q2 (ẑ, r̂) = ∥ẑ∥2 ẑ/(2π)+(∇2 h(0)+ r̂I)ẑ +v2 /(16π 2 ). In the first case, the remainders E1 , . . . , E4 extend to C 1 functions on a neighborhood of {ζ = 0}, including negative values of ζ; in the second case, the same holds with s in place of ζ. 58

Proof of Lemma D.7. By (A.9), h(z) = sin θ12 ⟨v2 , Fb (v2 )⟩ + (π − θ12 )⟨w̄1 , Fb (v2 )⟩ /(2π), where w̄1 = exp−v1 z. At z = 0, w̄1 = −v1 and θ12 = π − α, so h(0) = G⊤ 0 Fb (v2 ) = −να . Since w̄1 = −v1 + z + O(∥z∥22 ) and cos θ12 = − cos α + ⟨z, v2 ⟩ + O(∥z∥22 ), the differential of θ12 at z = 0 is −⟨v2 , dz⟩/ sin α. Differentiating h at z = 0, with ⟨v2 , Fb (v2 )⟩ = (sin α + (π − α) cos α)/(2π) and ⟨v1 , Fb (v2 )⟩ = ((π − α) + sin α cos α)/(2π), gives the formula for ∇h(0). When α = π/2, expanding h to second order in the same way gives h(z) = −να + ∥z∥22 /32 + ⟨v2 , z⟩2 /(8π 2 ) + O(∥z∥32 ), and hence the formula for ∇2 h(0). The terms K(exp−v1 z, v2 ) − K(exp−v1 z, χ(s, s3 ŷ)) of Ψχ(s,s3 ŷ) (exp−v1 z) are real analytic near (z, s) = (0, 0), because both angles stay near π − α, and they vanish at s = 0. Hence they equal s g(z, s, ŷ) with g real analytic, and g(z, 0, ŷ) = −Dw2 K(exp−v1 z, v2 )[Γ′ (0)] = h(z). Since ŷ enters χ(s, s3 ŷ) only at order s3 , a second-order expansion in s gives ∂s g(z, 0, ŷ) = −

1 2 K(exp z, v )[F (v ), F (v )] . Dw2 K(exp−v1 z, v2 )[Γ′′ (0)] + Dw 2 b 2 b 2 −v1 2 2

1 1 When α = π/2, we have Γ′′ (0) = DFb (v2 )Fb (v2 ) = ( 4π − 18 )v1 − 4π v2 , and at z = 0 the gradients in 2 z of the two terms in the bracket are −v2 /(8π ) and 0; hence ∂s ∇z g(0, 0, ŷ) = v2 /(16π 2 ). By the expansion of l(π − ∥z∥2 ) given at the beginning of this subsection and G(z)−1 z = z, the angular equation is z ′ = ∥z∥2 z a1 (∥z∥22 ) + s G(z)−1 ∇z g(z, s, ŷ), with y = s3 ŷ. Because G(0) = I and DG(0) = 0, the function G−1 ∇z g equals ∇h(0) at (0, 0, ŷ), and its derivatives in z and s there are ∇2 h(0) and ∂s ∇z g(0, 0, ŷ). Suppose that 0 < α < π/2, and substitute z = ζ ẑ and s = ζ 2 . The functions g(ζ ẑ, ζ 2 , ŷ)−g(0, 0, ŷ) and G−1 ∇z g(ζ ẑ, ζ 2 , ŷ)−∇h(0) are analytic and vanish at ζ = 0, so they are ζ times analytic functions; also, a1 (ζ 2 ∥ẑ∥22 )−1/(2π) is ζ 2 times an analytic function, and ẑ 7→ ∥ẑ∥32 and ẑ 7→ ∥ẑ∥2 ẑ are C 1 . Hence Ψw2 (w̄1 ) = ζ 2 (−να + ζk1 ) and z ′ = ζ 2 (q1 (ẑ) + ζk2 ), where k1 and k2 are C 1 functions of (ζ, ẑ, ŷ) on a neighborhood of ζ = 0. Suppose now that α = π/2, and substitute z = sẑ. Then G−1 ∇z g(sẑ, s, ŷ) vanishes at s = 0, and its derivative in s there is ∇2 h(0)ẑ + v2 /(16π 2 ). Taylor’s formula with integral remainder gives, in the same way, Ψw2 (w̄1 ) = s(−να + sk1 ) and z ′ = s2 (q2 (ẑ, 0) + sk2 ), with k1 and k2 C 1 on a neighborhood of s = 0. By (D.4), (D.9) and Lemma D.6, the remaining rescaled-time equations are bs + r1 Bs , b y + r 1 By , s′ = r1 −1 + s3 R y ′ = r1 s3 R r1′ = r1 Ψw2 (w̄1 ) − r1 /2 .

We substitute the blow-up variables and divide by ζ to pass to the time σ (by s when α = π/2), and we write By = β0 + ζ · (analytic) (or By = β0 + s · (analytic) when α = π/2). For example, bs − ζ 4 r̂Bs ), so dζ/dσ = ζ ′ /ζ = − 1 ζ 2 r̂ + ζ 6 E1 . The other equations s = ζ 2 gives 2ζζ ′ = −ζ 4 r̂(1 − ζ 6 R 2 follow in the same way from ẑ ′ = (z ′ − ζ ′ ẑ)/ζ, r̂′ = (r1′ − 4ζ 3 ζ ′ r̂)/ζ 4 and ŷ ′ = (y ′ − 6ζ 5 ζ ′ ŷ)/ζ 6 , and their analogues for α = π/2; the term r̂ẑ in q2 comes from −s′ ẑ/s. Each Ei is a sum of products of analytic functions, k1 , k2 , and polynomials in the blow-up variables. These expressions remain defined and C 1 for ζ ≤ 0 (for s ≤ 0 when α = π/2), which gives the stated extension. ■ Lemma D.8 (Rates of approach to (0, −v1 , v2 )). Let w(0) ∈ Ω with T < ∞, and suppose that r1 (t) → 0, w̄1 (t) → −v1 and w2 (t) → v2 as t ↑ T . Then, as t ↑ T , s(t) → 1, T −t

r1 (t) να → , (T − t)2 2

y(t) να → − β0 , (T − t)3 6 √ and there are M < ∞ and t0 < T such that ∥z(t)∥2 ≤ M T − t for t0 ≤ t < T . If α = π/2, then also ∥z(t)∥2 ≤ M (T − t) for t0 ≤ t < T . 59

Proof of Lemma D.8. Throughout the proof, t is close to T , and C denotes a constant that may change from line to line. By the third equation in (D.4), ẇ2 → Fb (v2 ), because r1 → 0 and ∇w2 Ψ is bounded near (−v1 , v2 ); so (w2 (t) − v2 )/(T − t) → −Fb (v2 ) and ∥w2 (t) − v2 ∥2 ≤ C(T − t). The difference K(exp−v1 z, v2 ) − K(exp−v1 z, w2 ) is real analytic near (z, w2 ) = (0, v2 ), vanishes at w2 = v2 , and has gradient −G0 in w2 at (0, v2 ). Together with the expansion of l(π − ∥z∥2 ) given at the beginning of this subsection, this gives δ > 0 and C1 < ∞ such that, for ∥z∥2 ≤ δ and ∥w2 − v2 ∥2 ≤ δ, ∥z∥32 5 2 + G⊤ 0 (w2 − v2 ) ≤ C1 ∥z∥2 + ∥z∥2 ∥w2 − v2 ∥2 + ∥w2 − v2 ∥2 , 6π ∥z∥2 z ≤ C1 ∥w2 − v2 ∥2 + ∥z∥32 . ∇z Ψw2 (exp−v1 z) − 2π 2

Ψw2 (exp−v1 z) −

(D.13)

By the first inequality, Ψw2 (t) (w̄1 (t)) ≥ −C(T − t), since ∥z∥32 /(6π) − C1 ∥z∥52 ≥ 0 for small ∥z∥2 . Solving the first equation in (D.4) backward from r1 (T ) = 0 gives Z T r1 (t) = − t

′

e(t −t)/2 Ψw2 (t′ ) (w̄1 (t′ )) dt′ ,

(D.14)

and hence 0 < r1 (t) ≤ C(T − t)2 . By (D.9), ṡ → −1, and s(t) → 0 because w2 → v2 = χ(0, 0). Integrating (D.9) from t to T , and using the bound on R in Lemma D.6 and the boundedness of B, we obtain Z T |s(t) − (T − t)| + ∥y(t)∥2 ≤ C r1 (t′ ) + (T − t′ )3 + ∥y(t′ )∥2 dt′ . t

Since r1 (t′ ) ≤ C(T − t′ )2 , Grönwall’s inequality gives |s(t) − (T − t)| + ∥y(t)∥2 ≤ C(T − t)3 .

In particular, s(t)/(T − t) → 1, and (T − t)/2 ≤ s(t) ≤ 2(T − t). Next we bound ∥z∥2 . By the second inequality in (D.13) and the identity for ⟨z, z ′ ⟩ given at the beginning of this subsection, the upper right Dini derivative of ∥z∥2 in rescaled time satisfies √ Dτ+ ∥z∥2 ≥ c2 ∥z∥22 − C2 s for some c2 , C2 > 0. Let m1 = ∥z∥2 / s. Since s′ = −r1 (1 − Rs − r1 Bs ) ≤ 0, √ the factor s−1/2 does not decrease, so Dτ+ m1 ≥ s (c2 m21 − C2 ). Choose M1 with c2 M12 ≥ 2C2 . If √ m1 ≥ M1 at some time t1 , then m1 stays above M1 , and Dτ+ m1 ≥ (c2 /2) s m21 gives Z p 1 c2 t s(t′ ) ′ 1 ≤ − dt . m1 (t) m1 (t1 ) 2 t1 r1 (t′ ) √ Since s/r1 ≥ C −1 (T − t)−3/2 , the integral tends to infinity as t ↑ T , and thepright-hand side √ becomes negative, which is impossible. Hence m1 < M1 , and ∥z∥2 ≤ M1 s ≤ M1 √ 2(T − t). Dividing the first inequality in (D.13) by T − t, and using ∥z∥32 /(T − t) ≤ C T − t → 0 and (w2 − v2 )/(T − t) → −Fb (v2 ), we obtain Ψw2 (w̄1 )/(T − t) → G⊤ 0 Fb (v2 ) = −να . Substituting t′ = t + λ(T − t) in (D.14) gives r1 (t) =− (T − t)2

Z 1

eλ(T −t)/2

Ψw2 (t′ ) (w̄1 (t′ )) dλ. T −t

For fixed λ ∈ [0, 1), the quotient in the integrand equals (1−λ)Ψw2 (t′ ) (w̄1 (t′ ))/(T −t′ ), which tends to R1 −(1−λ)να , and these quotients are bounded. Dominated convergence gives r1 (t)/(T −t)2 → να 0 (1− RT λ) dλ = να /2. Finally, y(t) = − t Ry (s, y) + r1 By (s, y, z) dt′ . The integral of Ry is O((T − t)4 ) by the bound on R. In the integral of r1 By , the substitution t′ = t + λ(T − t), the limit of r1 /(T − t)2 , R1 the limit By → β0 , and dominated convergence give y(t)/(T − t)3 → − ν2α β0 0 (1 − λ)2 dλ = − ν6α β0 . 60

It remains to treat α = π/2, where ∇h(0) = 0. Write y = s3 ŷ; by the bounds above, ŷ is bounded. By (D.10) and Taylor’s formula for ∇z g at (0, 0, ŷ), s2 v2 + O s∥z∥22 + s2 ∥z∥2 + s3 , 2 16π √ 2 uniformly in ŷ. Since ∇ h(0) is positive definite and ∥z∥2 ≤ M1 s, this gives DRτ+ ∥z∥2 ≥ c3 ∥z∥22 −C3 s2 t for some c3 , C3 > 0. Let m2 = ∥z∥2 /s. As before, Dτ+ m2 ≥ s(c3 m22 −C3 ), and t1 s(t′ )/r1 (t′ ) dt′ → ∞ because s/r1 ≥ C −1 (T − t)−1 . The argument used for m1 gives ∥z∥2 ≤ M2 s ≤ 2M2 (T − t). The lemma holds with M the largest of the constants obtained. ■ ∇z Ψw2 (exp−v1 z) = ∥z∥2 z a1 (∥z∥22 ) + s∇2 h(0)z +

Proposition D.9 (Trajectories reaching (0, −v1 , v2 )). There is a null set B ⊂ Ω with the following property. If w(0) ∈ Ω, T < ∞, and r1 (t) → 0, w̄1 (t) → −v1 and w2 (t) → v2 as t ↑ T , then w(t) ∈ B for all t sufficiently close to T . Proof of Proposition D.9. Suppose first that 0 < α < π/2, and consider the blow-up variables of √ (D.11). By Lemma D.8, as t ↑ T we have ζ = s → 0, r̂ = r1 /s2 → να /2, ŷ = y/s3 → −να β0 /6, √ √ and ẑ = z/ s stays bounded. Moreover, dσ/dt = s/r1 behaves like (2/να )(T − t)−3/2 , so σ → ∞. We apply Lemma D.7 with Λ larger than the bounds on ∥ẑ∥2 , |r̂| and ∥ŷ∥2 along the trajectory. Then Y = (ζ, ẑ, r̂, ŷ) is a bounded solution of (D.11) for all large σ, with ζ(σ) → 0. Since (π − 2α) sin α > 0, we have ∇h(0) ̸= 0, and q1 = ∇V1 for the strictly convex function V1 (ẑ) = ∥ẑ∥32 /(6π) + ⟨∇h(0), ẑ⟩, whose only critical point is ẑ1∗ = −(2π/∥∇h(0)∥2 )1/2 ∇h(0). Along a solution of dẑ/dσ = q1 (ẑ), the derivative of 12 ∥ẑ − ẑ1∗ ∥22 is ⟨ẑ − ẑ1∗ , ∇V1 (ẑ) − ∇V1 (ẑ1∗ )⟩. By the monotonicity of ∇V1 along rays from ẑ1∗ and the compactness of the unit sphere, this derivative has a positive lower bound on {∥ẑ − ẑ1∗ ∥2 ≥ a} for each a > 0. Hence no solution with ẑ(σ ′ ) ̸= ẑ1∗ at some σ ′ is defined and bounded on [σ ′ , ∞), so a solution that is defined and bounded on R is constant, equal to ẑ1∗ . Let σn → ∞. The translates Y (σn + ·) are bounded and have bounded derivatives, so a subsequence converges uniformly on compact intervals to a solution of (D.11) on R with ζ ≡ 0. Its ẑ-component is a bounded solution of dẑ/dσ = q1 (ẑ) on R, so it equals ẑ1∗ . Therefore ẑ(σ) → ẑ1∗ , and Y (σ) converges to Y1∗ = (0, ẑ1∗ , να /2, −να β0 /6). At Y1∗ , the partial derivatives of the right-hand sides of the ζ-, r̂- and ŷ-equations vanish: the first is O(ζ 2 ), and the other two are ζ times functions that vanish at Y1∗ , plus O(ζ 2 ). In the order (ζ, r̂, ŷ, ẑ), the Jacobian at Y1∗ is therefore block lower triangular, with a zero block of size d + 1 and the block Dq1 (ẑ1∗ ) = (2π)−1 (∥ẑ1∗ ∥2 I + ẑ1∗ (ẑ1∗ )⊤ /∥ẑ1∗ ∥2 ), which is positive definite. By the C 1 center-stable manifold theorem (Hirsch et al., 1977), there is a C 1 manifold M1 of dimension d + 1 through Y1∗ that contains every forward orbit staying in a small neighborhood of Y1∗ . Hence Y (σ) ∈ M1 for all large σ. Now let α = π/2, so that ∇h(0) = 0, and consider the blow-up variables of (D.12). By Lemma D.8, s → 0, r̂ = r1 /s2 → να /2, ŷ = y/s3 → −να β0 /6, and ẑ = z/s stays bounded. Moreover, dσ/dt = s/r1 behaves like (2/να )(T − t)−1 , so σ → ∞. At s = 0 and r̂ = να /2, the ẑ-equation is dẑ/dσ = ∇V2 (ẑ) for the strongly convex function ∥ẑ∥32 1 D 2 να E ⟨v2 , ẑ⟩ V2 (ẑ) = + ẑ, ∇ h(0) + I ẑ + , 6π 2 2 16π 2 whose only critical point ẑ2∗ is nonzero because ∇V2 (0) = v2 /(16π 2 ). The argument of the acute case shows that Y = (s, ẑ, r̂, ŷ) converges to Y2∗ = (0, ẑ2∗ , να /2, −να β0 /6). In the order (s, r̂, ẑ, ŷ), the Jacobian at Y2∗ is block lower triangular with diagonal blocks −να /2, να , Dẑ q2 (ẑ2∗ , να /2) and (3να /2)I, where 1 ∗ ẑ ∗ (ẑ ∗ )⊤ να Dẑ q2 (ẑ2∗ , να /2) = ∥ẑ2 ∥2 I + 2 ∗2 + ∇2 h(0) + I 2π ∥ẑ2 ∥2 2 61

is positive definite. Hence Y2∗ has one stable direction and 2d − 1 unstable directions, and its stable manifold M2 , a C 1 curve (Hirsch et al., 1977), contains Y (σ) for all large σ. Finally, we return to w. In the two cases, the maps (ζ, ẑ, r̂, ŷ) 7→ ζ 4 r̂ exp−v1 (ζ ẑ), χ(ζ 2 , ζ 6 ŷ) , (s, ẑ, r̂, ŷ) 7→ s2 r̂ exp−v1 (sẑ), χ(s, s3 ŷ) , defined for ζ > 0 and s > 0 respectively, are C 1 and send Y (σ) to w(t). The parts of M1 with ζ > 0 and of M2 with s > 0 are C 1 manifolds of dimensions d + 1 and 1, both less than 2d since d ≥ 2. Their images under these maps are therefore null in (Rd )2 . Let B be the intersection of the union of the two images with Ω. By the above, w(t) ∈ B for all t sufficiently close to T . ■ D.1.4

Avoidance of ∂Ω

The results above give null sets that contain w(t) for all t close to T . The next lemma turns them into null sets of initial points. Lemma D.10 (Preimages of null sets under the flow). Let A ⊂ Ω be a null set. The set of w(0) ∈ Ω for which w(t) ∈ A for all t in some nonempty open subinterval of [0, T ) is null. Proof of Lemma D.10. Such a trajectory lies in A at some rational time t∗ ≥ 0. Since L ∈ C 2 (Ω) by Lemma A.1, the time-t∗ map of gradient flow is a C 1 diffeomorphism from the open set of initial points with T > t∗ onto an open subset of Ω, with the backward flow as its inverse. Hence the set of w(0) with w(t∗ ) ∈ A is null, and the union over rational t∗ is null. ■ Proposition D.11 (Avoidance of ∂Ω in finite time). The set of w(0) ∈ Ω with T < ∞ is null. Proof of Proposition D.11. Let w(0) ∈ Ω with T < ∞. For i = 1, 2, 1/2 1/2 p ∥∇wi L(w)∥2 = E (f (x) − f ∗ (x)) 1(⟨wi , x⟩ > 0) x 2 ≤ E(f − f ∗ )2 E∥x∥22 ≤ 2dL(w(0)) along the trajectory, so w(t) is Lipschitz on [0, T ) and has a limit w(T ). By Lemma A.1, w(T ) ∈ / Ω, since otherwise the solution would extend beyond T ; and w(T ) ̸= (0, 0) by Lemma D.1. Exchanging the two students if necessary, we may assume that w1 (T ) = 0 ̸= w2 (T ). Then Lemma D.2 applies. If no limit point of (w̄1 (t), w2 (t)) is one of the two points in (D.5), then by Lemma D.5, w(t) ∈ Bn for some n and all t close to T . Otherwise, say, (−v1 , v2 ) is a limit point. Then w2 (T ) = v2 , and the limit ψ∗ in Lemma D.2 equals Ψv2 (−v1 ) = 0. Since Ψv2 (u) = l(∠(u, v1 )) vanishes only at u = −v1 by Lemma A.2, every limit point of w̄1 (t) equals −v1 . Hence w̄1 (t) → −v1 , and by Proposition D.9, w(t) lies in its null set B for all t close to T . The point (−v2 , v1 ) is handled in the same way, with the teachers exchanged. In all cases, w(t) lies, for all t close to T , in one of countably many null sets: the sets Bn , the set B of Proposition D.9 and its analogue with the teachers exchanged, and the images of all these sets under the exchange of the two students. Lemma D.10 completes the proof. ■ A trajectory that stays in Ω for all time can still approach ∂Ω as t → ∞. We first show that every such trajectory converges, and then that its limit lies on ∂Ω only for a null set of initializations. Proposition D.12 (Convergence of global solutions). Let w(0) ∈ Ω with T = ∞. Then the trajectory has finite length, it converges to some w∞ ∈ (Rd )2 , and 0 ∈ ∂L(w∞ ), where ∂L is the limiting (Mordukhovich) subdifferential. In particular, ∇L(w∞ ) = 0 if w∞ ∈ Ω.

Proof of Proposition D.12. Since L(w(t)) is nonincreasing and nonnegative, it has a limit L∞ , and since d L(w(t)) = −∥∇L(w(t))∥22 , (D.15) dt R∞ we have 0 ∥∇L(w(t))∥22 dt < ∞. By (D.1), the trajectory is bounded. Hence there are tn → ∞ with ∇L(w(tn )) → 0 and w(tn ) → w∞ for some w∞ . For nonzero arguments, the kernel K is built from norms, inner products, algebraic operations and arccos on [−1, 1], whose graph is the inverse graph of cos on [0, π]. Extending it by zero where an argument vanishes preserves subanalyticity, and the bound 0 ≤ K(u, v) ≤ ∥u∥2 ∥v∥2 /2, which follows from Lemma A.2, gives continuity there. Hence L is continuous and globally subanalytic on (Rd )2 . On Ω, ∂L(w) = {∇L(w)}, because L is C 1 there. By continuity, L(w∞ ) = L∞ , and since ∇L(w(tn )) ∈ ∂L(w(tn )) tends to zero while L(w(tn )) → L(w∞ ), the closedness of the limiting subdifferential gives 0 ∈ ∂L(w∞ ). Because w∞ may lie on ∂Ω, where L is not differentiable, we use the nonsmooth Łojasiewicz inequality for continuous subanalytic functions (Bolte et al., 2007, Theorem 3.1). It gives r0 , C, δ0 > 0 and µ ∈ [0, 1) such that |L(w) − L∞ |µ ≤ C dist 0, ∂L(w) whenever ∥w − w∞ ∥2 < r0 and 0 < |L(w) − L∞ | < δ0 . If L(w(t)) = L∞ at some finite time, then by (D.15) the trajectory is constant from that time on. Otherwise, whenever ∥w(t) − w∞ ∥2 < r0 and L(w(t)) − L∞ < δ0 , −

1−µ d 1−µ ∥∇L(w(t))∥22 L(w(t)) − L∞ = (1 − µ) ≥ ∥ẇ(t)∥2 . µ dt (L(w(t)) − L∞ ) C

C Choose n so large that ∥w(tn ) − w∞ ∥2 < r0 /2, L(w(tn )) − L∞ < δ0 , and 1−µ (L(w(tn )) − L∞ )1−µ < r0 /2. If the trajectory left the ball {∥w − w∞ ∥2 < r0 } after tn , integrating the last inequality up to the first exit time would show that its length after tRn is less than r0 /2, which is impossible. Hence ∞ it stays in the ball, and the same inequality gives tn ∥ẇ(t)∥2 dt < ∞. The trajectory therefore converges, and its limit is w∞ . ■

Proposition D.13 (Avoidance of ∂Ω as t → ∞). The set of w(0) ∈ Ω with T = ∞ whose trajectory converges to a point of ∂Ω is null. Proof of Proposition D.13. Let w be such a trajectory, with limit w∞ ∈ ∂Ω. By Lemma D.1, w∞ ̸= (0, 0); exchanging the two students if necessary, we may assume that w∞ = (0, w2,∗ ) with w2,∗ ̸= 0. By Proposition D.12, the trajectory has finite length, so Lemma D.2 applies with T = ∞. Moreover, by the third equation in (D.4), ẇ2 (t) → Fb (w2,∗ ); since w2 (t) converges, this limit is zero. On the other hand, Fb (v2 ) ̸= 0 by the formula in Appendix D.1.3, and Fb (v1 ) ̸= 0 by symmetry. Hence w2,∗ ∈ / {v1 , v2 }, no limit point of (w̄1 (t), w2 (t)) is one of the two points in (D.5), and by Lemma D.5, w(t) ∈ Bn for some n and all large t. Together with the images of the sets Bn under the exchange of the two students, Lemma D.10 completes the proof. ■

D.2

Saddle avoidance and global convergence

It remains to show that trajectories converge to strict saddles in Ω only for a null set of initializations. Proposition D.14 (Avoidance of strict saddles). The set of w(0) ∈ Ω with T = ∞ whose trajectory converges to a strict saddle of L in Ω, that is, a stationary point in Ω at which ∇2 L has a negative eigenvalue, is null. 63

Proof of Proposition D.14. For n ≥ 1, choose a function in Cc∞ (Ω) equal to one on a neighborhood of {w : ∥w∥2 ≤ n, min(r1 , r2 ) ≥ 1/n}, and let Ln be its product with L, extended by zero outside Ω. Then Ln is C 2 on (Rd )2 by Lemma A.1, and its gradient has compact support, so its gradient flow is defined for all time. By Bah et al. (2022, Theorem 28), the set Sn of initial points whose Ln -gradient flow converges to a strict saddle of Ln is null. Suppose that a trajectory converges to a strict saddle w∞ ∈ Ω of L. Then there are n and t1 such that w∞ and w(t) for all t ≥ t1 lie in the open set where Ln = L. Hence w∞ is a strict saddle of Ln , and for every t ≥ t1 the Ln -gradient flow starting at w(t) follows the original trajectory, so S w(t) ∈ Sn . Lemma D.10, applied to n Sn ∩ Ω, completes the proof. ■ Proof of Theorem 4.3. The initializations with w1 (0) = 0 or w2 (0) = 0 form a null set, so we consider w(0) ∈ Ω. Outside the null set of Proposition D.11, T = ∞, that is, the solution exists for all t ≥ 0 and both weights stay nonzero. By Proposition D.12, it converges to some w∞ with 0 ∈ ∂L(w∞ ). Outside the null set of Proposition D.13, w∞ ∈ Ω, and then ∇L(w∞ ) = 0. By Theorem 3.5, every stationary point in Ω outside G has a strictly negative Hessian direction, that is, it is a strict saddle, and by Proposition D.14 convergence to a strict saddle happens only on a null set. Hence, for almost every initialization, the solution exists for all t ≥ 0, both weights stay nonzero, and w(t) converges to (v1 , v2 ) or (v2 , v1 ). ■

Approach to the bisector saddle

As in Lemma 4.1, we consider α = ∠(v1 , v2 ) ∈ (0, π/2]. We recall the set Ω of pairs (w1 , w2 ) with w1 , w2 ̸= 0 and the notation ri = ∥wi ∥2 from Appendix D. For w(0) ∈ Ω, we write [0, Tmax ) for the maximal interval on which the gradient-flow solution exists in Ω; it is well defined by Lemma A.1. By (3.5) and (A.1b), r− = 4l(π − α/2) and r+ = 4l(α/2), so Lemma A.2 gives 0 < r− < 2 and r+ ≤ 2. For unit vectors p, q ∈ Rd , we also use ∠(p, q) = 2 arcsin

∥p − q∥2 π ≤ ∥p − q∥2 , 2 2

(E.1)

which holds because arcsin is convex on [0, 1], with arcsin 0 = 0 and arcsin 1 = π/2.

E.1

Norm bounds

Lemma E.1. Let w(0) ∈ Ω. For 0 ≤ t < Tmax and i = 1, 2, we have ri (t) ≤ ri (0) + t. If, in addition, ri (0) ≤ 2, then ri (t) ≤ 2 for 0 ≤ t < Tmax . Proof of Lemma E.1. By Lemma A.5, ṙi ≤ 1 − ri /2 ≤ 1, and integration gives the first bound. d The same inequality reads dt (ri − 2) ≤ −(ri − 2)/2, so Grönwall’s inequality gives ri (t) − 2 ≤ −t/2 (ri (0) − 2)e ≤ 0. ■ Lemma E.2. Let w(0) ∈ Ω and T > 0 satisfy maxi=1,2 ri (0) + T ≤ r− /4. Then Tmax > T , and ri (0) +

r− t ≤ ri (t) ≤ ri (0) + t, 4

0 ≤ t ≤ T,

i = 1, 2.

Proof of Lemma E.2. Let 0 ≤ t < min{T, Tmax }. By Lemma E.1, ri (t) ≤ ri (0) + t ≤ r− /4 for i = 1, 2, so Lemma A.5 gives ṙi (t) ≥ r− /2 − (r1 (t) + r2 (t))/2 ≥ r− /4. Integration proves the two bounds for these t. If Tmax ≤ T , the bounds keep w(t), 0 ≤ t < Tmax , in the compact subset of Ω on which ri (0) ≤ ∥wi ∥2 ≤ r− /4 for i = 1, 2. By Lemma A.1, the solution then extends beyond Tmax , which is impossible. Hence Tmax > T , and the bounds hold at t = T by continuity. ■ 64

E.2

Alignment with the bisector and balance of the norms

The alignment of w̄i with v̄ is driven by the component along v̄ of the projected teacher force in (A.13). Lemma E.3. For a unit vector q ∈ Rd , let D E ψ(q) := v̄, Pq⊥ (π − ∠(q, v1 ))v1 + (π − ∠(q, v2 ))v2 be the component along v̄ of the projected teacher force at q. Then ψ(q) ≥ 0, with equality if and only if q = v̄ or q = −v̄. Proof of Lemma E.3. We write θj = ∠(q, vj ) for j = 1, 2. Since v̄ = (v1 + v2 )/(2 cos(α/2)), we have ⟨v̄, vj ⟩ = cos(α/2) and ⟨v̄, q⟩ = (cos θ1 + cos θ2 )/(2 cos(α/2)). Expanding Pq⊥ = I − qq ⊤ therefore gives 2 cos

α α ψ(q) = 2 cos2 (2π − θ1 − θ2 ) − (cos θ1 + cos θ2 ) (π − θ1 ) cos θ1 + (π − θ2 ) cos θ2 . 2 2

By symmetry we may assume θ1 ≥ θ2 . We set x = θ1 + θ2 and y = θ1 − θ2 . The triangle inequality for angles, applied to q, v1 , v2 and to −q, v1 , v2 , gives α ≤ x ≤ 2π − α and 0 ≤ y ≤ α ≤ π/2. Since cos θ1 + cos θ2 = 2 cos(x/2) cos(y/2) and (π − θ1 ) cos θ1 + (π − θ2 ) cos θ2 = (2π − x) cos

x y x y cos + y sin sin , 2 2 2 2

the identity above becomes 2 cos

α α x y y ψ(q) = 2(2π − x) cos2 − cos2 cos2 − sin x sin y. 2 2 2 2 2

(E.2)

As x/2 ∈ [α/2, π − α/2], we have cos2 (x/2) ≤ cos2 (α/2), so the right-hand side is at least α y y y α y y y 2(2π − x) cos2 sin2 − sin x sin y = 2 sin (2π − x) cos2 sin − cos sin x . 2 2 2 2 2 2 2 2 Since 0 ≤ y/2 ≤ π/4 and tan u ≥ u for 0 ≤ u < π/2, we have sin(y/2) ≥ (y/2) cos(y/2), and hence y α α 2 cos ψ(q) ≥ sin y (2π − x) cos2 − sin x . (E.3) 2 2 2 For x ≤ π, the last factor is at least π cos2 (α/2) − 1 ≥ π/2 − 1 > 0, because cos2 (α/2) ≥ 1/2. For π < x ≤ 2π − α, it is positive because sin x < 0 and 2π − x > 0. Since sin y ≥ 0, it follows that ψ(q) ≥ 0. If q = ±v̄, then θ1 = θ2 , so the teacher force is a multiple of v1 + v2 and Pq⊥ annihilates it; thus ψ(q) = 0. Conversely, suppose that ψ(q) = 0. Then (E.3) forces y sin y = 0, that is, y = 0, and (E.2) reduces to 2(2π − x)(cos2 (α/2) − cos2 (x/2)) = 0. Since x < 2π, this gives x = α or x = 2π − α. With y = 0, we have ⟨q, v̄⟩ = cos(x/2)/ cos(α/2), which equals 1 in the first case and −1 in the second. Hence q = v̄ or q = −v̄. ■ Lemma E.4. For every η ∈ (0, π/2) and ε ∈ (0, 1], there is δ = δ(α, η, ε) > 0, independent of d, such that ψ(q) ≥ δ for every unit vector q ∈ Rd with ∥q − v̄∥2 ≥ ε and ∠(q, v̄) ≤ π − η.

Proof of Lemma E.4. Let a = ⟨q, v̄⟩ and b = ⟨q, v∆ ⟩, so that a2 + b2 ≤ 1. By (2.4), v1 = cos(α/2)v̄ + sin(α/2)v∆ and v2 = cos(α/2)v̄ − sin(α/2)v∆ , so cos ∠(q, v1 ) = a cos(α/2) + b sin(α/2) and cos ∠(q, v2 ) = a cos(α/2) − b sin(α/2). Expanding Pq⊥ = I − qq ⊤ in the definition of ψ shows that ψ(q) = ψ̃(a, b) for a continuous function ψ̃ on the closed unit disk D ⊂ R2 that depends only on α. The two conditions on q read a ≤ 1 − ε2 /2 and a ≥ − cos η. They define a compact set K ⊂ D, which is nonempty because ε ≤ 1. Every point of D is the pair (a, b) of some unit vector in R3 , so Lemma E.3, applied in R3 to two teachers at angle α, shows that ψ̃ is nonnegative on D and vanishes only at (1, 0) and (−1, 0). Neither point lies in K. Hence δ := minK ψ̃ is positive, and it depends only on α, η and ε. ■ Lemma E.5. Let w(0) ∈ Ω, and let Ei := 21 ∥w̄i − v̄∥22 for i = 1, 2. For 0 ≤ t < Tmax , Ėi ≤ − and

r3−i ψ(w̄i ) r3−i + ≤ , 2πri 2ri 2ri

(E.4)

i = 1, 2,

r3−j d (E1 + E2 ) ≤ 2 max (E1 + E2 ). j=1,2 rj dt

(E.5)

Proof of Lemma E.5. Since Ei = 1 − ⟨w̄i , v̄⟩, (A.13) and (A.14) give Ėi = −

ψ(w̄i ) π − γ r3−i v̄, Pw̄⊥i w̄3−i . + 2πri 2π ri

The first inequality in (E.4) follows from 0 ≤ π − γ ≤ π and |⟨v̄, Pw̄⊥i w̄3−i ⟩| ≤ 1, and the second from Lemma E.3. For (E.5), let ei = w̄i − v̄. Since Pw̄⊥i w̄i = 0, we have Pw̄⊥i v̄ = −Pw̄⊥i ei and Pw̄⊥i w̄3−i = Pw̄⊥i (e3−i − ei ). Since Pw̄⊥i is an orthogonal projection, the Cauchy–Schwarz inequality gives v̄, Pw̄⊥i w̄3−i = Pw̄⊥i ei , Pw̄⊥i (e3−i − ei ) ≤ ∥ei ∥2 ∥e1 ∥2 + ∥e2 ∥2 . Dropping the nonpositive first term and summing over i, we obtain 2 r3−j r3−j d 1 (E1 + E2 ) ≤ max ∥e1 ∥2 + ∥e2 ∥2 ≤ max ∥e1 ∥22 + ∥e2 ∥22 , j=1,2 rj dt 2 j=1,2 rj which is (E.5).

■

While the weights are small, the factor 1/ri in the first term of (E.4) makes the alignment fast. The next lemma shows that it takes place on a time scale proportional to rinit . Lemma E.6. Let η ∈ (0, π/2), 0 < m ≤ M and ε ∈ (0, 1]. There are constants Λ, r0 > 0, depending only on α, η, m, M, ε and not on d, with the following property. If w(0) ∈ Ω satisfies rinit ≤ r0 ,

mrinit ≤ ri (0) ≤ M rinit ,

∠(w̄i (0), v̄) ≤ π − η,

i = 1, 2,

then Tmax > Λrinit and ∥w̄i (Λrinit ) − v̄∥2 < ε for i = 1, 2. Proof of Lemma E.6. Let δ = δ(α, η, ε/2) be the constant of Lemma E.4, and set M +Λ r− δ ε2 6π/δ Λ := M e −1 , B := , r0 := min , , . 2m 4(M + Λ) 4πB(M + Λ) 8BΛ Since maxi ri (0) + Λrinit ≤ (M + Λ)rinit ≤ r− /4, Lemma E.2 gives Tmax > Λrinit and mrinit ≤ ri (t) ≤ M rinit + t ≤ (M + Λ)rinit , 66

0 ≤ t ≤ Λrinit ,

i = 1, 2.

In particular, r3−i /ri ≤ 2B on this interval, and Lemma E.5 gives Ėi ≤ −

ψ(w̄i ) + B ≤ B, 2πri

0 ≤ t ≤ Λrinit ,

i = 1, 2.

(E.6)

Fix i, and suppose that ∥w̄i (t)− v̄∥2 ≥ ε/2 for all t ∈ [0, Λrinit ]. At every such t with ∠(w̄i (t), v̄) ≤ π − η, Lemma E.4, the bound ri ≤ (M + Λ)rinit and the choice of r0 give Ėi ≤ −

δ + B ≤ −B < 0. 2π(M + Λ)rinit

Let t∗ be the supremum of the times t ≤ Λrinit such that ∠(w̄i , v̄) ≤ π − η on [0, t]. By continuity, the angle condition holds on [0, t∗ ], so Ei is decreasing there and Ėi (t∗ ) < 0. The angle ∠(w̄i , v̄) = arccos(1 − Ei ) increases with Ei . If t∗ < Λrinit , then Ei (t) < Ei (t∗ ) ≤ Ei (0) for t slightly larger than t∗ , and the angle condition holds beyond t∗ , contradicting the definition of t∗ . Hence t∗ = Λrinit , and ψ(w̄i ) ≥ δ on the whole interval. The first bound in (E.6), together with ri (t) ≤ M rinit + t, now gives Z Λrinit δ δ Λ ε2 Ei (0) − Ei (Λrinit ) ≥ − B dt = log 1 + − BΛrinit ≥ 3 − > 2. 2π(M rinit + t) 2π M 8 0 This is impossible, since Ei ≥ 0 and Ei (0) = 1 − cos ∠(w̄i (0), v̄) ≤ 1 + cos η < 2. Hence there is ti ∈ [0, Λrinit ] with ∥w̄i (ti ) − v̄∥2 < ε/2, that is, Ei (ti ) < ε2 /8. The second boundp in (E.6) and the choice of r0 then give Ei (Λrinit ) ≤ Ei (ti ) + BΛrinit < ε2 /4, so ∥w̄i (Λrinit ) − v̄∥2 = 2Ei (Λrinit ) < ε. ■ After this interval, the norms need a time of order one to grow. As long as the ratio r3−i /ri stays bounded, (E.5) allows the direction errors to grow only by a bounded factor over such a time. Lemma E.7. Let C ≥ 1, η ∈ (0, π/2) and Tα := r− /8. For every ε ∈ (0, 1] and T > Tα , there is r0 = r0 (α, C, η, ε, T ) > 0, independent of d, with the following property. If the initialization is C-balanced, 0 < rinit ≤ r0 , and ∠(w̄i (0), −v̄) ≥ η for i = 1, 2, then Tmax > T and ∥w̄i (t) − v̄∥2 < ε,

|r1 (t) − r2 (t)| < ε,

Tα ≤ t ≤ T,

i = 1, 2.

Proof of Lemma E.7. Let 256 T := K exp C + 2 , r−

ε∗ :=

min{ε, r− /2} . 1 + 2πT

Let Λ and ra be √ the constants of Lemma E.6 for m = 2/(C + 1), M = 2, the margin η and the tolerance ε∗ /(2 K), and set r− Tα min{ε, r− /2} r0 := min ra , , , . 32 Λ 2(2 + Λ) Let 0 < rinit ≤ r0 . Since r1 (0) + r2 (0) = 2rinit and maxi ri (0) ≤ C mini ri (0), we have 2rinit /(C + 1) ≤ ri (0) ≤ 2rinit . Since rinit ≤ r− /32, we get maxi ri (0)+Tα ≤ r− /16+r− /8 < r− /4, so Lemma E.2 2 /32. gives Tmax > Tα and ri (0) + r− t/4 ≤ ri (t) ≤ ri (0) + t for 0 ≤ t ≤ Tα . Hence ri (Tα ) ≥ r− Moreover, r3−i (t)/ri (t) ≤ (r3−i (0) + t)/(ri (0) + r− t/4). The right-hand side is a monotone function of t that equals r3−i (0)/ri (0) ≤ C at t = 0 and tends to 4/r− as t → ∞. Hence r3−i (t) 4 ≤ max C, , 0 ≤ t ≤ Tα , i = 1, 2. (E.7) ri (t) r− 67

The angular hypothesis is equivalent to ∠( √ w̄i (0), v̄) ≤ π − η, so Lemma E.62applies. At τa := Λrinit ≤ Tα , it gives ∥w̄i (τa ) − v̄∥2 < ε∗ /(2 K), and hence E1 (τa ) + E2 (τa ) < ε∗ /(4K), with Ei as in Lemma E.5. Let T ′ := min{T, Tmax }, and let τ be the supremum of the times t ∈ [Tα , T ′ ) such that r1 , r2 > 2 /64 on [T , t]. Since r (T ) ≥ r 2 /32, we have τ > T . Since r (0) ≤ 2r r− α i α α i init ≤ 2, Lemma E.1 − 2 on [T , τ ). Since r gives ri ≤ 2 on [0, Tmax ), so r3−i /ri ≤ 128/r− < 2 and C ≥ 1, we have α − 2Tα max{C, 4/r− } = max{Cr− /4, 1} ≤ C, and therefore Z t n r3−j 4 o 256 T 2 max du ≤ 2Tα max C, τa ≤ t < τ. + 2 ≤ log K, r− r− τa j=1,2 rj Hence (E.5), (E.7) and Grönwall’s inequality give E1 (t) + E2 (t) ≤ K(E1 (τa ) + E2 (τa )) < ε2∗ /4 for τa ≤ t < τ , and therefore ∥w̄i (t) − v̄∥2 < ε∗ ,

τa ≤ t < τ,

i = 1, 2.

(E.8)

We next bound r1 − r2 . By (A.15), (A.16) and (A.1b), 2 X 1 d (r1 − r2 ) = l(θ1j ) − l(θ2j ) − − l(γ) (r1 − r2 ). dt 2 j=1

Since l(γ) ≤ 1/2 by Lemma A.2, solving this linear equation gives |r1 (t) − r2 (t)| ≤ |r1 (0) − r2 (0)| +

Z tX 2 0

l(θ1j ) − l(θ2j ) du.

j=1

The integrand is at most 1, because 0 ≤ l ≤ 1/2. It is also at most γ, because |l′ | ≤ 1/2 by (A.2) and |θ1j − θ2j | ≤ γ by the triangle inequality for angles. On [τa , τ ), (E.1) and (E.8) give γ ≤ ∠(w̄1 , v̄) + ∠(w̄2 , v̄) < πε∗ . Using the bound 1 on [0, τa ], |r1 (0) − r2 (0)| ≤ 2rinit and the choice of ε∗ and r0 , we obtain n r o − |r1 (t) − r2 (t)| ≤ (2 + Λ)rinit + πε∗ T < min ε, , 0 ≤ t < τ. (E.9) 2 2 /64 for some i, while r > r 2 /64 on [T , τ ). By (E.9) and Suppose that τ < T ′ . Then ri (τ ) = r− i α − 2 /64 + r /2, and Lemma A.5 gives ṙ (τ ) ≥ r /2 − r 2 /64 − r /4 > 0, because continuity, r3−i (τ ) ≤ r− − i − − − 2 /64} ≤ r− < 2. This contradicts ri > ri (τ ) on [Tα , τ ), so τ = T ′ . If Tmax ≤ T , then min{ri (0), r− ri ≤ 2 on [0, Tmax ) for i = 1, 2, so the trajectory stays in a compact subset of Ω; Lemma A.1 then extends the solution beyond Tmax , which is impossible. Hence Tmax > T and τ = T . By continuity, (E.9) also holds at t = T , and ∥w̄i (T ) − v̄∥2 ≤ ε∗ . Since ε∗ < ε and τa ≤ Tα , this proves the lemma. ■

E.3

Proofs of Lemma 4.1 and Corollary 4.2

Proof of Lemma 4.1. If the conclusion holds for some ρ, it holds with the same t0 and r∗ for every larger ρ, so we may assume ρ ≤ 1. Let Tα = r− /8 be as in Lemma E.7, and set t0 := Tα + log

16 , ρ

ε :=

ρ , 20

r∗ := min{r0 (α, C, η, ε, t0 ), 1},

where r0 is given by Lemma E.7. Then t0 depends only on α and ρ, and r∗ only on α, C, η, ρ. Suppose that 0 < rinit ≤ r∗ . By Lemma E.7, the solution exists on [0, t0 ], both weights are nonzero 68

there, and the bounds of that lemma hold on [Tα , t0 ]. Moreover, ri (0) ≤ 2rinit ≤ 2, so ri ≤ 2 on [0, t0 ] by Lemma E.1. Adding (A.15) and (A.16), and using r+ = 4l(α/2) and (A.1b), we obtain 2 α 1 X d (r1 + r2 ) + (r1 + r2 ) − r+ = + − l(γ) (r1 + r2 ). l(θij ) − l dt 2 2 i,j=1

We bound the right-hand side on [Tα , t0 ]. Since |l′ | ≤ 1/2 by (A.2) and |θij − α/2| ≤ ∠(w̄i , v̄) by the triangle inequality for angles, (E.1) gives |l(θij )−l(α/2)| < πε/4. In the same way, γ < πε, and hence 0 ≤ 12 − l(γ) = l(0) − l(γ) < πε/2. Since r1 + r2 ≤ 4, the right-hand side is less than 3πε in absolute value. Solving the linear equation for r1 + r2 − r+ on [Tα , t0 ], and using |r1 (Tα ) + r2 (Tα ) − r+ | ≤ 4, we get |r1 (t0 ) + r2 (t0 ) − r+ | ≤ 4e−(t0 −Tα ) + 3πε = ρ/4 + 3πε. Recall from Section 4 that rs = r+ /2 and ws = rs v̄. Hence ri − rs = 12 (r1 + r2 − r+ ) ± 12 (r1 − r2 ), and therefore |ri (t0 ) − rs | < ρ/8 + 3πε/2 + ε/2. Since wi − ws = (ri − rs )w̄i + rs (w̄i − v̄) and rs ≤ 1, it follows that ∥w(t0 ) − wsaddle ∥2 ≤

2 X

∥wi (t0 ) − ws ∥2 <

i=1

ρ + (3π + 3)ε < ρ. 4

Thus tin = t0 has the required property.

■

Proof of Corollary 4.2. Let λ := min{1/2, (C 2 − 1)/(C 2 + 1)} > 0. For each i, ∥wi (0)∥22 /σ 2 has the χ2 distribution with d degrees of freedom, whose moment generating function is (1 − 2s)−d/2 for s < 1/2. Markov’s inequality applied to exp(s∥wi (0)∥22 /σ 2 ), with s = λ/(2(1 + λ)) and with s = −λ/(2(1 − λ)), gives d 2 P ∥wi (0)∥22 ≥ (1 + λ)σ 2 d ≤ exp − λ − log(1 + λ) ≤ e−dλ /8 , 2 d 2 P ∥wi (0)∥22 ≤ (1 − λ)σ 2 d ≤ exp − −λ − log(1 − λ) ≤ e−dλ /4 . 2 The last inequalities hold because, for 0 < λ ≤ 1/2, Z λ λ − log(1 + λ) = 0

Z λ

u λ2 du ≥ , 1+u 4

−λ − log(1 − λ) = 0

u λ2 du ≥ . 1−u 2

Let E be the event that (1 − λ)σ 2 d < ∥wi (0)∥22 < (1 + λ)σ 2 d and ∠(w̄i (0), −v̄) ≥ η for i = 1, 2. If the norm p condition holds but the angular condition fails for some i, then ⟨wi (0), v̄⟩ < −∥wi (0)∥2 cos η < −σ (1 − λ)d cos η. Here ⟨wi (0), v̄⟩/σ is standard normal, and a standard normal Z satisfies 2 P{Z ≤ −a} ≤ e−a /2 for a ≥ 0. A union bound therefore gives 2 λ (1 − λ) cos2 η c −dλ2 /8 −(1−λ)d cos2 η/2 −cd P(E ) ≤ 4e + 2e ≤ 6e , c := min , . 8 2 On E, both initial weights are nonzero, and the choice of λ gives r p √ 1+λ max ∥wi (0)∥2 ≤ min ∥wi (0)∥2 ≤ C min ∥wi (0)∥2 , rinit < σ (1 + λ)d ≤ 2σ d ≤ r∗ . i i 1−λ i Hence Lemma 4.1 applies on E and gives the conclusion.

■

Exclusion of saddles and global convergence

As in Theorem 3.5, we consider α = ∠(v1 , v2 ) ∈ (0, π/2]. We recall the set Ω of pairs (w1 , w2 ) with w1 , w2 ̸= 0 and the notation ri = ∥wi ∥2 from Appendix D. For w(0) ∈ Ω, we write [0, Tmax ) for the maximal interval on which the gradient-flow solution exists in Ω; it is well defined by Lemma A.1. We use the coordinates zi,j of Section 4.2.2 with respect to the orthonormal basis {v̄, v∆ , u3 , . . . , ud }. Since the teachers are unit vectors at angle α, (2.4) gives v1 = cos(α/2) v̄ + sin(α/2) v∆ and v2 = cos(α/2) v̄ − sin(α/2) v∆ . Hence cos θi1 − cos θi2 = ⟨w̄i , v1 − v2 ⟩ =

2 sin(α/2) zi,2 , ri

i = 1, 2.

(F.1)

Since arccos is strictly decreasing, θi2 − θi1 has the sign of zi,2 and vanishes with it. Taking inner products of (A.11) and (A.12) with v̄, v∆ and uj , and noting that the terms in w̄i and wi there combine to ai wi /(2πri ), with ai as in Lemma 3.3, we obtain, on Ω, 1 α a1 ż1,1 = (2π − θ11 − θ12 ) cos + z1,1 − (π − γ)z2,1 , 2π 2 r1 α a2 1 (2π − θ21 − θ22 ) cos + z2,1 − (π − γ)z1,1 , ż2,1 = 2π 2 r2 1 α a1 ż1,2 = (θ12 − θ11 ) sin + z1,2 − (π − γ)z2,2 , 2π 2 r1 (F.2) 1 α a2 ż2,2 = (θ22 − θ21 ) sin + z2,2 − (π − γ)z1,2 , 2π 2 r2 1 a1 z1,j − (π − γ)z2,j , 3 ≤ j ≤ d, ż1,j = 2π r1 1 a2 ż2,j = z2,j − (π − γ)z1,j , 3 ≤ j ≤ d. 2π r2 By (F.1), the first term in the equation for zi,2 , which comes from the teachers, vanishes together with zi,2 . At a zero of one planar splitting coordinate, the sign of its derivative is therefore determined by the planar splitting coordinate of the other student. Lemma F.1 (Signs of the planar splitting coordinates). Let w(0) ∈ Ω with z1,2 (0)z2,2 (0) < 0. Then z1,2 (t)z2,2 (t) < 0 for 0 ≤ t < Tmax . Proof of Lemma F.1. Exchanging the two students if necessary, we may assume that z1,2 (0) > 0 > z2,2 (0). By (F.1), the right-hand sides of the third and fourth lines of (F.2) vanish on the subspace where z1,2 = z2,2 = 0, so the vector field of the flow is tangent to this subspace. The field is locally Lipschitz on Ω by Lemma A.1, and uniqueness of solutions in both time directions shows that a trajectory that meets the subspace lies in it at all times. Since z1,2 (0) ̸= 0, the trajectory never meets it. Suppose now that z1,2 (t)z2,2 (t) = 0 for some t < Tmax , and let t1 be the first such time. By the previous paragraph, only one of the two coordinates vanishes at t1 . We treat the case z1,2 (t1 ) = 0 > z2,2 (t1 ); the other is symmetric. Then γ(t1 ) < π, since otherwise w2 (t1 ) would be a negative multiple of w1 (t1 ) and z2,2 (t1 ) would vanish as well. As θ11 = θ12 at t1 , the third line of (F.2) gives ż1,2 (t1 ) = −(π − γ(t1 ))z2,2 (t1 )/(2π) > 0. This is impossible: z1,2 is positive on [0, t1 ) and vanishes at t1 , so ż1,2 (t1 ) ≤ 0. ■

Under the hypothesis of Lemma F.1, z1,2 and z2,2 keep opposite signs, and θi2 − θi1 has the sign of zi,2 . The third and fourth lines of (F.2) then give, for 0 ≤ t < Tmax , α ai d 1 |θi2 − θi1 | sin + |zi,2 | + (π − γ)|z3−i,2 | , i = 1, 2. (F.3) |zi,2 | = dt 2π 2 ri The equations for the out-of-plane coordinates in (F.2) involve the same factors ai /ri and π − γ as (F.3), but no term coming from the teachers. Lemma F.2 (Comparison of out-of-plane and planar coordinates). Let w(0) satisfy Assumption 4.5. Then |zi,j (t)| ≤ C⊥ |zi,2 (t)| and |z1,j (t) − z2,j (t)| ≤ C⊥ |z1,2 (t) − z2,2 (t)| for 0 ≤ t < Tmax , i ∈ {1, 2} and 3 ≤ j ≤ d. Proof of Lemma F.2. By Lemma F.1, (F.3) holds on [0, Tmax ). Fix j with 3 ≤ j ≤ d. For i = 1, 2, ± we set p± i := C⊥ |zi,2 | ∓ zi,j . The first claim says that all four functions pi are nonnegative, and by Assumption 4.5 this holds at t = 0. By (F.3) and the last two lines of (F.2), α ai ± 1 ∓ ± C⊥ |θi2 − θi1 | sin + pi + (π − γ)p3−i . ṗi = 2π 2 ri By Lemma F.1 and (F.1), θi2 ̸= θi1 . Hence, if all four functions are nonnegative and one of them vanishes, its derivative is positive. Let t2 be the supremum of the times t < Tmax such that all four functions are nonnegative on [0, t], and suppose that t2 < Tmax . Then one of them is negative at times arbitrarily close to t2 from above. By continuity, it vanishes at t2 while the others are nonnegative there, so its derivative at t2 is positive and it is positive just after t2 , which is a contradiction. This proves the first claim. Since z1,2 and z2,2 have opposite signs, |z1,2 | + |z2,2 | = |z1,2 − z2,2 |, and the second claim follows from |z1,j − z2,j | ≤ |z1,j | + |z2,j |. ■ Proof of Theorem 4.6. The first claim is the second estimate of Lemma F.2. If d = 2, the family S⊥ is empty by Theorem 3.5, so we may assume that d ≥ 3. As in Proposition 3.7, we write Pd ⊥ wi = j=3 zi,j uj for the component of wi orthogonal to the teacher plane. Since u3 , . . . , ud are orthonormal, Lemma F.2 gives √ ∥w1⊥ (t) − w2⊥ (t)∥2 ≤ C⊥ d − 2 |z1,2 (t) − z2,2 (t)|. (F.4) Let w b ∈ S⊥ . By (3.10), w b1 and w b2 are orthogonal to v∆ , and ∥w b1⊥ − w b2⊥ ∥2 = 2rα (κα ) sin κα > 0. By the Cauchy–Schwarz inequality, |z1,2 (t) − z2,2 (t)| =

(w1 (t) − w b1 ) − (w2 (t) − w b2 ), v∆

≤ ∥w1 (t) − w b1 ∥2 + ∥w2 (t) − w b2 ∥2 ≤

√

2 ∥w(t) − w∥ b 2,

and since orthogonal projection does not increase norms, the same bound holds with wi (t) − w bi replaced by wi⊥ (t) − w bi⊥ . The triangle inequality and (F.4) now give √ √ √ 2rα (κα ) sin κα ≤ ∥w1⊥ (t) − w2⊥ (t)∥2 + 2 ∥w(t) − w∥ b 2 ≤ 2 1 + C⊥ d − 2 ∥w(t) − w∥ b 2. √ √ Hence the theorem holds with c = 2 rα (κα ) sin κα /(1 + C⊥ d − 2). ■ Every in-plane saddle in (3.9) has zero planar splitting coordinates, while near each of them (F.3) makes at least one of these coordinates grow. Lemma F.3 (Avoidance of in-plane saddles). Let w(0) ∈ Ω with z1,2 (0)z2,2 (0) < 0 and Tmax = ∞. Then w(t) does not converge to an in-plane saddle in (3.9). 71

Proof of Lemma F.3. Suppose that w(t) converges to such a saddle w∞ . Both components of w∞ are multiples of v̄, so z1,2 (t) and z2,2 (t) tend to zero. By Lemma F.1, (F.3) holds for all t ≥ 0. First suppose that w∞ has one of the first two forms in (3.9), that is, w∞ = (s1 v̄, s2 v̄) or w∞ = (−s1 v̄, −s2 v̄) with s1 , s2 > 0. At w∞ , we have γ = 0 and sin θij = sin(α/2) for all i, j, so ai /ri + π − γ = 2 sin(α/2)/si . By continuity, ai /ri + π − γ ≥ sin(α/2)/si for all sufficiently large t. Adding the two equations in (F.3) and dropping the nonnegative terms that contain |θi2 − θi1 |, we obtain, for such t, 2 d 1 X |z1,2 | + |z2,2 | ≥ dt 2π i=1

ai sin(α/2) + π − γ |zi,2 | ≥ |z1,2 | + |z2,2 | . ri 2π max{s1 , s2 }

Thus |z1,2 | + |z2,2 | grows exponentially for large t, contradicting its convergence to zero. Otherwise, w∞ has the third form in (3.9). Let i be the index of the student whose limit is −r− v̄. At w∞ , we have γ = π and sin θi1 = sin θi2 = sin(α/2), so ai (w∞ ) = 2 sin(α/2) − πr− , which is positive because r− < (2/π) sin(α/2) by (3.5). By continuity, ai /ri ≥ ai (w∞ )/(2r− ) for all d sufficiently large t, and (F.3) gives dt |zi,2 | ≥ ai (w∞ )|zi,2 |/(4πr− ). Again |zi,2 | grows exponentially, contradicting its convergence to zero. ■ The conclusion of the next corollary is contained in Theorem 4.3. What is new is the proof, in which Theorem 4.6 and Lemma F.3 take the place of the stable-manifold argument of Proposition D.14. Corollary F.4 (Global convergence under Assumption 4.5). For Lebesgue-almost every initialization satisfying Assumption 4.5, the solution exists for all t ≥ 0, both student weights remain nonzero, and w(t) converges to (v1 , v2 ) or (v2 , v1 ). Proof of Corollary F.4. As in the proof of Theorem 4.3, outside the null set of Proposition D.11 we have Tmax = ∞, and by Proposition D.12 the solution converges to some w∞ . Outside the null set of Proposition D.13, w∞ ∈ Ω, and then ∇L(w∞ ) = 0. Letting t → ∞ in Theorem 4.6 shows that w∞ ∈ / S⊥ , and Lemma F.3 shows that w∞ is not an in-plane saddle. By Theorem 3.5, w∞ is (v1 , v2 ) or (v2 , v1 ). ■

Approach to the bisector saddle under structured initialization

We prove Theorem 4.8 in this appendix and the next. Here we follow the trajectory from the initialization until it comes close to the bisector saddle; Appendix H.2 studies the escape from the saddle and completes the proof.

G.1

Coordinates and estimates near the bisector

As in Theorem 4.8, we consider α ∈ (0, π/2]. We use the set Ω, the norms ri = ∥wi ∥2 , the maximal time Tmax and the coordinates zi,j as in Appendix F, and, as in Proposition 3.7, we write P wi⊥ = dj=3 zi,j uj for the component of wi orthogonal to the teacher plane. We describe the pair (w1 , w2 ) by z1,1 + z2,1 z1,1 − z2,1 , δ := , a := z1,2 − z2,2 , 2 2 1/2 1/2 2 2 Y := ∥w1⊥ ∥22 + ∥w2⊥ ∥22 , N⊥ := z1,2 + z2,2 +Y2 . R :=

(G.1)

Here R and δ are the mean and half the difference of the two bisector coordinates, |a| = |⟨w1 −w2 , v∆ ⟩| is the planar splitting amplitude, and Y is the norm of the out-of-plane part of (w1 , w2 ). Since 72

P P 2 , the time T defined before Theorem 4.8 is the first time at which N reaches N⊥2 = 2i=1 dj=2 zi,j ε ⊥ ε. If z1,2 (0) > 0 > z2,2 (0), then Lemma F.1 gives z1,2 (t) > 0 > z2,2 (t),

a(t) = |z1,2 (t)| + |z2,2 (t)| > 0,

0 ≤ t < Tmax .

(G.2)

In this case a(0) = ainit by (2.5), and, since a = |z1,2 | + |z2,2 |, we have |zi,2 | ≤ a, ∥wi⊥ ∥2 ≤ Y and N⊥ ≤ a + Y . We recall the rate µα = 2 sin(α/2)/(πrs ) defined before Theorem 4.8. By (3.5) and rs = r+ /2, we have πrs = (π − α/2) cos(α/2) + sin(α/2). Hence µα < 1 is equivalent to tan(α/2) < π − α/2, which holds because tan(α/2) ≤ 1 < 3π/4 ≤ π − α/2 for α ≤ π/2. Thus (G.3)

0 < µα < 1. Three identities follow from (F.2). First, by the definition of ai in Lemma 3.3, zi,1 r3−i ai zi,1 = (sin θi1 + sin θi2 ) − πzi,1 − sin γ zi,1 . ri ri ri

We substitute this into the first two lines of (F.2) and use (2π − α) cos(α/2) + 2 sin(α/2) = 2πrs , which is (3.5). Since z1,1 + z2,1 = 2R, we obtain, for i = 1, 2, zi,1 α α 2π żi,1 − (rs − R) = (α − θi1 − θi2 ) cos + (sin θi1 + sin θi2 ) − 2 sin 2 ri 2 (G.4) r3−i + γz3−i,1 − sin γ zi,1 . ri Second, at each time at which z1,2 > 0 > z2,2 , the two equations (F.3) hold, and their sum gives 2π ȧ =

2 h X i=1

|θi2 − θi1 | sin

i α ai + + π − γ |zi,2 | . 2 ri

(G.5)

Third, the last two lines of (F.2) read 2π ẇ1⊥ = (a1 /r1 )w1⊥ − (π − γ)w2⊥ and 2π ẇ2⊥ = (a2 /r2 )w2⊥ − (π − γ)w1⊥ . They give π

d 2 a1 ⊥ 2 a2 ⊥ 2 Y = ∥w1 ∥2 + ∥w2 ∥2 − 2(π − γ)⟨w1⊥ , w2⊥ ⟩ dt r r2 1 ai ≤ max + π − γ Y 2 ; i=1,2 ri

(G.6)

the inequality in the second line follows from 0 ≤ π − γ ≤ π and −2⟨w1⊥ , w2⊥ ⟩ ≤ Y 2 . The equations for w1⊥ and w2⊥ are linear with scalar coefficients, so w1⊥ (t) and w2⊥ (t) remain in span{w1⊥ (0), w2⊥ (0)}. In particular, Y ≡ 0 if Y (0) = 0. The estimates below involve the out-of-plane components only through Y and the norms ∥wi⊥ ∥2 , which is why their constants do not depend on d. The bisector makes the angle α/2 with both teachers, so a student with zi,1 > 0 that is close to 2 + ∥w ⊥ ∥2 )1/2 for the the bisector axis Rv̄ has both teacher angles close to α/2. We write ρi := (zi,2 i 2 distance of wi from Rv̄. The next lemma quantifies this in terms of the ratio ρi /zi,1 . Lemma G.1 (Angles near the bisector). Fix α ∈ (0, π/2] and set Cangle := 6/ sin5 (3α/8). Let d ≥ 2 and i ∈ {1, 2}, and suppose that zi,1 > 0 and ρi ≤ (α/8) zi,1 , with ρi as above. Then ∠(w̄i , v̄) ≤ ρi /zi,1 , θi1 , θi2 ∈ [3α/8, 5α/8], 0 ≤ ri − zi,1 ≤ ρ2i /(2zi,1 ), and θi2 − θi1 − |θi1 + θi2 − α| + (sin θi1 + sin θi2 ) 73

2zi,2 |zi,2 | ρ2i , ≤ Cangle 3 zi,1 zi,1

zi,1 α ρ2 − 2 sin ≤ Cangle 2i . ri 2 zi,1

(G.7) (G.8)

Proof of Lemma G.1. Since zi,1 > 0, the angle ∠(w̄i , v̄) is acute. Its tangent is ρi /zi,1 , which is at most α/8. As arctan y ≤ y for y ≥ 0, this gives the first claim. Since ∠(v̄, v1 ) = ∠(v̄, v2 ) = α/2, the triangle inequality for angles gives |θij − α/2| ≤ ∠(w̄i , v̄) ≤ α/8, which is the second claim. Since ri ≥ zi,1 , the identity ri − zi,1 = ρ2i /(ri + zi,1 ) gives the third claim and 0≤1−

zi,1 ρ2 ≤ i2 . ri 2zi,1

As recalled at the start of Appendix F, v1 = cos(α/2) v̄ + sin(α/2) v∆ and v2 = cos(α/2) v̄ − sin(α/2) v∆ . Hence cos θi1 = mi + ∆i and cos θi2 = mi − ∆i , where mi := cos

α zi,1 , 2 ri

∆i := sin

α zi,2 . 2 ri

By the second claim, the numbers mi ± ∆i , their midpoint mi , and cos(α/2) all lie in the interval [cos(5α/8), cos(3α/8)] ⊂ (0, 1). For y in this interval, 1 − y 2 ≥ sin2 (3α/8), and therefore | arccos′ (y)| =

1 1 ≤ , 1/2 2 sin(3α/8) (1 − y )

| arccos′′ (y)| =

y 1 , ≤ 3 3/2 2 (1 − y ) sin (3α/8)

and | arccos′′′ (y)| = (1 + 2y 2 )(1 − y 2 )−5/2 ≤ 3/ sin5 (3α/8). We also note that |∆i | ≤ |zi,2 |/zi,1 , since ri ≥ zi,1 . R m +∆ For the difference, θi2 − θi1 = arccos(mi − ∆i ) − arccos(mi + ∆i ) = mii−∆ii (− arccos′ (y))dy. The error of the midpoint rule on an interval of length 2|∆i | is at most (2|∆i |)3 /24 times the maximum of the second derivative of the integrand, so θi2 − θi1 + 2∆i arccos′ (mi ) ≤

|zi,2 |3 |∆i |3 ≤ . 3 sin5 (3α/8) sin5 (3α/8) zi,1

2 sin2 (α/2) + ρ2 , we have Since ri2 (1 − m2i ) = zi,1 i

−2∆i arccos′ (mi ) =

−1/2 2zi,2 ρ2i . 1+ 2 zi,1 zi,1 sin2 (α/2)

As 0 ≤ 1 − (1 + y)−1/2 ≤ y/2 for y ≥ 0 and sin(α/2) ≥ sin(3α/8), it follows that θi2 − θi1 −

|z | ρ2 2zi,2 1 1 i,2 i ≤ + , 3 zi,1 zi,1 sin2 (3α/8) sin5 (3α/8)

which implies (G.7). For the sum, Taylor’s formula with the Lagrange remainder gives arccos(mi + ∆i ) + arccos(mi − ∆i ) − 2 arccos mi ≤

2 zi,2 ∆2i ≤ . 2 sin3 (3α/8) sin3 (3α/8) zi,1

Moreover, 0 ≤ cos(α/2) − mi ≤ 1 − zi,1 /ri , so the mean value theorem gives |2 arccos mi − α| = 2 arccos mi − arccos cos

2(1 − zi,1 /ri ) α ρ2i ≤ ≤ 2 . 2 sin(3α/8) zi,1 sin(3α/8)

2 sin3 (3α/8)). Together, the last two bounds give |θi1 + θi2 − α| ≤ 2ρ2i /(zi,1

Finally, sin θi1 + sin θi2 = 2 sin (θi1 + θi2 )/2 cos (θi2 − θi1 )/2 . Since the sine is 1-Lipschitz and 0 ≤ 1 − cos y ≤ y 2 /2, | sin θi1 + sin θi2 − 2 sin(α/2)| ≤ |θi1 + θi2 − α| +

(θi2 − θi1 )2 . 4

By the mean value theorem, |θi2 − θi1 | ≤ 2|∆i |/ sin(3α/8) ≤ 2|zi,2 |/(zi,1 sin(3α/8)). The right side 2 sin3 (3α/8)). We multiply by z /r ≤ 1 and add 2 sin(α/2)(1 − above is therefore at most 3ρ2i /(zi,1 i,1 i 2 . This bounds the second term on the left of (G.8) by 4ρ2 /(z 2 sin3 (3α/8)), and zi,1 /ri ) ≤ ρ2i /zi,1 i i,1 (G.8) follows. ■ On the bisector axis, where N⊥ = 0 and z1,1 , z2,1 > 0, both teacher angles equal α/2, γ = 0 and ri = zi,1 , so (G.4) gives ż1,1 = ż2,1 = rs − R: so R relaxes to rs and δ stays constant. The next lemma shows that near the axis this remains true up to quadratic errors, while a grows at the rate µα rs /R and Y at most at the rate µα rs /(2R). At R = rs these are the two unstable rates of Lemma 4.4. Lemma G.2 (Equations of motion near the bisector). Fix α ∈ (0, π/2] and set ν0 := min{1/8, 7α/64}. There is Cbis ≥ 1, depending only on α, such that the following holds for every d ≥ 2. Let w(0) ∈ Ω, and let t ∈ [0, Tmax ) be a time at which z1,2 > 0 > z2,2 , 0 < R ≤ 2rs and a + Y + |δ| ≤ ν0 R. Then 7R/8 ≤ zi,1 ≤ ri ≤ 9R/8 for i = 1, 2, and a2 + Y 2 , R2 a + Y 2 sin(α/2) |δ| a2 + Y 2 ȧ − a ≤ Cbis + 2+ a, πR R R R3 a + Y d 2 h 2 sin(α/2) |δ| a2 + Y 2 i 2 Y ≤ + Cbis + 2+ Y . dt πR R R R3

|Ṙ − (rs − R)| + |δ̇| ≤ Cbis

Proof of Lemma G.2. In this proof, Cα denotes a constant depending only on α, whose value may change from line to line. Since |δ| + a + Y ≤ ν0 R ≤ R/8, |zi,2 | ≤ a and ∥wi⊥ ∥2 ≤ Y , 9 7 R ≤ R − |δ| ≤ zi,1 ≤ ri ≤ zi,1 + |zi,2 | + ∥wi⊥ ∥2 ≤ R + |δ| + a + Y ≤ R, 8 8 which is the first claim. Moreover, ρi ≤ a + Y ≤ ν0 R ≤ (8ν0 /7)zi,1 ≤ (α/8)zi,1 , so Lemma G.1 applies to both students, with ρ2i ≤ a2 + Y 2 . By that lemma and the triangle inequality for angles, γ ≤ ∠(w̄1 , v̄) + ∠(w̄2 , v̄) ≤

16(a + Y ) 16ν0 2(a + Y ) = ≤ . 7R/8 7R 7

We first prove the bound for Ṙ and δ̇. Let i ∈ {1, 2} and j = 3 − i. By Lemma G.1, the first two terms on the right of (G.4) are at most Cα (a2 + Y 2 )/R2 in absolute value. We write the remaining term as rj rj rj γzj,1 − sin γ zi,1 = γ zj,1 − zi,1 + (γ − sin γ) zi,1 . ri ri ri Since zj,1 ri −rj zi,1 = zj,1 (ri −zi,1 )−zi,1 (rj −zj,1 ), Lemma G.1 and the first claim give |zj,1 −rj zi,1 /ri | ≤ Cα (a2 + Y 2 )/R. Also 0 ≤ γ − sin γ ≤ γ 3 /6, rj zi,1 /ri ≤ rj ≤ 9R/8 and (a + Y )2 ≤ 2(a2 + Y 2 ). With the bound for γ, we obtain γzj,1 − sin γ

rj (a + Y )(a2 + Y 2 ) a2 + Y 2 zi,1 ≤ Cα ≤ C , α ri R2 R2 75

where the last step uses a + Y ≤ ν0 R ≤ 2ν0 rs . Hence |żi,1 − (rs − R)| ≤ Cα (a2 + Y 2 )/R2 for i = 1, 2, and the first bound follows from Ṙ = (ż1,1 + ż2,1 )/2 and δ̇ = (ż1,1 − ż2,1 )/2. For a, we estimate the two terms in (G.5). By (G.7), |θi2 − θi1 | −

|zi,2 | (a2 + Y 2 ) 2|zi,2 | ≤ Cα . zi,1 R3

By the definition of ai in Lemma 3.3, rj sin θi1 + sin θi2 ai +π−γ = − sin γ − γ. ri ri ri Dividing (G.8) by zi,1 shows that the first term on the right differs from 2 sin(α/2)/zi,1 by at most Cα (a2 + Y 2 )/R3 , and the other two terms are at most γ(rj /ri + 1) ≤ (16(a + Y )/(7R))(9/7 + 1) ≤ 6(a + Y )/R in absolute value. Finally, zi,1 = R ± δ gives |1/zi,1 − 1/R| ≤ 8|δ|/(7R2 ). Therefore a + Y ai a2 + Y 2 2 sin(α/2) |δ| ≤ Cα +π−γ− + 2+ , ri R R R R3 |δ| 2|zi,2 | a2 + Y 2 ≤ Cα + |zi,2 |. |θi2 − θi1 | − R R2 R3 Substituting these bounds into (G.5) and using |z1,2 | + |z2,2 | = a proves the second bound. The bound for ai /ri + π − γ holds for both i. Together with (G.6), it proves the third bound. ■

G.2

Alignment with the bisector

At a small initialization, the ratios a/R, Y /R and |δ|/R need not be small, so Lemma G.2 does not yet apply. By Lemma E.6, the directions w̄i first align with v̄ in a time of order rinit . Afterwards R grows at a rate close to rs , while a, Y and δ change little. The next lemma follows the trajectory until R is a large multiple of rinit , when all three ratios are small. Lemma G.3 (Alignment with the bisector). Fix α ∈ (0, π/2], C ≥ 1 and ν > 0. There are r0 ∈ (0, 1], B > 1 and Calign ≥ 1, depending only on α, C and ν, with Br0 < rs /4, such that the following holds for every d ≥ 2. Suppose that the initialization is C-balanced, that 0 < rinit ≤ r0 , and that z1,1 (0), z2,1 (0) > 0 and z1,2 (0) > 0 > z2,2 (0). Then there is a first time τ0 ∈ [0, Tmax ) at which R(τ0 ) = Brinit ; it satisfies τ0 ≤ Calign rinit , −1 Calign ainit ≤ a(t) ≤ Calign ainit ,

Y (t) + |δ(t)| ≤ Calign rinit ,

0 ≤ t ≤ τ0 ,

and a(τ0 ) + Y (τ0 ) + |δ(τ0 )| ≤ νR(τ0 ). Proof of Lemma G.3. We first collect bounds that hold on a time interval of length proportional to rinit . Since r1 (0) + r2 (0) = 2rinit and the initialization is C-balanced, 2rinit ≤ (1 + C) mini ri (0), so rinit /C ≤ ri (0) ≤ 2rinit . Also ainit ≤ r1 (0) + r2 (0) = 2rinit , Y (0) ≤ r1 (0) + r2 (0) = 2rinit and |δ(0)| ≤ rinit . Let λ > 0, and suppose that rinit ≤ r− /(4(2 + λ)), with r− as in (3.5). Since r− = 4l(π − α/2) < 2 by (A.1b) and Lemma A.2, this gives rinit < 1/4. Then maxi ri (0) + λrinit ≤ r− /4, and Lemma E.2 gives Tmax > λrinit and rinit /C ≤ ri (t) ≤ (2 + λ)rinit for 0 ≤ t ≤ λrinit . On this interval, |ai | ≤ 2 + πri + r3−i and rinit < 1 give |ai |/ri ≤ (2C + π + C(2 + λ))/rinit . Let J be the reflection that fixes v̄, u3 , . . . , ud and maps v∆ to −v∆ , so that Jv2 = v1 . The triangle inequality for angles and (E.1) give |θi2 − θi1 | = |∠(J w̄i , v1 ) − ∠(w̄i , v1 )| ≤ ∠(J w̄i , w̄i ) ≤ 76

π|zi,2 | π ∥J w̄i − w̄i ∥2 = . 2 ri

d 2 Y ≤ 4C(1 + λ)Y 2 /rinit . Since By (G.5) and (G.6), these bounds give |ȧ| ≤ 3C(1 + λ)a/rinit and dt ri ≤ (2 + λ)rinit ≤ r− /4 < 1/2, we have |ai | ≤ 2 + (π + 1)/2, and the first two lines of (F.2) give 2π|żi,1 | ≤ 2π + |ai | + πr3−i < 4π. Hence |δ̇| ≤ 2. Integrating, we find that, for 0 ≤ t ≤ λrinit ,

e−3C(1+λ)λ ainit ≤ a(t) ≤ e3C(1+λ)λ ainit ,

Y (t) ≤ 2e2C(1+λ)λ rinit ,

|δ(t)| ≤ rinit + 2t.

(G.9)

Next we consider times at which both directions are close to v̄. Let n o rs νrs εa := min ν0 , , , 4Cangle + 4rs 4Cangle + 12rs with ν0 as in Lemma G.2, and suppose that at some time ∥w̄i − v̄∥2 ≤ εa for i = 1, 2 and R ≤ rs /4. Then zi,1 /ri = ⟨w̄i , v̄⟩ = 1 − ∥w̄i − v̄∥22 /2 ≥ 7/8 and ρi /ri = ∥w̄i − ⟨w̄i , v̄⟩v̄∥2 ≤ εa , so ρi ≤ (8εa /7)zi,1 ≤ (α/8)zi,1 . By Lemma G.1, the first two terms on the right of (G.4) are at most 2Cangle ε2a in absolute value, and the triangle inequality for angles gives γ ≤ 16εa /7. Since zj,1 ≤ 2R and rj ≤ (8/7)zj,1 ≤ 16R/7 for j = 1, 2, the last two terms are at most γ(z3−i,1 + r3−i ) ≤√11εa R. We use R ≤ rs /4 and the choice of εa for the first bound below, and we sum |zi,2 | + ∥wi⊥ ∥2 ≤ 2 εa ri over i for the second: √ rs |żi,1 − (rs − R)| ≤ (Cangle + rs )εa ≤ , a + Y ≤ 2 εa (r1 + r2 ) ≤ 4εa R. (G.10) 4 Consequently Ṙ ≥ rs /2 and |δ̇| ≤ (Cangle + rs )εa at such times. √ Let Λ and ra be the constants of Lemma E.6 for η = π/3, m = 1/C, M = 2 and ε = εa / 2. We set n o 2(1 + 2Λ) 4B r− ε2a B := 3 + Λ + , λ1 := Λ + , r0 := min ra , , , ν rs 4(2 + λ1 ) 4C(2 + λ1 )λ1 and Calign := (3 + 2λ1 )e3C(1+λ1 )λ1 . These depend only on α, C and ν, and Calign ≥ λ1 . We note that Br0 < Br− /(4λ1 ) ≤ r− rs /16 < rs /4, because λ1 ≥ 4B/rs and r− < 2. Suppose now that rinit ≤ r0 . With T1 := λ1 rinit , (G.9) holds with λ = λ1 on [0, T1 ]. Since ∠(w̄i (0), v̄) < π/2, Lemma E.6 applies, and at ta := Λrinit we have Ei (ta ) < ε2a /4, where Ei = 12 ∥w̄i −v̄∥22 as in Lemma E.5. By (E.4), Ėi ≤ r3−i /(2ri ) ≤ C(2 + λ1 )/2 on [0, T1 ], so the choice of r0 gives Ei < ε2a /4 + ε2a /8 < ε2a /2 on [ta , T1 ]. Thus ∥w̄i − v̄∥2 < εa on [ta , T1 ]. For 0 ≤ t ≤ ta , the bound ri (t) ≤ ri (0) + t of Lemma E.2 gives R ≤ (r1 + r2 )/2 ≤ (2 + Λ)rinit < Brinit , so R does not reach Brinit before ta . As long as ta ≤ t ≤ T1 and R ≤ Brinit , we have ∥w̄i − v̄∥2 < εa and R ≤ rs /4, so Ṙ ≥ rs /2. If R stayed below Brinit on [0, T1 ], we would obtain R(T1 ) ≥ R(ta )+(rs /2)(T1 −ta ) = R(ta )+2Brinit , where R(ta ) > 0 because zi,1 ≥ (7/8)ri there. This is impossible, so τ0 exists, ta < τ0 ≤ T1 ≤ Calign rinit , and τ0 − ta ≤ 2(Brinit − R(ta ))/rs ≤ 2Brinit /rs . The bounds for a, Y and δ on [0, τ0 ] follow from (G.9) with λ = λ1 . At τ0 , (G.10) gives a + Y ≤ 4εa R(τ0 ). Integrating the bound for δ̇ from ta , and using |δ(ta )| ≤ (1 + 2Λ)rinit , we obtain a(τ0 ) + Y (τ0 ) + |δ(τ0 )| 1 + 2Λ 2(Cangle + rs )εa ν ν ≤ 4εa + + ≤ + , R(τ0 ) B rs 2 2 by the choices of εa and B.

■

G.3

Growth of the bisector coordinates up to rs /2

Once the ratios are small, Lemma G.2 applies. Then R grows at a rate close to rs − R, δ changes R little, and, since µα rs dt/R ≈ µα log R, a grows like Rµα while Y grows at most like Rµα /2 . Since µα < 1 by (G.3), the ratios a/R and Y /R decrease, and the trajectory stays near the bisector until R reaches rs /2. Lemma G.4 (Growth of the bisector coordinates). Fix α ∈ (0, π/2], and let ν0 be as in Lemma G.2. There are ν ∈ (0, ν0 ] and Cscale ≥ 1, depending only on α, such that the following holds for every d ≥ 2. Let w(0) ∈ Ω satisfy z1,2 (0) > 0 > z2,2 (0), 0 < R0 < rs /2 and a0 + Y0 + |δ0 | ≤ νR0 , where R0 , a0 , Y0 and δ0 denote the values of R, a, Y and δ at t = 0. Then there is a first time t∗ ∈ [0, Tmax ) at which R(t∗ ) = rs /2; it satisfies 0 < t∗ < 2, −1 Cscale a0

and

R(t) µα R0

≤ a(t) ≤ Cscale a0

R(t) µα R0

Y (t) ≤ Cscale Y0

R(t) µα /2 R0

0 ≤ t ≤ t∗ ,

a2 + Y02 1 |δ(t∗ )| ≤ Cscale |δ0 | + 0 + a(t∗ )2 log . R0 R0

Proof of Lemma G.4. Let Cbis be the constant of Lemma G.2, and let ν1 := min{ν0 , (rs /(4Cbis ))1/2 }. We first derive bounds that hold for every ν ≤ ν1 /4; the value of ν, together with a constant Q ≥ 2, is fixed afterwards. Let τ be the supremum of the times s ∈ [0, Tmax ) such that R < rs /2 and a+Y +|δ| < ν1 R on [0, s]. Both conditions hold at t = 0, so τ > 0. On [0, τ ), we have R > 0, because a > 0 and a+Y +|δ| < ν1 R, and the signs (G.2) hold, so Lemma G.2 applies. Its first bound and (a2 + Y 2 )/R2 ≤ ν12 give Ṙ ≥ rs − R − rs /4 ≥ rs /4. Hence R is increasing, R ≥ R0 , and t < 4(rs /2 − R0 )/rs < 2 for t < τ . Moreover 7R0 /8 ≤ ri ≤ 9rs /16 on [0, τ ). If τ = Tmax , the solution would stay in a compact subset of Ω on a bounded interval, which Lemma A.1 excludes. Thus τ < min{2, Tmax }, the estimates above extend to [0, τ ] by continuity, and at τ either R = rs /2 or a + Y + |δ| = ν1 R. Since Ṙ ≥ rs /4, we have ds ≤ (4/rs )dR, so an integral in time of a nonnegative quantity over [0, t] ⊂ [0, τ ] is at most 4/rs times the corresponding integral in R over [R0 , R(t)]. Moreover, by the first bound of Lemma G.2, 1 1 |Ṙ − (rs − R)| 8Cbis a2 + Y 2 − ≤ = . rs2 R2 Ṙ rs − R Ṙ (rs − R) Since 1/(R(rs − R)) = (1/R + 1/(rs − R))/rs and 0 ≤ log((rs − R0 )/(rs − R)) ≤ log 2 for R ≤ rs /2, it follows that Z t Z ds R(t) 8Cbis R(t) a2 + Y 2 µα rs − µα log ≤ µα log 2 + dR. (G.11) R0 rs R3 0 R R0 Let τQ be the supremum of the times s ∈ [0, τ ] such that, on [0, s], a ≤ Qa0

R µα R0

Y ≤ QY0

R µα /2 R0

(G.12)

If Y0 > 0, both inequalities are strict at t = 0. If Y0 = 0, then Y ≡ 0 and the second inequality holds throughout, while the first is strict at t = 0. In either case τQ > 0, and if τQ < τ , then, by

continuity, the first inequality, or the second with Y0 > 0, is an equality at τQ . On [0, τQ ], since R ≥ R0 , µα < 1 and a0 , Y0 ≤ νR0 , (G.12) gives Z ∞ Z R(t) 2 a2 Z ∞ Y02 Q2 ν 2 a +Y2 2µα −3 µα −3 2 0 R dR + R dR ≤ dR ≤ Q . R3 1 − µα R0µα R0 R02µα R0 R0 R R(t) Similarly, R2µα −2 ≤ R02µα −2 and Rµα −2 ≤ R0µα −2 for R ≥ R0 give R0 (a2 +Y 2 )R−2 dR ≤ 2Q2 ν 2 R(t), and the first bound of Lemma G.2 yields Z 8Cbis Q2 ν 2 4Cbis R(t) a2 + Y 2 dR ≤ ν + R(t). (G.13) |δ(t)| ≤ |δ0 | + rs R2 rs R0 Rs For the term with δ in Lemma G.2, we use |δ(s)| ≤ |δ0 | + 0 |δ̇| and exchange the order of integration. Rt R∞ Since s′ ds/R(s)2 ≤ (4/rs ) R(s′ ) dR/R2 = 4/(rs R(s′ )), we find Z t

|δ| 4 |δ0 | ds ≤ + 2 rs R0 0 R

Z t

Z t 2 |δ̇| 4 a +Y2 ds ≤ ν + Cbis ds . rs R3 0 R 0

Combining these bounds with (G.11), the second and third bounds of Lemma G.2, (a+Y )/R < ν1 ≤ 1 and t < 2, we obtain, for 0 ≤ t ≤ τQ , R(t) µα R(t) a(t) 2 2 − µα log ≤ Γ1 + Γ2 (ν + Q2 ν 2 ), Y (t)2 ≤ Y02 log eΓ1 +Γ2 (ν+Q ν ) , a0 R0 R0 where Γ1 := µα log 2 + 2Cbis and Γ2 depends only on α; the second bound follows from Grönwall’s inequality for Y 2 . We now choose Q := 2eΓ1 +1 and then ν ∈ (0, ν1 /4] so small that Γ2 (ν + Q2 ν 2 ) ≤ 1,

(2Q + 1)ν +

8Cbis Q2 ν 2 ν1 ≤ . rs 2

Then a ≤ (Q/2)a0 (R/R0 )µα and Y ≤ (Q/2)Y0 (R/R0 )µα /2 on [0, τQ ], so at τQ the first inequality in (G.12) is strict, and so is the second when Y0 > 0. Hence τQ = τ . Moreover, a0 (R/R0 )µα ≤ νR and Y0 (R/R0 )µα /2 ≤ νR, so (G.13) gives a + Y + |δ| ≤ ν1 R/2 on [0, τ ]. At τ , therefore, R = rs /2, and t∗ = τ ∈ (0, 2). The bounds for a and Y hold with Cscale ≥ Q. For δ(t∗ ), we use the first inequality in (G.13) with (G.12). The contribution of Y 2 to the integral is at most Q2 Y02 /((1 − µα )R0 ). For the R ∞ −2µ contribution of a2 , if µα < 1/2, then a20 R0 α R0 R2µα −2 dR = a20 /((1 − 2µα )R0 ). If µα ≥ 1/2, then R2µα −2 ≤ 1/R for R ≤ rs /2, because rs = 2l(α/2) ≤ 1 by (3.5), (A.1b) and Lemma A.2. R r /2 Hence a20 R0−2µα Rs0 R2µα −2 dR ≤ a20 R0−2µα log(1/R0 ). In this case, the lower bound for a at t∗ gives a0 R0−µα ≤ (Q/2)(2/rs )µα a(t∗ ) ≤ (Q/rs )a(t∗ ). Hence the bound for δ(t∗ ) holds with Cscale large enough, depending only on α. ■

G.4

Entry into a neighborhood of the bisector saddle

We fix c0 := rs /64 and consider the segment Cc0 := {((rs + s)v̄, (rs − s)v̄) : |s| ≤ c0 } of the first family in (3.9); it contains wsaddle . For |s| ≤ c0 , ∥w1 − (rs + s)v̄∥22 + ∥w2 − (rs − s)v̄∥22 = (R + δ − rs − s)2 + (R − δ − rs + s)2 + N⊥2 . Hence 2 dist (w1 , w2 ), Cc0 = 2(R − rs )2 + 2 min (δ − s)2 + N⊥2 . (G.14) |s|≤c0

The analysis in Appendix H.2 starts once the trajectory is within a fixed distance εentry of Cc0 . By Lemma G.4, the quantities a, Y and δ are small when R reaches rs /2. Afterwards R relaxes exponentially to rs , and over the bounded time this takes, a and Y change at most by bounded factors and δ changes by O(a2 + Y 2 ). The next lemma combines this with Lemmas G.3 and G.4. Lemma G.5 (Entry into a neighborhood of the bisector saddle). Fix α ∈ (0, π/2], C ≥ 1 and εentry ∈ (0, c0 ]. There are centry ∈ (0, 1] and Centry ≥ 1, depending only on α, C and εentry , such that the following holds for every d ≥ 2. Suppose that √ the initialization is C-balanced, that z1,1 (0), z2,1 (0) > 0, z1,2 (0) > 0 > z2,2 (0) and ainit ≥ rinit /(C d), and that √ √ √ d rinit d ≤ centry , rinit d 1 + log + log d ≤ centry . (G.15) rinit We write εinit for the left side of the second inequality. Then there is a first time tentry ∈ [0, Tmax ) at which dist((w1 , w2 ), Cc0 ) ≤ εentry . Moreover tentry ≤ Centry , and ae := a(tentry ) and qe := √ rinit d + a2e (1 + log(1/rinit )) satisfy √ −µα −µα −1 , Y (tentry )2 ≤ Centry rinit d ae , ≤ ae ≤ Centry ainit rinit Centry ainit rinit (G.16) 1 α |δ(tentry )| ≤ Centry qe , sup N⊥ (t) + qe log ≤ Centry ε1−µ init . a e 0≤t≤tentry Proof of Lemma G.5. Let ν and Cscale be the constants of Lemma G.4, let r0 , B and Calign be the constants of Lemma G.3 for α, C and this ν, and let Cbis be the constant of Lemma G.2. The constants C1 , . . . , C6 below depend only on α, C and εentry . We require centry ≤ r0 ; a √ further condition is √ imposed below. Since √ centry ≤ 1, the first inequality in (G.15) gives rinit ≤ rinit d ≤ 1; hence log( d/rinit ) ≥ 0 and rinit d ≤ εinit ≤ centry . We first follow the trajectory until R reaches rs /2. By Lemma G.3, the time τ0 exists, and R(τ0 ) = Brinit ≤ Br0 < rs /4. We apply Lemma G.4 to the solution t 7→ w(τ0 + t); its hypotheses hold by (G.2) and Lemma G.3. Let t1 := τ0 + t∗ be the first time after τ0 at which R = rs /2, and −1 let R0 = Brinit , a0 , Y0 , δ0 be the values of R, a, Y , δ at τ0 . Then Calign ainit ≤ a0 ≤ Calign ainit and √ Y0 + |δ0 | ≤ Calign rinit . We also use ainit ≤ r1 (0) + r2 (0) = 2rinit , ainit ≥ rinit /(C d), rs /(2R0 ) = rs /(2Brinit ) and rs /(2B) < 1. On [0, τ0 ] we bound a and Y by Lemma G.3, and on [τ0 , t1 ] we −µα 2 use (R/R0 )µα ≤ (rs /(2Brinit ))µα ≤ rinit . For δ(t1 ), we use (a20 + Y02 )/R0 ≤ 5Calign rinit /B and log(1/R0 ) ≤ log(1/rinit ). With these facts, Lemma G.4 gives −µα −µα C1−1 ainit rinit ≤ a(t1 ) ≤ C1 ainit rinit , 1−µα a(t) + Y (t) ≤ C1 rinit

(0 ≤ t ≤ t1 ),

√ Y (t1 )2 Y02 3 ≤ Cscale ≤ C1 rinit d, a(t1 ) a0 1 |δ(t1 )| ≤ C1 rinit + a(t1 )2 1 + log . rinit

(G.17)

Since sup0<r≤1 r1−µα (1 + log(1/r)) ≤ 1/(1 − µα ), these bounds give 1−µα a(t1 ) + Y (t1 ) + |δ(t1 )| ≤ C2 rinit .

(G.18)

Next we follow the trajectory from t1 to tentry . We consider the region where rs /4 < R < 2rs and a + Y + |δ| < ν0 rs /4. There Lemma G.2 applies, and it gives |Ṙ − (rs − R)| + |δ̇| ≤

16Cbis 2 (a + Y 2 ), rs2 80

|ȧ| ≤ C3 a,

d 2 Y ≤ C3 Y 2 , dt

with C3 := (3+Cbis (1+rs ))/rs , because 2 sin(α/2)/(πR) ≤ 3/rs , (a+Y )/R ≤ 1, |δ|/R2 ≤ 4ν0 /rs and (a2 + Y 2 )/R3 ≤ 4ν02 /rs in this region. Let T2 := log(2rs /εentry ) > 0, C4 := eC3 T2 (1 + 16Cbis rs−2 (1 + T2 )eC3 T2 ) and S := a(t1 ) + Y (t1 ) + |δ(t1 )|. We now impose the remaining condition on centry : α C2 c1−µ entry ≤ min{1, ν0 rs /(8C4 ), εentry /(8C4 )}. By (G.18) and rinit ≤ centry , S satisfies the same bound, so w(t1 ) lies in the region. Let τ1 be the supremum of the times t ∈ [t1 , Tmax ) with t ≤ t1 + T2 such that the trajectory lies in the region on [t1 , t]. For t1 ≤ t < τ1 , Grönwall’s inequality gives a(t) + Y (t) ≤ eC3 T2 (a(t1 ) + Y (t1 )) and e−C3 T2 a(t1 ) ≤ a(t) ≤ eC3 T2 a(t1 ). Integrating the bound for δ̇, and applying the variationd of-constants formula to dt (R − rs ) = −(R − rs ) + (Ṙ − (rs − R)) with R(t1 ) − rs = −rs /2, we obtain |δ(t) − δ(t1 )| + R(t) − rs +

2 16Cbis rs −(t−t1 ) 2C3 T2 ≤ e (1 + T )e a(t ) + Y (t ) . 2 1 1 2 rs2

Since S ≤ 1, the last two bounds give, for t1 ≤ t < τ1 , a(t) + Y (t) + |δ(t)| ≤ C4 S,

R(t) − rs +

rs −(t−t1 ) ≤ C4 S 2 . e 2

(G.19)

Hence, on [t1 , τ1 ), we have a + Y + |δ| ≤ ν0 rs /8 and 3rs /8 ≤ R ≤ 9rs /8, since C4 S 2 ≤ C4 S ≤ rs /8. By Lemma G.2, these bounds also give 21rs /64 ≤ ri ≤ 81rs /64. If τ1 = Tmax , the solution would therefore stay in a compact subset of Ω on the bounded interval [t1 , Tmax ), which Lemma A.1 excludes. Hence τ1 < Tmax , the bounds extend to τ1 by continuity, and they keep the trajectory inside the region at τ1 . Thus τ1 = t2 := t1 + T2 , and (G.19) holds on [t1 , t2 ]. At t2 , r2s e−T2 = εentry /4, so |R − rs | ≤ εentry /4 + εentry /8. By (G.14) with s = 0 and N⊥ ≤ a + Y , dist (w1 , w2 ), Cc0

ε 3ε entry 2 entry 2 +3 < ε2entry . ≤ 2(R − rs )2 + 2δ 2 + (a + Y )2 ≤ 2 8 8

Before √ t1 , we have R < rs /2, by the definitions of τ0 and t∗ , and (G.14) gives dist((w1 , w2 ), Cc0 ) > rs / 2 > εentry . Hence tentry exists and t1 ≤ tentry ≤ t2 ≤ Calign + 2 + T2 . It remains to prove (G.16). Let C5 := C1 C42 . On [t1 , tentry ], a changes at most by the factor C T e 3 2 ≤ C4 , Y 2 increases at most by this factor, and a + Y ≤ C4 (a(t1 ) + Y (t1 )). With (G.17), this gives the bounds for ae and Y (tentry )2 with C5 in place of Centry , and the bound N⊥ ≤ 1−µα a + Y ≤ C5 rinit on [0, for δ̇ above, |δ(tentry ) − δ(t1 )| ≤ C4 (a(t1 )2 + Y (t1 )2 ), √ tentry ]. By the bound √ 2 and Y (t1 ) ≤ C1 rinit d a(t1 ) ≤ C1 rinit d because a(t1 ) ≤ S ≤ 1. Together with (G.17) and a(t1 ) ≤ eC3 T2 ae ≤ C4 ae , this gives the bound for δ(tentry ). −µα For the last bound in (G.16), we note that ae ≤ C4 S ≤ 1 and that rinit ≥ √ 1, so ae ≥ √ C5−1 ainit ≥ r√init /(CC5 d). Hence, with√ C6 := 1 + log(CC5 ), log(1/ae ) ≤ log(CC ) + log( d/rinit ) ≤ 5 √ −2 C6 (1 + log( d/rinit ) + log d), and rinit d log(1/ae ) ≤ C6 εinit . Since rinit d ≤ 1, we have d ≤ rinit √ 1−µα and 1 + log( d/rinit ) + log d ≤ 4(1 + log(1/rinit )). With ae ≤ 2C5 rinit , it follows that 64C52 C6 1−µα 1 1 1 2 2−2µα a2e 1 + log log ≤ 16C52 C6 rinit 1 + log ≤ r , rinit ae rinit (1 − µα )2 init where we used sup0<r≤1 r1−µα (1 + log(1/r))2 ≤ 4/(1 − µα )2 . Since rinit ≤ εinit ≤ 1, the terms 1−µα α C5 rinit , C6 εinit and the last bound are all at most a constant times ε1−µ init . This proves (G.16) with Centry large enough. ■

Logarithmic escape from the bisector saddle

H.1

Hessian at the bisector saddle

Proof of Lemma 4.4. At wsaddle , both students equal ws = rs v̄. By the continuous extension in Lemma A.6, the off-diagonal blocks Q2 (ws , ws ) of the Hessian equal 12 Id , and the term Q1 (ws , ws ) in the diagonal blocks vanishes. As recalled at the start of Appendix F, v1 = cos(α/2)v̄+sin(α/2)v∆ and v2 = cos(α/2)v̄ − sin(α/2)v∆ . Hence the angle between ws and each teacher is α/2, n̄v1 ,ws = v∆ and ⊤ ), where µ = 2 sin(α/2)/(πr ) n̄v2 ,ws = −v∆ , so Q1 (ws , v1 ) = Q1 (ws , v2 ) = (µα /4)(Id − v̄v̄ ⊤ + v∆ v∆ α s is the rate defined before Theorem 4.8. Both diagonal blocks therefore equal 1 µα ⊤ A := Id − Id − v̄v̄ ⊤ + v∆ v∆ . 2 2 The vectors v̄, v∆ and u3 , . . . , ud are eigenvectors of A, with eigenvalues 12 , 12 − µα and 12 − µα /2, ⊤ maps them to 0, 2v and u . The Hessian maps (x, x) to respectively, because Id − v̄v̄ ⊤ + v∆ v∆ j ∆ ((A + 12 Id )x, (A + 12 Id )x) and (x, −x) to ((A − 21 Id )x, −(A − 12 Id )x). Hence the vectors (x, ±x) with x ∈ {v̄, v∆ , u3 , . . . , ud } are eigenvectors of the Hessian, with the following eigenvalues: x eigenvalue of (x, x) eigenvalue of (x, −x) v̄ 1 0 v∆ 1 − µα −µα uj , 3 ≤ j ≤ d 1 − µα /2 −µα /2 By (G.3), µα < 1, so the eigenvalues in the middle column are positive. The 2d vectors (x, ±x) are orthogonal, so they form a basis of (Rd )2 , and the table gives the negative, positive and zero eigenspaces stated in Lemma 4.4. ■

H.2

Proof of Theorem 4.8

By Lemma G.5, the trajectory comes within a small fixed distance of the segment Cc0 after a bounded time, with a small planar splitting amplitude ae . Afterwards a grows at a rate close to µα until N⊥ reaches ε, which takes a time of order µ−1 α log(1/ae ). Over so long a time, a constant error in the growth rate would change the escape time by an amount that grows with log(1/ae ), so we need errors that vanish at the saddle. Once the rate 2 sin(α/2)/(πR) in Lemma G.2 is compared with µα , the bounds of that lemma take this form near wsaddle . Lemma H.1 (Equations of motion near the bisector saddle). Fix α ∈ (0, π/2], and let ν0 be as in Lemma G.2. There is Clocal ≥ 1, depending only on α, such that the following holds for every d ≥ 2. Let w(0) ∈ Ω, and let t ∈ [0, Tmax ) be a time at which z1,2 > 0 > z2,2 and |R − rs | + |δ| + a + Y ≤ ν0 rs .

(H.1)

Then rs /2 ≤ ri ≤ 2rs for i = 1, 2, and |ȧ − µα a| ≤ Clocal |R − rs | + |δ| + a + Y a, d 2 Y ≤ µα + Clocal (|R − rs | + |δ| + a + Y ) Y 2 , dt |Ṙ − (rs − R)| + |δ̇| ≤ Clocal (a2 + Y 2 ).

(H.2) (H.3) (H.4)

Proof of Lemma H.1. Let Cbis be the constant of Lemma G.2. We first check the hypotheses of that lemma. Since ν0 ≤ 1/8, (H.1) gives 7rs /8 ≤ R ≤ 9rs /8. If R ≥ rs , then a + Y + |δ| ≤ ν0 rs ≤ ν0 R; if R < rs , then a + Y + |δ| ≤ ν0 rs − (rs − R) ≤ ν0 R, because ν0 ≤ 1. Hence the lemma applies. Its first claim gives rs /2 ≤ 7R/8 ≤ ri ≤ 9R/8 ≤ 2rs , and its first bound, together with R ≥ 7rs /8, gives (H.4) whenever Clocal ≥ 2Cbis /rs2 . For the other two bounds, we compare the error terms of Lemma G.2 with |R − rs | + |δ| + a + Y . Since R ≥ 7rs /8 and a + Y ≤ rs /8, a+Y |δ| a2 + Y 2 2(a + Y ) 2|δ| a + Y + 2+ ≤ + 2 + ≤ 3 1 + rs−2 |δ| + a + Y , 3 2 R R R rs rs 4rs where the last step uses 1/rs ≤ 1 + rs−2 . Moreover, 2 sin(α/2)/(πR) = µα rs /R, and µα < 1 by (G.3), so 2 sin(α/2) µα |R − rs | 2|R − rs | ≤ 2 1 + rs−2 |R − rs |. − µα = ≤ πR R rs Substituting these two bounds into the second and third bounds of Lemma G.2 gives (H.2) and (H.3) with Clocal := 4Cbis (1 + rs−2 ), since Cbis ≥ 1. This constant also satisfies the requirement above. ■ For each α ∈ (0, π/2], with ν0 and√Clocal as in Lemma G.2 and Lemma H.1, we set ζ := min{ν0 rs /8, µα /(16Clocal )} and εentry := 2 ζ/100, which depend only on α. This choice is made so that, whenever |R − rs | + |δ| + a + Y ≤ 4ζ, the inequality (H.1) holds and (H.2) gives ȧ ≥ µα a/2. Since ν0 ≤ 1/8, we have εentry < ζ ≤ rs /64 = c0 , so Lemma G.5 applies with this εentry . Below, tentry , ae and qe are as in that lemma. Lemma H.2 (Escape from a neighborhood of the bisector saddle). Fix α ∈ (0, π/2] and C ≥ 1, and let centry and Centry be the constants of Lemma G.5 for α, C and εentry . There is ε0 > 0, depending only on α and C, such that for every ε ∈ (0, ε0 ] there is cpass ∈ (0, centry ], depending only on α, C and ε, with the following property. For every d ≥ 2 and every initialization that satisfies the hypotheses of Lemma G.5 with cpass in place of centry , the time Tε is finite and Tε − tentry −

1 1 Clocal + log(4/ε) log ≤ . µα ae µα

(H.5)

Proof of Lemma H.2. We set CY := 4Centry e4Clocal , Cδ := 8(Centry + Clocal (1 + 2CY )/µα ) and ε0 := ζ/(1 + 2Cδ ζ). Then Cδ ≥ 16Clocal , since CY ≥ 4 and µα < 1, and, for 0 < ε ≤ ε0 , (H.6)

2Cδ ε2 + ε ≤ ζ.

We fix ε ∈ (0, ε0 ]. Since µα < 1, we have Clocal ζ ≤ µα /16 < 1/16; as Clocal ≥ 1, also ε ≤ ζ < 1/16. We choose cpass ∈ (0, centry ] so small that n ε ζ ζ2 o α , , . β := 2Centry c1−µ ≤ min pass 8CY 2Cδ 8CY Now let d ≥ 2, and consider an initialization that satisfies the hypotheses of Lemma G.5 with cpass in place of centry . Since√cpass ≤ centry , that lemma applies, and εinit ≤ cpass . The first inequality in α (G.15) gives rinit ≤ rinit d ≤ cpass ≤ 1, and cpass ≤ c1−µ pass ≤ β/2. Since ainit ≤ r1 (0) + r2 (0) = 2rinit , 1−µα α (G.16) gives ae ≤ 2Centry rinit ≤ β and sup0≤t≤tentry N⊥ (t) + qe log(1/ae ) ≤ Centry ε1−µ ≤ β/2. init Since β ≤ ε/(8CY ) < 1/e, we have log(1/ae ) ≥ 1. Hence ae ≤ β,

√ β rinit d ≤ , 2

qe ≤ qe log 83

1 β ≤ , ae 2

sup 0≤t≤tentry

N⊥ (t) ≤

β . 2

(H.7)

√ By (G.16), Y (tentry )2 ≤ Centry rinit d ae and |δ(tentry )| ≤ Centry qe . Since √ the distance from (w1 , w2 ) to Cc0 is at most εentry at tentry , (G.14) gives |R(tentry ) − rs | ≤ εentry / 2 = ζ/100. By (G.2), z1,2 > 0 > z2,2 on [0, Tmax ), so Lemma H.1 applies wherever (H.1) holds. Let τ be the supremum of the times t ∈ [tentry , Tmax ) such that, on [tentry , t], we have a < 2ε, the inequality (H.1) holds strictly, and Z t √ |R(u) − rs | du < 1. (H.8) Y 2 < CY rinit d a, |δ| < Cδ (qe + a2 ), |R − rs | < ζ, tentry

These conditions hold at tentry : ae ≤ β < 2ε; the first two inequalities follow from the bounds at tentry above, because CY , Cδ > Centry ; and |R(tentry ) − rs | ≤ ζ/100. Moreover, by (H.7) and the choice of β, we have Centry qe ≤ Cδ β/16 ≤ ζ/32, ae ≤ ζ/32 and Y (tentry )2 ≤ Centry β 2 /2 ≤ CY β/8 ≤ ζ 2 /64, so the left side of (H.1) is less than ζ < ν0 rs at tentry . Hence τ > tentry . We now show that, on [tentry , τ ), each of these conditions other than a < 2ε holds with room to spare. On this interval, a < 2ε ≤ 1, so (H.7) and (H.8) give Y 2 ≤ CY β ≤ ζ 2 /8 and |δ| ≤ Cδ β/2+4Cδ ε2 . Since Cδ β/2 ≤ ζ/4 and, by (H.6), 4Cδ ε2 +2ε ≤ 2ζ, it follows that |R−rs |+|δ|+a+Y ≤ ζ + (ζ/4 + 2ζ) + ζ/2 ≤ 4ζ. Since 16Clocal ζ ≤ µα , (H.2) gives ȧ ≥

µα a 2

on [tentry , τ ).

(H.9)

Hence a is strictly increasing, and √ dt ≤ 2da/(µα a). Changing the variable of integration from t to a, and using a ≤ 1 and Y 2 ≤ CY rinit d a, we obtain, for tentry ≤ t < τ , Z t Z t 2 1 2a(t) a(t)2 t − tentry ≤ log , a du ≤ , a2 du ≤ , µα ae µα µα tentry tentry Z t Z t √ √ 1/2 4 2CY 2 . Y du ≤ rinit d a(t), CY rinit d a(t) Y du ≤ µα µα tentry tentry

(H.10)

√ √ Integrating (H.4) and using (H.10) together with rinit d a ≤ rinit d ≤ qe , we obtain |δ(t)| ≤ Centry qe +

√ Cδ Clocal a(t)2 + 2CY rinit d a(t) ≤ qe + a(t)2 . µα 8

By (H.8), (H.10) and (H.7), Z t (|δ| + a + Y ) du ≤ tentry

1/2 1 13ζ 1 1 2Cδ qe log + Cδ a(t)2 + 2a(t) + 4 CY β ≤ < , µα ae 2µα 2

where we used Cδ β ≤ ζ/2, 4Cδ ε2 + 4ε ≤ 4ζ, (CY β)1/2 ≤ ζ/2 and 16ζ ≤ µα . Since a > 0, Y 2 /a is continuously differentiable, and (H.2) and (H.3) give Y 2 d Y2 ≤ 2Clocal |R − rs | + |δ| + a + Y . dt a a Rt By Grönwall’s inequality, the last integral bound and the bound tentry |R − rs | du < 1, we obtain √ √ Y (tentry )2 3Clocal Y (t)2 CY ≤ e ≤ Centry e3Clocal rinit d < rinit d. a(t) ae 4

d Applying the variation-of-constants formula to dt (R − rs ) = −(R − rs ) + (Ṙ − (rs − R)) and using (H.4), we obtain

|R(t)−rs | ≤ e−(t−tentry )

ζ +Clocal 100

Z t

ζ ζ e−(t−u) a(u)2 +Y (u)2 du ≤ +Clocal 4ε2 +CY β < , 100 2 tentry

because 32Clocal ε2 ≤ 2Cδ ε2 ≤ ζ and Clocal CY β ≤ Clocal ζ 2 /8 ≤ ζ/128. Integrating the middle expression over [tentry , t] and exchanging the order of integration, we find in the same way Z t |R(u) − rs | du ≤ tentry

√ 1 ζ Clocal ζ Clocal a(t)2 + 2CY rinit d a(t) ≤ 4ε2 + 2CY β < , + + 100 µα 100 µα 2

by the same two bounds and 16ζ ≤ µα . Finally, with the bounds just proved, |R − rs | + |δ| + a + Y ≤

ζ ζ ζ ζ ν0 rs ζ Cδ β Cδ 2 + + ε + 2ε + ≤ + + 2ζ + < . 2 16 2 2 2 32 2 2

These bounds are stronger than the conditions defining τ , other than a < 2ε. By Lemma H.1, also rs /2 ≤ ri ≤ 2rs on [tentry , τ ), and (H.9) gives τ − tentry ≤ (2/µα ) log(2ε/ae ). If τ = Tmax , the solution would stay in a compact subset of Ω on a bounded interval, which Lemma A.1 excludes. Hence τ < Tmax . By continuity the stronger bounds hold at τ , so a(τ ) = 2ε. Since ae ≤ β < ε/4, there is a first time t− ∈ (tentry , τ ) at which a = ε/4. On [tentry , t− ], the bound for Y 2 /a and CY β ≤ ε/8 give Y 2 ≤ CY βε/32 ≤ (ε/16)2 , so N⊥ ≤ a + Y ≤ 5ε/16. Together 2 +z 2 ≥ a2 /2 = 2ε2 , with (H.7), this shows that N⊥ < ε on [0, t− ], so Tε ≥ t− . At τ , we have N⊥2 ≥ z1,2 2,2 so Tε ≤ τ < ∞. Rτ It remains to estimate t− and τ . By (H.2) and the integral bounds above, tentry |ȧ/a − µα | du ≤ Clocal . Since a(t− ) = ε/4 and a(τ ) = 2ε, it follows that 1 ε 1 1 1 1 1 log ≤ τ − tentry − log ≤ log(2ε) + Clocal . log − Clocal ≤ t− − tentry − µα 4 µα ae µα ae µα As t− ≤ Tε ≤ τ and log(2ε) < 0, this proves (H.5).

■

Proof of Theorem 4.8. Let ε0 be as in Lemma H.2 for the given α and C, fix ε ∈ (0, ε0 ], and let cpass be as in that lemma. We show that the theorem holds with c := cpass /2 (so that c < 1) and K := Centry +

Clocal + log(4/ε) + log Centry , µα

which depend only on α, C and ε. Let d ≥ 2, and consider an initialization as in the theorem. Exchanging the two students if necessary, we may assume that z1,2 (0) > 0 > z2,2 (0); this changes none of Tε , rinit and ainit . Since c ≤ cpass , the initialization then satisfies the hypotheses of Lemma G.5 with cpass in place of centry . By Lemma H.2, Tε is finite and satisfies (H.5). By Lemma G.5, µα 0 ≤ tentry ≤ Centry , and (G.16) gives | log(1/ae ) − log(rinit /ainit )| ≤ log Centry . These three bounds µα −1 give |Tε − µα log(rinit /ainit )| ≤ K. ■

Gradient Descent on Two ReLU Neurons: Global Landscape and Bifurcation Dynamics

Analyze & explore related research in Conceptio

Related documents