Random Quadratic Form with random forcing: Metastable synchronization by noise

Random Quadratic Form with random forcing: Metastable synchronization by noise Anna Shalova Korteweg-de Vries Institute for Mathematics, University of Amsterdam, a.shalova@uva.nl

arXiv:2608.16664v1 [math.PR] 17 Aug 2026

August 18, 2026 Abstract We study the Random Quadratic Form (RQF) on a sphere in the presence of random Brownian forcing. We show that the forcing does not effectively change the law of the process but affects the synchronization properties of the system. While the RQF without forcing exhibits partial synchronization due to the intrinsic symmetries, the introduction of an arbitrarily small forcing results in long-term symmetry breaking and leads to full synchronization. In this work we focus on the small forcing regime and recover the multiscale behavior of the two-point process. We show that in the first stage the model converges to an anti-polar configuration due to the symmetries of the RQF and in the second stage the two clusters meet due to the symmetry breaking phenomenon. The model is motivated by continuous-time machine learning models such as Neural ODEs and continuous-time formulations of transformers. In particular, the results of this work explain the role of the bias and the scale of its initialization.

Contents 1 Introduction 1.1 Neural ODEs and Transformers . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Literature overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 3 5 6

2 Notation and Preliminaries 7 2.1 Preliminaries on SDEs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2 Random Dynamical Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Random Attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 3 Main results 3.1 Random attractor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Metastable synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Auxiliary lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1

11 12 15 16

4 Multi-cluster and multiscale dynamics 21 4.1 Multiple clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 4.2 Multiple time-scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

1

Introduction

In this work we study the Random Quadratic Form (RQF) on a sphere Sn−1 with Brownian forcing given by the following stochastic differential equation: dXt = PXt ∂Qt Xt + γPXt ∂Wt ,

(1)

where PX denotes the projection onto the tangent space of Sn−1 at X PX := PTX Sn−1 = I − XX T , the noisy process Qt : (0, ∞) × Ω → Symn is given by 1 Qt = √ (Bt + BtT ), 2 where {Btij : i, j ∈ 1 . . . n} are independent Brownian motions and Wt : (0, ∞) × Ω → Rn is an n-dimensional Brownian motion independent of Qt . We use the notation ∂Qt , ∂Wt to specify that the equation is understood in the Stratonovich sense. The RQF model was introduced in [ES26] as a stochastic counterpart of a gradient flow of a quadratic functional and only included the multiplicative noise, namely the case γ = 0 was considered. Analogously to the deterministic setting, the system was shown to exhibit clustering behaviour in the sense that any two solutions of Eq. (1) with γ = 0 driven by the same noisy process in the long-time limit become either aligned or anti-polar. At the same time, the one point motion of the system is a Brownian motion and has no preferred direction, so the nontrivial behavior only appears on the level of the two-point motion. Such a phenomenon is known as synchronization by noise and we provide a rigorous formulation of synchronization by noise result for the RQF in Section 1.3. The anti-polar limiting configuration can be explained by the intrinsic symmetry of the RQF without forcing, which is violated in the presence of an arbitrarily small forcing γ. As a result of that, the system with any non-zero γ in the long-time limit synchronizes to a single point, namely the anti-polar state is no longer stable. At the same time, in the small forcing regime γ ↓ 0, the strong attraction to the symmetric configuration dominates on the time scale t ∼ log γ −1 and before converging to the random attractor consisting of a single point, the system approaches the anti-polar configuration defined by the dynamics of the non-forced RQF. We call this phenomenon metastable synchronization because both of the limiting configurations, namely stable and metastable ones, are synchronizing. In this work we characterize both of the attractive configurations and the corresponding rates of convergence. Despite changing the symmetry properties of the system, the given forcing preserves the qualitative behaviour of the single-point process: for an arbitrary γ, the RQF is a (rescaled) spherical Brownian motion. This, in particular, implies that the effect of the forcing is also only noticeable at the level of the two-point process. The rest of the paper is structured as follows. In the rest of the introduction we discuss the main driving application from machine learning, give a literature overview 2

Figure 1: Ensemble of solutions of eq. (1) with different initial conditions driven by the same realization of the noisy processes Qt and Wt with the forcing parameter γ = 0.1. In the first stage (the first row), the particles cluster into an anti-polar configuration and follow the dynamics of the ’meta’-attractor AQ (ω). In the second stage (the second row), the particles converge to the global random attractor A(ω) and continue moving as a single cluster. and conclude the section with a schematic statement of the main results. In Section 2 we give the necessary theoretical background on random dynamical systems and random attractors. We introduce and prove the main results in Section 3. Finally, in Section 4 we discuss how the results of this work can be extended to recover the metastable behaviour with multiple (> 2) scales.

1.1

Neural ODEs and Transformers

Neural ODEs, introduced in [CRBD18], are a class of neural networks in which the features x(t) evolve continuously in time according to an ordinary differential equation ẋ(t) = f (θ(t), x(t), t),

x0 ∈ Xin ,

where x0 is the input data defined on the input space Xin and θ are the parameters of the neural network. This is in contrast to the classical feed-forward networks, in which the features evolution is defined by a discrete-time dynamical system xk+1 = f (θk , xk ),

3

x0 ∈ Xin .

Considering a specific parametrization of a Neural ODE, consisting of a single feed-forward layer with a normalization step, the corresponding dynamics of features takes the form
ẋ = Px s M (t)x + B(t) , x(0) = x0 ∈ Sn−1 , (2) where s is an activation function and the time-dependent parameters M (t) and B(t) are the weights and the biases of the linear layer. Define the cumulative weight and biases processes as Z t Z t Qt = M (s)ds and Wt = B(s)ds. 0

0

Then, considering the linear activation function s(x) = x, we note that features in (2) follow the RQF dynamics as in Eq. (1) and therefore the RQF can be understood as a simplified model of a Neural ODE model. To justify the white-noise structure of the driving processes we remark that in discrete-time neural networks, the parameters θk of every layer k are usually initialized randomly and independently from layer to layer. Hence, we argue that the RQF driven by the diffusion processes Qt and Wt as defined in (1) is a natural continuous-time proxy for a neural ODE at initialization. The relative scale of the weight and the bias initialization is then encoded in the single parameter γ, and, as we show below, this scale determines the long-time behaviour of the system. Moreover, recently introduced continuous-time models of transformers [SABP22, GLPR25] can be understood as an extension of the Neural ODE framework and are specifically concerned with the joint dynamics of features corresponding to multiple inputs. In particular, the input of a continuous time transformer is a sequence of vectors (xi (0) ∈ Xin )i=1...N , which in language modeling problems correspond to different words in a text. In contrast to Neural ODEs, the dynamics of every feature vector xi (t) in transformers is additionally coupled to the states of all the other vectors xj (t) through the so-called Self-Attention mechanism and takes the form ẋi = Pxi (FF(xi ) + Attn(xi ; x1 , x2 . . . xN )) , FF(xi ) = s(M xi + B), X 1 T exi Q Kxj V xj , Attn(xi ; x1 , x2 . . . xN ) = P x QT Kx i j je j see [VSP+ 17, GLPR25] for an extensive description of the architecture. Such a structure can be interpreted as an interacting particle system, where the Feed-Forward layers act as an effective potential and Self-Attention layers describe the interaction between vectors. In the absence of the interaction force, the tokens (xi )1≤i≤d are driven by the common noise coming from the shared parameters M and B and thus follow the multipoint motion of the RQF with forcing. Recently, clustering and metastability of tokens in transformers have been extensively studied for the self-attention driven dynamics [GLPR24, GLPR25], see Section 1.2 for details. In this work we follow the approach of [ES26] and provide a counterpart of these results for the dynamics defined purely by the feed-forward layers. In particular, our results imply the existence of metastable clustering of the multi-point motion in Neural ODEs when the variance of the bias initialization is smaller than the variance of the weights initialization.

4

1.2

Literature overview

Diffusions on a sphere. Diffusion processes on Sn−1 naturally arise in the context of stability analysis of stochastic differential equations by means of the multiplicative ergodic theorem. In particular, the Furstenberg–Khasminskii formula reduces the calculation of the leading Lyapunov exponent to the ergodic averaging of a specific functional over the ergodic measure of the projective process [Fur63, Kha11], see also [Arn98, Chapter 6] and [IL01]. Therefore, synchronizing behaviour of diffusions on the sphere is closely related to the Lyapunov stability of SDEs in Euclidean spaces and has been studied in various formulations. Specifically, synchronization of diffusions on Sn−1 has been established: for the canonical Brownian motion in [Bax86], for general isotropic Brownian flows in [Rai99, CGS16] and for the RQF formulation in [ES26]. We remark that for each of these models the one-point motion is a spherical Brownian motion and therefore carries no distinctive information about the flow, which motivates the study of the two-point process. We remark that the Euclidean counterpart of this picture is classical: the full Lyapunov spectrum for isotropic Brownian flows on Rd is computed in [LJ85], see also [BH86]. And the random attractor in the stable regime is shown to be a singleton [LJ85, DLJ88]. Synchronization by noise. Synchronization of diffusion processes is an example of a more general phenomenon which is known as synchronization by noise. For a general class of random dynamical systems it has been formalized in [FGS17a], and we refer the reader to this work for an extensive literature review on the topic. The approaches allowing to establish synchronization by noise include multiplicative ergodic theorem [ACW83, Bax91, CR04] and the Feller explosion test [Sch02, CGS16] which was also used for the RQF without forcing in [ES26]. Alternatively, synchronization can be deduced from the order-preserving properties of the dynamics [CF98, FGS17b]. Local synchronization can also be established using large-deviations theory, see e.g. [MHM96, Tea08]. Metastability in particle systems. Classically, metastability of a stochastic system refers to the phenomena observed in gradient-type systems when the system spends a large time near the local minima of the corresponding energy. In such a setting the metastability can be studied using a large deviations-based approach and, in particular, Eyring-Kramers asymptotics [FW12, BG06, BdH15]. This work is concerned with a different setting, namely small perturbation of a stochastic process but not of a deterministic flow. At the same time, our result can be understood as an analog of the classical metastability phenomenon but on the level of the two-point motion. Since we characterize the relative convergence of multi-point motion, another related phenomenon is the transient clustering in interacting particle systems where the fast scale corresponds to the clusters formation. In particular, on a finite-particle level, the metastable clustering of interacting diffusions and the corresponding Lyapunov exponents are studied in [AEG26]. In a mean-field setting, the cluster formation and merging dynamics is discussed in [GGH+ 26, WSHW26]. We also remark that the mean-field formulation is a special case of the dynamic metastability framework [OR07]. However, the clustering mechanism studied in these works is of a different nature and is caused by the pairwise interactions. Note that interactions are not present in our setting.

5

Clustering and metastability in transformers. The continuous-time models of transformers can be interpreted as interacting particle systems [SABP22, GLPR24, GLPR25] and have been shown to exhibit (metastable) clustering dynamics. In particular, the longtime clustering of tokens has been established in various settings, see [Rig26] for an overview of the results and recent works [BKK+ 25, ABRR26, LMP+ 26]. In particular, the rate of convergence to a single cluster in this context is established in [CLPR25]. The transient clustering dynamics in the context of transformers is studied in [GKPR24, BPA25, BPA26]. Closest to the present work are the stochastic formulations of Neural ODEs and transformers, in which the randomness appears due to the random initialization of the parameters. In particular, the pathwise synchronization of the flow of randomly initialized Neural ODEs is characterized in [ES26, ABG+ 26]. Similarly, clustering in random attentionbased models is established in [FSE+ 26, KGR26]. We also remark that the pathwise synchronization is conceptually different from the clustering in noisy transformer models [SS26, BBR25], which correspond to interacting particle systems with independent noises.

1.3

Main Results

In this work we study the effect of additive noise on the two-point process of the RQF and the metastable behaviour arising in the small-forcing regime γ → 0. We characterize both the global attractor and the transient clustering dynamics appearing on the faster time-scale. We complement the analysis by studying the one-point motion and showing that no metastability arises on the level of one-point dynamics. To introduce the results, we will require some preliminary facts. Recall that the processes Qt and Wt are independent and therefore the probability space generated by (Qt , Wt ) is in fact a product space Ω = ΩQ × ΩW , where Qt = Qt (ω Q ) and Wt = Wt (ω W ). In case γ = 0, the space ΩW is redundant and easily factors out. In addition, the SDE (1) has smooth coefficients and thus admits a path-wise solution, see Proposition 2.6. We denote the corresponding solution map by Xtω (x0 ), where x0 ∈ Sn−1 denotes the initial condition X0 = x0 . With this notation we are ready to present the main results. In particular, recall the characterization of the two-point process without forcing from [ES26]. Theorem 1.1 (γ = 0, [ES26, Theorem 4.8]). There exists a random set AQ (ω) = AQ (ω Q ) consisting of two anti-polar points AQ (ω) = {aQ (ω), −aQ (ω)}, where aQ (ω) is measurable with respect to the past, and for all x0 ∈ Sn−1 the pathwise solution Xtω (x0 ) in the long-time limit converges to AQ (θt ω): dist(Xtω (x0 ), AQ (θt ω)) → 0

Ω − a.s.,

where θt is the time-shift defined in Eq. (3). The random set AQ is in fact the forward attractor of the corresponding random dynamical system as defined in Section 2.3. In case of the non-zero forcing, AQ (ω) no longer attracts all the trajectories in the long-time limit due to the effect of the additive noise. Instead, all the trajectories synchronize to a single-valued attractor A(ω). However, on a time-scale t ∼ log |γ|−1 , the set AQ (ω) is still attractive, and thus in the forced regime we call AQ (ω) the ’meta’-attractor after the ’meta’-stable behaviour it describes. We, therefore, obtain the following characterization of the dynamics for γ ̸= 0. 6

Theorem 1.2 (γ = ̸ 0). There exists a random set AQ (ω Q ) as in Theorem 1.1 and a random singleton A(ω) = {a(ω)} such that the dynamics consists of the two stages: • (convergence to the ’meta’-attractor AQ ): 1

E dist(Xtω (x0 ), AQ (θt ω)) ≲ e− 2 t + |γ|C(n, t), • (convergence to the global attractor A): E dist(Xtω (x0 ), A(θt ω)) ≲ e−λ(n,γ)t , where θt is the time-shift as in Eq. (3) and C(n, t) ≲ holds for an arbitrary choice of x0 .

1 λ(n, γ) ∼ γ 2 (n − 1), 2

√ 2t ne for large t. The characterization

In particular we obtain explicit convergence rates in expectation to the random attractor in both forced and non-forced regimes, which allows us to study the intermediate regime of convergence to the anti-polar configuration. We highlight that the two attractors AQ and A are structurally different and therefore the long-term behaviour of the two-point motion of the system with forcing differs from the non-forced case and exhibits a multi-scale behavior, see Figure 1. At the same time, this difference cannot be detected on the level of one point motion as follows from the following result, see also Theorem 3.1. Theorem 1.3 (RQF is a Brownian motion). For any γ ∈ R the process Xt/(1+γ 2 ) is an Sn−1 -valued Brownian motion. Acknowledgments and use of AI. I am grateful to Maximilian Engel and Enrique Carro Garrido for many insightful discussions on random attractors. The work was supported by the Dutch Research Council (NWO), in the framework of the program VI.Vidi.233.133 ‘A Rigorous Framework for Transient Random Dynamics’. I used AI at different stages of the manuscript preparation. In particular, the use of a regularized function in Lemma 3.2 and Proposition 3.8 was suggested by an AI and the code used to produce illustrations was generated by an LLM. The paper was written entirely by me and all mistakes are mine.

2

Notation and Preliminaries

In this section we introduce the notation and give the necessary theoretical background on SDEs and random dynamical systems. We refer the reader to [Arn98] for a detailed introduction to RDS. For a separable metric space X we denote the Borel σ-algebra on X by B(X ). Let Q Ω = C0 (R, Rn×n ) and ΩW = C0 (R, Rn ) be the spaces of continuous functions satisfying ω(0) = 0 equipped with the metric d: d(ω, ω b ) :=

∞ X 1 N =1

∥ω − ω b ∥N N 2 1 + ∥ω − ω b∥

∥ω − ω b ∥N := sup ∥ω(t) − ω b (t)∥,

,

|t|≤N

N

7

and let F Q = B(ΩQ ), F W = B(ΩW ). Let PQ and PW be the Wiener probability measures on F Q and F W , where the Wiener probability measure of an Rm -valued Brownian motion is given by Z xm Z x1 1 −∥y∥2 /2|t| e dy1 · · · dym , P ({ω ∈ Ω : ω1 (t) ≤ x1 , . . . , ωm (t) ≤ xm }) = · · · (2π|t|)m/2 −∞ −∞ for all x ∈ Rm . We denote the product probability space by (Ω, F, P) = (ΩQ × ΩW , F Q × F W , PQ × PW ). Finally, we define the family of time shifts (θt )t∈R on the product space (Ω, F) by (3)

θt ω(·) := ω(t + ·) − ω(t),

and remark that this family preserves the Wiener measure of a set A ∈ F, namely P(θt−1 A) = P(A) for all t ∈ R, A ∈ F.

2.1

Preliminaries on SDEs.

Let M be a smooth Riemannian manifold without boundary. Consider an M-valued Ito SDE of the form dXt = F0 (Xt ) dt +

m X

Fj (Xt ) dWtj ,

X0 = x ∈ M, t ∈ R.

(4)

j=1

Fixing a probability space of the Brownian motion Wt we define the pathwise solution of (4) as follows. Definition 2.1 (Pathwise solution). Given a probability space generated by an mdimensional two-sided Brownian motion (Ω, F, P), an SDE (4) is said to admit a pathwise solution on (Ω, F, P) with initial condition x0 ∈ M if there exists a map X ω : Ω → C(R, M) satisfying ω-a.s Xtω = x0 +

Z t 0

F0 (Xsω (x0 )) ds +

m Z t X j=1

Fj (Xsω (x0 )) dWsω,j ,

X0ω = x0 ,

0

where Wtω,j is the j-th component of the sample path Wtω . Definition 2.2 (Infinitesimal generator). Let the diffusion Xt be a weak solution of (4), the operator L : C ∞ (M) → C ∞ (M)  1  (Lf )(x) := lim E f (Xt ) − f (X0 ) X0 = x t↓0 t is called the infinitesimal generator of Xt . The infinitesimal generator describes the evolution of statistics of a diffusion process, and thus can be used to track the expected distance between two solutions along the dynamics. This is the key component of the proof of Theorems 3.4 and 3.9, where we will require the following elementary version of Dynkin’s formula. 8

Proposition 2.3 (Dynkin’s formula, [Øks03, Theorem 7.4.1]). Let L be the infinitesimal generator of a diffusion process Xt on R with X0 = x0 , then for any f ∈ Cc∞ and for all t ∈ R+ Z  t

Ef (Xt ) = f (x0 ) + E

(Lf )(Xs )ds . 0

Finally, we remark that infinitesimal generator L and its L2 adjoint L∗ define the backward and forward Kolmogorov evolutions respectively. The latter, ∂t ρt − L∗ ρt = 0, describes the evolution of ρt = law Xt and is also known as the Fokker-Planck equation.

2.2

Random Dynamical Systems

Definition 2.4 (Random dynamical system (RDS)). Let (Ω, F, P) be an abstract probability space and (M, B(M)) be a compact Riemannian manifold with the corresponding Borel σ-algebra. An RDS consists of the two components: 1. model of the noise: a family of F-measurable measure-preserving maps (θt : Ω → Ω)t∈R , satisfying: θ0 ω = ω, ∀ω ∈ Ω, θt+s ω = θt θs ω, ∀t, s ∈ R, ω ∈ Ω, 2. model of the dynamics: a (B(R) ⊗ F ⊗ B(M))-measurable map φ : R × Ω × M → M which, for all ω ∈ Ω and x ∈ M satisfies the cocycle property φ(0, ω, x) = x, and φ(t + s, ω, x) = φ(t, θs ω, φ(s, ω, x)), ∀s, t ∈ R. Given an RDS (φ, θ), for any u, v ∈ R with u < v, we denote by Fu,v the sub-σ-algebra generated by the random variables φ(t, θs ω, x) for x ∈ M and t, s ∈ R with u ≤ s ≤ v and 0 < t ≤ v − s. We say that the RDS satisfies the Markov property if its future is independent of its past, namely: Definition 2.5 (Markov RDS). The RDS (θ, φ) is called Markov if F−∞,0 and F0,∞ are independent. Due to the following result, any SDE with sufficiently regular coefficients can be uniquely reformulated in the form of a Markov RDS. Proposition 2.6 (SDE as an RDS, [Arn98, Theorem 2.3.42]). Consider an Ito SDE (4) or the corresponding Stratonovich SDE and assume that F0 ∈ C 1,δ (M, T M) and every Fj ∈ C 2,δ (M, T M) for some δ ∈ (0, 1]. Let (Ω, F, P) be a (fixed) probability space generated by a two-sided m-dimensional Brownian motion. Then there exists a unique (B(R) ⊗ F ⊗ B(M))-measurable map φ(t, ω, x) such that • Xtω = φ(t, ω, x) is a pathwise solution of (4), • (θ, φ) is a Markov RDS, where (θt )t∈R is the family of time shifts defined in Eq. (3).

9

Due to their equivalence, we switch to the RDS notation φ(t, ω, x) for the pathwise solutions of Eq. (4). Moreover, to use the probabilistic estimates of Dynkin’s type for Markov RDS with random initial conditions we will need the following property of conditional expectation. Lemma 2.7 (Freezing lemma). Let (θ, φ̂) be a Markov RDS on a manifold M over the probability space (Ω, F, P) and let β : Ω → M be an F−∞,0 -measurable random point. Let ψ : C(R+ , M) → R be a Borel-measurable bounded map and define Ψ(x) := E[ψ(φ̂(·, ω, x))],

∀x ∈ M.

Then Ψ is Borel-measurable, bounded and E[ψ(φ̂(·, ω, β(ω))) F−∞,0 ] = Ψ(β(ω)),

Ω − a.s..

In particular, Ψ(β(ω)) is F−∞,0 -measurable and satisfies inf Ψ(x) ≤ E[Ψ(β(ω))] ≤ sup Ψ(x). x

x

Proof. Since φ̂ is Markov, the map (ω, x) 7→ φ̂(·, ω, x) is F0,∞ ×B(M)-measurable, implying that the same is true for the composition function ψ(φ̂(·, ω, x)). With this remark, the result is a special case of the ’freezing’ lemma [Bal17, Lemma 4.1]. In particular, the freezing lemma decouples expectations with respect to the past and future when the corresponding σ-algebras are independent.

2.3

Random Attractors

Given a state space M and a probability space (Ω, F, P), we say that a set-valued map A : Ω → B(M) is a random compact set if it is Ω-a.s. compact and the function dist(x0 , A(ω)) is F-measurable for all x0 ∈ M. With this, we define a random point attractor of an RDS as the random set attracting all trajectories under the forward dynamics, namely Definition 2.8 (Random Point Attractor). A random compact set (A(ω))ω∈Ω is called the forward point attractor, if • it is φ-invariant: φ(t, ω, A(ω)) = A(θt ω), • for every x ∈ M
dist φ(t, ω, x), A(θt ω) → 0,

∀t ∈ R, Ω − a.s.

A random attractor A(ω) is called minimal if for any other random attractor Ã(ω) we have A(ω) ⊆ Ã(ω), Ω-a.s. Finally, replacing the a.s. convergence with convergence in probability we obtain the definition of a weak random point attractor. We note that if a strong point attractor exists, it is also a weak point attractor but the converse is not true. Existence of a weak point attractor for ergodic Markov RDS on compact state spaces is guaranteed by the correspondence theorem, see [KS12, Theorem 4.2.9]. In particular, the following holds. 10

Proposition 2.9 (Existence of a weak point attractor). Let (θ, φ) be a Markov RDS on a compact Riemannian manifold which admits a unique ergodic measure ρ, then the weak limit w φ(t, θ−t ω, ·)∗ ρ → µω , as t → ∞, exists almost surely and the random set A(ω) := supp(µω ) is the minimal weak point attractor of the RDS (θ, φ). As follows from the result by Le Jan [LJ87], the sample measures can either consist entirely of atoms or be fully continuous with no atoms at all, namely the following characterization holds. Proposition 2.10 (Discrete vs. continuous sample measures [LJ87]). The sample measures µω are either Ω-a.s. continuous, namely for every x ∈ M satisfy µω ({x}) = 0, or are Ω-a.s. discrete measures supported on N ∈ N atoms and given by N

µω =

1 X δa (ω) , N i=1 i

where each ai : Ω → M is an F−∞,0 -measurable random point.

3

Main results

We begin by recovering the Fokker-Planck equation of the RQF SDE with forcing. We establish that RQF is a rescaled spherical Brownian motion, which, in particular, implies that it is ergodic and the unique invariant distribution is the uniform measure on Sn−1 . Theorem 3.1 (RQF is a Brownian motion). Let ρt = law(Xt ) as in the eq. (1), then for any γ ∈ R, ρt is the unique classical solution of the rescaled heat flow on Sn−1 : 1 + γ2 w ∆ρt = 0, ρt → law(X0 ) as t ↓ 0, 2 where ∆ is the Laplace-Beltrami operator on Sn−1 . In particular, the uniform measure ρ̄ = vol n−11(Sn−1 ) volSn−1 is the unique ergodic measure of the RQF, where volSn−1 is the S volume measure of the n − 1-dimensional sphere. ∂t ρt −

Proof. Consider the following Stratonovich diffusions on Sn−1 : dUt = PUt ∂Qt Ut , dVt = PVt ∂Wt . The process Vt is a natural definition of a Brownian motion on Sn−1 . Moreover, the generator of the process Ut is LU = 12 ∆, see [ES26, Theorem 4.3]. Since the driving processes Qt and Wt are independent, we conclude that the generator of the RQF with forcing takes the form (1 + γ 2 ) ∆f, ∀f ∈ C ∞ . 2 Finally, since the Laplace-Beltrami operator on Sn−1 is essentially self-adjoint we conclude that law Xt solves the given Fokker-Planck equation and hence the result. Lf = (LU + γ 2 LV )f =

11

3.1

Random attractor

We now move to studying the ω-pointwise properties of the solutions to (1). For this we define an essential component of the proof, namely the auxiliary process Zt = ⟨Xt , Yt ⟩, where the couple (Xt , Yt ) solving: dXt = PXt ∂Qt Xt + γPXt ∂Wt , dYt = PYt ∂Qt Yt + γPYt ∂Wt ,

X0 ∈ Sn−1 , Y0 ∈ Sn−1

(5)

is the two-point motion of the RQF with forcing. Note that any two points on a sphere x, y ∈ Sn−1 coincide iff ⟨x, y⟩ = 1 and are anti-aligned iff ⟨x, y⟩ = −1. Therefore, convergence to either an anti-polar configuration or a singleton of a stochastic process can be both characterized in terms of the dynamics of the corresponding process Zt , namely its boundary behaviour at ±1. In particular, for γ ̸= 0 we will require the following moment-bound on (1 − Zt ) to upper bound the distance to the attractor. Lemma 3.2 (Lyapunov function for γ ̸= 0). Let (Xt , Yt ) be the two-point motion of the 2 (n−1)+4 RQF with forcing γ ̸= 0 and let Zt = ⟨Xt , Yt ⟩. Then for all p ∈ (0, γ 2(γ 2 +4) ) and every z0 = ⟨X0 , Y0 ⟩ ∈ [−1, 1]: E(1 − Zt )p ≤ (1 − z0 )p e−pαt , where α = min{γ 2 (n − 1), γ 2 (n − 1) + 4 − 2p(γ 2 + 4))}, in particular, α > 0. The proof of the bound is largely technical and therefore is deferred to Section 3.3. In addition, we will need the following pointwise bound. Lemma 3.3. For x, y ∈ Sn−1 let z = ⟨x, y⟩, then ∀p ∈ (0, 12 ]: dist(x, y) ≤

π (1 − z)p . 2p

Proof. First note that the geodesic distance on a sphere is upper bounded by the Euclidean distance dist(x, y) ≤ π2 ∥x − y∥. At the same time, expanding the norm of the difference we get ∥x − y∥2 = ∥x∥2 + ∥y∥2 − 2 ⟨x, y⟩ = 2(1 − z), and since x, y ∈ Sn−1 implies 1 − z ∈ [0, 2]: √ √ 1 1 ∥x − y∥ = 2(1 − z) 2 = 2(1 − z) 2 −p (1 − z)p ≤ 21−p (1 − z)p . Combining the inequalities we get the result. We now proceed to characterizing attractors of the RQF with forcing. Theorem 3.4 (Random attractor with forcing). Let γ ̸= 0, then there exists an F−∞,0 measurable map a : Ω → Sn−1 such that A(ω) = {a(ω)} is the strong forward point attractor of the RQF RDS. Moreover, for all x0 ∈ Sn−1 : E(dist(φ(t, ω, x0 ), A(θt ω))) ≤ πe−λγ t , 2

where λγ = γ2γ 2 +4 (n − 1). 12

(6)

Proof. The proof is structured as follows. First we prove almost sure collapse dist(Xt , Yt ) → 0 using the Lemma 3.2. Then we relate the two-point dynamics to the weak forward random point attractor of the corresponding RDS. Finally, we apply the freezing Lemma 2.7 to establish almost sure convergence and the estimate (6). Step 1: Lyapunov function. Applying Lemma 3.2 with p = γ 22+4 , we obtain α(γ, n, p) = γ 2 (n − 1) and therefore: E(1 − Zt )p ≤ (1 − z0 )p e−λγ t , 2

where λγ = γ2γ 2 +4 (n−1). Since the bound holds for an arbitrary choice of the initial condition z0 ∈ [−1, 1], the stochastic process ξt = (1 − Zt )p is a non-negative supermartingale. Moreover, it is uniformly bounded by construction, namely ξt ≤ 2p . Thus, by Doob’s supermartingale convergence theorem, the convergence ξt → 0 holds Ω-almost surely. Hence, using the pointwise comparison dist(x, y) ≤ 2πp (1 − z)p from Lemma 3.3, we conclude that dist(Xt , Yt ) → 0 almost surely for every X0 , Y0 ∈ Sn−1 . Step 2: Random attractor. By Proposition 2.9, there exist the sample measures µω and a weak point attractor A(ω). Since the sphere is a compact manifold and the RQF is ergodic, by [Bax91, Proposition 2.6]: (Pt )∗ (ρ̄ × ρ̄) → E(µω (dx) × µω (dy)) where Pt∗ is the adjoint of the semigroup of the two-point process (5). At the same time, for the test function f (x, y) = dist(x, y) using dist(Xt , Yt ) → 0 we obtain Z Z lim Pt f (x, y)d(ρ̄ × ρ̄) = 0 = dist(x, y)E(µω (dx) × µω (dy)). t→∞

and thus µω is Ω-a.s. supported on a single point, which we denote by a(ω). Applying Proposition 2.10, we conclude that the set A(ω) := {a(ω)} is the minimal weak point attractor of the RQF with forcing and a(ω) is measurable with respect to the past. Step 3: Almost sure convergence. First note that by the φ-invariance of the random attractor we necessarily have A(θt ω) = {a(θt ω)} = {φ(t, ω, a(ω))}. Thus, to upgrade to an almost sure convergence we argue as follows. Consider the Markov RDS φ̂ corresponding to the two-point process of the RQF: φ̂(t, ω, (x, y)) = (φ(t, ω, x), φ(t, ω, y)), and the function ψ : C(R+ , Sn−1 ) × C(R+ , Sn−1 ) → {0, 1} defined as ψ(u, v) := 1{dist(u(t), v(t)) → 0}, with the corresponding function Ψ : Sn−1 × Sn−1 → [−1, 1]: h i Ψ(x, y) := E(ψ(φ(·, ω, x), φ(·, ω, y))) = P dist(Xt , Yt ) → 0 X0 = x, Y0 = y . From step 1 we know that Ψ((x, y)) ≡ 1. Consider the random F−∞,0 -measurable point β(ω) = (x0 , a(ω)), then applying freezing Lemma 2.7 to the RDS φ̂, we deduce P[dist(φ(t, ω, x0 ), φ(t, ω, a(ω)) → 0] = E[Ψ(β(ω))] ≥ inf Ψ(x, y) = 1. x,y

Step 4: Convergence rate. Arguing analogously to step 4 we obtain the expected rate of convergence to the random attractor. In particular, consider the same φ̂ and β(ω) as 13

in the previous step and define a family of functions κt : C(R+ , Sn−1 ) × C(R+ , Sn−1 ) for t ∈ R+ : κt (u, v) = dist(u(t), v(t)), which is bounded by construction. The correspondning function Ψ then takes the form Ψ(x, y) := E[κt (φ̂(·, ω, (x, y)))] = E[κt (φ(·, ω, x), φ(·, ω, y))] h i = E dist(Xt , Yt ) X0 = x, Y0 = y . Then, applying the freezing Lemma 2.7 and using the exponential convergence from Lemma 3.2 we obtain E[dist(φ(t, ω, x0 ), φ(t, ω, a(ω))] = E[Ψ(β(ω))] ≤ sup Ψ((x, y)) = πe−λγ t , x,y

where the bound π sup Ψ((x, y)) ≤ sup (1 − z)p p e−λγ t = πe−λγ t . 2 x,y z∈[−1,1] follows from Lemmas 3.2 and 3.3. Remark 3.5 (Contraction rate). We remark that the top Lyapunov exponent classically provides local exponential convergence rate for points in the ω-dependent neighborhood of the attractor; namely bounds of the form dist(φ(t, ω, x), A(θt ω)) ≤ C(ω, x)eΛmax t dist(x, A(ω)),

∀x : dist(x, A(ω)) < r(ω, x),

which do not generally imply exponential convergence to the attractor in expectation. Notably, for the classical Brownian flow on Sn−1 given by dXt = PXt ∂Wt ,

(7)

[Bax86, Theorem 5.3(ii)] establishes almost sure exponential convergence with the rate 1 e− 2 (n−1)t . We also remark that our rate −λγ /γ 2 recovers the result of [Bax86] λγ 2(n − 1) 1 − 2 =− → − (n − 1) 2 γ 4+γ 2 in the limit γ → 0.

◁

Remark 3.6 (Rate for large γ). For large γ the second branch of α(γ, n, p) becomes 2 (n−1)+4 optimal in Lemma 3.2, giving the optimal value p̂ = γ 4(γ 2 +4) . At the same time, for n ≥ 4 √ 1 and large enough γ 2 we have p̂ ≥ 2 . Thus we can take p = 12 directly estimating 1 − z to get the contraction rate 1 (n − 2) 2 λγ = pα(p, γ, n) = (γ 2 (n − 1) + 4 − γ 2 − 4) = γ 2 2 for γ ≫ 1.

◁

14

Remark 3.7 (Generalizations). We expect similar estimates to hold for a larger class of isotropic diffusions on Sn−1 . In particular, for the classical formulation of a spherical Brownian motion (7) the proof applies in a simplified form since the γ-independent components are not present. In particular, taking p ↑ 12 and following the steps of the proof above, yields the following estimate 1

E dist(φ̃(t, ω, x0 ), A(θt ω)) ≤ C0 e− 2 (n−2)t , which is consistent with the rate for the large γ regime in Remark 3.6. We also note that 1 the rate in expectation is weaker than the almost sure rate e− 2 (n−1)t established in [Bax86, Theorem 5.3(ii)]. To cover the case n = 2, it is also possible to choose p ∈ (0, 12 ) still giving exponential contraction and therefore almost sure convergence. For example, taking p = 14 we get the estimate 1 E dist(φ̃(t, ω, x0 ), A(θt ω)) ≤ C0 e− 8 t . ◁

3.2

Metastable synchronization

Finally, we consider the small forcing regime γ → 0 and establish the multi-scale behaviour of the two-point process (5). In this Section we slightly change the notation and use the upper index γ to denote the RQF process with the forcing γ, namely Xtγ and, analogously, φγ for the corresponding RDS. In particular, Xt0 denotes the RQF without forcing as studied in [ES26]. Since the dynamics for small t is dominated by quadratic component, the proof relies on a coupling argument between the processes Xtγ and Xt0 . First, arguing analogously to the Theorem 3.4, we establish convergence rates for the non-forced RQF φ0 . Proposition 3.8 (Exponential convergence for γ = 0). Let γ = 0, then there exists a random set AQ (ω) = AQ (ω Q ) consisting of two anti-polar points AQ (ω) = {aQ (ω), −aQ (ω)}, Q where aQ (ω) is an F−∞,0 -measurable random point and the RQF RDS satisfies: 1 π E(dist(φ0 (t, ω, x0 ), AQ (θt ω))) ≤ √ e− 2 t , 2

(8)

for all x0 ∈ Sn−1 The proof of Proposition 3.8 is analogous to the proof of Theorem 3.4 and is postponed to Section 3.3. We are now moving to the main result. Theorem 3.9 (Metastability of the two-point motion). For any γ ∈ R there exists a random set AQ (ω Q ) = {aQ (ω Q ), −aQ (ω Q )}, Q where aQ (ω Q ) is an F−∞,0 -measurable function, and the RQF RDS (φγ , θ) satisfies √ π − 1 t π n − 1 √ 4t γ Q E dist(φ (t, ω, x0 ), A (θt ω)) ≤ √ e 2 + |γ| e − 1, 4 2

for all x0 ∈ Sn−1 . 15

(9)

Remark 3.10 (The time scale). The two terms of the upper bound (9) balance at t ∈ R+ solving √ n − 1 √ 4t − 12 t e = |γ| e − 1, 4 which corresponds to 1 2 t = log |γ|−1 − log n + O(1) 5 5 as γ → 0. This estimate supports the statement of the ’meta’-attractor AQ being attractive on the time scale of order log |γ|−1 . ◁ Proof. We consider the coupling of the RQFs with and without forcing (φ0 , φγ ) and by triangle inequality obtain E dist(φγ (t, ω, x0 ), AQ (θt ω)) ≤ E dist(φ0 (t, ω, x0 ), AQ (θt ω)) + E dist(φ0 (t, ω, x0 ), φγ (t, ω, x0 )) := A(t) + B(t). 1

By Proposition 3.8, the first term decays exponentially A(t) ≤ √π2 e− 2 t , and thus we only need to bound the distance between φ0 and φγ . Let Vt := Xt0 − Xtγ , V0 = 0, then Vt solves the following SDE dVt =

n−1 2 γ γ Xt − Vt dt − γPXtγ dWt + (PXt0 dQt Xt0 − PXtγ dQt Xtγ ), 2

(10)

and applying Ito formula to f (x) = ∥x∥2 we conclude that the evolution of ∥Vt ∥2 follows the dynamics d∥Vt ∥2 = (γ 2 (n − 1) + λV ∥Vt ∥2 )dt + dMt , (11) where

γ 2 (n − 1) ≤ 4, 2 and EMt = 0 is a martingale, see Lemma 3.12 for details. Integrating Eq. (11) and taking expectation on both sides then gives Z t 2 2 E∥Vt ∥ ≤ ∥V0 ∥ + 4E∥Vs ∥2 ds + γ 2 (n − 1)t. λV = 2 Xtγ , Xt0

2

+ 2 Xtγ , Xt0 −

0

Applying standard Gronwall argument we obtain the upper bound γ 2 (n − 1) 4t γ 2 (n − 1) 4t (e − 1) ≤ (e − 1). 4 4 p Finally, note that by Cauchy-Schwartz inequality B(t) ≤ E∥Vt ∥2 . Therefore, using dist(x, y) ≤ π2 ∥x − y∥ and combining the estimates for A(t) and B(t) we get the result. E∥Vt ∥2 ≤

3.3

Auxiliary lemmas

We begin with characterizing the dynamics of the process Zt , which is used to establish the moment bound in Lemma 3.2 and Proposition 3.8.

16

Lemma 3.11 (Dynamics of Zt ). Let (Xt , Yt ) be the two-point motion of the RQF with forcing and let Zt = ⟨Xt , Yt ⟩. Then Zt solves the Ito stochastic differential equation
dZt = γ 2 (1 − Zt )(n − 2 − Zt ) − 2Zt (1 − Zt2 ) dt − Zt (XtT dQt Xt + YtT dQt Yt ) + 2XtT dQt Yt + γ(1 − Zt )(Xt + Yt )T dWt . In particular, the infinitesimal generator of the process Zt , takes the form
(Lγ f )(z) = (1 − z) γ 2 (n − 2 − z) − 2z(1 + z) ∂z f + (1 − z)2 (1 + z)(2(1 + z) + γ 2 )∂zz f,

(12)

for any f ∈ C ∞ ([−1, 1]). Proof. We start with reformulating the RQF (1) in the Ito form: dXt = −

(n − 1)(1 + γ 2 ) Xt dt + PXt dQt Xt + γPXt dWt , 2

which follows from [ES26, Lemma 4.9] and the classical representation of a Brownian motion on Sn−1 . Then, applying Ito’s lemma to the function f (x, y) = ⟨x, y⟩ we obtain dZt = df (Xt , Yt ) = XtT dYt + YtT dXt + d[Xt , Yt ] = −(n − 1)(1 + γ 2 )Zt dt + d[Xt , Yt ] + dMtγ ,

(13)

where the martingale Mtγ is given by the following equation dMtγ = −Zt (XtT dQt Xt + YtT dQt Yt ) + 2Xt dQt Yt + γ(1 − Zt )(Xt + Yt )T dWt . Rt Calculating the quadratic covariation [Xt , Yt ] = 0 q(Zs )ds we obtain the following expression for the function q: X X q(z) = qγ=0 (z) + γ 2 Pxi,j Pyi,j = qγ=0 (z) + γ 2 (δi,j − xi xj )(δi,j − y i y j ) i,j 3

i,j 2

2

= nz − 3z + 2z + γ (n + z − 2), where

qγ=0 (z) = nz − 3z + 2z 3

is the quadratic covariation of the quadratic noise component of the RQF as follows from the proof of [ES26, Lemma 4.9]. Plugging the expression into (13) we obtain the cumulative drift −(n − 1)(1 + γ 2 )z + nz − 3z + 2z 3 + γ 2 (n + z 2 − 2) = γ 2 (1 − z)(n − 2 − z) − 2z(1 − z 2 ). Finally, to obtain (12) we calculate the covariance of the process Zt : X Σ(z) = Σγ=0 (z) + γ 2 (1 − z)2 (xi + y i )2 = 4(1 − z 2 )2 + 2γ 2 (1 − z)2 (1 + z) i

where Σγ=0 (z) = 4(1 − z 2 )2 is the covariance of the quadratic part derived in [ES26, Theorem 4.8], and hence the result. 17

We are now ready to prove Lemma 3.2 and Proposition 3.8. Proof of Lemma 3.2. By Lemma 3.11, the generator of the process Zt takes the closed form and is therefore decoupled from the dynamics of (Xt , Yt ). We consider the test function gp,δ = (1 − z + δ)p for δ, p ∈ (0, 1). Note that gp,δ is smooth on [−1, 1] and since ′ the process is restricted to the interval [−1, 1] the discontinuity of gp,δ at z = 1 + δ does not play any role. For gp,δ we calculate
(Lγ gp,δ )(z) = −p(1 − z) γ 2 (n − 2 − z) − 2z(1 + z) (1 − z + δ)p−1 + p(p − 1)(1 − z)2 (1 + z)(2(1 + z) + γ 2 )(1 − z + δ)p−2 := −pΛp (n, γ, z)gp,δ + Rp (δ, γ, z), where Λp (n, γ, z) = γ 2 (n − 2 − z) − 2z(1 + z) − (p − 1)(1 + z)(2(1 + z) + γ 2 )
= γ 2 (n − 1) + (z + 1) 2 − p(γ 2 + 2(1 + z)) , Rp (n, δ, γ, z) = pδ(1 − z + δ)p−1 (γ 2 (n − 2 − z) − 2z(1 + z)) − p(p − 1)(1 − z + δ)p−2 (1 + z)(2(1 + z) + γ 2 )δ(δ + 2(1 − z)) =: A + B Let s = 1 + z and rewrite Λp as Λp (n, γ, z) = γ 2 (n − 1) + s(2 − p(γ 2 + 2s)). Since it is a concave function of s for any given interval it is minimized at one of the end points. Since z ∈ [−1, 1], and therefore s ∈ [0, 2] we conclude that min Λp (n, γ, z) = min{γ 2 (n − 1), γ 2 (n − 1) + 4 − 2p(γ 2 + 4))} := α(γ, n, p),

z∈[−1,1]

2

(n−1)+4 implying that for every p ∈ (0, p∗ ), where p∗ = γ 2(γ 2 +4) , α(·, ·, p) is a strictly positive function. At the same time, using z ∈ [−1, 1], p ∈ (0, p∗ ) and p∗ < 1 we obtain the following upper bounds:

|A| ≤ δ(1 − z + δ)p−1 (γ 2 (n − 1) + 4) ≤ δ p (γ 2 (n − 1) + 4) |B| ≤ δ 2 (1 − z + δ)p−2 2(4 + γ 2 ) ≤ 2δ(1 − z + δ)p−1 (4 + γ 2 ) ≤ 2δ p (4 + γ 2 ), hence,

|Rp (n, δ, γ, ·)| ≤ δ p (γ 2 n + 12) := δ p β(γ, n)

on [−1, 1]. Applying Dynkin’s formula (Proposition 2.3) to gp,δ for all δ > 0 we obtain Z t  p Egp,δ (Zt ) ≤ gp,δ (z0 ) + E (−pα(γ, n, p)gp,δ (Zs ) + δ β(γ, n))ds . 0

Using a standard variation of constants estimate we therefore conclude that Egp,δ (Zt ) ≤ gp,δ (z0 )e−pα(γ,n,p)t + δ p

β(γ, n) . pα(γ, n, p)

Since the inequality gp,0 (z) ≤ gp,δ (z) holds for all δ > 0 pointwise on [−1, 1], we can bound the p-th moment of (1 − Zt ) by: E(1 − Zt )p ≤ E(1 − Zt + δ)p ≤ gp,δ (z0 )e−pα(γ,n,p)t + δ p Taking limit δ → 0 concludes the proof. 18

β(γ, n) pα(γ, n, p)

Proof of Proposition 3.8. First note that the existence of the strong forward point attractor AQ (ω) follows directly from [ES26, Theorem 4.8]. The proof of the estimate (8) follows the structure of the proof of Theorem 3.4 with the Lyapunov function hp,δ (z) = (1 − z 2 + δ)p . In particular the generator of Zt0 takes the form (L0 f )(z) = −2z(1 − z 2 )∂z f (z) + 2(1 − z 2 )2 ∂zz f (z), for all f ∈ C ∞ ([−1, 1]). Plugging in hp,δ we obtain the expression L0 hp,δ = 4pz 2 (1 − z 2 )(1 − z 2 + δ)p−1 + 4p(1 − z 2 )2 (1 − z 2 + δ)p−2 ((p − 1)2z 2 − (1 − z 2 + δ)) = 4p(1 − z 2 + δ)p (z 2 + (p − 1)2z 2 − (1 − z 2 ))
+ 4pδ(1 − z 2 + δ)p−2 (1 − 2z 2 )(1 − z 2 + δ) + 2(1 − p)z 2 [2(1 − z 2 ) + δ] := −4pΛ̂p (z)hp,δ + R̂p (z, δ), where Λ̂p (z) = (1 − 2pz 2 )
R̂p (z, δ) = 4pδ(1 − z 2 + δ)p−2 (1 − z 2 )(1 + 2(1 − 2p)z 2 ) + δ(1 − 2pz 2 ) , and for p ∈ (0, 21 ] |R̂p | ≤ 4pδ(1 − z 2 + δ)p−2 (3(1 − z 2 ) + δ) ≤ 12pδ p . Choosing p = 14 we get Λ̂ 1 (z) ≥ 12 and reproducing the variation of constants estimate 4

1

from Lemma 3.2 we bound the growth of the random variable (1 − Zt2 ) 4 by 1

1

1

E(1 − Zt2 ) 4 ≤ (1 − z02 ) 4 e− 2 t . Since the points converge to either polar or anti-polar configuration, we define ξt := min(∥Xt − Yt ∥, ∥Xt + Yt ∥), p and, using ∥Xt ± Yt ∥ = 2(1 ± z), obtain p p √ 1 ξt ≤ ∥Xt + Yt ∥∥Xt − Yt ∥ = 4 4(1 − Zt )(1 + Zt ) = 2(1 − Zt2 ) 4 , which immediately yields Eξt ≤

√ −1t 2e 2 .

Similarly to the proof of Theorem 3.4, applying the freezing argument and using dist(x, y) ≤ π ∥x − y∥ we conclude that 2 E dist(φ(t, ω, x0 ), AQ (θt ω)) ≤ Where

1 π π Eξt ≤ √ e− 2 t . 2 2

AQ (θt ω) = {φ0 (t, ω, aQ (ω)), −φ0 (t, ω, aQ (ω))}

by invariance of the random attractor. 19

Finally, we derive the dynamics of the process ∥Vt ∥2 used in the proof of Theorem 3.9. Lemma 3.12 (Dynamics of ∥Vt ∥2 ). Consider Xt0 , Xtγ and Vt as in Eq. (10), and let Ut = ∥Vt ∥2 , then Ut solves dUt = b(Xt0 , Xtγ , Ut )dt + dMt , where Mt is a martingale with EMt = 0 and   γ 2 (n − 1) γ γ γ 0 2 0 2 0 b(Xt , Xt , Ut ) = γ (n − 1) + Ut 2 Xt , Xt + 2 Xt , Xt − . 2 Proof. Applying Ito’s lemma to f (x) = ∥x∥2 = ⟨x, x⟩ we obtain df (Vt ) = 2 ⟨Vt , dVt ⟩ + d[Vt , Vt ] The first term gives 2 ⟨Vt , dVt ⟩ = (n − 1)(γ

2

⟨Vt , Xtγ ⟩ − Ut )dt + dMt = −(n − 1)Ut

  1 2 1 + γ dt + dMt , 2

where we used ⟨Vt , Xtγ ⟩ = Xt0 − Xtγ , Xtγ = Xt0 , Xtγ − 1 = −

1 1 Xt0 − Xtγ , Xt0 − Xtγ = − ∥Vt ∥2 . 2 2

Rt Calculating the quadratic covariation [Vt , Vt ] = 0 q(s)ds we obtain 2 X i,j 1 X  i,j γ k i,j i,k i,k 0 k γ j 0 j PXsγ (Xs ) − PXs0 (Xs ) + PXsγ (Xs ) − PXs0 (Xs ) q(s) = + γ2 (PXsγ )2 2 i,j,k i,j := I + II. The second term is essentially the trace of the projection matrix and therefore II = γ 2 (n − 1). k Let σi,j (x) := √12 (Pxi,j xk + Pxi,k xj ), then, simplifying the first term, we obtain I=

X

k (σi,j (Xtγ ))2 +

i,j,k

X

k (σi,j (Xt0 ))2 − 2

i,j,k

X

k k σi,j (Xt0 )σi,j (Xtγ ),

i,j,k

where for all a, b ∈ Sn−1 X k k σi,j (a)σi,j (b) = (n − 3) ⟨a, b⟩ + 2 ⟨a, b⟩3 . i,j,k

as follows from the proof of [ES26, Lemma 4.9]. In particular, for a = b we obtain X k (σi,j (a))2 = (n − 3) + 2 = n − 1. i,j,k

Plugging it into (14) and using ∥Vt ∥2 = 2 − 2 ⟨Xt0 , Xtγ ⟩ we obtain I = 2(n − 1) − 2(n − 3) Xt0 , Xtγ − 4 Xt0 , Xtγ = ∥Vt ∥2 ((n − 1) + 2 Xt0 , Xtγ 20

2

3

+ 2 Xt0 , Xtγ ),

(14)

Figure 2: On the left: the ensemble of solutions of Eq. (17) driven by the same noise from different initial conditions. The system gradually approaches each of the random attractors for k = 7, 3 and 1. On the right: the dynamics of pairwise scalar products between the particles showing the formation and dissolvement of the clusters. and hence the total drift takes the form   1 2 2 γ 0 2 b(Xt , Xt , Ut ) = γ (n − 1) − (n − 1)Ut 1 + γ + Ut ((n − 1) + 2 Xt0 , Xtγ + Xt0 , Xtγ ) 2   γ 2 (n − 1) γ γ 2 0 2 0 = γ (n − 1) + Ut 2 Xt , Xt + 2 Xt , Xt − . 2 Finally note that the process Mt satisfies M0 = 0 and is given by X X i,j i,j dMt = hit dWti + gt dBt , |hit |, |gti,j | < C, i

i,j

where all hit , gti,j are adapted processes. Thus Mt is a martingale and EMt = 0.

4

Multi-cluster and multiscale dynamics

In this section we discuss how the same approach can be extended to the cases of multiclustered attractors and multi-scale dynamics. From now on we only consider the dynamics on the circle, namely the case n = 2. Formulating a Brownian motion on a sphere of n > 2 dimensions with a discrete random attractor consisting of k > 2 points is generally an open question.

4.1

Multiple clusters

To construct a system with a random attractor consisting of k > 1 points we consider the angular representation of the RQF on the circle as in [ES26, Section 5.1]. In particular, consider the Stratonovich SDE with k-harmonic coefficients of the form dϕt = cos(kϕt )∂Bt1 + sin(kϕt )∂Bt2 , 21

(15)

where Bt1,2 are independent Brownian motions. Notice that for any k ∈ N the variable ψt = kϕt satisfies the equation (16)

dψt = k cos ψt ∂Bt1 + k sin ψt ∂Bt2 .

The model (16) is the angular representation of the (rescaled) classical Brownian motion as in Eq. (7) on S1 , implying that so is (15). Moreover, the RDS corresponding to (16) is fully synchronizing, namely its random point attractor is almost surely a singleton [Bax86, Section 5]. Therefore, the random attractor of the k-harmonic model (15) consists of at most k points and the following extension of [ES26, Proposition 5.1] holds. Proposition 4.1 (Harmonic model on S1 ). Let k ∈ N and ϕt be the harmonic model on R/2πZ ∼ = S1 defined in Eq. (15), then (i) ϕt is a Brownian motion (ii) the minimal weak random point attractor A(ω) of the corresponding RDS is supported on exactly k equidistant F−∞,0 -measurable random points:   2mπ A(ω) = a(ω) + : m = 0...k − 1 . k 2

2 Proof. The infinitesimal generator of the process (16) is L = k2 ∂ψψ , changing the variables back to ϕ gives (i). To show (ii) note that the proof of [ES26, Proposition 5.1] applies with the reparametrization ψt = kϕt .

At the same time, according to the Remark 3.7, we expect the model (16) to satisfy at least k2 E dist(ψ(t, ω, x0 ), A(θt ω)) ≤ C0 e− 8 t . The fact dist(ϕ1 , ϕ2 ) = k −1 dist(ψ 1 , ψ 2 ) therefore implies exponential convergence to the k2

random attractor in expectation for the k-harmonic model at the same rate e− 8 t .

4.2

Multiple time-scales

To construct a model with multiple time scales note that the RQF with forcing in angular coordinates is written as dϕt = (cos(2ϕt )∂Bt1 + sin(2ϕt )∂Bt2 ) + γ(cos(ϕt )∂Bt3 + sin(ϕt )∂Bt4 ), where the harmonic noise with k = 1 is treated as perturbation of a harmonic model with k = 2. We emphasize that the perturbation corresponds to the lower harmonic and that the multiscale dynamics would not be present in the opposite case, as follows from the estimates in Theorem 1.2 for γ ≫ 1. Indeed, if a higher harmonic serves as a small perturbation, then the unperturbed system synchronizes to a singleton on the fast time-scale and the perturbation will not be able to split the mass into two points again. Thus, to construct an model exhibiting a multiscale synchronization we consider a cascade of m harmonic noises of a decreasing order, namely the SDE of form: dϕt =

m−1 X

γ i (cos(ki ϕt )∂Bti,1 + sin(ki ϕt )∂Bti,2 ),

ki > ki+1 ,

(17)

i=0

where B , B are independent Brownian motions. We first note that ϕ is a rescaled Brownian motion on the circle. i,1

i,2

22

Proposition 4.2 (Multiharmonic model is a Brownian motion). Let ϕt be the multiharmonic model as in Eq. (17), then ϕt/w(γ,m) is a Brownian motion, where w(γ, m) = Pm−1 2i i=0 γ . Proof. As follows from Proposition 4.1 and the independence of P the driving processes 1 2i i,1 i,2 B , B , the generator of the multiharmonic process ϕt is Lf = 2 m−1 i=0 γ ∆ and hence the result. We remark that the time scale appearing in the Fokker-Planck equation of the multiharmonic model does not depend on the harmonics ki but only on the number of components m and the small parameter γ. At the same time, the parameters ki define the structure of the attractive configuration. In particular, the model with m harmonics admits m − 1 ’meta’-attractors Ai (ω) : i = 0 . . . m − 2 and one global attractor Am−1 (ω) where the structure the j-th attractor depend on the harmonics (ki )i≤j . To show the existence of m different random attractors and the corresponding mscale behaviour we argue as follows. We define the first random attractor A0 (ω) as the attractor of k0 -harmonic model (15). According to Proposition 4.1, it consists of exactly k0 equidistant points and is exponentially attractive with a rate independent on γ as discussed above. By a coupling argument analogous to the proof of Theorem 3.9, we also conclude that the divergence from A0 (ω) is linear in γ. To recover the second scale we define A1 (ω) as the random attractor of the truncated system driven by the first two harmonic noises: dϕ1t =

1 X

γ i (cos(ki ϕ1t )∂Bti,1 + sin(ki ϕ1t )∂Bti,2 ).

i=0

By Proposition 2.9, there exists a weak point attractor A1 (ω) of the truncated system bi-harmonic model ϕ1t and its structure only depends on the first two harmonic numbers k0 and k1 . Iterating this construction and considering the next truncated model dϕ2t =

2 X

γ i (cos(ki ϕ2t )∂Bti,1 + sin(ki ϕ2t )∂Bti,2 ).

i=0

we obtain a cascade of random weak point attractors. We also expect that the explicit exponential rates of convergence to the random point attractors Ai (ω) are available when ki mod ki+1 = 0 for all i < m. In this case Ai (ω) consists of exactly ki equidistant points and, by analogy with the RQF, the Lyapunov function of the two-point process at the i-th time scale is the corresponding harmonic function f (ξ 1 , ξ 2 ) = (1 − cos(ki (ξ 1 − ξ 2 )))p . We conjecture that in this case an analog of Theorem 1.2 could be formulated iteratively for each truncation. At the same time, we expect the multiscale behavior to appear in a general multiharmonic setting and we illustrate it on Figure 2. We show the dynamics of the multi-point motion of the system (17) with m = 3 and k = 7, 3, 1. As expected, at three different scales the model concentrates around different number of clusters. It is important to note that the harmonics are not divisible and thus the random attractor A1 (ω) is not an equidistant triplet. In this work we do not specify the structure ’meta’-attractors of a general multi-harmonic model. In addition, we remark that the transition between the ’meta’-attractors follows a non-trivial dynamics which is also outside of the scope of this article. 23

References [ABG+ 26] Andrea Agazzi, Giuseppe Bruno, Eloy Mosig García, Samuele Saviozzi, and Marco Romito. Stochastic scaling limits and synchronization by noise in deep transformer models. arXiv preprint arXiv:2604.26898, 2026. [ABRR26] Albert Alcalde, Leon Bungert, Konstantin Riedl, and Tim Roith. Quantifying concentration phenomena of mean-field transformers in the low-temperature regime. arXiv preprint arXiv:2605.10931, 2026. [ACW83]

L. Arnold, H. Crauel, and V. Wihstutz. Stabilization of linear systems by noise. SIAM J. Control Optim., 21(3):451–461, 1983.

[AEG26]

Zachary P Adams, Maximilian Engel, and Rishabh S Gvalani. Separation of time scales in weakly interacting diffusions. Archive for rational mechanics and analysis, 250(3):33, 2026.

[Arn98]

Ludwig Arnold. Random Dynamical Systems. Springer, Berlin, 1998.

[Bal17]

Paolo Baldi. Stochastic calculus. In Stochastic Calculus: An Introduction Through Theory and Exercises, pages 215–254. Springer, 2017.

[Bax86]

Peter H. Baxendale. Asymptotic behaviour of stochastic flows of diffeomorphisms: two case studies. Probab. Theory Related Fields, 73(1):51–85, 1986.

[Bax91]

Peter H Baxendale. Statistical equilibrium and two-point motion for a stochastic flow of diffeomorphisms. In Spatial Stochastic Processes: A Festschrift in Honor of Ted Harris on his Seventieth Birthday, pages 189–218. Springer, 1991.

[BBR25]

Krishnakumar Balasubramanian, Sayan Banerjee, and Philippe Rigollet. On the structure of stationary solutions to McKean–Vlasov equations with applications to noisy transformers. arXiv preprint arXiv:2510.20094, 2025.

[BdH15]

Anton Bovier and Frank den Hollander. Metastability: A Potential-Theoretic Approach, volume 351 of Grundlehren der mathematischen Wissenschaften. Springer, 2015.

[BG06]

Nils Berglund and Barbara Gentz. Noise-Induced Phenomena in Slow-Fast Dynamical Systems: A Sample-Paths Approach. Springer, 2006.

[BH86]

Peter H. Baxendale and Theodore E. Harris. Isotropic stochastic flows. Ann. Probab., 14(2):1155–1179, 1986.

[BKK+ 25] Martin Burger, Samira Kabri, Yury Korolev, Tim Roith, and Lukas Weigand. Analysis of mean-field models arising from self-attention dynamics in transformer architectures with layer normalization. Philosophical Transactions A, 383(2298), 2025. [BPA25]

Giuseppe Bruno, Federico Pasqualotto, and Andrea Agazzi. Emergence of meta-stable clustering in mean-field transformer models. In The Thirteenth International Conference on Learning Representations, 2025. 24

[BPA26]

Giuseppe Bruno, Federico Pasqualotto, and Andrea Agazzi. A multiscale analysis of mean-field transformers in the moderate interaction regime. Advances in Neural Information Processing Systems, 38:133305–133341, 2026.

[CF98]

Hans Crauel and Franco Flandoli. Additive noise destroys a pitchfork bifurcation. J. Dynam. Differential Equations, 10(2):259–274, 1998.

[CGS16]

Michael Cranston, Benjamin Gess, and Michael Scheutzow. Weak synchronization for isotropic flows. Discrete and Continuous Dynamical Systems-B, 21(9):3003–3014, 2016.

[CLPR25]

Shi Chen, Zhengjiang Lin, Yury Polyanskiy, and Philippe Rigollet. Quantitative clustering in mean-field transformer models. arXiv:2504.14697, 2025.

[CR04]

Tomás Caraballo and James C Robinson. Stabilisation of linear PDEs by Stratonovich noise. Systems & control letters, 53(1):41–50, 2004.

[CRBD18] Ricky T. Q. Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems, volume 31, 2018. [DLJ88]

Richard W. R. Darling and Yves Le Jan. The statistical equilibrium of an isotropic stochastic flow with negative Lyapunov exponents is trivial. In Séminaire de Probabilités XXII, volume 1321 of Lecture Notes in Math., pages 175–185. Springer, 1988.

[ES26]

Maximilian Engel and Anna Shalova. Random quadratic form on a sphere: Synchronization by common noise. arXiv preprint arXiv:2603.06187, 2026.

[FGS17a]

Franco Flandoli, Benjamin Gess, and Michael Scheutzow. Synchronization by noise. Probability Theory and Related Fields, 168(3):511–556, 2017.

[FGS17b]

Franco Flandoli, Benjamin Gess, and Michael Scheutzow. Synchronization by noise for order-preserving random dynamical systems. Ann. Probab., 45(2):1325–1350, 2017.

[FSE+ 26]

Lev Fedorov, Michaël E Sander, Romuald Elie, Pierre Marion, and Mathieu Laurière. Clustering in deep stochastic transformers. arXiv preprint arXiv:2601.21942, 2026.

[Fur63]

Harry Furstenberg. Noncommuting random products. Transactions of the American Mathematical Society, 108(3):377–428, 1963.

[FW12]

Mark I. Freidlin and Alexander D. Wentzell. Random Perturbations of Dynamical Systems, volume 260 of Grundlehren der mathematischen Wissenschaften. Springer, 3 edition, 2012.

[GGH+ 26] NJ Gerber, Rishabh S Gvalani, Martin Hairer, Grigorios A Pavliotis, and André Schlichting. Formation of clusters and coarsening in weakly interacting diffusions. Nonlinearity, 39(7):075023, 2026.

25

[GKPR24] Borjan Geshkovski, Hugo Koubbi, Yury Polyanskiy, and Philippe Rigollet. Dynamic metastability in the self-attention model. arXiv:2410.06833, 2024. [GLPR24]

Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. The emergence of clusters in self-attention dynamics. Advances in Neural Information Processing Systems, 36, 2024.

[GLPR25]

Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. A mathematical perspective on transformers. Bulletin of the American Mathematical Society, 62(3):427–479, 2025.

[IL01]

Peter Imkeller and Christian Lederer. Some formulas for lyapunov exponents and rotation numbers in two dimensions and the stability of the harmonic oscillator and the inverted pendulum. Dynamical Systems: An International Journal, 16(1):29–61, 2001.

[KGR26]

Hugo Koubbi, Borjan Geshkovski, and Philippe Rigollet. Homogenized transformers. arXiv preprint arXiv:2604.01978, 2026.

[Kha11]

Rafail Khasminskii. Stochastic stability of differential equations. Springer Science & Business Media, 2011.

[KS12]

Sergei Kuksin and Armen Shirikyan. Mathematics of two-dimensional turbulence, volume 194 of Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, 2012.

[LJ85]

Yves Le Jan. On isotropic Brownian motions. Z. Wahrscheinlichkeitstheorie verw. Gebiete, 70:609–620, 1985.

[LJ87]

Y. Le Jan. Équilibre statistique pour les produits de difféomorphismes aléatoires indépendants. Ann. Inst. H. Poincaré Probab. Statist., 23(1):111–120, 1987.

[LMP+ 26]

Sixu Li, Thomas Jacob Maranzatto, Jan Peszek, Trevor Teolis, Semih Akkoc, Konstantin Riedl, Sennur Ulukus, and Nicolás García Trillos. On the diverse dynamical behaviors arising in deep linear transformers. arXiv preprint arXiv:2607.18584, 2026.

[MHM96]

R.E. Mahony, U Helmke, and J.B. Moore. Gradient algorithms for principal component analysis. The ANZIAM Journal, 37(4):430–450, 1996.

[Øks03]

Bernt Øksendal. Stochastic differential equations. In Stochastic differential equations: an introduction with applications, pages 65–84. Springer, 2003.

[OR07]

Felix Otto and Maria G Reznikoff. Slow motion of gradient flows. Journal of Differential Equations, 237(2):372–420, 2007.

[Rai99]

Olivier Raimond. Flots browniens isotropes sur la sphere. In Annales de l’Institut Henri Poincare (B) Probability and Statistics, volume 35, pages 313–354. Elsevier, 1999.

26

[Rig26]

Philippe Rigollet. The mean-field dynamics of transformers. In International Congress of Mathematicians 2026. SIAM, 2026.

[SABP22]

Michael E Sander, Pierre Ablin, Mathieu Blondel, and Gabriel Peyré. Sinkformers: Transformers with doubly stochastic attention. In International Conference on Artificial Intelligence and Statistics, pages 3515–3530. PMLR, 2022.

[Sch02]

Michael Scheutzow. Comparison of various concepts of a random attractor: A case study. Archiv der Mathematik, 78(3):233–240, 2002.

[SS26]

Anna Shalova and André Schlichting. Solutions of stationary McKean–Vlasov equation on a high-dimensional sphere and other Riemannian manifolds. Adv. Nonlinear Anal., 15(1):Paper No. 20250141, 2026.

[Tea08]

Oliver M. Tearne. Collapse of attractors for ODEs under small random perturbations. Probab. Theory Related Fields, 141(1-2):1–18, 2008.

[VSP+ 17]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.

[WSHW26] Nathalie Wehlitz, Richard Scherzer, Carsten Hartmann, and Stefanie Winkelmann. Energetic characterisation of transient clustering dynamics in aggregation-diffusion systems. arXiv preprint arXiv:2605.30243, 2026.

27

Related documents

• Random Quadratic Form with random forcing: Metastable synchronization by noise#625229
• Random Quadratic Form with random forcing: Metastable synchronization by noise#622956
• Quadratic Perturbations of Markov Systems#985187
• Scaling limit for the pinning model in correlated Gaussian environment beyond the $L^2$-regime#926511
• Incompressibility and spectral gaps of random circuits#1051965