Conceptio › Archive › arXiv CS

arXiv CSopen access

Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

neural-networks

machine learning, deep learning, neural networks

Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction

arXiv:2605.20107v1 [cs.LG] 19 May 2026

Robert Jenkinson Alvarez 3 March 2026

Abstract JEPAs often regularize one-view embeddings toward an isotropic Gaussian, implicitly baking Euclidean symmetry into the representation. We show that this is not merely a benign default. For a known structured downstream geometry H ≻ 0, the minimax and maximum-entropy covariance under a Hamiltonian energy budget is (c/d)H −1 , and Euclidean isotropy incurs a closed-form price of isotropy. More importantly, when the downstream geometry is unknown, no geometry-independent fixed marginal target is canonical: every fixed covariance shape can be maximally misaligned for some structured geometry. We further show that even oracle one-view marginals do not identify the JEPA view-to-view predictive coupling. These results suggest that the structural bias in JEPAs should enter the cross-view coupling rather than a fixed encoder marginal. We instantiate this principle with HamJEPA, which encodes each view as a phase-space state (q, p) and predicts view-to-view transitions with a learned Hamiltonian leapfrog map, while non-isotropic scale and spectral floors prevent collapse. In a deliberately headless token protocol, HamJEPA improves over SIGReg on CIFAR-100 by +4.89 kNN@20 and +3.52 linear-probe points at 30 epochs, and by +6.45 kNN@20 and +10.64 linear-probe points at 80 epochs, while a matched MLP predictor ablation shows that the symplectic coupling is the ingredient driving the neighborhood-geometry gain. On ImageNet-100, HamJEPA-q improves by +4.82 kNN@20 and +7.52 linear-probe points at 45 epochs.

Introduction

Joint Embedding Predictive Architectures (JEPAs) learn representations by predicting one embedding from another. For a fixed predictive loss, downstream performance depends on embedding geometry: scale, anisotropy, and collapse. We study objectives of the form minθ LJEPA (θ) + λ Lreg (Zθ ), where Lreg regularizes the embedding distribution induced by the encoder. A standard target is an isotropic Gaussian embedding distribution. LeJEPA motivates this choice under a “no downstream structure” viewpoint and enforces it with Sketched Isotropic Gaussian Regularization (SIGReg) [3]. Isotropy is a symmetry assumption: it treats all embedding directions as exchangeable. In many settings (patch-token grids, sequences, graphs), the index set carries known structure that induces preferred correlations and invariances; enforcing Euclidean isotropy can misalign the representation with this geometry. This paper makes three linked contributions. First, for a fixed structured downstream geometry H, we derive the minimax and maximum-entropy covariance (c/d)H −1 and the price paid by Euclidean isotropy. Second, we show that this oracle result should not be interpreted as a new universal marginal target: when H is unknown, no geometry-independent fixed covariance shape is canonical, and even oracle marginals do not identify predictive coupling. Third, we instantiate the resulting design principle with a phase-space JEPA (HamJEPA) that places Hamiltonian structure in the cross-view predictor through a symplectic leapfrog map while using non-isotropic scale and spectral floors to prevent collapse.

Code and models: GitHub: HamJEPA-HamSIGReg. 1.1

Positioning relative to prior regularizers and dynamics priors

LeJEPA regularizes Euclidean embeddings toward an isotropic Gaussian using SIGReg, a sliced characteristic-function (CF) matching objective [3]. KerJEPA broadens this marginal-discrepancy viewpoint with kernel-based discrepancies [22]. HamJEPA departs from this line by asking when Euclidean isotropy is misaligned with structured task geometry and by placing Hamiltonian structure in the predictive map rather than prescribing a Euclidean-isotropic encoder marginal. Several SSL objectives prevent collapse through low-order or spectral statistics, including Barlow Twins, VICReg, MCR2 , and coding-rate-regularized self-distillation [4, 18–20]. HamJEPA uses related scale and spectral non-degeneracy controls, but couples them to a phase-space JEPA with a symplectic predictor; we do not claim these anti-collapse penalties are novel in isolation. Hamiltonian Neural Networks, Hamiltonian Generative Networks, and Symplectic ODE-Net impose energy-based or symplectic structure on learned dynamics, typically for temporal or control data [8, 15, 21]. HamJEPA uses the same geometric bias in a different role: a predictor prior for paired augmentations, not a physics model of observed trajectories.

Setup

Data, views, and randomness. Let x ∼ D be a data sample and let τ ∼ A be a random augmentation. A view is v := τ (x). We sample paired views of the same sample: va = τa (x),

vb = τb (x).

(1)

We use two representation parameterizations: (A) Euclidean embedding. An encoder fθ : X → Rd maps a view to an embedding random variable Z := fθ (V ) ∈ Rd ,

(2)

where V denotes a generic random view. Assume finite second moments. Let µθ := E[Z] and e := Z − µθ so E[Z] e = 0 and Cov(Z) e = Σθ . Σθ := Cov(Z) ⪰ 0. We write centered embeddings as Z (B) Phase-space state (HamJEPA). A phase-space encoder Eθ : X → R2d0 outputs a state S := Eθ (V ) = (Q, P ),

Q, P ∈ Rd0 .

(3)

We interpret Q as a content coordinate and P as an auxiliary momentum coordinate used to model structured view-to-view transitions. We write projections πq (Q, P ) = Q and πp (Q, P ) = P . Batch (row) convention. For a minibatch of size n, we stack embeddings/states as row vectors: ⊤ ⊤ Z ∈ Rn×d has rows zi⊤ , and S ∈ Rn×2d0 has rows s⊤ i = (qi , pi ). We center batches by subtracting e = Z − 1 11⊤ Z. the row-mean, e.g. Z n Two Hamiltonians (to avoid notation collisions). Section 3 uses an SPD operator H ≻ 0 to specify a geometry on embedding space. HamJEPA uses a scalar energy Hϕ (q, p) generating a symplectic map on phase space. We reserve H for the SPD operator and Hϕ for the scalar energy.

A Theory of Marginal Geometry and Predictive Coupling in JEPAs

This section is pure theory: given a structured notion of task complexity and an energy budget, what covariance/distribution is canonical, and what is the cost of forcing Euclidean isotropy? Throughout this section, fix an SPD operator H ≻ 0, which defines the structured embedding geometry through the quadratic energy z ⊤ Hz; Hamiltonian-isotropic covariance in this geometry has the form αH −1 . e := Z − E[Z] and write Σ := Cov(Z). e We work with the centered embedding Z 2

3.1

Setup: structured tasks and a Hamiltonian budget

e and constrain task complexity by the H −1 -unit ball We model downstream linear tasks by y = w⊤ Z WH := {w ∈ Rd : w⊤ H −1 w ≤ 1}.

(4)

We pair this with a representation energy budget in the H-geometry: e = tr(HΣ) = c, E[Ze⊤ H Z]

c > 0.

(5)

Given Σ ⪰ 0, define the worst-case task variance functional V (Σ) := sup w⊤ Σw.

(6)

w∈WH

The next lemma gives a convenient spectral form. Lemma 3.1 (Spectral form of V (Σ)). For any Σ ⪰ 0, V (Σ) = λmax H 1/2 ΣH 1/2 .

(7)

Moreover, if umax is a unit top eigenvector of H 1/2 ΣH 1/2 , then wmax := H 1/2 umax attains the supremum in (6). □

Proof. See Appendix B.1.1. 3.2

Minimax covariance under a Hamiltonian budget

We ask for the covariance that minimizes the worst-case task variance (6) under the H-budget (5). Theorem 3.2 (Minimax covariance under a Hamiltonian budget). Fix H ≻ 0 and c > 0. Consider min

V (Σ).

(8)

Σ⪰0: tr(HΣ)=c

The unique minimizer is Σ⋆ =

c −1 H , d

(9)

and the optimal value is min

V (Σ) =

Σ⪰0: tr(HΣ)=c

c . d

(10)

Equivalently, H 1/2 Σ⋆ H 1/2 = dc I. □

Proof. See Appendix B.1.2. 3.3

Maximum entropy: the canonical distribution

The minimax result selects a canonical covariance. A complementary principle selects a canonical distribution. Theorem 3.3 (Maximum entropy under a Hamiltonian energy constraint). Let Z be an Rd -valued random vector with a density, E[Z] = 0, and finite second moments. Among all such distributions satisfying E[Z ⊤ HZ] = c, (11) the unique maximizer of differential entropy is the Gaussian c Z ∼ N 0, H −1 . d

(12) □

Proof. See Appendix B.1.3. 3

3.4

Price of Euclidean isotropy

Many practical objectives encourage Euclidean isotropy, i.e., Σ ∝ I in the original coordinates. This subsection quantifies the worst-case gap incurred by imposing Euclidean isotropy under the same Hamiltonian budget. Under the restriction Σ = σ 2 I, the constraint tr(HΣ) = c forces the unique feasible Euclideanisotropic covariance c Σiso = I. (13) tr(H) Theorem 3.4 (Price of Euclidean isotropy). Fix H ≻ 0 and c > 0. Let Σ⋆ be as in Theorem 3.2 and let Σiso be as in (13). Then c V (Σiso ) = λmax (H), (14) tr(H) V (Σiso ) d λmax (H) = ρ(H) := . (15) V (Σ⋆ ) tr(H) Moreover 1 ≤ ρ(H) ≤ d, with ρ(H) = 1 if and only if H ∝ I. Proof. See Appendix B.1.4. □ Proposition 3.5 (Metric isotropy equivalence). Let H ≻ 0 and let Z ∈ Rd be a random vector with finite second moments and covariance Σ := Cov(Z). Define the H-whitened representation Z ′ := H 1/2 Z. Then Cov(Z ′ ) = H 1/2 ΣH 1/2 . In particular, for any α > 0, Σ = αH −1 ⇐⇒ Cov(Z ′ ) = αI. □

Proof. See Appendix B.1.5. 3.5

A quadratic Hamiltonian lift of the oracle structured Gaussian

The structured Gaussian selected by Theorems 3.2 and 3.3 admits a natural lift to a Gibbs law on phase space. Proposition 3.6 (Quadratic Hamiltonian lift of the oracle covariance). Fix H ≻ 0 and c > 0, and define the quadratic Hamiltonian HH (q, p) := 12 q ⊤ Hq + 12 ∥p∥22 , q, p ∈ Rd . (16) Consider the Gibbs density on phase space d (17) πH (q, p) ∝ exp − HH (q, p) . c Then q and p are independent under πH , with c c q ∼ N 0, H −1 , p ∼ N 0, I . (18) d d Moreover, EπH [q ⊤ Hq] = c,

EπH [∥p∥22 ] = c,

EπH [HH (q, p)] = c.

(19)

In particular, the q-marginal of πH is exactly the minimax-optimal and maximum-entropy law from Theorems 3.2 and 3.3. Proof. See Appendix B.1.6. □ Proposition 3.7 (The oracle family spans the SPD cone). Fix c > 0. As H ranges over SPD operators, the family c (20) Σ⋆ (H) = H −1 d ranges over the entire SPD cone. Equivalently, for every Σ ≻ 0, choosing c H = Σ−1 (21) d yields Σ⋆ (H) = Σ. □

Proof. See Appendix B.1.7. 4

From the oracle covariance to a practical unknown-H relaxation. Proposition 3.6 shows that when H is known, the oracle covariance Σ⋆ = dc H −1 is exactly the q-marginal of a quadratic Hamiltonian Gibbs model on phase space. In that quadratic oracle case, the induced dynamics are q̇ = p,

ṗ = −Hq,

(22)

so a Hamiltonian predictor is not an unrelated construction, but a natural dynamical realization of the same structured geometry. This does not mean that the practical training mechanism is uniquely determined by Section 3. Rather, the theory should be read as an oracle statement: when H is known, the correct geometry is Hamiltonian-anisotropic rather than Euclidean-isotropic. When H is unknown, Proposition 3.7 shows that the oracle family {Σ⋆ (H) : H ≻ 0} spans the entire SPD cone, so the theory no longer singles out one fixed covariance target. Why not learn another fixed marginal target? The previous results might suggest replacing Euclidean isotropy by a better fixed target covariance. The next result shows why this is brittle when downstream geometry is unknown. Theorem 3.8 (No universal fixed marginal target). Fix any SPD target shape M ≻ 0 independent of the downstream geometry. Under geometry H ≻ 0, the budget-feasible covariance with shape M is ΣM (H) :=

c M. tr(HM )

Its regret relative to the oracle covariance Σ⋆ (H) = (c/d)H −1 is VH (ΣM (H)) d λmax (H 1/2 M H 1/2 ) = . ⋆ VH (Σ (H)) tr(HM ) This quantity equals 1 iff M ∝ H −1 , and for every fixed M ≻ 0, VH (ΣM (H)) = d. ⋆ H≻0 VH (Σ (H)) sup

Thus no geometry-independent fixed covariance shape is canonical for all structured downstream geometries. □

Proof. See Appendix B.1.8. d

Proposition 3.9 (Oracle marginals do not identify the predictive coupling). Let Za , Zb ∈ R be centered jointly Gaussian random vectors with Cov(Za ) = Cov(Zb ) = Σ ≻ 0 and cross-covariance Cov(Zb , Za ) = C. Then E[Zb | Za ] = CΣ−1 Za . Therefore, the same oracle marginal covariance Σ admits a continuum of different predictive maps as the admissible cross-covariance C varies. Marginal matching alone cannot determine the JEPA view-to-view prediction problem. □

Proof. See Appendix B.1.9.

Together, Theorem 3.8 and Proposition 3.9 show that the unknown-geometry JEPA problem should not be reduced to choosing a single fixed encoder marginal. The structural bias must instead enter the view-to-view coupling. HamJEPA instantiates this principle by using a Hamiltonian phase-space coupling: the quadratic known-H case recovers the oracle q-marginal, while the practical unknown-H method uses a learned symplectic predictor and geometry-agnostic non-collapse constraints. Takeaway. Section 3 does not merely say that Euclidean isotropy can be suboptimal. It shows that, under unknown downstream geometry, fixed marginal matching is the wrong object: no geometryindependent covariance target is universally canonical, and even an oracle marginal does not determine the JEPA predictive coupling. The structural bias must therefore enter the view-to-view map. 5

HamJEPA: Symplectic Predictive Structure Without Isotropic Marginals

Section 3 shows that fixed marginal matching is brittle under unknown structured geometry and that marginals alone do not determine JEPA prediction. HamJEPA therefore places the structural bias in the view-to-view coupling: a phase-space encoder is trained with a symplectic, volumepreserving predictor, while encoder-side scale and spectral constraints prevent collapse without enforcing isotropic Gaussian marginals. A visual overview of HamJEPA is provided in Appendix E. 4.1

Phase-space encoder

Given paired views va , vb as defined in Section 2, the encoder Eθ : X → R2d0 outputs phase-space states sa = Eθ (va ) and sb = Eθ (vb ) with s = (q, p) ∈ R2d0 . q, p ∈ Rd0 .

s = Eθ (v) = (q, p),

(23)

We interpret q as a content coordinate used for prediction and evaluation, and p as an auxiliary momentum coordinate enabling structured transitions. We write πq (q, p) = q and πp (q, p) = p. Observable coordinate vs. conjugate latent. Section 3 is formulated in terms of the representation used by downstream tasks; in HamJEPA, that conceptual role is played by q. The auxiliary coordinate p is the conjugate latent that makes Hamiltonian transport possible. For non-invertible augmentations, this makes q-matching the cleanest observable formulation, since the target need not contain a uniquely recoverable dynamical state. In practice, however, the exact matching choice is a stability knob rather than a theorem: on CIFAR-100 we primarily use q-matching, while on ImageNet-100 we found full-state (q, p)-matching more stable and therefore use it there. 4.2

Symplectic predictor via a separable Hamiltonian and leapfrog

Canonical symplectic structure.

Let J :=

0 −I

I 0

∈ R2d0 ×2d0

(24)

denote the canonical symplectic matrix. Hamilton’s equations and flow map. Given a differentiable scalar energy Hϕ : R2d0 → R, Hamilton’s ODE on phase space is ṡ(t) = J ∇s Hϕ s(t) , s(0) = s0 , (25) where J is the canonical symplectic matrix (24). We denote its time-t flow map by Φϕ,t , i.e. s(t) = Φϕ,t (s0 ). Separable energy (the theory-tight choice). To obtain a discrete predictor map with rigorous symplectic guarantees under an explicit integrator, we use a separable Hamiltonian energy Hϕ (q, p) = T (p) + Vϕ (q), T (p) = 12 ∥p∥22 , (26) where Vϕ : Rd0 → R is a learnable potential. Fixing T (p) = 12 ∥p∥2 is stabilizing and prevents scale-control constraints from being satisfied by rescaling a learnable kinetic metric. Leapfrog (velocity Verlet) predictor map. Given step size ∆t > 0 and K ∈ N steps, define the leapfrog update pk+ 21 = pk − ∆t 2 ∇q Vϕ (qk ), qk+1 = qk + ∆t pk+ 21 ,

k = 0, . . . , K − 1.

pk+1 = pk+ 12 − ∆t 2 ∇q Vϕ (qk+1 ), 2d0 This defines a one-step map Ψϕ,∆t : R → R2d0 and a K-step predictor (K) K Φϕ := Ψϕ,∆t .

(27)

(28)

Given sa = (qa , pa ), we predict ŝb = (q̂b , p̂b ) by (K)

ŝb = Φϕ (sa ). 6

(29)

4.3

Objective: content prediction plus non-isotropic anti-collapse

The projected log-det and participation-ratio penalties should be understood as an encoder-side anti-collapse bridge, not as the primary novelty of the method; similar covariance-volume objectives appear elsewhere, whereas our contribution is to couple such non-collapse controls with a phase-space JEPA and a symplectic predictive prior. (K)

Let sa = (qa , pa ) = Eθ (va ), sb = (qb , pb ) = Eθ (vb ), and ŝb = (q̂b , p̂b ) = Φϕ (sa ). HamJEPA optimizes min Lpred + λbi Lbi + λbudget Lbudget + λvol Lvol + λpr Lpr + λwd Rwd (θ, ϕ). (30) θ,ϕ

All expectations are implemented as minibatch averages; sg(·) denotes stop-gradient. (1) Matching loss. Let m ∈ {q, qp} denote the matching mode, with Πq (q, p) = q and Πqp (q, p) = (q, p). CIFAR-100 uses m = q, while ImageNet-100 uses m = qp for stability. We define Lpred := E ∥Πm (ŝb ) − sg(Πm (sb ))∥22 . (31) (2) Bidirectional prediction. step size −∆t:

Define the backward prediction by running the same integrator with (K)

ŝa = Φϕ (sb ; −∆t). The bidirectional h loss is

i (K) (K) Lbi := E ∥πq (Φϕ (sa ; +∆t)) − sg(qb )∥22 + ∥πq (Φϕ (sb ; −∆t)) − sg(qa )∥22 .

(32)

Time reversibility of leapfrog (Corollary A.5) makes this an “exact inverse in-model” constraint rather than an independent predictor. (3) Fixed-units scale budget (anti-gauge). Symplecticity constrains the predictor map but does not prevent the encoder from collapsing all views to a point. We therefore enforce scale budgets in fixed units (no Gaussianity, no isotropy): E∥q∥2 2 E∥p∥2 2 2 2 Lbudget := − αq + − αp , αq , αp > 0. (33) d0 d0 (4) Projected log-det volume floor. A scale budget alone can be satisfied by concentrating variance in a low-dimensional subspace. To prevent rank collapse without forcing isotropy, we impose a projected log-det floor on the content batch. Let Q ∈ RB×d0 contain centered content codes: PB Qi = qi − B1 j=1 qj . Sample a random projection R ∈ Rd0 ×k with orthonormalized columns (k ≪ d0 ) and form Y = QR ∈ RB×k . Define the regularized projected covariance 1 ΣY := Y ⊤ Y + εIk , ε > 0, (34) B−1 and per-dimension log-volume 1 ℓvol := log det(ΣY ). (35) k We use the hinge penalty 2 Lvol := max(0, τ − ℓvol ) , τ ∈ R. (36) (5) Participation ratio floor (prevents “one-spike” solutions). Log-det alone can be satisfied with one very large eigenvalue and many tiny ones when scale is not tightly controlled, or when minibatch effects dominate. To explicitly rule out the “one-spike” regime, we add a participation ratio (PR) floor on ΣY : 2 2 Pk tr(ΣY ) i=1 λi PR(ΣY ) := = Pk ∈ [1, k], (37) 2 tr(Σ2Y ) i=1 λi where λ1 , . . . , λk are the eigenvalues of ΣY . We penalize violations of a target effective rank r0 ∈ (1, k]: 2 Lpr := max(0, r0 − PR(ΣY )) . (38) Together, these terms prevent collapse while permitting anisotropy; HamJEPA does not constrain representations to a unit sphere or isotropic Gaussian target. 7

4.4

Structural guarantees

The formal structural theory of HamJEPA is deferred to Appendix A. At a high level, the separable (K) Hamiltonian (26) and leapfrog update (27) imply that the K-step predictor Φϕ is symplectic: ⊤ (K) (K) (K) DΦϕ (s) J DΦϕ (s) = J, det DΦϕ (s) = 1. (39) Thus the predictor is locally volume-preserving and is inverted by running leapfrog with step size −∆t. Since symplecticity constrains only the predictor, encoder collapse is controlled separately by the fixed-unit scale, projected log-det, and participation-ratio floors. Appendix A gives reciprocal singular-value pairing, shadow-Hamiltonian stability, and the anti-collapse lemmas.

Results

Controlled protocol. We evaluate frozen representations on CIFAR-100 and ImageNet-100 using cosine kNN and linear probing. The goal is not a projector-heavy SOTA recipe, but an objective-level comparison: all methods share the same backbone, augmentations, optimizer, batch size, schedule, two global views, no local views, the encoder runs in a headless token regime using ResNet stage-3 (layer3) features, and identity projector. For HamJEPA, the encoder output s = (q, p) is split channel-wise; q is the primary readout, while p and (q, p) test auxiliary-state and dimension effects. Full details and diagnostics are in Appendix D–F. SIGReg is the isotropic-Gaussian JEPA baseline closest to our theoretical comparison. This protocol is intentionally underpowered by modern SSL standards. By removing projection heads, local crops, EMA teachers, and recipe-specific tuning, the comparison isolates whether changing the predictive geometry changes the frozen representation. 5.1

CIFAR-100: controlled gains and predictor ablation

HamJEPA improves over SIGReg after only 30 epochs under the same backbone, views, optimizer, and identity projector. Using the primary q readout, HamJEPA gains +4.89 points in kNN@20 (31.45 vs. 26.56) and +3.52 points in linear probing (33.95 vs. 30.43). The p readout is also predictive (29.71 kNN@20; 33.07 linear), but remains behind q in nearest-neighbor accuracy, matching the intended role of q as the content coordinate and p as an auxiliary dynamical coordinate. Concatenating (q, p) gives the strongest linear probe at 30 epochs (34.18), while q alone gives the strongest kNN geometry. The 80-epoch predictor-family ablation gives the sharper control. MLP-HJEPA keeps the phasespace encoder, channel-wise q/p split, identity projector, prediction loss, anti-collapse regularizers, optimizer, batch size, and training schedule fixed, and replaces only the symplectic leapfrog map with a non-symplectic residual MLP predictor. This ablation is intentionally strong: MLP-HJEPA reaches high linear accuracy (42.26–42.77), showing that the phase-space split and regularizers can produce linearly useful features. However, its kNN@20 accuracy remains essentially at the SIGReg level (28.10–28.30 vs. 27.98). By contrast, HamJEPA with the symplectic predictor reaches 34.43 kNN@20 and 44.59 linear accuracy using the q readout. Thus the main geometric gain is not explained by the q/p split, anti-collapse terms, or generic MLP capacity: replacing the MLP by a symplectic Hamiltonian predictor yields a +6.33 point kNN@20 gain over MLP-HJEPA-q and a +6.45 point gain over SIGReg at the same 80-epoch budget. 5.2

ImageNet-100: same minimal recipe, larger-scale signal

On ImageNet-100, HamJEPA again improves over SIGReg under the same minimal recipe (45 epochs, single seed). Using the primary q readout, HamJEPA gains +4.82 points in kNN@20 (24.92 vs. 20.10) and +7.52 points in linear probing (31.92 vs. 24.40). The best HamJEPA linear readout is p at 32.08, giving a +7.68 point gain over SIGReg-q. These gains are not explained by feature dimension. SIGReg benefits modestly from concatenating its two 1024-d blocks: q → (q, p) improves by +0.84 kNN@20 points (20.10 → 20.94) and +1.14 linear-probe points (24.40 → 25.54). In contrast, HamJEPA gains essentially nothing from concatenation in kNN (24.92 → 24.64) and only +0.12 in linear probing (31.92 → 32.04). Crucially, the 1024-d HamJEPA-q readout already outperforms the 2048-d SIGReg-(q, p) readout by +3.98 points in kNN@20 and +6.38 points in linear probing. We match the full phase space (q, p) during pretraining for stability. Consistent with this objective, p is highly predictive and slightly exceeds q on the linear probe (32.08 vs. 31.92), while remaining 8

kNN@20 (%)

Linear (%)

kNN best (%)

30 epochs, mean±std over 3 seeds SIGReg 26.56 ± 0.18 HamJEPA-q 31.45 ± 0.27 HamJEPA-p 29.71 ± 0.25 HamJEPA-(q, p) 30.88 ± 0.39

30.43 ± 0.30 33.95 ± 0.24 33.07 ± 0.36 34.18 ± 0.17

– – – –

33.95 42.26 41.90 42.77 44.59 44.44 44.52

28.71 @ k=1 28.10 @ k=20 28.30 @ k=20 28.15 @ k=20 34.43 @ k=20 32.96 @ k=20 33.66 @ k=20

Setting

Method / readout

80 epochs, predictor-family ablation SIGReg 27.98 MLP-HJEPA-q 28.10 MLP-HJEPA-p 28.30 MLP-HJEPA-(q, p) 28.15 HamJEPA-q 34.43 HamJEPA-p 32.96 HamJEPA-(q, p) 33.66

Table 1: CIFAR-100 controlled results. The 30-epoch block shows short-budget performance; the 80-epoch block isolates the predictor family. MLP-HJEPA differs from HamJEPA only by replacing the symplectic leapfrog predictor with a non-symplectic residual MLP.

kNN@20 (%)

Linear probe (%)

SIGReg (q-eval) SIGReg (p-eval) SIGReg ((q, p)-eval)

20.10 18.78 20.94

24.40 23.86 25.54

HamJEPA (q-eval) HamJEPA (p-eval) HamJEPA ((q, p)-eval)

24.92 24.72 24.64

31.92 32.08 32.04

Method

Table 2: ImageNet-100 downstream performance after 45 epochs of pretraining, single seed. For HamJEPA, q and p are learned phase-space coordinates. For SIGReg, the same split is a representation-block / dimension control rather than a literal position–momentum decomposition.

comparable in kNN (24.72 vs. 24.92). However, concatenating (q, p) does not improve kNN, suggesting that the nearest-neighbor geometry is not simply a consequence of evaluating a larger vector. The result is therefore a controlled objective-level delta: HamJEPA improves frozen-feature geometry in a deliberately headless setting where neither projection-head capacity nor representation dimension explains the gain. Geometry diagnostics. The accuracy gains are accompanied by broader, non-collapsed covariance spectra without enforcing an isotropic Gaussian marginal. On ImageNet-100, SIGReg-q has effective rank ≈ 38.33 and PR ≈ 24.91, while HamJEPA-q reaches ≈ 94.73 and ≈ 45.27; even the larger SIGReg-(q, p) representation remains lower (≈ 51.16 and ≈ 36.17). CIFAR-100 shows the same qualitative pattern, with HamJEPA-q increasing effective rank from ≈ 109.7 to ≈ 124.5. Full spectra and training-time diagnostics are deferred to Appendix D.

Conclusion

We showed that Euclidean-isotropic JEPA regularization can be misaligned with structured task geometry, and that fixed marginal targets are brittle when the downstream geometry is unknown. HamJEPA instead places Hamiltonian structure in the view-to-view predictor: views are encoded as phase-space states, predicted with a symplectic leapfrog map, and kept non-collapsed by nonisotropic scale and spectral floors. Under a deliberately headless token protocol, HamJEPA improves frozen-feature kNN and linear-probe performance over SIGReg on CIFAR-100 and ImageNet-100. Limitations. This is a controlled geometry study rather than a SOTA SSL recipe. The main baseline is SIGReg, ImageNet-100 is single-seed, and we do not learn or recover the downstream operator H; broader baselines, multi-seed ImageNet runs, and learned structured geometries remain future work. 9

References [1] Shun’ichi Amari. Differential-geometrical methods in statistics, 1985. [2] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a jointembedding predictive architecture, 2023. [3] Randall Balestriero and Yann LeCun. Lejepa: Provable and scalable self-supervised learning without the heuristics, 2025. URL https://arxiv.org/abs/2511.08544. [4] Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning, 2021. [5] Ludwig Boltzmann. Weitere studien über das wärmegleichgewicht unter gasmolekülen, 1872. [6] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations, 2020. [7] J. Willard Gibbs. Elementary principles in statistical mechanics: Developed with especial reference to the rational foundation of thermodynamics, 1902. [8] Sam Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks, 2019. URL https://arxiv.org/abs/1906.01563. [9] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent: A new approach to self-supervised learning, 2020. [10] William Rowan Hamilton. On a general method in dynamics, 1834. [11] E. T. Jaynes. Information theory and statistical mechanics, 1957. [12] Pierre-Simon Laplace. Traité de mécanique céleste, 1799. [13] Joseph Liouville. Sur la théorie de la variation des constantes arbitraires, 1838. [14] Siméon-Denis Poisson. Sur la variation des constantes arbitraires dans les questions de mécanique, 1809. [15] Peter Toth, Danilo Jimenez Rezende, Andrew Jaegle, Sébastien Racanière, Aleksandar Botev, and Irina Higgins. Hamiltonian generative networks, 2019. [16] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding, 2018. [17] Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere, 2020. [18] Ziyang Wu, Jingyuan Zhang, Druv Pai, XuDong Wang, Chandan Singh, Jianwei Yang, Jianfeng Gao, and Yi Ma. Simplifying dino via coding rate regularization, 2025. URL https://arxiv. org/abs/2502.10385. [19] Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. In Advances in Neural Information Processing Systems, volume 33, 2020. [20] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Selfsupervised learning via redundancy reduction, 2021. [21] Yaofeng Desmond Zhong, Biswadip Dey, and Amit Chakraborty. Symplectic ode-net: Learning hamiltonian dynamics with control, 2019. [22] Eric Zimmermann, Harley Wiltzer, Justin Szeto, David Alvarez-Melis, and Lester Mackey. Kerjepa: Kernel discrepancies for euclidean self-supervised learning, 2025. URL https: //arxiv.org/abs/2512.19605. 10

Theory: symplecticity, volume preservation, and anti-collapse

This appendix records the structural guarantees enjoyed by HamJEPA when using the separable Hamiltonian (26) and the leapfrog integrator (27). These results justify the summary in Section 4.4: leapfrog gives a symplectic, volume-preserving, time-reversible predictor, while the encoder-side scale, log-det, and participation-ratio constraints prevent collapse without imposing Euclidean isotropic Gaussian marginals. All proofs are in Appendix B.2. A.1

Statements

Row/column convention (linear transforms). The paper uses column vectors for single states/embeddings and row-stacked matrices for minibatches. The following lemma formalizes the equivalence. Lemma A.1 (Row/column convention for left-multiplication by H 1/2 ). Let H ∈ Rd×d and let z ∈ Rd be a column vector. Define z ′ := H 1/2 z. Let Z ∈ Rn×d be a row-stacked minibatch with rows Zi,: = zi⊤ . Define Z′ := ZH 1/2 . Then for every i, 1/2 (Z′ )⊤ zi . i,: = H

□

Proof. See Appendix B.2.1.

Symplecticity of the continuous flow. We first state the standard symplecticity property of Hamiltonian flows. Theorem A.2 (Symplecticity of Hamiltonian flows). Assume Hϕ ∈ C 2 (R2d0 ) and let Φϕ,t denote the flow map of Hamilton’s ODE (25). Then Φϕ,t is symplectic: for all s and t, ⊤ DΦϕ,t (s) J DΦϕ,t (s) = J. (40) Consequently, det(DΦϕ,t (s)) = 1 for all s, t, and Φϕ,t preserves phase-space volume (Liouville). Proof. See Appendix B.2.2. □ Corollary A.3 (Volume-preserving maps preserve differential entropy). Let F : Rm → Rm be a C 1 diffeomorphism and let X admit a density. Then h(F (X)) = h(X) + E log | det(DF (X))| . In particular, if | det(DF (x))| = 1 for all x (e.g., F = Φϕ,t from Theorem A.2), then h(F (X)) = h(X). □

Proof. See Appendix B.2.3.

Symplecticity of the discrete predictor. HamJEPA uses the leapfrog/Verlet map (27) as a discrete predictor. For separable Hamiltonians this map is symplectic. Theorem A.4 (Leapfrog is symplectic and volume-preserving for separable Hamiltonians). Let Hϕ (q, p) = T (p) + Vϕ (q) with T (p) = 12 ∥p∥22 and Vϕ ∈ C 2 . Then the one-step leapfrog map Ψϕ,∆t (K) K defined by (27) is symplectic. Therefore, for any K ≥ 1, the K-step predictor Φϕ = Ψϕ,∆t is symplectic and satisfies (K) det DΦϕ (s) = 1 for all s. (41) Proof. See Appendix B.2.4. □ Corollary A.5 (Invertibility and time reversibility). Under the assumptions of Theorem A.4, the leapfrog map is bijective and satisfies Ψ−1 ϕ,∆t = Ψϕ,−∆t .

(42)

In particular, the K-step predictor admits an exact inverse by running the same integrator with step (K) (K) size −∆t: (Φϕ (·; +∆t))−1 = Φϕ (·; −∆t). □

Proof. See Appendix B.2.5. 11

Proposition A.6 (Approximation of separable Hamiltonian flows by HamJEPA predictors). Fix a horizon T > 0 and a compact set K ⊂ R2d0 . Let Kq := {q ∈ Rd0 : ∃p with (q, p) ∈ K}. Let V : Rd0 → R be C 3 on an open neighborhood of a compact set Ωq ⊇ Kq , and assume ∇V is Lipschitz on Ωq . Let ΦVt denote the time-t flow map of Hamilton’s ODE q̇ = p,

ṗ = −∇V (q)

(i.e., the separable Hamiltonian H(q, p) = 12 ∥p∥22 + V (q)). For a learnable potential Vϕ ∈ C 3 with ∇Vϕ Lipschitz on Ωq , let Ψϕ,∆t be one leapfrog step (27) (K) and let Φϕ := ΨK ϕ,∆t with K∆t = T (28). Assume that for all initial states in K, both the exact trajectory under V and the exact trajectory under Vϕ remain inside Ω := Ωq × Ωp for all t ∈ [0, T ]. Then there exists a constant C = C(T, Ω, Vϕ ) > 0 such that (K)

sup (q,p)∈K

Φϕ (q, p) − ΦVT (q, p) 2 ≤

sup ∥∇Vϕ (q) − ∇V (q)∥2 + q∈Ωq

integrator error

∆t2 |{z}

}

modeling error

In particular, if the potential class can approximate ∇V uniformly on Ωq (and admits bounded derivatives on Ωq ), then for every ε > 0 there exist ϕ and ∆t > 0 (hence K = T /∆t) such that sup (q,p)∈K

(K)

Φϕ (q, p) − ΦVT (q, p) 2 ≤ ε.

Proof. See Appendix B.2.6. □ 2d0 ×2d0 ⊤ Lemma A.7 (Reciprocal singular-value pairing). Let A ∈ R be symplectic: A JA = J. Then the singular values of A occur in reciprocal pairs: if σ is a singular value, so is 1/σ (with the same multiplicity). Equivalently, A cannot be contractive in all directions; any local contraction must be compensated by expansion. □

Proof. See Appendix B.2.7.

Long-horizon stability (shadow Hamiltonians). A core practical advantage of symplectic integrators is the existence of a nearby modified energy that explains stable multi-step rollouts. We record a standard backward-error-analysis statement. Theorem A.8 (Shadow Hamiltonian / backward error analysis (standard)). Assume Vϕ is sufficiently smooth (e.g., analytic with bounded derivatives on the region visited by the numerical trajectory). e [N ] such that: Then for any integer N ≥ 1 there exists a modified Hamiltonian H ϕ,∆t e [N ] (s) = Hϕ (s) + O(∆t2 ) uniformly on the region of interest. 1. (Closeness) H ϕ,∆t e [N ] up to 2. (Modified-flow representation) One leapfrog step equals the time-∆t flow of H ϕ,∆t local error O(∆tN +1 ). [N ]

e 3. (Near-conservation) Along the leapfrog iterates (sk ), the modified energy H ϕ,∆t (sk ) is nearly conserved, and the original energy error Hϕ (sk ) − Hϕ (s0 ) remains bounded over long horizons when ∆t is small. Proof (sketch via BCH/backward error analysis). See Appendix B.2.8.

□

Anti-collapse: what each ingredient does (and does not do). HamJEPA uses an encoder-side non-degeneracy bridge that avoids isotropic Gaussianity. It is useful to separate (i) scale control, (ii) rank/volume control, and (iii) spike control. Lemma A.9 (Scale budgets do not prevent rank collapse). Fix c > 0 and consider PSD matrices Σ ⪰ 0 with tr(Σ) = c. There exist such Σ with rank 1 (hence complete collapse except for one direction), e.g. Σ = diag(c, 0, . . . , 0). 12

Proof. See Appendix B.2.9. □ Lemma A.10 (Log-det alone does not prevent “one-spike” spectra). Fix k ≥ 2 and τ ∈ R. There exist positive definite Σ ∈ Rk×k with k1 log det(Σ) = τ but with PR(Σ) arbitrarily close to 1 (spike regime). Proof. Appendix B.2.10. □ Lemma A.11 (Participation ratio alone does not fix scale). For any PSD Σ ⪰ 0 and any scalar a > 0, PR(aΣ) = PR(Σ). Thus a PR constraint alone does not prevent collapse by uniformly shrinking all eigenvalues. Proof. See Appendix B.2.11. □ d0 Lemma A.12 (Projected log-det detects full-space rank collapse). Let Q ∈ R be a zero-mean random vector with covariance ΣQ ⪰ 0 and let R ∈ Rd0 ×k have full column rank k (e.g., orthonormal columns as in Sec. 4.3). Define the population projected covariance ΣY,0 := R⊤ ΣQ R ∈ Rk×k . Then rank(ΣY,0 ) ≤ rank(ΣQ ). In particular, if rank(ΣQ ) < k then ΣY,0 is singular and therefore det(ΣY,0 ) = 0 and log det(ΣY,0 ) = −∞. Moreover, if one uses the ridge-regularized version from (34), ΣY := ΣY,0 + εIk ,

ε > 0,

and if r := rank(ΣY,0 ), then ΣY has exactly k − r eigenvalues equal to ε and hence log det(ΣY ) = (k − r) log ε +

r X

log(λi + ε),

i=1

where λ1 , . . . , λr > 0 are the nonzero eigenvalues of ΣY,0 . □

Proof. See Appendix B.2.12.

Projected log-det floor. Recall the projected covariance ΣY from (34). The following proposition quantifies how a log-det floor plus a trace/scale control lower-bounds the smallest eigenvalue. Proposition A.13 (Projected log-det floor prevents rank collapse). Let ΣY ∈ Rk×k be SPD with eigenvalues λ1 ≥ · · · ≥ λk > 0. Suppose log det(ΣY ) ≥ kτ and tr(ΣY ) ≤ m for some m > 0. Then (k − 1)k−1 . (43) λmin (ΣY ) ≥ exp(kτ ) mk−1 In particular, the representation cannot collapse onto a subspace of dimension < k in the projected sense. □

Proof. See Appendix B.2.13.

Joint spectral non-degeneracy (budget + log-det + PR). The next theorem formalizes the “LogDet + PR + budget” bridge: it rules out both needle collapse and the one-spike “L-shaped” failure mode. Theorem A.14 (Spectral non-degeneracy via joint constraints). Let Σ ∈ Rk×k be SPD with eigenvalues λ1 ≥ · · · ≥ λk > 0. Assume the three constraints 1 tr(Σ) = c, PR(Σ) ≥ r0 , log det(Σ) ≥ τ, (44) k for some c > 0, r0 ∈ (1, k], and τ ∈ R. Then √ k−1 r0 c λmax (Σ) ≤ √ ; λmin (Σ) ≥ exp(kτ ) . (45) r0 c Consequently, Σ cannot realize the spike regime λ1 ≫ λ2 , . . . , λk and cannot approach rank collapse λmin → 0 under the stated floors. 13

□

Proof. See Appendix B.2.14.

Corollary A.15 (Bounded conditioning under joint non-degeneracy constraints). Under the assumptions of Theorem A.14, the condition number satisfies k λmax (Σ) c κ(Σ) := ≤ exp(−kτ ) √ . λmin (Σ) r0 Proof. See Immediate by dividing the eigenvalue bounds in Theorem A.14.

□

Why symplecticity is still meaningful despite encoder regularizers. Symplecticity constrains the predictor map, while the budget/volume/PR constraints constrain the encoder outputs. The next proposition states the correct conditional claim: if the encoder occupies nontrivial volume locally, then the symplectic predictor cannot destroy it. Proposition A.16 (Symplectic predictors cannot induce collapse). Fix ϕ and consider the predictor (K) map Φϕ from (28). If the encoder outputs occupy a set U ⊂ R2d0 with positive Lebesgue volume, (K)

(K)

then Φϕ (U) has the same volume as U. In particular, Φϕ cannot map a positive-volume set to a set of zero volume; any global representational collapse must arise from Eθ , not from the symplectic predictor. □

Proof. See Appendix B.2.15.

Takeaway. Theorems A.2 and A.4 ensure the predictor map is symplectic and volume-preserving (hence non-contracting in a precise geometric sense), with exact time reversibility (Corollary A.5) and reciprocal singular-value pairing (Lemma A.7). Since symplecticity alone does not prevent encoder collapse, HamJEPA uses a fixed-units scale budget plus projected log-det and PR floors; Theorem A.14 formalizes why the combination rules out both rank collapse and one-spike pathologies without enforcing isotropic Gaussian marginals.

Mathematical Proofs

B.1

Proofs for Section 3

B.1.1

Proof of Lemma 3.1

Let Σ ⪰ 0 and H ≻ 0. For w ∈ WH , define u := H −1/2 w. Then w⊤ H −1 w = ∥H −1/2 w∥22 = ∥u∥22 ≤ 1, so WH = {H 1/2 u : ∥u∥2 ≤ 1}. Substituting w = H 1/2 u into the objective yields w⊤ Σw = u⊤ H 1/2 ΣH 1/2 u. Therefore V (Σ) = sup u⊤ H 1/2 ΣH 1/2 u = λmax H 1/2 ΣH 1/2 , ∥u∥2 ≤1

where the last equality is the Rayleigh quotient characterization of the top eigenvalue. If umax is a unit top eigenvector of H 1/2 ΣH 1/2 , then the corresponding wmax := H 1/2 umax satisfies ⊤ wmax H −1 wmax = 1 and attains the supremum. B.1.2

Proof of Theorem 3.2

Let Σ ⪰ 0 and define A := H 1/2 ΣH 1/2 ⪰ 0. Then tr(HΣ) = tr(A), so the constraint tr(HΣ) = c becomes tr(A) = c. By Lemma 3.1, the objective is V (Σ) = λmax (A). Thus (8) is equivalent to min A⪰0: tr(A)=c

λmax (A).

Pd Let the eigenvalues of A be λ1 ≥ · · · ≥ λd ≥ 0. Then i=1 λi = c, hence λ1 ≥ c/d. Therefore, for every feasible A, c λmax (A) ≥ . d Equality λmax (A) = c/d holds if and only if λ1 = · · · = λd = c/d, i.e., A = (c/d)I, which is unique. Mapping back, c Σ⋆ = H −1/2 AH −1/2 = H −1 , d and the optimum value is c/d. B.1.3

Proof of Theorem 3.3

Let Z have a density, E[Z] = 0, and satisfy E[Z ⊤ HZ] = c. Define the invertible linear change of variables Y := H 1/2 Z. Then E[Y ] = 0 and E∥Y ∥22 = E[Z ⊤ HZ] = c. By the change-of-variables formula for differential entropy, h(Z) = h(Y ) + log | det(H −1/2 )| = h(Y ) − 21 log det(H). Hence maximizing h(Z) under (11) is equivalent to maximizing h(Y ) under E∥Y ∥22 = c. Let ΣY := Cov(Y ) ⪰ 0. A standard entropy bound states that for any random vector with covariance ΣY , 1 h(Y ) ≤ log (2πe)d det(ΣY ) , 2 with equality if and only if Y is Gaussian with covariance ΣY . Next, the constraint E∥Y ∥22 = c implies tr(Σ PY ) = c (since E[Y ] = 0). For any PSD matrix with eigenvalues ν1 , . . . , νd ≥ 0 and fixed sum i νi = c, the arithmetic–geometric mean inequality gives d d d c d 1 X Y νi = , det(ΣY ) = νi ≤ d i=1 d i=1 with equality if and only if ν1 = · · · = νd = c/d, i.e., ΣY = (c/d)I. Combining the two inequalities yields 1 log (2πe)d (c/d)d , 2 with equality if and only if Y ∼ N (0, (c/d)I). Therefore the unique maximizer is Y ∼ N (0, (c/d)I), and hence c Z = H −1/2 Y ∼ N 0, H −1 , d as claimed. h(Y ) ≤

B.1.4

Proof of Theorem 3.4

Under Euclidean isotropy Σ = σ 2 I, the budget tr(HΣ) = c forces c σ 2 tr(H) = c ⇒ σ 2 = , tr(H) c so Σiso = tr(H) I. Using Lemma 3.1, c c H = λmax (H), tr(H) tr(H) which is (14). From Theorem 3.2, V (Σ⋆ ) = c/d, so the ratio is c V (Σiso ) d λmax (H) tr(H) λmax (H) = = = ρ(H). ⋆ V (Σ ) c/d tr(H) Since λmax (H) ≥ tr(H)/d for any SPD H, we have ρ(H) ≥ 1. Also λmax (H) ≤ tr(H) implies ρ(H) ≤ d. Finally, ρ(H) = 1 holds if and only if λmax (H) = tr(H)/d, which occurs if and only if all eigenvalues of H are equal, i.e. H ∝ I. V (Σiso ) = λmax H 1/2 Σiso H 1/2 = λmax

B.1.5

Proof of Proposition 3.5

Let µ := E[Z] so that Σ = E[(Z − µ)(Z − µ)⊤ ]. Define Z ′ := H 1/2 Z and note E[Z ′ ] = H 1/2 µ. Then Cov(Z ′ ) = E (Z ′ − E[Z ′ ])(Z ′ − E[Z ′ ])⊤ = E H 1/2 (Z − µ)(Z − µ)⊤ (H 1/2 )⊤ = H 1/2 E (Z − µ)(Z − µ)⊤ (H 1/2 )⊤ = H 1/2 ΣH 1/2 , where we used that H 1/2 is symmetric for SPD H. For the equivalence: if Σ = αH −1 then Cov(Z ′ ) = H 1/2 (αH −1 )H 1/2 = αI. Conversely, if Cov(Z ′ ) = αI, multiply left and right by H −1/2 to obtain Σ = H −1/2 (αI)H −1/2 = αH −1 . □ B.1.6

Proof of Proposition 3.6

By definition, HH (q, p) = 21 q ⊤ Hq + 12 ∥p∥22 , so

d d 1 exp − q ⊤ Hq exp − ∥p∥22 . ZH 2c 2c Hence the density factorizes into a q-term and a p-term, so q and p are independent. πH (q, p) =

The partition function also factorizes: Z Z d ⊤ ZH = exp − 2c q Hq dq

d exp − 2c ∥p∥22 dp .

Using the standard Gaussian integral Z exp − 12 x⊤ Ax dx = (2π)d/2 det(A)−1/2

(A ≻ 0),

with A = (d/c)H for the q-integral and A = (d/c)I for the p-integral, we obtain c d ZH = 2π det(H)−1/2 . d Therefore c −d/2 c −d/2 d d πH (q, p) = 2π det(H)1/2 exp − q ⊤ Hq · 2π exp − ∥p∥22 . 2c 2c | d {z } | d {z } q-marginal

p-marginal

Thus q is Gaussian with precision matrix (d/c)H, hence covariance (c/d)H −1 , and p is Gaussian with precision (d/c)I, hence covariance (c/d)I: c c q ∼ N 0, H −1 , p ∼ N 0, I . d d Next, Similarly,

c c EπH [q ⊤ Hq] = tr H Cov(q) = tr H H −1 = tr(I) = c. d d c EπH [∥p∥22 ] = tr Cov(p) = tr I = c. d 16

Hence

1 1 1 1 Eπ [q ⊤ Hq] + EπH [∥p∥22 ] = c + c = c. 2 H 2 2 2

EπH [HH (q, p)] =

Finally, the q-marginal is exactly the Gaussian c N 0, H −1 , d which is the minimax-optimal covariance from Theorem 3.2 and the maximum-entropy law from Theorem 3.3. □ B.1.7

Proof of Proposition 3.7

Let Σ ≻ 0 be arbitrary and define H :=

c −1 Σ . d

Since Σ−1 ≻ 0, we have H ≻ 0. Then Σ⋆ (H) =

c −1 c c −1 −1 c d H = Σ = · Σ = Σ. d d d d c

Thus every Σ ≻ 0 arises as Σ⋆ (H) for some H ≻ 0, so the family {Σ⋆ (H) : H ≻ 0} is exactly the SPD cone. □ B.1.8

Proof of Theorem 3.8

First note that ΣM (H) is feasible for the H-budget: tr(HΣM (H)) = tr H

c M tr(HM )

= c.

By the spectral form of the worst-case task variance, VH (Σ) = λmax H 1/2 ΣH 1/2 . Therefore VH (ΣM (H)) = λmax H

1/2

c M H 1/2 tr(HM )

For the oracle covariance Σ⋆ (H) =

c λmax H 1/2 M H 1/2 . tr(HM )

c −1 H , d

we have H 1/2 Σ⋆ (H)H 1/2 = so VH (Σ⋆ (H)) =

c I, d

c . d

Dividing the two expressions gives c

VH (ΣM (H)) tr(HM ) λmax (H = ⋆ VH (Σ (H)) c/d Since

1/2

M H 1/2 ) =

d λmax (H 1/2 M H 1/2 ) . tr(HM )

tr(HM ) = tr H 1/2 M H 1/2 ,

writing A := H 1/2 M H 1/2 ≻ 0 gives VH (ΣM (H)) d λmax (A) = . VH (Σ⋆ (H)) tr(A) 17

For any A ≻ 0, λmax (A) ≤ tr(A) ≤ d λmax (A), and hence 1≤

d λmax (A) ≤ d. tr(A)

The ratio equals 1 if and only if tr(A) = d λmax (A), which holds if and only if all eigenvalues of A are equal, i.e. A = αI for some α > 0. Since A = H 1/2 M H 1/2 , this is equivalent to H 1/2 M H 1/2 = αI, or, multiplying on the left and right by H −1/2 , M = αH −1 . Thus the regret is 1 if and only if M ∝ H −1 . It remains to show that, for every fixed M ≻ 0, the supremum over H ≻ 0 is d. The upper bound d was just shown. To see that it is sharp, fix δ > 0 and define Aδ := diag(1, δ, . . . , δ) ≻ 0. We claim there exists Hδ ≻ 0 such that 1/2

1/2

Hδ M H δ

= Aδ .

Indeed, let 1/2 Bδ := M −1/2 M 1/2 Aδ M 1/2 M −1/2 ≻ 0 and set

Hδ := Bδ2 . 1/2

Then Hδ

= Bδ and 1/2

1/2

Hδ M Hδ

= Bδ M Bδ = Aδ .

For this choice, VHδ (ΣM (Hδ )) d d λmax (Aδ ) = . = ⋆ VHδ (Σ (Hδ )) tr(Aδ ) 1 + (d − 1)δ Letting δ ↓ 0 gives VH (ΣM (H)) = d. ⋆ H≻0 VH (Σ (H)) sup

For d = 1, the same conclusion is immediate since the ratio is identically 1 = d. Thus no geometryindependent fixed covariance shape M is canonical for all structured geometries. B.1.9

Proof of Proposition 3.9

Because (Za , Zb ) is jointly Gaussian, any affine transformation of (Za , Zb ) is also Gaussian. Define the residual R := Zb − CΣ−1 Za . Then Cov(R, Za ) = Cov(Zb , Za ) − CΣ−1 Cov(Za , Za ) = C − CΣ−1 Σ = 0. Since (R, Za ) is jointly Gaussian and uncorrelated, R and Za are independent. Also, because the variables are centered, E[R] = 0. Therefore E[Zb | Za ] = E[CΣ−1 Za + R | Za ] = CΣ−1 Za + E[R | Za ] = CΣ−1 Za . 18

It remains to show that the same marginal covariance Σ admits many different predictive maps. For any t ∈ (−1, 1), consider the block covariance Σ tΣ Γt := . tΣ Σ For any u, v ∈ Rd ,

⊤ u u Γt = u⊤ Σu + 2t u⊤ Σv + v ⊤ Σv. v v

Equivalently, 1−t 1+t (u + v)⊤ Σ(u + v) + (u − v)⊤ Σ(u − v). 2 2 Since Σ ≻ 0 and t ∈ (−1, 1), this quadratic form is strictly positive for every nonzero (u, v). Hence Γt ≻ 0, so it defines a valid centered joint Gaussian law with both marginals equal to Σ and cross-covariance Ct = tΣ. For this joint law, the Bayes predictor is E[Zb | Za ] = Ct Σ−1 Za = tZa . As t varies over (−1, 1), the marginal covariance of both views remains exactly Σ, but the predictive map varies continuously from nearly −Za to nearly Za . Thus even an oracle marginal covariance does not identify the view-to-view coupling. Marginal matching alone therefore cannot determine the JEPA predictive problem. u⊤ Σu + 2t u⊤ Σv + v ⊤ Σv =

B.2

Proofs for Section 4.4

B.2.1

Proof of Lemma A.1

By definition, Z′ = ZH 1/2 , so its ith row is (Z′ )i,: = (Zi,: )H 1/2 = zi⊤ H 1/2 . Taking transpose gives ⊤ 1/2 ⊤ (Z′ )⊤ ) = (H 1/2 )⊤ zi = H 1/2 zi , i,: = (zi H

where we used symmetry of H 1/2 (valid for SPD H). B.2.2

Proof of Theorem A.2

Fix s0 ∈ R2d0 and let s(t) = Φϕ,t (s0 ) solve ṡ = J∇Hϕ (s). Let A(t) := DΦϕ,t (s0 ) ∈ R2d0 ×2d0 denote the Jacobian of the flow with respect to the initial condition. Differentiating the ODE with respect to s0 yields the variational equation Ȧ(t) = J ∇2 Hϕ (s(t)) A(t),

A(0) = I.

(46)

Define M (t) := A(t)⊤ JA(t). Using (46) and the facts J ⊤ = −J and ∇2 Hϕ is symmetric, we compute Ṁ (t) = Ȧ(t)⊤ JA(t)+A(t)⊤ J Ȧ(t) = A(t)⊤ ∇2 Hϕ (s(t))⊤ J ⊤ JA(t)+A(t)⊤ JJ∇2 Hϕ (s(t))A(t). Since ∇2 Hϕ (s(t))⊤ = ∇2 Hϕ (s(t)) and J ⊤ = −J and JJ = −I, this becomes

Ṁ (t) = A(t)⊤ ∇2 Hϕ (s(t)) (−J)JA(t)+A(t)⊤ (−I)∇2 Hϕ (s(t))A(t) = A(t)⊤ ∇2 Hϕ (s(t)) A(t)−A(t)⊤ ∇2 Hϕ (s(t)) A(t) = Thus M (t) is constant in t. At t = 0, A(0) = I so M (0) = J, hence M (t) = J for all t, proving (40). For the determinant claim, take determinants of A(t)⊤ JA(t) = J: det(A(t))2 det(J) = det(J) ⇒ det(A(t))2 = 1. Since A(0) = I has det(A(0)) = 1 and det(A(t)) varies continuously with t, we must have det(A(t)) = 1 for all t. Finally, volume preservation follows from det(DΦϕ,t (s)) ≡ 1 by the change-of-variables theorem: for any measurable set U , Z Z Vol(Φϕ,t (U )) = | det(DΦϕ,t (s))| ds = 1 ds = Vol(U ). U

B.2.3

Proof of Corollary A.3

Proof of Theorem A.4

Write the leapfrog step as a composition of two “kicks” and one “drift”. Define for h ∈ R: Kh (q, p) := (q, p − h∇Vϕ (q)),

Dh (q, p) := (q + hp, p),

where the drift uses ∇T (p) = p for T (p) = 21 ∥p∥2 . Then the one-step leapfrog map satisfies Ψϕ,∆t = K∆t/2 ◦ D∆t ◦ K∆t/2 . We show that both Kh and Dh are symplectic. Their Jacobians have block forms I 0 I 0 I ⊤ DKh (q, p) = = , B = B, DD (q, p) = h B I 0 −h∇2 Vϕ (q) I

hI I

I 0

A direct calculation shows that any matrix of the form ( BI I0 ) with B symmetric satisfies A⊤ JA = J, and similarly for ( I0 CI ) with C symmetric. Therefore DKh and DDh are symplectic matrices at every point, so Kh and Dh are symplectic maps. The composition of symplectic maps is symplectic, hence Ψϕ,∆t is symplectic, and iterating yields (K) K symplecticity of Φϕ = Ψϕ,∆t . The determinant claim follows as in the proof of Theorem A.2. B.2.5

Proof of Corollary A.5

Using the decomposition Ψ∆t = K∆t/2 ◦ D∆t ◦ K∆t/2 , we have Kh−1 = K−h and Dh−1 = D−h by inspection. Hence −1 −1 −1 Ψ−1 ∆t = K∆t/2 ◦ D∆t ◦ K∆t/2 = K−∆t/2 ◦ D−∆t ◦ K−∆t/2 = Ψ−∆t . K −1 K Taking K-fold compositions gives (Ψ∆t ) = Ψ−∆t .

B.2.6

Proof of Proposition A.6

Write the phase-space state as s = (q, p) ∈ R2d0 and define the two vector fields p p f (s) := , fϕ (s) := . −∇V (q) −∇Vϕ (q) V

Let ΦVt denote the flow of ṡ = f (s) and Φt ϕ the flow of ṡ = fϕ (s). Throughout, work on the compact set Ω = Ωq × Ωp from the proposition statement. Since ∇V and ∇Vϕ are Lipschitz on Ωq and Ωp is compact, both f and fϕ are Lipschitz on Ω; let L be any Lipschitz constant of fϕ on Ω: ∥fϕ (x) − fϕ (y)∥2 ≤ L∥x − y∥2 ,

∀x, y ∈ Ω.

Define the uniform force mismatch on Ωq : δ := sup ∥∇Vϕ (q) − ∇V (q)∥2 . q∈Ωq

C , I

C ⊤ = C.

Step 1: Continuous dependence of flows on the force field. Fix s0 ∈ K and let s(t) := ΦVt (s0 ) V and s̃(t) := Φt ϕ (s0 ). By assumption, both trajectories remain in Ω for t ∈ [0, T ]. Let ∆(t) := s̃(t) − s(t). Then ˙ ∆(t) = fϕ (s̃(t)) − f (s(t)) = fϕ (s̃(t)) − fϕ (s(t)) + fϕ (s(t)) − f (s(t)) . Taking norms and using Lipschitzness of fϕ on Ω plus the definition of δ gives ˙ ∥∆(t)∥ 2 ≤ L∥∆(t)∥2 + δ. Since ∆(0) = 0, Grönwall’s inequality yields, for all t ∈ [0, T ],   δ eLt − 1, L > 0, ∥∆(t)∥2 ≤ L  δt, L = 0. In particular, at t = T we obtain a uniform (over s0 ∈ K) modeling-error bound ( LT e −1 , L > 0, Vϕ L sup ∥ΦT (s0 ) − ΦVT (s0 )∥2 ≤ C1 δ, C1 := T, L = 0. s0 ∈K Step 2: Leapfrog global error is O(∆t2 ). Fix ϕ and consider the ODE ṡ = fϕ (s). Let Ψϕ,h denote one leapfrog step with step size h (Eq. (29)), and define the numerical iterates sn+1 := Ψϕ,h (sn ),

s0 ∈ K,

tn := nh,

(K)

so that sK = (Ψϕ,h )K (s0 ) = Φϕ (s0 ) with Kh = T . (2a) Local truncation error bound. Because Vϕ ∈ C 3 on a neighborhood of Ωq , the vector fields p 0 A(q, p) := , B(q, p) := 0 −∇Vϕ (q) are C 2 on Ω, and their exact flows are exp(hA) : (q, p) 7→ (q + hp, p),

exp(hB) : (q, p) 7→ (q, p − h∇Vϕ (q)).

The leapfrog step is exactly the Strang splitting: h h Ψϕ,h = exp B exp(hA) exp B . 2 2 Standard Strang splitting theory (derivable via the BCH expansion for the Lie operators LA , LB ) implies that the local truncation error is third order: there exists a constant Cloc = Cloc (Ω, Vϕ ) such that V sup Ψϕ,h (x) − Φhϕ (x) 2 ≤ Cloc h3 for all sufficiently small h. x∈Ω

(Concretely, the BCH expansion gives exp( h2 LB ) exp(hLA ) exp( h2 LB ) = exp h(LA + LB ) + 3 O(h ) as operators on smooth test functions, and boundedness of the commutators on Ω yields the uniform Cloc .) (2b) Error recursion and discrete Grönwall. Define the global error at step n by V

en := sn − Φtnϕ (s0 ). Then V

ϕ en+1 = Ψϕ,h (sn ) − Φtn+1 (s0 )

V V = Ψϕ,h (sn ) − Φhϕ Φtnϕ (s0 ) . V

Add and subtract Φhϕ (sn ): V V V V en+1 = Ψϕ,h (sn ) − Φhϕ (sn ) + Φhϕ (sn ) − Φhϕ (Φtnϕ (s0 )) . | {z } | {z } local truncation

stability

Taking norms, using the local error bound from (2a), and using Lipschitz stability of the flow V V ∥Φhϕ (x) − Φhϕ (y)∥2 ≤ eLh ∥x − y∥2 on Ω yields ∥en+1 ∥2 ≤ Cloc h3 + eLh ∥en ∥2 . Iterating this recursion with e0 = 0 gives  LKh −1 e , L > 0, Lhj 3 3 Lh ∥eK ∥2 ≤ Cloc h e = Cloc h · e −1  j=0 K, L = 0. K−1 X

Since Kh = T and eLh − 1 ≥ Lh for Lh ≥ 0, we obtain   Cloc eLT − 1 h2 , L ∥eK ∥2 ≤  Cloc T h2 ,

L > 0, L = 0.

Taking the supremum over s0 ∈ K (all iterates stay in Ω by assumption), we conclude that there exists C2 = C2 (T, Ω, Vϕ ) such that (K)

sup Φϕ (s0 ) − ΦTϕ (s0 ) 2 ≤ C2 h2 .

s0 ∈K

Step 3: Combine modeling and integrator errors. (K)

By the triangle inequality, V

(K)

∥Φϕ (s0 ) − ΦVT (s0 )∥2 ≤ ∥Φϕ (s0 ) − ΦTϕ (s0 )∥2 + ∥ΦTϕ (s0 ) − ΦVT (s0 )∥2 . Taking suprema over s0 ∈ K and applying the bounds from Steps 1–2 yields (K)

sup ∥Φϕ (s0 ) − ΦVT (s0 )∥2 ≤ C2 h2 + C1 δ.

s0 ∈K

This is the claimed inequality with C := max{C1 , C2 } (or C := C1 + C2 ). Finally, if the model class can make δ arbitrarily small (uniform approximation of ∇V on Ωq ), choose ϕ so that C1 δ ≤ ε/2, then choose h = ∆t so that C2 h2 ≤ ε/2, and conclude (K)

sup ∥Φϕ (s0 ) − ΦVT (s0 )∥2 ≤ ε.

s0 ∈K

□ B.2.7

Proof of Lemma A.7

Let A be symplectic: A⊤ JA = J. Using J −1 = −J and J ⊤ = −J, we obtain A−1 = −JA⊤ J. Then

(A⊤ A)−1 = A−1 A−T = (−JA⊤ J)(−JAJ) = J −1 (A⊤ A)J,

so (A⊤ A)−1 is similar to A⊤ A and thus has the same eigenvalues. Therefore the eigenvalues of √ A⊤ A occur in reciprocal pairs (µ, 1/µ), and the singular values σ = µ occur in reciprocal pairs (σ, 1/σ). B.2.8

Proof sketch of Theorem A.8

This is a standard result from backward error analysis for symplectic integrators; we provide the BCH-based sketch specialized to leapfrog. Let LT and LV denote the Lie derivatives (Poisson bracket operators) corresponding to the Hamiltonians T and V , acting on smooth observables. The exact time-h flow of T (resp. V ) is exp(hLT ) (resp. exp(hLV )). The leapfrog step can be written operator-theoretically as the symmetric composition exp h2 LV exp(hLT ) exp h2 LV . 22

By the Baker–Campbell–Hausdorff (BCH) formula, this equals exp h(LT + LV ) + h3 C3 + h5 C5 + · · · , where C3 , C5 , . . . are nested commutators of LT and LV . For Hamiltonian systems, commutators of Hamiltonian vector fields correspond to Hamiltonian vector fields of Poisson brackets; equivalently, each C2j+1 is itself a Lie derivative LH2j for some scalar function H2j built from iterated Poisson brackets of T and V . Thus the exponential above is the exact time-h flow of a modified Hamiltonian e∆t = H + h2 H2 + h4 H4 + · · · H e [N ] at the level of formal series. Truncating this series at order N yields a modified Hamiltonian H ∆t whose flow matches one leapfrog step up to local error O(hN +1 ). Under analyticity, classical results show that the truncation can be chosen so that the resulting modified energy is nearly conserved over exponentially long times in 1/h. (See, e.g., standard references on geometric numerical integration for full details.) B.2.9

Proof of Lemma A.9

Take Σ = diag(c, 0, . . . , 0). Then Σ ⪰ 0, tr(Σ) = c, and rank(Σ) = 1. B.2.10

Proof of Lemma A.10

Fix k ≥ 2 and τ . For ε > 0, define eigenvalues λ2 = · · · = λk = ε,

λ1 = exp(kτ ) ε−(k−1) .

Qk Then i=1 λi = exp(kτ ), hence k1 log det(Σ) = τ . But as ε → 0, λ1 → ∞ and λ2 , . . . , λk → 0. The participation ratio satisfies P ( λi )2 λ2 PR(Σ) = Pi 2 −→ 12 = 1, λ1 i λi so PR(Σ) can be made arbitrarily close to 1 while maintaining the log-det constraint. B.2.11

Proof of Lemma A.11

Let λi be the eigenvalues of Σ. Then the eigenvalues of aΣ are aλi , so P P ( i aλi )2 a2 ( i λi )2 PR(aΣ) = P = 2 P 2 = PR(Σ). 2 a i (aλi ) i λi B.2.12

Proof of Lemma A.12 1/2

Let ΣQ ⪰ 0 and R ∈ Rd0 ×k have full column rank. Write ΣQ = A⊤ A with A := ΣQ (PSD square root). Then ΣY,0 = R⊤ ΣQ R = R⊤ A⊤ AR = (AR)⊤ (AR). Therefore ΣY,0 is PSD and rank(ΣY,0 ) = rank (AR)⊤ (AR) = rank(AR) ≤ rank(A) = rank(ΣQ ). If rank(ΣQ ) < k, then rank(ΣY,0 ) < k, so ΣY,0 is singular. Hence det(ΣY,0 ) = 0 and log det(ΣY,0 ) = −∞. For the ridge statement, let r := rank(ΣY,0 ) and let λ1 , . . . , λr > 0 be its nonzero eigenvalues. Then ΣY,0 has exactly k − r zero eigenvalues. Adding εIk shifts each eigenvalue by ε, so ΣY = ΣY,0 + εIk has eigenvalues λ1 + ε, . . . , λr + ε, ε, . . . , ε . | {z } k−r times

Thus log det(ΣY ) =

r X

log(λi + ε) + (k − r) log ε,

i=1

□

which is exactly the claimed formula. 23

B.2.13

Proof of Proposition A.13

Let λ1 ≥ · · · ≥ λk > 0 be eigenvalues of ΣY . From log det(ΣY ) ≥ kτ we have k Y

λi ≥ exp(kτ ).

i=1

Let λmin = λk . Using

Pk−1

i=1 λi ≤ tr(ΣY ) − λk ≤ m, the AM–GM inequality yields k−1 Y

λi ≤

Pk−1 λ k−1 i=1

k−1

i=1

Therefore

k Y

exp(kτ ) ≤

λi = λk

i=1

k−1 Y

≤

λi ≤ λk

i=1

so λmin (ΣY ) = λk ≥ exp(kτ ) B.2.14

m k−1 . k−1 m k−1 , k−1

(k − 1)k−1 . mk−1

Proof of Theorem A.14

P Let λ1 ≥ · · · ≥ λk > 0 be eigenvalues of Σ and c = i λi . The PR constraint implies P k X ( i λi )2 c2 PR(Σ) = P 2 ≥ r0 ⇒ λ2i ≤ . r0 i λi i=1 P Since λ2max = λ21 ≤ i λ2i , we get c λmax (Σ) ≤ √ . r0 Next, the log-det constraint implies k Y

λi = det(Σ) ≥ exp(kτ ).

i=1

Using λi ≤ λmax for i = 1, . . . , k − 1 gives exp(kτ ) ≤ λmin λk−1 max , B.2.15

exp(kτ ) ≥ exp(kτ ) λmin ≥ λk−1 max

√

r0 c

k−1 .

Proof of Proposition A.16 (K)

(K)

By Theorem A.4, Φϕ is a C 1 diffeomorphism with | det(DΦϕ (s))| = 1 for all s. For any measurable U with finite volume, the change-of-variables theorem gives Z Z (K) (K) Vol Φϕ (U) = det(DΦϕ (s)) ds = 1 ds = Vol(U). U

(K) Hence Vol(U) > 0 implies Vol(Φϕ (U)) > 0.

Therefore the predictor cannot map a positivevolume set to a zero-volume set; any collapse to a lower-volume region must already be present in the encoder outputs.

Supplementary theory: maximum entropy on phase space and symplectic factorization

Theorem C.1 (Maximum entropy on phase space under an expected energy constraint). Let H : Rm → R be measurable and assume there exists β > 0 such that the partition function Z Z(β) := e−βH(s) ds Rm

is finite. Fix c ∈ R and consider the set of densities Z Z Pc := p density on Rm : p(s) ds = 1, H(s)p(s) ds = c, h(p) > −∞ , where h(p) := − for

p log p is differential entropy. If there exists β ⋆ > 0 such that Epβ⋆ [H(S)] = c

1 −βH(s) e , Z(β) then the unique maximizer of h(p) over Pc is p⋆ = pβ ⋆ . pβ (s) =

Proof. Fix β > 0 with Z(β) < ∞ and define pβ (s) = Z(β)−1 e−βH(s) . For any p ∈ Pc , Z p(s) ds ≥ 0, KL(p∥pβ ) = p(s) log pβ (s) with equality iff p = pβ a.e. Expanding log pβ (s) = −βH(s) − log Z(β) gives Z Z KL(p∥pβ ) = p log p ds − p log pβ ds Z = −h(p) − p(s) − βH(s) − log Z(β) ds Z Z = −h(p) + β H(s)p(s) ds + log Z(β) p(s) ds | {z } | {z } =c

= −h(p) + βc + log Z(β). Thus h(p) ≤ βc + log Z(β) for all p ∈ Pc , with equality iff p = pβ . If there exists β ⋆ such that pβ ⋆ ∈ Pc (i.e., it matches the energy constraint), then pβ ⋆ attains the upper bound and is therefore a maximizer. Uniqueness follows from the equality condition for KL divergence. Corollary C.2 (Energy-based densities are invariant under Hamiltonian flow). Let H ∈ C 2 (R2d0 ) and let Φt be the Hamiltonian flow of ṡ = J∇H(s), assumed to be a C 1 diffeomorphism for the times of interest. If S has density p(s) = g(H(s)) for some measurable g : R → R+ , then for all t, d

Φt (S) = S. In particular, the Gibbs density p(s) ∝ e−βH(s) is invariant. Proof. Energy is conserved along Hamiltonian trajectories: if s(t) = Φt (s0 ), then d H(s(t)) = ∇H(s(t))⊤ ṡ(t) = ∇H(s(t))⊤ J∇H(s(t)) = 0, dt since J ⊤ = −J implies u⊤ Ju = 0 for all u. Thus H(Φt (s)) = H(s). Also Hamiltonian flows are volume-preserving (Liouville): | det(DΦt (s))| = 1 for all s, t. Let Y = Φt (S). By change of variables, pY (y) = p(Φ−t (y)) · det DΦ−t (y) = g(H(Φ−t (y))) · 1 = g(H(y)) = p(y). Hence Y and S have the same density. Proposition C.3 (Generic kick–drift–scaling factorization of linear symplectic maps). Let A = a b ∈ Sp(2d ) and assume the block d ∈ Rd0 ×d0 is invertible. Define 0 c d B := bd−1 ,

C := d−1 c.

Then B and C are symmetric and

B A= I | {z } I 0

I 0 d−⊤ 0 . C I 0 d | {z } | {z }

upper shear (drift) symplectic scaling lower shear (kick)

Proof. Since A ∈ Sp(2d0 ), it satisfies A⊤ JA = J, which implies the standard block identities b⊤ d = d⊤ b and a⊤ d − c⊤ b = I. From b⊤ d = d⊤ b and invertibility of d, B ⊤ = (bd−1 )⊤ = d−⊤ b⊤ = bd−1 = B, so B is symmetric. Similarly, AJA⊤ = J also holds for symplectic A, and its block identities include cd⊤ = dc⊤ . Then C ⊤ = (d−1 c)⊤ = c⊤ d−⊤ = d−1 c = C, so C is symmetric. Now multiply the three proposed factors: I B I d−⊤ 0 0 I C 0 d

0 I

d−⊤ + BdC dC

Bd . d

The bottom-right block matches d. The top-right block matches b because Bd = (bd−1 )d = b. The bottom-left block matches c because dC = d(d−1 c) = c. For the top-left block, use the symplectic identity ad⊤ − bc⊤ = I (from AJA⊤ = J) and rightmultiply by d−⊤ : a − bc⊤ d−⊤ = d−⊤ . Using c⊤ d−⊤ = (d−1 c)⊤ = C ⊤ = C, this becomes a = d−⊤ + bC = d−⊤ + (Bd)C = d−⊤ + BdC, which matches the computed top-left block. Hence the factorization holds.

Further results and figures

We refer to the method uniformly as HamJEPA; in the code/config files, the suffix hjepa_mv simply denotes the minimal multi-view implementation used in our experiments. D.1

Frozen-feature diagnostic summaries

Figures 1 (CIFAR-100) and 3 (ImageNet-100) summarize the geometry and non-parametric usefulness of frozen encoder representations for the SIGReg baseline and HamJEPA latent states. Each method panel contains four diagnostics computed on raw encoder features (i.e., no whitening or other post-processing): 1. kNN sweep: top-1 accuracy as a function of neighborhood size k. 2. Covariance eigenspectrum: top-256 eigenvalues (log-scale) of the mean-centered empirical covariance. 3. Random-pair cosine similarities: histogram of cosine similarities after per-vector ℓ2 normalization without mean subtraction. 4. Feature norms: histogram of ℓ2 feature norms. A key subtlety is that the covariance spectrum is computed after mean-centering, whereas the cosine histogram is computed on normalized features without centering. Consequently, strongly positive random-pair cosines can arise from a dominant mean direction even when the mean-centered covariance remains high-rank. We exploit this complementarity in the discussions below. D.2

Discussion CIFAR-100

D.2.1

30 epoch results

These results are for the 30 epoch pretraining. 26

(1) Neighborhood quality: kNN sweeps. In Fig. 1a, the SIGReg baseline peaks at small k and degrades as k grows, consistent with semantics being locally present but becoming less class-pure as neighborhoods expand. Quantitatively, SIGReg peaks at 27.46 ± 0.31% (at k=1) and falls to 23.10 ± 0.32% (at k=200). HamJEPA improves the entire sweep for all latent variants in Fig. 1. Notably, the single q state (1024-D) outperforms the SIGReg baseline (2048-D) across the plotted k values: 31.45 ± 0.27% at k=20 for q, versus 26.56 ± 0.18% for SIGReg at k=20 (a +4.88 ± 0.11 point gain). The p and (q, p) variants achieve 29.71 ± 0.25% and 30.88 ± 0.39% at k=20, respectively. That (q, p) does not exceed q suggests that, under the default weighting, p contributes less additional metric-consistent neighborhood structure than q, even though it remains informative for linear decoding (see below). (2) Linear probe vs. kNN: separability without neighborhood purity. While kNN distinguishes the latent variants, the linear probe largely does not: SIGReg reaches 30.43 ± 0.30% top-1, whereas HamJEPA reaches 33.95 ± 0.24%, 33.07 ± 0.36% and 34.18 ± 0.17% top-1 for q, p, and (q, p) respectively. A useful interpretation is that p is linearly discriminative but induces less clean neighborhoods under the raw metric, so its gains show up more clearly in the probe than in kNN. (3) Covariance eigenspectrum: rank proxies and collapse checks. The top-right panels show mean-centered covariance spectra (top 256 eigenvalues, log-scale). A convenient summary is the effective-rank proxy computed from these eigenvalues: SIGReg has effective rank ≈ 109.73 ± 0.61 (top-256), whereas HamJEPA-q increases to 125.58 ± 3.33. In contrast, HamJEPA-p is more concentrated (effective rank 79.46 ± 3.99), and (q, p) lies in between (93.10 ± 4.00). Two points follow: (i) HamJEPA-q does not achieve its downstream gain via low-rank collapse (the centered covariance is high-rank), and (ii) the q/p split is geometrically meaningful: q and p allocate variance differently even when their linear probe scores are similar. (4) Cosine similarities: diagnosing mean-dominance. The bottom-left panels reveal the largest qualitative difference. SIGReg’s random-pair cosines are centered near zero (mean 0.01699±0.00089) with substantial negative mass, which is consistent with its isotropy-encouraging design. HamJEPA’s cosines shift sharply positive: mean cosine is 0.71561±0.00129 for q, 0.61687±0.00409 for p, and 0.66613 ± 0.00270 for (q, p). Crucially, this strong alignment coexists with a high-rank mean-centered covariance (especially for q). This combination is most naturally explained by a large non-zero mean feature direction that dominates dot products when features are not explicitly centered: the covariance spectrum removes that mean direction by construction, while the cosine histogram does not. In other words, the CIFAR-100 HamJEPA embeddings look “cone-like” in raw normalized space primarily due to mean-dominance rather than variance collapse. (5) Feature norms: scale control and the (q, p) energy split. SIGReg feature norms are concentrated around 44.891 ± 0.090, whereas HamJEPA’s q and p have smaller typical norms around 31.358 ± 0.017 and 31.281 ± 0.029. The concatenated (q, p) pnorm shifts back to 44.303 ± 0.027, closely matching the Pythagorean combination ∥[q; p]∥2 ≈ ∥q∥22 + ∥p∥22 , which is what one expects if q and p maintain comparable energy rather than one collapsing to near-zero. This provides a simple consistency check that the phase-space parameterization retains a balanced contribution from both coordinates. D.2.2

80 epoch results

By design, the CIFAR-100 pretraining curves tend to plateau because the experiment is set up as a minimal, controlled training regime, whose goal is to isolate representational geometry effects rather than to chase absolute accuracy with an aggressively optimized recipe. CIFAR-100 is small and low-resolution, so the encoder typically learns the dominant semantic invariances early; after that point, additional epochs mainly yield diminishing returns (small refinements in neighborhood structure or feature scaling) that often do not translate into large gains in linear-probe/(k)NN top-1 under a fixed evaluation protocol. In other words, once both methods reach a “good-enough” feature space, the remaining headroom is mostly governed by recipe choices we intentionally avoid here (stronger augmentation, longer schedules, larger models, distillation/EMA, more hyperparameter search), since those would confound the attribution: we want any observed differences to come primarily from the geometry induced by the training dynamics, not from extra optimization budget. 27

Method

Variant

Linear probe (%)

kNN@20 (%)

kNN best (%)

SIGReg

single

33.95

27.98

28.71 @ k=1

MLP-HJEPA (non-symplectic) MLP-HJEPA (non-symplectic) MLP-HJEPA (non-symplectic)

q p (q, p)

42.26 41.90 42.77

28.10 28.30 28.15

28.10 @ k=20 28.30 @ k=20 28.15 @ k=20

HamJEPA (symplectic) HamJEPA (symplectic) HamJEPA (symplectic)

q p (q, p)

44.59 44.44 44.52

34.43 32.96 33.66

34.43 @ k=20 32.96 @ k=20 33.66 @ k=20

Table 3: CIFAR-100, 80-epoch predictor-family ablation. MLP-HJEPA keeps the phase-space encoder, channel-wise q/p split, identity projector, prediction loss, anti-collapse regularizers, optimizer, batch size, and 80-epoch schedule fixed, and replaces only the symplectic leapfrog predictor by a non-symplectic residual MLP predictor.

Predictor-family ablation: what changes and what stays fixed. In addition to SIGReg and symplectic HamJEPA, we evaluate a non-symplectic predictor ablation, MLP-HJEPA, that keeps the phase-space encoder, channel-wise q/p split, identity projector, prediction loss, anti-collapse regularizers, optimizer, batch size, and training schedule fixed, and replaces only the leapfrog Hamiltonian rollout by a generic residual MLP rollout predictor. Because Fig. 2 visualizes only SIGReg and symplectic HamJEPA, we summarize the MLP ablation numerically in Table 3. This is therefore a clean apples-to-apples test of whether the gains come mainly from the phase-space representation and encoder-side regularization, or from the symplectic predictor itself. (1) Neighborhood quality: kNN sweeps and local semantic structure. At 80 epochs, SIGReg reaches 27.98% at kNN@20 and peaks at 28.71% over the full sweep (at k=1). It peaks at small k and degrades as k grows, consistent with semantics being locally present but becoming less class-pure as neighborhoods expand. The non-symplectic MLP-HJEPA ablation barely changes this picture: its best kNN@20 is only 28.30% (from the p readout), a gain of just +0.32 points over SIGReg, and its best-over-k score is in fact slightly below SIGReg (28.30% vs. 28.71%). By contrast, symplectic HamJEPA is qualitatively different: the q readout reaches 34.43% at kNN@20 and also attains the best-over-k value of 34.43% (at k=20), namely gains of +6.45 points at kNN@20 and +5.72 points best-over-k relative to SIGReg. Notably, the single q state (1024-D) outperforms the SIGReg baseline (2048-D) across the plotted k values. Best-vs.-best, the symplectic predictor outperforms the MLP predictor by +6.13 points at kNN@20. This is the sharpest evidence that the symplectic rollout is doing something specifically geometric: the phase-space encoder alone does not automatically produce high-purity local neighborhoods. (2) Linear probe: phase-space structure already helps, but symplecticity still matters. The linear-probe comparison is more nuanced. MLP-HJEPA already substantially outperforms SIGReg, reaching 42.77% top-1 in its best variant ((q, p)) versus 33.95% for SIGReg, a gain of +8.82 points. Symplectic HamJEPA pushes this further to 44.59% (best q readout), i.e. another +1.82 points over MLP-HJEPA and +10.64 over SIGReg. This suggests that a large part of the linear-separability gain comes from the phase-space parameterization plus encoder-side anti-collapse bridge, not only from symplecticity. However, once nonparametric neighborhood quality is examined, the predictor family becomes decisive: the MLP predictor narrows much of the probe gap, but essentially none of the kNN gap. (3) q/p decomposition: structured phase space versus generic feature blocks. Within MLPHJEPA, the three readouts are almost indistinguishable: linear probe is 42.26%/41.90%/42.77% for q/p/(q, p), and kNN@20 is 28.10%/28.30%/28.15%. In other words, once the symplectic predictor is removed, the q/p split behaves much more like a generic two-block partition of the feature vector than like a structured position-momentum state space. This contrasts with symplectic HamJEPA, where q is the cleanest neighborhood readout, p is slightly weaker, and (q, p) remains close but does not dominate. The natural interpretation is that the Hamiltonian rollout induces a genuine functional differentiation between the two coordinates, whereas the MLP rollout does not. 28

(4) Mean-centered covariance geometry: high-rank symplectic q versus concentrated MLP states. The scalar covariance diagnostics sharpen this distinction. SIGReg has effective rank ≈ 99.80 and participation ratio ≈ 58.32 on the top-256 eigenspectrum. Symplectic HamJEPA-q is substantially broader still, with effective rank ≈ 128.98 and participation ratio ≈ 70.34, confirming that its downstream gains are not coming from collapse onto a narrow subspace. By contrast, MLPHJEPA is much more spectrally concentrated: its effective rank is only ≈ 42.06 for q, ≈ 43.95 for p, and ≈ 42.85 for (q, p), with participation ratios only ≈ 23.65–24.99. So although the MLP ablation yields good linear-probe accuracy, it does so with a much more concentrated mean-centered covariance geometry than symplectic HamJEPA. Put differently: the MLP predictor can produce a linearly usable representation, but it does not organize a broad, high-rank content geometry in the way the symplectic predictor does. (5) Cosine similarities and feature norms: the predictor changes geometry, not just scale. The random-pair cosine statistics again place MLP-HJEPA much closer to SIGReg than to symplectic HamJEPA. SIGReg has cosine mean ≈ 0.006 with standard deviation ≈ 0.121, while MLP-HJEPA remains near zero as well (cosine means ≈ 0.042 for q, ≈ 0.053 for p, and ≈ 0.047 for (q, p), all with standard deviation around 0.19). Symplectic HamJEPA, in contrast, exhibits strongly positive cosine means (≈ 0.625 for q, ≈ 0.443 for p, and ≈ 0.534 for (q, p)), reflecting the markedly more aligned raw geometry visible in Fig. 2. Importantly, this difference is not explained by trivial scale effects: the feature norms of MLP-HJEPA are nearly the same as those of symplectic HamJEPA (means ≈ 32.21, 31.89, and 45.32 for MLP-HJEPA versus ≈ 31.81, 31.88, and 45.06 for symplectic HamJEPA). Thus the predictor family is primarily changing how variance is organized and aligned, not simply how large the features are. Takeaway (CIFAR-100, 80 epochs). The MLP ablation is informative in exactly the right way. It shows that the phase-space encoder, q/p split, and non-isotropic anti-collapse regularization already explain a large fraction of the linear-probe improvement over SIGReg. However, they do not explain the major improvement in nonparametric neighborhood quality. That gain appears to be specifically tied to the symplectic predictor: relative to the non-symplectic MLP rollout, symplectic HamJEPA produces much better kNN structure, a broader and less concentrated mean-centered covariance for the content coordinate, and a more distinctive raw embedding geometry. In this sense, the symplectic component is not merely a mild architectural preference; it is the ingredient that turns a linearly useful phase-space representation into one with substantially cleaner local semantic geometry. D.3

Discussion ImageNet-100

(1) Frozen-feature quality: kNN and probe. On ImageNet-100, HamJEPA again improves nonparametric and linear evaluation. In Fig. 3, the dimension-matched comparison (2048-D) is SIGReg (q, p) versus HamJEPA (q, p). At the final checkpoint, SIGReg (q, p) reaches a best kNN top-1 of 21.7% (peak near k=50), whereas HamJEPA (q, p) reaches 25.9% (a +4.2 point gain). For linear probing, SIGReg (q, p) reaches 25.5% top-1, whereas HamJEPA (q, p) reaches 32.0% (a +6.5 point gain). Figure 5 additionally provides a strict 1024-D comparison on q: at epoch 45, HamJEPA-q improves linear probe by +7.52 points and best kNN by +4.90 points over SIGReg-q (as annotated). (2) Covariance rank: HamJEPA spreads variance across many directions. The covariance spectra (top-right) and the effective-rank proxy show a stark contrast: SIGReg-q and SIGReg-p have low effective rank (≈ 38–39 on the top-256 eigenvalues), whereas HamJEPA-q and HamJEPA-p are much higher (≈ 95–98). Thus, on ImageNet-100, HamJEPA’s gains are associated with more variance being spread across principal directions (higher rank), not less. (3) Cosine similarities: mild global alignment without extreme cone structure. Unlike CIFAR100, ImageNet-100 does not show an extreme cone effect for HamJEPA. SIGReg’s random-pair cosines remain centered near zero (mean ≈ 0), while HamJEPA shifts to moderately positive cosines (mean ≈ 0.27–0.28 depending on q/p/(q, p)). Given the simultaneously high effective rank of the mean-centered covariance, this again suggests a shared direction and/or non-zero mean, but the effect is substantially weaker than on CIFAR-100. 29

(4) q/p decomposition: complementarity for SIGReg, redundancy for HamJEPA. Figure 4 makes the structural difference explicit. For SIGReg+tokens, (q, p) is consistently best throughout training, indicating that q and p halves are not individually sufficient but become competitive when concatenated. For HamJEPA, q and p are individually as strong as (q, p) (both for probe and kNN), suggesting that both coordinates carry similar semantic content and that concatenation adds little. (5) Dynamics: accuracy tracks geometry for HamJEPA, not for SIGReg. In Fig. 5, HamJEPA’s effective rank increases over training while probe accuracy improves, yielding a strong positive association in the probe-versus-rank plot (annotated ρ). SIGReg shows the opposite: effective rank decreases while accuracy rises slightly, yielding a negative association. This supports the interpretation that HamJEPA’s ImageNet gains co-occur with a progressively less concentrated (mean-centered) covariance structure. Takeaway (ImageNet-100). On ImageNet-100, HamJEPA improves kNN and linear probing while producing substantially higher-rank mean-centered covariance than SIGReg and only moderately more positive cosine alignment. In contrast to SIGReg+tokens, HamJEPA makes both q and p individually strong, with little benefit from concatenation.

(a) SIGReg (raw).

(b) HamJEPA q (raw).

(d) HamJEPA (q, p) (raw).

Figure 1: Frozen-feature diagnostic summaries (raw) 30 epochs. Each panel contains: (top-left) kNN sweep; (top-right) covariance eigenspectrum (top 256, log scale); (bottom-left) cosine similarity histogram (random pairs); (bottom-right) feature norm histogram.

(a) SIGReg (raw).

(b) HamJEPA q (raw).

(d) HamJEPA (q, p) (raw).

Figure 2: CIFAR-100 frozen-feature diagnostic summaries (raw) 80 epochs. Each panel contains: (top-left) kNN sweep; (top-right) mean-centered covariance eigenspectrum (top 256, log scale); (bottom-left) random-pair cosine similarity histogram; (bottom-right) feature norm histogram.

(a) SIGReg (raw).

(b) HamJEPA q (raw).

(d) HamJEPA (q, p) (raw).

Figure 3: ImageNet-100 frozen-feature diagnostic summaries (raw). Each panel contains: (topleft) kNN sweep; (top-right) covariance eigenspectrum (top 256, log scale); (bottom-left) cosine similarity histogram (random pairs); (bottom-right) feature norm histogram.

SIGReg+tokens: q/p/qp decomposition

(a)

kNN best over k

24.0 23.5 23.0

kNN best over k

29 28 27

22.5

22.0

Top-1 (%)

24.5

(b)

Linear probe top-1

25.0

Top-1 (%)

MV-HJEPA: q/p/qp decomposition

(b)

Linear probe top-1 q

Top-1 (%)

(a) 25.5

21 25

Pretrain epoch

(a) SIGReg (tokens).

Pretrain epoch

(b) HamJEPA.

Figure 4: q/p decomposition across pretraining (ImageNet-100). We evaluate frozen features extracted at sparse pretraining checkpoints using (left) a linear probe top-1 and (right) the best kNN top-1 across the evaluated k values. For SIGReg+tokens, concatenating (q, p) is consistently better than either half alone, indicating that the split halves are complementary (and/or that doubling feature dimension helps). For HamJEPA, q, p, and (q, p) track each other closely, suggesting that both phase-space coordinates are individually informative and largely redundant under the symplectic objective. (Left) SIGReg tokens. (Right) HamJEPA.

ImageNet: training dynamics from sparse checkpoints (a)

(b)

Linear probe top-1

Delta@45: +7.52

Delta@45: +4.90 24

Top-1 (%)

kNN best over k

22 20 18

24 16

MV-HJEPA (q) SIGReg+tokens (q)

22 10

Pretrain epoch (c)

Pretrain epoch (d)

Effective rank (top-q)

Probe vs geometry 45

32 MV-HJEPA (q): rho=0.90 30

Probe top-1 (%)

Effective rank (top-q)

70 60 50

SIGReg+tokens (q): rho=-0.90

10 45

22 10

Pretrain epoch

Effective rank (top-q)

Figure 5: ImageNet-100 training dynamics from sparse checkpoints (geometry vs. accuracy). Using the q representation at a set of pretraining epochs, we plot: (a) linear probe top-1; (b) best kNN top-1 over k (shaded region indicates the spread over the k sweep); (c) effective-rank proxy computed from the top-256 eigenvalues of the mean-centered q covariance; (d) probe accuracy versus effective rank (points annotated by epoch), with the plotted correlation coefficient ρ. HamJEPA improves steadily and shows increasing effective rank, while SIGReg+tokens improves modestly while its effective rank decreases; the deltas at epoch 45 are annotated in panels (a,b).

D.4

Further figures

Figures 6–8 provide an intuition for a LeJEPA-inspired sliced-projection diagnostic: rather than comparing rollout distributions only in aggregate, we compare many 1D projections (a Cramér–Wold viewpoint) to expose direction-dependent geometric distortion.

LeJEPA-inspired projection sketch for HamJEPA rollouts

Projected point densities

Input support (conceptual)

error target (ref) HamJEPA SigReg proxy

Embedding distribution 2

0 2 4 6 15.0

12.5

10.0

7.5

5.0

2.5

0.0

2.5

1 0 1 Projection coordinate (along direction)

Figure 6: LeJEPA-inspired sliced-projection diagnostic for HamJEPA rollouts (cartoon). Left: conceptual support of the input distribution (manifold-structured data). Middle: an example embedding distribution in a 2D slice with several projection directions u(θ). Right: estimated 1D projected densities along representative directions; black denotes the reference/target, green denotes HamJEPA rollouts, orange denotes a SIGReg proxy baseline, and the shaded band visualizes the per-direction projection error. Matching many 1D projections (Cramér–Wold/sliced view) makes geometric distortion from non-structure-preserving dynamics visible, while symplectic rollouts remain close to the reference across directions. LeJEPA-style projection sketch for Hamiltonian rollouts Target=high-accuracy leapfrog (dt=0.010, steps=2500); Models=Euler vs leapfrog (dt=0.100, steps=250), T=25.0 density

0.3

0.0

0.1

2 4

0.0

0.2

0.3

0.4

0.5 q

0.6

0.7

5 z1 (whitened)

0.8

5 <u, z>

Projection u(theta), theta=0.45 rad

0.4 density

0.3

5 <u, z>

0.2

Projection u(theta), theta=2.93 rad

0.4

density

z2 (whitened)

0.0

target p_t (reference) SigReg proxy (Euler) HamJEPA (leapfrog)

0.1

target (ref) SigReg proxy HamJEPA

0.2

Final distributions in whitened space

0.2

Projection u(theta), theta=0.12 rad

0.4

Initial distribution p_x

0.2 0.0

5 <u, z>

Figure 7: LeJEPA-style projection sketch on a toy Hamiltonian transport problem. Left: initial phase-space distribution px over (q, p). Middle: final distributions in whitened coordinates after rolling out to a fixed horizon T ; the reference target is generated by a high-accuracy leapfrog integrator, while the two model curves compare a coarse Euler rollout (SIGReg proxy) against a coarse leapfrog rollout (HamJEPA). Right: 1D densities of projections ⟨u(θ), z⟩ for representative angles θ, showing that leapfrog-based rollouts track the reference distribution across directions, whereas Euler produces large direction-dependent mismatch. 35

g(theta) = W1(projected model, projected reference)

Directional mismatch to the reference distribution (Cramer-Wold sketch) HamJEPA stays close across directions; Euler shows larger mismatch Directional Cramer-Wold sketch: mismatch to reference

SigReg proxy (Euler) mean=4.901 max=7.215 H1~0.067 HamJEPA (leapfrog) mean=0.0137 max=0.0206 H1~0.0002

7 6 5 4 3 2 1 0 0.0

0.5

1.0

1.5 direction angle theta (period pi)

2.0

2.5

3.0

Figure 8: Directional Cramér–Wold discrepancy (sliced mismatch) versus projection angle. For each direction angle θ (period π), we compute a 1D mismatch g(θ) = W1 (⟨u(θ), Zmodel ⟩, ⟨u(θ), Zref ⟩) between the projected model and reference distributions. HamJEPA (leapfrog) stays uniformly close across directions, while the Euler/SIGReg proxy exhibits large, anisotropic deviation; the legend reports summary statistics over θ (mean/max and the aggregate score used in the plot).

A visual guide to HamJEPA

This appendix gives a diagrammatic explanation of HamJEPA for readers who want a more visual account than the equation-level presentation in the main text. The figures here are explanatory schematics rather than empirical results. Their purpose is to separate the method into four conceptual layers: (i) the overall two-view prediction pipeline, (ii) how the encoder constructs the phase-space state s = [q; p], (iii) how the predictor evolves that state by a symplectic Hamiltonian rollout, and (iv) why encoder-side anti-collapse regularization is still required even when the predictor itself is symplectic.

ŝb sa q̂b qa τa

symplectic flow ΦK ϕ

p̂b

view va shared encoder Eθ

image x

qb τb

view vb

prediction loss

sb encoder-side anti-collapse regularizers

Figure 9: HamJEPA at the highest level. Two augmented views of the same image are encoded by a shared encoder into phase-space states sa = [qa ; pa ] and sb = [qb ; pb ]. The predictor is not an unconstrained regression head: it is a structured Hamiltonian rollout ΦK ϕ that evolves sa into a predicted target state ŝb . Training combines a prediction loss between ŝb and sb with separate encoder-side anti-collapse regularizers applied across the batch of states. 36

one token

token grid

q half

view v

ResNet-18 token mode

q ∈ Rd

layer3 feature map

s = [q; p] ∈ R2d p half

p ∈ Rd df channels identity projector preserves the q/p split

Figure 10: How the encoder constructs the phase-space state. HamJEPA is not just taking an arbitrary vector and renaming its halves as q and p. In the implementation, the encoder runs in token mode: an intermediate ResNet feature map is converted into a token grid, the channel stack of each token is split into a q half and a p half, and those halves are flattened across the spatial grid to form q ∈ Rd and p ∈ Rd . These are then concatenated into s = [q; p] ∈ R2d . This is why HamJEPA uses an identity projector: a learned MLP after the encoder would remix the coordinates and destroy the intended q/p semantics.

Hϕ (q, p) = T (p) + Vϕ (q) q̇ = p,

ṗ = −∇q Vϕ (q)

T (p) = 12 ∥p∥2

Vϕ (q)

ŝb one leapfrog step

q p

q̂b half-kick

drift

half-kick

p ← p − ∆t ∇V (q) 2

q ← q + ∆t p

p ← p − ∆t ∇V (q) 2

p̂b

repeat K times

Figure 11: What the predictor is doing. HamJEPA does not use an arbitrary predictor za 7→ ẑb . Instead, it rolls the state forward with a learned separable Hamiltonian Hϕ (q, p) = T (p) + Vϕ (q) and a leapfrog integrator. The potential Vϕ (q) drives the kick updates through ∇q Vϕ (q), while the kinetic term T (p) = 21 ∥p∥2 yields the drift update q ← q + ∆t p. Repeating this structured step K times produces the predicted target state ŝb = ΦK ϕ (sa ).

generic learned map

symplectic map

| det DΦ| = 1: bends and shears, but preserves phase-space volume

can arbitrarily squash local volume

Figure 12: What “symplectic” means geometrically. A generic learned map can bend, stretch, and arbitrarily collapse local geometry. A symplectic map can still deform the state space, but it preserves total phase-space volume locally, i.e. | det DΦ| = 1. This gives the predictor a structured notion of reversibility and ensures that the predictor itself is not the source of volume collapse. Symplecticity constrains the predictor, but it does not by itself prevent encoder collapse. A collapsed encoder could still map many different views to almost the same latent state, and a perfectly symplectic predictor would then only move around an already-degenerate state cloud. The next two figures 37

therefore separate these two roles: first, why predictor structure alone is insufficient; second, how the encoder-side regularizers keep the latent space alive.

many different views

collapsed states

still low-volume

shared encoder

symplectic predictor

predictor structure cannot rescue an already-collapsed encoder output cloud

Figure 13: Why the predictor alone is not enough. Even if the predictor is perfectly symplectic, the encoder could still map many different views to nearly the same latent state. In that situation the rollout acts on an already-collapsed representation. This is why HamJEPA needs a second component beyond the predictor: encoder-side anti-collapse regularization.

Why the predictor alone is not enough. The symplectic constraint belongs to the predictor map, not to the encoder. This means it prevents the rollout from crushing phase-space volume, but it does not stop the encoder itself from mapping many different views to nearly the same latent state. A collapsed encoder would therefore still give a degenerate state cloud, even with a perfectly symplectic predictor. The next figures isolate the role of each encoder-side anti-collapse regularizer.

under-scale

target

over-scale

Figure 14: Energy budget. The energy budget keeps the average per-dimension scale of q and p near a target range. It penalizes both under-scaled states (which risk collapse toward zero) and over-scaled states (which can destabilize dynamics), so the latent cloud stays at a controlled overall radius.

dead coordinate

all coordinates active

Figure 15: Variance floor. The variance floor prevents individual coordinates from becoming nearly constant across the batch. Visually, it revives dead axes: instead of allowing the latent cloud to collapse onto a lower-dimensional direction, it forces each coordinate to retain a minimum amount of variation. 38

healthy volume

thin sheet

line-like collapse

Figure 16: Projected log-det floor. The projected log-det floor prevents the batch representation from flattening into a very low-volume set. It does not require isotropy, but it does force the projected covariance to retain nontrivial volume, ruling out extreme degeneracies such as line-like or sheet-like collapse.

one dominant spike

variance spread across modes

Figure 17: Participation-ratio floor and top-eigenvalue control. These penalties prevent the representation from surviving only through one huge dominant direction. The participation-ratio floor encourages variance to remain spread across multiple modes, while the top-eigenvalue ceiling prevents a single spike from taking over the spectrum.

mean near zero

large offset direction

Figure 18: Mean penalty. A large global mean can create a dominant offset direction that makes random-pair cosine similarities look artificially cone-like. The mean penalty discourages this by keeping the batch mean closer to zero, so the geometry is not dominated by a single global shift. 39

CIFAR-100

q̂b p̂b

ImageNet-100

q̂b

p̂b

qb pb

match mainly q

match full [q; p]

q is the main content coordinate; p is more auxiliary

both coordinates are directly constrained

Figure 19: Why CIFAR-100 and ImageNet-100 use different matching choices. On CIFAR-100 the training objective emphasizes the content coordinate q, so q is the cleanest downstream readout and p mainly supports the dynamics. On ImageNet-100 the stable regime was to match the full state [q; p], which directly constrains both coordinates and makes p much more informative. How to read the sequence. Taken together, Figs. 9–19 decompose the full method into three conceptual layers: the encoder constructs a structured phase-space state, the predictor evolves that state with a symplectic Hamiltonian rollout, and the regularizers ensure that the encoder supplies a non-degenerate cloud of states for those dynamics to act on. The distinctive point is therefore twofold: HamJEPA changes both the representation parameterization (from a single embedding to a phase-space state) and the predictor family (from arbitrary regression to Hamiltonian dynamics), while avoiding the need to force the encoder marginal toward Euclidean isotropy.

Implementation Details and Algorithms

In this section (as in the code) we refer to HamJEPA as MV-HJEPA, denoting the minimal multi-view implementation used in our experiments. F.1

Compute resources

All reported experiments were run on NVIDIA A100 GPUs. CIFAR-100 pretraining took approximately 40 seconds per epoch, corresponding to roughly 20 minutes for a 30-epoch run and roughly 53 minutes for an 80-epoch run. ImageNet-100 pretraining took approximately 5 hours for a 45-epoch run. Downstream evaluation was also run on A100 GPUs and required under 5 minutes for CIFAR100 and under 15 minutes for ImageNet-100 per checkpoint/readout evaluation. These timings are approximate wall-clock measurements; peak GPU memory was not separately profiled. F.2

Two training modes: MV-HJEPA vs. baseline (LeJEPA + regularizer)

Our training scripts are mode-switched by the presence of a hjepa: block in the YAML: • MV-HJEPA mode (hjepa present): we treat the representation as a phase-space state z = [q; p] ∈ R2d and train a Hamiltonian flow predictor that maps between two global views. We additionally apply scale/diversity regularizers on all views. • Baseline mode (hjepa absent): we use the LeJEPA prediction objective where all views predict the mean of the global views, and we optionally add a distributional regularizer such as SIGReg. F.3

Data pipeline and multi-view batching (CIFAR100 vs. ImageNet)

Multi-crop dataset contract. Each dataset example yields a list of V augmented views (multi-crop), plus labels: ({x(v) }Vv=1 , y, ycoarse ). In our configurations, we set num_global_views=2 and num_local_views=0, so V = 2. 40

CIFAR100MultiCrop. eval/datasets/cifar100_multicrop.py applies the full augmentation in the worker and returns float tensors already normalized (using ToTensor() + Normalize). Notable defaults: • out_size=32 (CIFAR resolution) • blur uses a small kernel (kernel_size=3) ImageNetMultiCrop. eval/datasets/imagenet_multicrop.py applies the same high-level augmentation pattern, but returns uint8 tensors using PILToTensor() to reduce dataloader IPC bandwidth. We then convert to float and normalize on device inside the ImageNet training loop. Notable defaults: • out_size=224 • blur uses a larger kernel (kernel_size=23) appropriate for higher resolution • global crop scale defaults match the YAML (global_scale=[0.4,1.0]) Multi-view batching optimization. For computational efficiency, we concatenate views along the batch dimension and run a single encoder+projector forward: xcat = concat x(1) , . . . , x(V ) ∈ R(V B)×C×H×W , zcat = P (E(xcat )) ∈ R(V B)×K . We then reshape: zviews ∈ RV ×B×K ,

zviews = reshape(zcat , (V, B, K)).

This is exactly how both train_cifar_hamjepa.py and train_imagenet_hamjepa.py operate. F.4

Encoder and token-state parameterization

ResNet18 backbone with dataset-specific stem. We use torchvision.models.resnet18(weights=None) and alter the stem depending on the dataset: • CIFAR stem (encoder_stem=cifar): conv1 is replaced by a 3 × 3 stride-1 conv and maxpool is removed. • ImageNet stem (encoder_stem=imagenet): keep the default 7 × 7 stride-2 conv and maxpool. Token mode (used in all shown configs). With encoder_mode=tokens, we expose an intermediate feature map at a chosen ResNet layer (token_layer in {layer2, layer3, layer4}), project its channels with a 1 × 1 conv to token_d_f, optionally pool to a fixed spatial grid (token_hw), then flatten. Concretely, if the token feature map has shape [B, token_d_f, h, w], the flattened representation is z ∈ RB×(hw token_d_f) . This is why embed_dim must match hw token_d_f: • CIFAR100: layer3 naturally yields an 8×8 map, and with token_d_f=32 we get 8·8·32 = 2048. • ImageNet: layer3 yields 14 × 14 pre-pool; we set token_hw=8 so adaptive pooling yields 8 × 8, again giving 2048. Projector. We support either an MLP projector or an identity projector. In MV-HJEPA we require projector_type=identity and enforce it in code, because we interpret the representation channels as structured phase-space coordinates and do not want a learned MLP to arbitrarily mix them. 41

Phase-space split: z = [q; p]. In MV-HJEPA we require an even representation dimension K = 2d, and interpret: q z= ∈ R2d , q ∈ R d , p ∈ Rd . p This split is implemented channel-wise per token in the encoder when split_qp=true: the token channel dimension token_d_f is split into equal halves (first half → q, second half → p) before flattening and concatenation. F.5

Hamiltonian flow predictor (MV-HJEPA)

Hamiltonian dynamics. We predict one view’s state from another by integrating Hamiltonian dynamics in latent phase space. Given a Hamiltonian H(q, p), the vector field is: dq ∂H = , dt ∂p

dp ∂H =− . dt ∂q

In code, ∂H/∂q and ∂H/∂p are obtained via automatic differentiation (torch.autograd.grad(create_graph=True)), so gradients propagate through the integrator. Hamiltonian parameterizations.

hamjepa/hamiltonian.py supports:

1. Quadratic: H(q, p) = 21 (∥q∥2 + ∥p∥2 ) (no learnable parameters). 2. Learnable (residual MLP): Hθ (q, p) = 21 (∥q∥2 + ∥p∥2 ) + s fθ ([q, p]), where s (residual_scale) is a stabilizing residual multiplier. 3. Separable (used for MV-HJEPA): Hϕ (q, p) = T (p) + Vϕ (q),

T (p) = 12 ∥p∥2 ,

Vϕ (q) = 21 α∥q∥2 + s fϕ (q),

where α (base_coeff) is a fixed quadratic base coefficient and s (residual_scale) is a residual scaling factor. The separable form enables an explicit velocity-Verlet/leapfrog update that is symplectic. Residual-scale warmup. In MV-HJEPA, we linearly warm up residual_scale over residual_scale_warmup_epochs. Early in training the potential is close to purely quadratic (stable), and the learned residual is introduced gradually. Step size and fp32 integration. We integrate for steps=S steps with step size dt (fixed in MV-HJEPA; learn_dt=false). For mixed precision runs (bf16), we optionally do the dynamics computations in fp32 and cast back (integrate_fp32=true) to stabilize higher-order autodiff through the integrator. F.6

Symplectic integrators

We implement: • Symplectic Euler for general H(q, p). • Leapfrog / velocity-Verlet: – a general variant using autodiff for ∇q H and ∇p H – a specialized separable variant (used by MV-HJEPA when H = T + V ) using only ∇V (q) For separable H(q, p) = 12 ∥p∥2 + V (q), the specialized update is: p1/2 = p − ∆t 2 ∇V (q),

q ← q + ∆t p1/2 , 42

p ← p1/2 − ∆t 2 ∇V (q).

F.7

MV-HJEPA prediction loss

Given two global views’ states za = [qa ; pa ] and zb = [qb ; pb ], we predict: ẑb = FlowH (za ; ∆t, S), and compute a MSE matching term either on q only or the full (q, p) state: MSE(q̂b , qb ) + wp MSE(p̂b , pb ) (match = q) Lmatch = MSE(ẑb , zb ) (match = qp). We typically stop-gradient the target branch (detach_target=true) to prevent trivial collapse. Optionally, we add an energy-consistency term that encourages the flow to preserve the input energy: Lenergy = MSE H(qa , pa ), H(q̂b , p̂b ) , and the full prediction loss is: Lpred = Lmatch + λE Lenergy . Bidirectional training.

If enabled, we symmetrize by predicting in both directions. Lpred ← 21 (Lpred (z0 , z1 ) + Lpred (z1 , z0 )) .

In the most physically consistent variant, the reverse direction uses reverse-time integration (negated ∆t), which is supported by the predictor’s direction flag. F.8

Regularizers (applied across all views)

We compute regularizers on the concatenation of all views’ states: zall ∈ R(V B)×2d . Phase-space energy budget (scale control). We stabilize scale in fixed units by enforcing percoordinate second moments: 2 2 Lbudget = E[q 2 ] − qtgt + E[p2 ] − ptgt , where expectations are taken over batch and coordinates in fp32 and can be Distributed Data Parallel (DDP)-synchronized. Variance floor (anti-collapse). Let σi be the batch standard deviation of dimension i (fp32 stats). We apply: d 1X Lvar = max(0, σmin − σi )2 , d i=1 typically on q and optionally on p depending on the experiment. Projected log-determinant floor (spectral diversity). We enforce a lower bound on the logdeterminant of a randomly projected covariance. Given x ∈ RB×d , sample an orthonormal projection R ∈ Rd×k , compute y = (x − x̄)R, then: 2 C = Cov(y) + εI, Llog det = max 0, τ − k1 log det(C) . We optionally add constraints on the participation ratio PR = tr(C)2 /tr(C 2 ) and the top-eigenvalue fraction λmax /tr(C). Mean penalty.

We penalize non-zero feature means using the squared mean (fp32): Lmean ∝ ∥E[z]∥22 . 43

F.9

Training step summary

MV-HJEPA.

At each step:

1. Build V views per image, concatenate them, and compute zviews ∈ RV ×B×2d . 2. Select the two global views (z0 , z1 ) and compute Lpred via Hamiltonian flow prediction. 3. Compute Lbudget , Lvar , Llog det , and Lmean on all views. 4. Optimize encoder + (identity) projector + predictor parameters with AdamW, optional gradient clipping, and cosine LR schedule with warmup. Baseline (LeJEPA + SIGReg/HamSIGReg).

If hjepa is absent:

1. Compute zviews as above. 2. Apply LeJEPA prediction loss: all views match the per-sample mean of global views. 3. Add a distributional regularizer (e.g. SIGReg) with weight lambda_reg. 4. If the regularizer includes a learnable metric H, optionally update H on a slower/periodic schedule. F.10

Algorithms

Notation. Let V be the number of augmented views per image, B the batch size, and K the representation dimension. In MV-HJEPA we enforce K = 2d and interpret each representation as a phase-space state z = [q; p] ∈ R2d with q, p ∈ Rd (channel-wise split, split_qp=true). We write concatv x(v) for concatenation of views along the batch dimension, and reshape for a view/batch reshape with the same underlying memory order as the implementation. Algorithm 1 Multi-crop view generation (used by CIFAR100MultiCrop and ImageNetMultiCrop) Input: Image I, multi-crop config: (Vg , Vℓ ), output size s, scales (αg , βg ) and (αℓ , βℓ ) Output: List of views {x(v) }Vv=1 where V = Vg + Vℓ 1: Initialize empty list views ← [ ] 2: for i = 1 to Vg do 3: x ← Augment(I; RandomResizedCrop(s, scale [αg , βg ]), HFlip, CJ, Gray, Blur, Solarize) 4: Append x to views 5: end for 6: for i = 1 to Vℓ do 7: x ← Augment(I; RandomResizedCrop(s, scale [αℓ , βℓ ]), HFlip, CJ, Gray, Blur, Solarize) 8: Append x to views 9: end for 10: return views

Implementation note (dataset-specific): CIFAR views are produced as float tensors with normalization applied in the dataset pipeline; ImageNet views are emitted as compact uint8 tensors (via PILToTensor) and normalized on-device in the training loop. Algorithm 2 ResNet token encoder forward pass (token mode, optional q/p split) Input: Image batch x ∈ RB×3×H×W , ResNet stem+layers, token layer ℓ ∈ {layer2, layer3, layer4}, channel projection to df , optional pooling to token_hw = h = w, split_qp flag Output: Representation z ∈ RB×K where K = h · w · df 1: u ← stem(x) 2: u ← layer1(u) 3: u ← layer2(u); set fmap ← u if ℓ = layer2 4: u ← layer3(u); set fmap ← u if ℓ = layer3 44

5: u ← layer4(u); set fmap ← u if ℓ = layer4 6: if token_hw is set then 7: fmap ← AdaptiveAvgPool2d(fmap, (h, w)) 8: end if 9: t ← Conv1 × 1 + BN(fmap) ∈ RB×df ×h×w 10: t ← permute(t) ∈ RB×h×w×df 11: if split_qp = false then 12: z ← flatten(t) ∈ RB×(hwdf ) 13: else 14: Split channels: t 7→ (tq , tp ) with tq , tp ∈ RB×h×w×(df /2)

q ← flatten(tq ) ∈ RB×(hw(df /2)) ; p ← flatten(tp ) ∈ RB×(hw(df /2)) 16: z ← [q; p] ∈ RB×(hwdf ) 17: end if 18: return z

15:

Algorithm 3 MV-HJEPA training step (matches train_*_hamjepa.py when hjepa: is present) Input: Views {x(v) }Vv=1 , encoder E, projector P (identity for MV-HJEPA), predictor FlowH , loss/regularizer weights λ• Input: MV-HJEPA requires split_qp=true and K = 2d 1: xcat ← concatv x(v) ∈ R(V B)×C×H×W 2: zcat ← P (E(xcat )) ∈ R(V B)×2d 3: zviews ← reshape(zcat ) ∈ RV ×B×2d 4: z0 ← zviews [0], z1 ← zviews [1] 5: Lpred ← HamConsistency(z0 , z1 ) 6: if bidirectional then 7: Lpred ← 21 (Lpred + HamConsistency(z1 , z0 )) 8: end if 9: zall ← zviews .reshape(V B, 2d) 10: Split zall 7→ (qall , pall ) along the last dimension 11: Lbudget ← EnergyBudget(zall ) 12: Lvar ← VarFloor(qall ) (+ VarFloor(pall ) if enabled in that script) 13: Llog det ← LogDetFloor(qall ) + LogDetFloor(pall ) 14: Lmean ← mean(zall )⊙2 averaged over dimensions ▷ matches mean().square().mean() 15: L ← Lpred + λbudget Lbudget + λvar Lvar + λlog det Llog det + λmean Lmean 16: Update parameters of {E, P, FlowH } with AdamW; optionally clip gradients and step LR schedule Algorithm 4 Hamiltonian Consistency loss (HamiltonianConsistencyLoss) Input: States za , zb ∈ RB×2d , predictor FlowH , options: detach_target, match ∈ {q, qp}, p_weight, energy_weight, bidirectional Output: Scalar loss L 1: z ba→b ← FlowH (za ; direction = +1) 2: zb⋆ ← zb if not detach_target else stopgrad(zb ) 3: if match=qp then 4: ℓa→b ← MSE(b za→b , zb⋆ ) 5: else ▷ match=q 6: ℓa→b ← MSE(b za→b [:, : d], zb⋆ [:, : d]) 7: if p_weight > 0 then 8: ℓa→b ← ℓa→b + p_weight · MSE(b za→b [:, d :], zb⋆ [:, d :]) 9: end if 10: end if 11: if energy_weight > 0 then 12: Split za 7→ (qa , pa ) and zba→b 7→ (q̂, p̂) 13: Ha ← stopgrad H(qa , pa ) ; Hẑ ← H(q̂, p̂) 14: ℓa→b ← ℓa→b + energy_weight · MSE(Hẑ , Ha ) 15: end if 16: L ← ℓa→b 45

17: if bidirectional then 18: Compute ℓb→a identically using direction = −1 and target za (optionally detached) 19: L ← 21 (ℓa→b + ℓb→a ) 20: end if 21: return L

Algorithm 5 Hamiltonian Flow Predictor (forward pass, HamiltonianFlowPredictor) Input: State z0 ∈ R···×2d , Hamiltonian module H, method, steps S, base step size ∆t > 0, direction s ∈ {+1, −1}, integrate_fp32, learn_dt Output: State zT ∈ R···×2d 1: Split z0 7→ (q0 , p0 ) 2: ∆t ← s · ∆t; if learn_dt, set ∆t ← s · softplus(raw_dt) 3: if integrate_fp32 enabled and dtype(z0 ) ∈ {fp16, bf16} then 4: Cast (q0 , p0 ) to fp32 5: end if 6: if H is separable and method is leapfrog then 7: (qT , pT ) ← IntegrateSeparableLeapfrog(Vϕ , q0 , p0 , ∆t, S) 8: else 9: (qT , pT ) ← IntegrateHamiltonian(H, q0 , p0 , ∆t, S, method) 10: end if 11: zT ← [qT ; pT ] 12: if integrate_fp32 enabled and original dtype was bf16/fp16 then 13: Cast zT back to the original dtype 14: end if 15: return zT Algorithm 6 Symplectic integration routines (as in hamjepa/integrators.py) Input: General Hamiltonian H(q, p) or separable potential V (q), initial (q, p), step size ∆t, steps S, method ∈ {leapfrog, symplectic_euler} Output: Final (q, p) after S steps 1: function I NTEGRATE H AMILTONIAN(H, q, p, ∆t, S, method) 2: for n = 1 to S do 3: if method = symplectic_euler then 4: p ← p − ∆t ∇q H(q, p) 5: q ← q + ∆t ∇p H(q, p) ▷ ∇p uses updated p 6: else ▷ method = leapfrog (velocity Verlet) 7: p1/2 ← p − ∆t 2 ∇q H(q, p) 8: q ← q + ∆t ∇p H(q, p1/2 ) 9: p ← p1/2 − ∆t 2 ∇q H(q, p1/2 ) 10: end if 11: end for 12: return (q, p) 13: end function 14: function I NTEGRATE S EPARABLE L EAPFROG(V, q, p, ∆t, S) 15: for n = 1 to S do 16: p1/2 ← p − ∆t 2 ∇q V (q) 17: q ← q + ∆t p1/2 18: p ← p1/2 − ∆t ▷ ∇q V uses updated q 2 ∇q V (q) 19: end for 20: return (q, p) 21: end function Algorithm 7 Phase-space energy budget regularizer (PhaseSpaceEnergyBudget) Input: States z ∈ R···×2d , targets (qtgt , ptgt ), ddp_sync Output: Scalar loss Lbudget 1: Flatten z → zf ∈ RN ×2d and cast to fp32

2: Split zf 7→ (q, p) with q, p ∈ RN ×d

3: q2_sum ← q ⊙2 , p2_sum ← p⊙2 , count ← N d 4: if ddp_sync and DDP initialized then 5: All-reduce (SUM) q2_sum, p2_sum, and count 6: end if 7: q2_µ ← q2_sum/count; p2_µ ← p2_sum/count 8: Lbudget ← (q2_µ − qtgt )2 + (p2_µ − ptgt )2 9: return Lbudget

Algorithm 8 Variance floor regularizer (VarianceFloor) Input: Features x ∈ RN ×d , std floor σmin , ε, ddp_sync Output: Scalar loss Lvar 1: Cast x to fp32; if N < 2 return P 0 P ⊙2 2: Compute sufficient stats: s1 ← n xn , s2 ← n xn , and n ← N 3: if ddp_sync and DDP initialized then 4: All-reduce (SUM) s1 , s2 , and n 5: end if p 6: µ ← s1 /n; var ← s2 /n − µ⊙2 ; σ ← max(var, 0) + ε Pd 1 2 7: Lvar ← d i=1 ReLU(σmin − σi ) 8: return Lvar Algorithm 9 Projected log-det floor (ProjectedLogDetFloor) Input: Features x ∈ RB×D , projection dim k, log-det floor ℓmin , ε, refresh interval r, optional pr_norm_floor and eigmax_frac_ceiling, ddp_sync Output: Scalar loss Llog det 1: Cast x to fp32; if B < 2 return 0 2: k ← min(k, D, B − 1) 3: if need resample (first call / device change / shape change / step mod r = 0) then 4: Sample R ∼ N (0, 1)D×k and orthonormalize columns via QR: R ← qr(R) 5: end if 6: Center and project: xc ← x − mean(x); y ← xc R ∈ RB×k P ⊤ 7: Compute sufficient stats: sum_y ← b yb , sum_yy ← y y, n ← B 8: if ddp_sync and DDP initialized then 9: All-reduce (SUM) sum_y, sum_yy, and n 10: end if 11: n ← max(n, 2); µy ← sum_y/n sum_yy−n·(µy µ⊤ y ) ; n−1 1 13: ℓ ← k log det(Σ)

12: Σ ←

Σ ← 12 (Σ + Σ⊤ ) + εI ▷ implemented via slogdet

14: Llog det ← ReLU(ℓmin − ℓ)2 tr(Σ)2

15: Compute PR = tr(Σ2 ) , pr_norm = PR/k, and eigmax_frac = λmax /tr(Σ) 16: if pr_norm_floor is set then 17: Llog det ← Llog det + ReLU(pr_norm_floor − pr_norm)2 18: end if 19: if eigmax_frac_ceiling is set then 20: Llog det ← Llog det + ReLU(eigmax_frac − eigmax_frac_ceiling)2 21: end if 22: return Llog det

Algorithm 10 SIGReg forward pass (random slicing + Epps–Pulley statistic) Input: Batch z ∈ R···×N ×D , slices K, knots t1 , . . . , tT (with weights w), refresh interval r, ddp_sync Output: Scalar statistic S(z) (mean-reduced in our configs) 1: (Slicing) If step mod r = 0, sample A ∈ RD×K with i.i.d. N (0, 1) columns and normalize each column to unit norm

2: if ddp_sync and DDP initialized then 3: Ensure all ranks share the same sampling seed for A (broadcast seed from rank 0) 4: end if 5: Project: y ← zA ∈ R···×N ×K 6: (Epps–Pulley)P Compute empirical CF on each P slice: 7: ĉ(t) = N1 n cos(t yn ), ŝ(t) = N1 n sin(t yn ) (optionally globally averaged via all-

reduce if ddp_sync) 2 8: Target CF for N (0,h1) is ϕ(t) = e−t /2 i PT 9: Sk ← N j=1 wj (ĉ(tj ) − ϕ(tj ))2 + ŝ(tj )2 for each slice k

10: Reduce over slices (and leading dims) by mean: S(z) ← meank (Sk ) 11: return S(z)

Algorithm 11 LeJEPA baseline step with SIGReg regularization (non-MV branch in train_*_hamjepa.py) Input: Views {x(v) }Vv=1 , encoder E, projector P , number of global views Vg , regularizer R (SIGReg), weight λreg 1: xcat ← concatv x(v) 2: zcat ← P (E(xcat )) ∈ R(V B)×D 3: zviews ← reshape(zcat ) ∈ RV ×B×D PVg B×D 4: z̄ ← V1 ▷ mean of global views v=1 zviews [v] ∈ R g 1 2 5: Lpred ← 2 mean ∥zviews − z̄∥2 ▷ matches lejepa_prediction_loss 6: Lreg ← R(zviews ) ▷ SIGReg expects shape (· · · , N, D) 7: L ← Lpred + λreg Lreg ▷ spectral term omitted here (zero in our SIGReg configs) 8: Update {E, P } with AdamW; optionally step LR schedule and log diagnostics F.11

YAML Configuration Knobs

The repository is fully config-driven: the training scripts read a YAML file and route into either (i) MV-HJEPA mode when the hjepa: stanza is present (and split_qp=true), or (ii) the LeJEPA+regularizer baseline when hjepa: is absent (e.g. regularizer.type=sigreg). Below we document every key appearing in the provided configs, exactly as it is consumed by the code. We keep seed=42 for reproducibility across runs. F.11.1

Data (data:) • root: Dataset root path passed to CIFAR100 / ImageFolder loader. • batch_size: DataLoader batch size (batch_size=batch_size). We use larger batches on CIFAR (512) and smaller on ImageNet (256) to balance GPU memory and throughput. • num_workers: DataLoader worker count. Higher values improve input pipeline throughput when CPU-bound. • drop_last: If true, drops the last incomplete batch. This avoids smaller last batches, which can destabilize batch statistics and covariance-based regularizers (and helps DDP batch-size consistency). • pin_memory (ImageNet configs): Enables pinned host memory to speed CPU→GPU transfers. • persistent_workers (ImageNet configs): Keeps workers alive across epochs to reduce worker spin-up overhead. • prefetch_factor (ImageNet configs): Worker prefetch depth; increases overlap between CPU augmentation and GPU training. • num_global_views, num_local_views: Multi-crop configuration. In all provided runs we set two global views and no local views (2/0), matching the two-view training objective in both MV-HJEPA and the LeJEPA baseline. • out_size (ImageNet configs): Output crop resolution for RandomResizedCrop. We use 224 (standard ImageNet scale). 48

• global_scale, local_scale (ImageNet configs): Scale ranges for global/local crops. Only global crops are used when num_local_views=0. • CIFAR note: the CIFAR YAMLs omit out_size/global_scale/local_scale, so the dataset defaults are used (out_size=32, global_scale=(0.3,1.0), local_scale=(0.05,0.3)). F.11.2

Model (model:) • encoder_mode: global uses GAP (global average pooling)+FC; tokens exposes an intermediate feature map and flattens it into tokens. All provided configs use tokens. • encoder_stem: cifar modifies ResNet-18 stem (3×3 stride 1, no maxpool) for 32×32 inputs; imagenet keeps the default 7×7 stride 2 + maxpool stem. • token_layer: Which ResNet stage to expose as the token grid (layer2/layer3/layer4). We use layer3, giving an 8×8 grid on CIFAR and a 14×14 grid on ImageNet before any optional pooling. • token_d_f: Number of channels after the 1×1 token projection (Conv1x1+BN). In token mode, the representation dimension is K = h · w · token_d_f. • token_hw (ImageNet configs): Optional adaptive pooling size for the token grid. We set token_hw=8 on ImageNet so that with token_d_f=32 we obtain K = 8 · 8 · 32 = 2048 (matching embed_dim). • embed_dim: Output dimension K of the encoder in token mode. For our settings this is tied to the token grid geometry (e.g. 8 · 8 · 32 = 2048). • proj_hidden, proj_out: Hidden/output sizes for the optional MLP projector. These are read from the YAML but are only used when projector_type=mlp. • projector_type: identity returns z unchanged; mlp applies MLPProjector. MVHJEPA requires identity so the channel-wise q/p semantics are preserved. • split_qp: If true, the token channels are split into (q, p) halves per token and concatenated, enforcing K = 2d and enabling the phase-space interpretation required by MV-HJEPA.

F.11.3

MV-HJEPA predictor (hjepa:)

(Only present in *_hjepa_mv.yaml configs.) • hamiltonian: Choice of Hamiltonian parameterization. We use separable, i.e. H(q, p) = 21 ∥p∥2 + Vϕ (q), to enable explicit symplectic leapfrog integration. • method: Integrator choice. We use leapfrog (velocity Verlet), implemented in integrate_separable_leapfrog for separable H. • steps: Number of integrator steps S per forward pass. We use 2 to keep compute low while still giving a nontrivial flow. • dt: Fixed step size ∆t. Used as a constant buffer when learn_dt=false. • learn_dt: If true, ∆t becomes learnable via a softplus parameterization. MV-HJEPA disables this (false) and the training script explicitly rejects learnable-dt to avoid degenerate solutions (“dt-collapse”). • hidden_dim, depth: MLP size/depth used inside the Hamiltonian residual network (for Vϕ in the separable case). • residual_scale: Multiplier on the Hamiltonian residual output. This is a stability knob: the code is designed so the Hamiltonian has a quadratic base plus a small learned residual. • residual_scale_warmup_epochs: Linearly ramps residual_scale from 0 to its target value over the first epochs (implemented by overwriting predictor.H.residual_scale each epoch). • base_coeff: Coefficient on the quadratic base potential term in V (q) (i.e. 0.5 base_coeff ∥q∥2 ). Keeping this nonzero prevents the “trivial potential” loophole where dynamics can collapse. • integrate_fp32: If true, the predictor casts z to fp32 during integration when training in bf16/fp16, improving numerical stability of gradient-based integration. 49

F.11.4

MV-HJEPA loss (loss:) • match: Which part of the state to match in the consistency loss: q matches positions only (optionally with a small p penalty), while qp matches the full state. • p_weight: Only used when match=q. Adds an auxiliary MSE on p with weight p_weight. • detach_target: If true, the target branch is detached in the prediction loss (safer default to prevent the target moving to satisfy the predictor). • energy_weight: Optional extra term matching H(z) before/after the flow. Set to 0.0 in all provided configs (disabled). • bidirectional: If true, averages forward and backward consistency (z0 → z1 and z1 → z0 ). In the ImageNet training script this averaging is done explicitly in the loop; in the CIFAR script it can be handled inside the loss module.

F.11.5

MV-HJEPA anti-collapse regularizers (regularizer:)

(These are the phase-space regularizers used in the MV-HJEPA branch; they are not the SIGReg baseline.) • q_per_dim_target, p_per_dim_target: Targets for the phase-space energy budget: they control the desired per-dimension mean-square magnitude of q and p. • q_std_floor: Per-dimension standard deviation floor used by VarianceFloor on q (and on p in the CIFAR MV script; ImageNet MV applies the variance floor to q only). • ddp_sync: If true and DDP is initialized, the sufficient statistics for the budget/variance/logdet terms are aggregated across ranks using all-reduce. This makes the regularizers behave as if computed on the global batch. • q_logdet_proj_dim, p_logdet_proj_dim: Projection dimension k for the log-det regularizer on q and p. • q_logdet_floor, p_logdet_floor: Minimum allowed log-determinant per projected dimension (a volume floor) used by ProjectedLogDetFloor. • q_logdet_eps, p_logdet_eps: Diagonal jitter εI added to the covariance before computing slogdet. • q_logdet_refresh_interval, p_logdet_refresh_interval: How often the random projection matrix is resampled (larger values reduce overhead and noise by reusing projections). • q_pr_norm_floor, p_pr_norm_floor: Optional lower bounds on normalized participation ratio PR/k (encourages higher effective rank). • q_eigmax_frac_ceiling, p_eigmax_frac_ceiling: Optional upper bounds on λmax /tr(Σ) to prevent a single dominant direction. F.11.6

Training (train:) • epochs: Total training epochs. • lr: Base AdamW learning rate for encoder/projector (and predictor in CIFAR MV). • h_lr (ImageNet MV config): Separate learning rate used specifically for the Hamiltonian parameters inside the predictor (predictor.H.parameters()); the ImageNet training script builds a two-group optimizer when these parameters exist. • weight_decay: AdamW weight decay for the main optimizer. • warmup_epochs: Warmup duration used by the linear warmup + cosine decay scheduler (when enabled by the script). • min_lr_ratio: Sets ηmin = lr · min_lr_ratio for cosine annealing. • precision: Mixed precision mode. bf16 enables CUDA autocast bf16 in training. • ckpt_dir: Output directory for checkpoints. • checkpoint_every (ImageNet configs): Save checkpoints every k epochs (ImageNet script only). 50

• grad_clip: If > 0, applies global-norm gradient clipping (clip_grad_norm_). • log_every: Print training diagnostics every log_every steps. • profile_timing, profile_every (ImageNet MV config): Optional lightweight profiling of data time vs step time (only used in the ImageNet script). • MV-HJEPA loss weights: lambda_budget, lambda_var, lambda_logdet, lambda_mean weight the corresponding regularizers in Algorithm 3. • SIGReg baseline weights: lambda_reg is the coefficient on the SIGReg term in Algorithm 11. • h_update_interval, h_grad_clip (SIGReg YAMLs): These knobs are parsed by the training scripts for the learnable Hamiltonian regularizer path (regularizer.type=ham_sigreg with learnable H). In the provided SIGRegonly configs (type=sigreg), there are no H parameters, so these settings are effectively inert.

Record · ID 204778 · SHA-256 a9f9034ad99c76cd
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.