Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning

Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning

arXiv:2608.21147v1 [cs.LG] 21 Aug 2026

Blaise Delaney∗ TimeTrace Labs

Dominic Dootson∗ TimeTrace Labs

Andrew Pfaff TimeTrace Labs

Juan Jose Juan Castella TimeTrace Labs University of Cambridge

Yuji Xing TimeTrace Labs

Jonny Hancox NVIDIA

Salil Patel TimeTrace Labs University of Oxford Karin Sevegnani NVIDIA

Abstract The cyclic structure of physiological processes offers a natural prior for selfsupervised representation learning, and the cardiac cycle provides a particularly well-defined setting in which to exploit it. We derive a phase-equivariant selfsupervised objective and introduce W INDER, a joint-embedding architecture that organises representations into phase-invariant coordinates and phase-rotating harmonic subspaces. Its transport operator is fixed and closed-form, derived from the cycle’s geometry rather than learned, and adds no parameters. Evaluated on PTB-XL under a frozen linear-probe protocol, W INDER attains diagnostic accuracy within the range reported by state-of-the-art self-supervised methods at a ∼1 M parameter footprint, while exhibiting phase-equivariant latent geometry. These findings demonstrate that explicitly encoding cardiac-phase symmetry can preserve diagnostically useful information while yielding a latent geometry that is legible, parameter-efficient, and directly tied to a measurable physiological quantity.

Φ : R → S1 ϕ

t Ri

Time, t ∈ R

Ri+1

Phase, ϕ ∈ S1

Figure 1: Phase reparameterisation of the cardiac cycle. A heart beat between consecutive R peaks is mapped from linear time t ∈ R to phase ϕ ∈ S1 . Identifying ϕ = 0 with ϕ = 2π makes the periodic structure of the heartbeat explicit while preserving within-cycle ordering.

1

Introduction

interpretation depends on the morphology and timing of the P wave, QRS complex, T wave, The 12-lead electrocardiogram (ECG) provides a and intervening intervals [1]. Despite variation rapid, non-invasive measurement of cardiac elec- in rate and morphology, cardiac electrical activtrical activity at millisecond-scale resolution. Its ity follows a recurrent sequence of depolarisation ∗ Equal contribution; joint first authors.

only 1.2 M deployed parameters, while producing phase-ordered latent geometry.

and repolarisation [2]. Mapping successive R–R intervals onto a common phase coordinate preserves this ordering while normalising beat duration, a principle reflected in QT rate correction [3]. Conduction delay and ischaemic deformation can then be viewed as changes to within-cycle timing and morphology, while ectopic activity disrupts the recurring pattern across cycles.

The remainder of this paper is organised as follows. Next, the current state of the art of selfsupervised learning, time-series representation learning, and equivariant deep learning is discussed. Inspired by these ideas, Section 3 derives the required loss function to enforce phaseequivariant representations. In Section 4, the implementation of W INDER in a compact causal architecture is described. Our experiments in Section 5 assess diagnostic performance, cardiacphase equivariance, latent geometry, and the contribution of the transport objective. Finally, the implications and limitations of these findings are discussed in Section 6 before concluding in Section 7.

Continuous physiological recordings capture both patient-state dynamics and sensor-level noise— motion, electrode contact, baseline wander— in a single observed trace. Scored in input space, autoregressive and reconstruction objectives are compelled to reward reproducing the sensor observation in full, noise included. Jointembedding predictive architectures (JEPAs) offer an alternative prescription, whereby predictions are scored against a latent target, training an encoder–predictor system to anticipate a future or masked representation of the signal state rather than the raw sample [4, 5]. Under this paradigm, sensor variation need not be reproduced sample by sample; insofar as the target encoder suppresses variation unnecessary for predicting the latent signal state, representational capacity can concentrate on physiological dynamics.

2

Related Work

W INDER sits at the intersection of self-supervised representation learning and geometric deep learning, using a phase-aware equivariant latent space to capture the cyclic structure of the cardiac cycle in ECG time series. Accordingly, this work draws on three active fields: predictive selfsupervised learning, time-series representation learning at constrained model scale, and equivariant (symmetry-structured) representation learning.

Learning the heart’s recurring structure requires exactly this kind of objective: one that preserves cardiac-cycle timing and morphology without being dominated by nuisance signals. Pointwise reconstruction can overemphasise high-amplitude motion and channel artefacts; the corresponding advantage of latent prediction has been reported for ECG [6] and EEG [7]. The relaxation does not, however, confer invariance to artefacts: it withdraws the explicit reward for reproducing them without penalising their encoding. Standard latent-prediction objectives also leave the organisation of cardiac phase unspecified. A model may therefore encode phase without organising latent motion consistently across beats.

2.1

Predictive Self-Supervised Learning

A central challenge in self-supervised learning is preventing non-contrastive objectives from converging to uninformative representations. Recently, LeJEPA [8] prevented representation collapse by regularising the embedding distribution directly, rather than through heuristics such as stop-gradients or teacher–student networks [10]. LeJEPA pairs a predictive loss with the Sketched Isotropic Gaussian Regulariser (SIGReg), which tests Gaussian isotropy of the embedding distribution along randomly sampled projection directions

In this work, we introduce W INDER, a phaseequivariant extension of LeJEPA [8, 9]. W INDER retains LeJEPA’s regularisation against representational collapse and adds a transport objective that penalises deviations from phase equivariance. This objective encourages latent trajectories to follow cardiac-phase order consistently across beats, grounding the representation in the heart’s physiological cycle.

M

SIGRegT (z) := iid

1 X T (am ; z), M m=1

(1)

am ∼ Unif(SK−1 ), Empirically, enforcing this periodic symmetry improves diagnostic performance over a matched where T (a; z) is a univariate goodness-of-fit transport-free control. W INDER also attains perstatistic applied to the projection of the latent formance in the same broad range as substanembeddings, z ∈ RK , along the unit vector, a. tially larger self-supervised ECG models with To target an isotropic Gaussian distribution, the natural univariate test is the Epps–Pulley (EP) 2

goodness-of-fit statistic [11]: Z 2 EP(a; z) := N |φ ba (τ ) − φ0 (τ )| φ0 (τ ) dτ.

port oscillatory modes [24, 25]. In both cases, cyclicity is predicted or accommodated rather than imposed as a symmetry of the latent space.

(2) The empirical and standard Gaussian characteristic functions are defined as N 2 1 X iτ a⊤ zn , φ0 (τ ) := e−τ /2 . φ ba (τ ) := e N n=1 (3) Equation (1) replaces the maximum over projection directions in LeJEPA’s theoretical criterion with an average, preventing the gradient from concentrating on only the worst-performing direction. Collapse is thus excluded by construction of the SIGReg regulariser rather than by tuning, leaving only a single trade-off hyperparameter that tunes the relative weight between regulariser and predictive loss. LeWorldModel [9] carries this recipe to the temporal setting: next-embedding prediction from pixels under the same SIGReg regularisation.

Other ECG self-supervised methods either predate SIGReg [6] or use world models for intervention simulation [26]. Moreover, a recent scaling study finds contrastive predictive coding modestly ahead of JEPA-style objectives at comparable scale [27]. Thus, neither the appropriate anti-collapse regulariser nor the correct representation of cardiac phase is settled. Section D places W INDER alongside these methods on PTB-XL, with their parameter counts and pre-training corpora, and sets out why no two published entries share an evaluation protocol.

R

2.2

Time-Series Foundation Models: Scale, Adaptation, and Efficiency

Outside this lineage, general-purpose time-series foundation models continue to diversify along three largely independent axes [28]: scale, incontext adaptation, and architectural efficiency. Pretraining now extends into wearable sensing at trillion-minute scale, with SensorFM’s own scaling ablations spanning up to a hundred million parameters and transferring across dozens of downstream health tasks [29]. A parallel direction replaces retraining with adaptation at inference time: Baguan-TS conditions a sequencenative forecaster on a retrieved, covariate-aware context rather than updating any weight [30]. At the other end of the spectrum, with only 0.2M– 2.6M parameters, Reverso [31] closes much of the performance gap to models one to three orders of magnitude larger. It does this through a hybrid architecture alternating gated long convolutions with DeltaNet’s linear-recurrent state updates. W INDER sits at a similar scale to Reversoby imposing structure on the latent space rather than architecturaly innovations.

Since LeJEPA, the recipe’s every component has been revised in turn: the target geometry, rectified toward sparsity [12], or proven suboptimal to a uniform sphere on manifolds [13]; the test statistic [14]; and the constraint’s ambient scope, which Sub-JEPA restricts to random, deliberately frozen subspaces [15]. A parallel wave targets the marginal’s blindness to temporal order: SD-JEPA learns a progression coordinate [16]; letting one emerge from temporally centered residuals [17]; or pools the statistic within-sample across time [18]. Yet to our knowledge the regularised objective remains a set in every case, invariant to permuting the time index. The conditions under which the base recipe’s guarantees hold are only now being characterized, so far confined to a restricted world class of independent, Ornstein–Uhlenbeck latent dynamics under isotropic sampling [19].

Two related time-series architectures, Olivia and RoMAE, also apply rotations to continuous coordinates, but toward different ends than W INDER. Olivia harmonises cross-domain spectral heterogeneity by learning an orthogonal transform of the raw time axis, parameterised, as here (Equation (12)), as a product of Householder reflections [32]; RoMAE instead fixes the rotation, applying rotary position embeddings to continuous positions within a masked-autoencoder objective [33]. In both cases the rotated coordinate is domainCardioState-JEPA is the closest phase-aware agnostic — dataset identity or sequence position ECG method, but treats phase as an auxiliary — and the rotation removes or encodes a nuisance; supervised target within an objective that retains W INDER’s rotation instead acts on a single physan EMA teacher [23]. Sonata models another pe- iologically derived coordinate, cardiac phase, to riodic biosignal—gait and tremor dynamics—by impose a symmetry the latent representation is expanding the recurrence eigenspectrum to sup- required to respect.

SIGReg has recently entered biosignal representation learning, primarily through EEG and broader multi-domain time-series models [7, 20, 21]. Its application to ECG remains limited. LeNEPA regularises temporal embeddings within individual PTB-XL recordings [18], while Beyond Patient Invariance represents pathology as a transition in latent space and reports that SIGReg outperforms VICReg on physiological signals [22]. Neither method imposes a cardiac phase coordinate.

3

2.3

Equivariant Representation Learning for Time Series

dow is equivariant by construction [47]: that is a cyclic shift of the recording, not a phase advance through a physiological cycle. Echo-POSED is equivariant to probe pose and invariant to cardiac phase, so the representation does not track left-ventricular twist [48]. This phase invariance inverts the constraint required here.

Equivariant networks encode a known symmetry as a group action the model is constrained to respect: transforming the input by g transforms the representation by a corresponding Rg . The canonical construction is group-equivariant convolution [34]; Bronstein et al. [35] place this construction in the geometric deep learning catalogue of grids, groups, graphs, geodesics, and gauges. Two symmetries spring to mind for time series: time-translation (sliding the window) and dilation (changing sampling rate). Both have been realised as group-equivariant layers [36, 37], including sampling-rate and discrete-scale forms [38, 39]. A time shift and a phase advance are not, however, the symmetry that defines cardiac cycle: phase is periodic in the beat, not a shift or a stretch of the recording.

To our knowledge, a derived cardiac-phase action on a learned ECG latent remains unoccupied. This work fills this gap.

3

Theory

The recurring physiological organisation of the cardiac cycle motivates treating phase not merely as a timestamp, but as a symmetry that the latent representation should preserve. Given this observation, instead of parameterising the heartbeat as a function of time, t ∈ R, it can be reparame∼ 1 The constructions above constrain the geometry terised as a function of phase, ϕ ∈ [0, 2π) = S , representing the closed-loop cardiac cycle. of the input: the layer is built so that a transformation of x is matched by a transformation of Given detected R-peak timestamps {Ri }, the the features. A second literature constrains the phase map Φ : R → S1 is defined by geometry of the latent space instead, requiring t − Ri , t ∈ [Ri , Ri+1 ). z itself to transform under G. A disentangled t 7−→ ϕ(t) := 2π Ri+1 − Ri representation is defined as a split of z into an in(4) variant factor and a factor that transforms with the This reparameterisation introduces the phasegroup [40]. Unsupervised group-equivariant au- translation group T := {T | ∆ ∈ [0, 2π)}, ∆ toencoders learn that split from a known action on which is isomorphic to U (1). Let X denote a the input, producing invariant and equivariant la- space of phase-parameterised multivariate time tent coordinates [41]. Class–pose decomposition series on which T acts — in this work a 12-lead writes z as an invariant class code times a group ECG signal. For T ∈ T , the action on x ∈ X is ∆ element [42]. The same move appears outside defined by vision: POEM rotates paired latent features of a time-series forecaster by a learned SO(2) phase
(T∆ x)(ϕ) := x (ϕ − ∆) mod 2π . (5) under periodicity drift [43], and PHALAR realises time-shift as a U (1) rotation via the Fourier This definition corresponds to the canonical left shift theorem, in audio [44]. These methods show action of U (1) shifting functions along the cirthat a group action can be imposed on z, but they cular domain. The central idea of W INDER is are developed for images, objects, generic time simple: the latent space should obey the same series, and audio, not for healthcare. symmetry as the cardiac cycle. The following On ECG, physically grounded latent geometry ex- derivation establishes the equivariance condition ists without a group action. LVCG maps 12-lead required for the latent representation to satisfy the recordings to a latent Frank vectorcardiogram and periodic symmetry. trains view-invariant tokens; R-peaks segment beats, but they are not used as a symmetry [45]. 3.1 Phase-Equivariant Latent Space Nef-Net queries an implicit electrocardiac field by viewpoint angle to synthesise new leads [46]. To model latent evolution over cardiac phase, In both cases the geometry is of the source as seen the encoder is required to preserve the phasefrom different electrodes. Invariance to view is translation symmetry of the input. Specifically, not equivariance to cardiac phase: transforming the action T∆ on the input space X should induce the input should move z in a controlled way, not a corresponding action R∆ on the latent space Z, leave it fixed. such that fθ (T∆ x) = R∆ fθ (x), ∆ ∈ [0, 2π). (6) Similar approaches appear in health, but none have imposed phase equivariance on a learned Thus, translating a signal in phase before encodECG representation. CatEquiv uses circular 1- ing is equivalent to translating its latent represenD convolution so that rolling an inertial win- tation after encoding (see Figure 2). 4

x

fθ

T∆ T∆ x

Accordingly, the transport action is block diagonal in the coordinates Q⊤ z:

fθ (x)

R∆ fθ

R∆ = QB∆ Q⊤ , n max M B∆ = IK0 ⊕ (Icn ⊗ ρ(n∆)) .

R∆ fθ (x)

(10a) (10b)

n=1

Figure 2: Commutative diagram of phase equivariance. Encoding after a phase translation T∆ is equivalent to applying the corresponding latent transformation R∆ after encoding, fθ (T∆ x) = R∆ fθ (x).

Here, cn ∈ Z≥0 is the multiplicity of harmonic n: the number of independent two-dimensional feature channels that transform at that rate. The latent dimension is therefore K=

The natural building blocks of the latent group symmetry are the irreducible representations (irreps) of the underlying group, U (1). For convenience, the analysis uses real representations. Under the isomorphism U (1) ∼ = SO(2), each conjugate pair of non-trivial complex characters is represented by a two-dimensional real rotation block. The real irreducible representations are of two types:

K0 |{z}

invariant capacity

+ 2

nX max

cn .

(11)

n=1

| {z }

harmonic capacity

The harmonic orders n determine which phasedependent transformations the latent space can represent, whereas the multiplicities cn determine how much feature capacity is allocated to each order. Figure 4 makes the transformation of an individual latent vector explicit. In the canonical coordinates Q⊤ z, the K0 invariant coordinates re ω = 0, main fixed, while each of the cn two-dimensional  1,   pairs assigned to harmonic n rotates by n∆, so ρω (∆) = cos(ω∆) − sin(ω∆)  , ω > 0. different components of the same latent vector  sin(ω∆) cos(ω∆) advance at different integer multiples of cardiac (7) phase. ! Imposing 2π-periodicity, R2π = IK , gives More generally, the orthogonal basis Q may also be learned. One parameterisation uses a product χω (2π) = 1 ⇐⇒ ω = n, n ∈ Z. (8) of J Householder reflections,

J Y vv ⊤ The zero-frequency character is trivial. Over R, Q = H(vj ), H(v) = I − 2 ⊤ , (12) v v the characters n and −n form equivalent twoj=1 dimensional rotation representations, so only the canonical representatives n ∈ Z>0 are retained. which guarantees Q⊤ Q = I throughout training [49]. The selected architecture in Section 4 inBecause U (1) is compact, averaging an inner stead fixes the canonical basis, so its transport product over the group yields an invariant inner operator has no learnable parameters. product. Consequently, every finite-dimensional continuous real representation is equivalent, after a change of basis, to an orthogonal representa- 3.2 Learning Equivariant Transport tion. Because the encoder’s latent coordinates are learned, their axes need not align with the The W INDER objective combines latent predicinvariant subspaces on which the irreducible rep- tion, phase-equivariant transport, and spectral regresentations act. An orthogonal change of basis ularisation: Q ∈ O(K) is therefore introduced, with Q⊤ z ex- LWinder = Lpred + λSIG LSIG + λtrans Ltrans . pressing the learned latent in the canonical irrep (13) basis. Here, the blue terms are inherited from LeJEPA: In this canonical basis, the group action decom- Lpred trains the latent predictor by requiring the poses into a trivial component and harmonic iso- latent magnitude and content to support nexttypic components, so the latent space admits the token prediction, while LSIG prevents dimensional collapse by controlling the full latent disdirect-sum decomposition tribution. The orange term, Ltrans , is specific to W INDER and enforces phase equivariance by conn max M
K ∼ K0 cn 2 straining how the unit-normalised latent direction R =R ⊕ R ⊗R . (9) changes with cardiac phase. n=1

5

Rπ/2

Rπ/2

Rπ/2

Rπ/2

QT z

Figure 3: Fourier decomposition of a phase-equivariant latent trajectory. Local Cartesian panels show the cumulative sum of the first three Fourier components at four cardinal phases. Dashed circles indicate component magnitudes, blue segments show the component-wise construction, orange arrows show the resultant latent vectors, and curved Rπ/2 arrows denote quarter-cycle group actions.



 I  K0  0    0   .  ..   0

0 Ic1 ⊗ ρ(∆) 0 .. . 0

3.2.2

 ··· 0   ··· 0   . .. .  . Ic2 ⊗ ρ(2∆) .   .. ..  . . 0  ··· 0 Icnmax ⊗ ρ(nmax ∆) 0 0

Spectral Regularisation

The prediction and transport objectives do not prevent the encoder from collapsing onto a lowdimensional representation. SIGReg therefore constrains the latent distribution towards a reference distribution p0 .

Figure 4: Canonical block structure of the transport operator B∆ . The IK0 block (orange) is fixed under transport; the harmonic blocks (teal) rotate by n∆ via ρ(n∆), each acting on cn independent two-dimensional channels.

SIGReg follows LeJEPA’s standard randomprojection prescription. Rather than comparing distributions directly in the K-dimensional latent space, SIGReg compares their one-dimensional sketches along random unit directions am ∈ SK−1 . Matching random one-dimensional projec3.2.1 Latent Prediction tions provides a scalable surrogate for matching Given a context ending at token t, the causal pre- the full latent distribution. In practice, the intedictor gψ maps the projected latent prefix z≤t to gral is evaluated by numerical quadrature and the a prediction of the next latent, projection directions are resampled throughout training. ẑt+1 := gψ (z≤t )t+1 . (14) Once a distribution can be targeted, the remaining The prediction loss is the mean-squared error be- question is which distribution to use. To avoid tween the predictor output ẑ and the projection competing with the transport objective, the refhead’s own output z, averaged over latent dimen- erence distribution must be invariant under the sions and target positions. latent group action. A target is called admissible 6

ECG structure

Physiological process

Typical duration [ms]

Characteristic order

P wave PR interval QRS complex ST segment T wave QT interval R–R interval

Atrial depolarisation Atrial-to-ventricular conduction Ventricular depolarisation Ventricular plateau phase Ventricular repolarisation Total ventricular electrical activity Complete cardiac cycle

80–110 120–200 70–110 80–120 100–250 350–450 600–1000

n = 10–13 n = 5–9 n = 10–15 n = 9–13 n = 4–10 n=3 n=1

Table 1: Characteristic timescales of ECG structure. Physiological durations are representative adult values [2]. Characteristic orders follow Equation (20), using T = 1000 ms; sharper morphology may require higher orders.

latent space. Second, because the isotropic target assigns variance to every latent coordinate, it penalises concentration in the invariant block and encourages the non-trivial harmonic irreps to remain populated. SIGReg does not determine what those harmonic coordinates encode; the transport For a centred Gaussian target, p0 = N (0, Σ), a objective supplies the phase-dependent structure. linear transformation gives
⊤ z ∼ N (0, Σ) =⇒ R∆ z ∼ N 0, R∆ ΣR∆ . 3.2.3 Phase-Equivariant Transport if

(R∆ )# p0 = p0 , ∀∆, (15) where (R∆ )# p0 denotes the pushforward of p0 under R∆ , namely the distribution obtained by applying R∆ to samples from p0 .

For two tokens t and t′ from the same recording, let ∆tt′ := (ϕ(t′ ) − ϕ(t)) mod 2π and ⊤ ! R∆ ΣR∆ = Σ, ∀∆. (16) ẑt := zt /∥zt ∥2 . Because R∆ ′ is orthogonal, the tt In the canonical basis, the most general symmet- squared directional equivariance defect satisfies ric admissible covariance is 2 " δtt′ := R∆tt′ ẑt − ẑt′ 2   n max M (18)
1 0 Q⊤ ΣQ = S0 ⊕ An ⊗ = 2 1 − R∆tt′ ẑt , ẑt′ . 0 1 n=1  # (17) The empirical transport loss is therefore 0 −1   + Cn ⊗ . 1 0 Ltrans = E(t,t′ ) 1 − R∆tt′ ẑt , ẑt′ . (19)

Admissibility therefore requires

The An term represents covariance between corresponding coordinates of different copies of harmonic n. The Cn term represents covariance between orthogonal coordinates—one copy’s cosine-like coordinate and another’s sinelike coordinate—corresponding to a phase offset of π/2. Here,

The transport loss enforces equivariance of unitnormalised latent directions, while latent magnitudes are controlled by the prediction and spectral objectives.

Different members of this admissible family can encode covariance structures motivated by the underlying mechanism or quantity of interest. For simplicity, however, the LeJEPA choice Σ = IK is adopted, giving p0 = N (0, IK ), which is invariant under every orthogonal transformation.

3.3

This directional objective has a phase-blind failure mode: representations can concentrate in the invariant block, on which R∆ acts as the identity. ⊤ Such representations may satisfy the transport S0 = S0⊤ , An = A⊤ , C = −C , n n n constraint without using the non-trivial harmonic and each covariance block is required to be pos- irreps to encode cardiac phase. The transport obitive definite. When cn = 1, skew symmetry jective alone therefore does not distinguish useful forces Cn = 0, reducing the harmonic block to phase equivariance from trivial phase invariance. σn2 I2 . Bounding the Harmonic Spectrum

The quantisation condition in Equation (8) determines that the admissible harmonic orders are integers. The maximum retained order, nmax , is bounded from below by the clinical timescales needed to be resolved and from above by the resSIGReg consequently supplies two compleolution of the sensor used to estimate phase. mentary anti-collapse pressures. First, it penalises concentration of z in a constant or low- Let δt denote the shortest clinically relevant dimensional subspace, forcing latents from dif- timescale within a cardiac cycle of duration T . ferent sources to occupy distinct regions of the Since harmonic n varies on a timescale propor7

n=1 n=2

half height

n=3 n=4 n=5 n=6 δt n=7

Time

Time

(a)

(b)

Figure 5: Localisation from a gapped harmonic band. (a) Harmonics n = 1, . . . , 7, with the retained blocks {1, 4, 7} shown in orange. (b) Their normalised sum over a cardiac cycle. The shaded full width at half maximum, δt, denotes the temporal scale that this harmonic band can resolve, while the gaps leave pronounced side lobes.

tional to T /n, resolving δt requires

partition held out from the sealed evaluation (Section 4).

T . (20) δt To ground this estimate physiologically, Table 1 relates standard ECG structures to their characteristic Fourier orders. The practical upper bound on nmax is set by the Nyquist limit of the ECG sampling rate. At clinical ECG sampling rates the instrumental ceiling lies well above the orders in Table 1, so the clinical requirement is ordinarily the active constraint. This means that the number of harmonics is specified by the data rather than found by searching over model capacity. nmax ≳

4

Method

4.1

Dataset

The PTB-XL dataset [50, 51] was selected because its acquisition and annotation structure match both the mechanism under study and its downstream evaluation. It provides fixed-length, 10-second, 12-lead clinical ECGs at 500 Hz, giving sufficient temporal resolution to construct a cardiac-phase coordinate from R-peak timing. Its patient-aware stratified folds and hierarchical diagnostic labels provide a reproducible route from self-supervised pre-training to diagnostic evaluation. PTB-XL therefore provides an established ECG dataset whose structure directly supports the phase-indexed latent space learning problem.

The highest retained harmonic and the feature capacity assigned to each harmonic are separate design choices. For a contiguous spectrum n = 1, . . . , nmax with one copy of each harmonic, every non-zero harmonic requires a twodimensional real rotation block. The resulting minimum latent dimension is

(21) Version 1.0.3 of the corpus comprises 21,799 recordings from 18,869 patients [52]: clinical Additional copies, cn > 1, provide more inde- resting ECGs, each 10 s long and sampled at pendent feature channels at harmonic n without 500 Hz across the 12 standard leads. Each recordincreasing the highest supported frequency. Con- ing carries one or more of 71 SCP-ECG1 stateversely, omitting intermediate harmonics reduces ments, assigned by cardiologists and spanning the latent dimension while preserving the scale diagnostic, form and rhythm categories. associated with the highest retained harmonic, but produces less uniform localisation and stronger Classification of the PTB-XL diagnostic subside lobes. Figure 5 illustrates this trade-off for classes provides the primary evaluation of the learned latent representations. The “23-subclass” the band {1, 4, 7}. metric averages performance across the 17 classes In practice, nmax is chosen from the required 1‘Standard Communications Protocol for Computerphysiological resolution and acquisition rate, while K0 and {cn } are selected on a validation Assisted Electrocardiography’. Kmin = K0 + 2nmax .

8

with at least 20 positive fold-9 examples. The 4.2 Model architecture five-group macro-average follows PTB-XL’s published diagnostic hierarchy, which maps these labels into normal ECG, myocardial infarction, We implement W INDER as a causal jointST/T change, conduction disturbance, and hyperembedding predictive architecture for 10-second, trophy superclasses. 12-lead ECGs. Each recording is represented by The corpus is used in two stages, separated so that 125 non-overlapping tokens, each corresponding nothing reported on the sealed fold was chosen to 80 ms of signal sampled at 100 Hz. The distinwith reference to it. In the development stage, guishing component is not the JEPA framework models are fitted on folds 1–8 and scored on fold itself, but the fixed phase-indexed transport opera9. That stage fixes the design choices by grid tor that constrains the projected representation to search: the number of encoder layers, the number transform according to cardiac phase (cf. Equaof predictor layers, and the loss-weight prefac- tion (10b)). Figure 6 shows the complete JEPA artors of Equation (13). It also fixes the isotypic chitecture, distinguishing the deployed path from capacity allocation, meaning the split of the 256- components used only during training. dimensional latent budget between the invariant The encoder partitions each input into 125 noncapacity K0 and the harmonic multiplicities {cn } overlapping eight-sample patches. Each 12 × 8 of Equation (11). The highest retained harmonic patch is flattened to 96 values and passed through nmax is not tuned in this stage; it is set in advance the same per-patch MLP, with dimensions 96 → from the physiological resolution requirement of 512 → 256 and a GELU nonlinearity. The reSection 3.3. Fold 9 is the only partition consulted sulting 256-dimensional tokens pass through two in making any of these choices, and it is likewise residual convolutional blocks with kernel size where the eligible subclass list is defined. 3 and dilations 1 and 2 respectively. Left-only In the confirmatory stage, those settings are padding preserves causality. Across the four confrozen and the model is pre-trained on the patient- volutions, each output depends on the current disjoint folds 1–9 (19,601 recordings; 16,965 pa- token and the preceding 12 tokens, giving a 0.96 s tients), over which the per-lead normalisation look-back. statistics are also fitted. A linear probe on the Within each context block, the residual branch frozen representation is fitted over the same folds applies channel-wise LayerNorm, GELU, and and evaluated on fold 10 (2,198 recordings; 1,904 causal convolution twice beneath a single idenpatients), which remained sealed until unblindtity connection. No normalisation or activation ing; results reported on it are this paper’s headline follows the residual addition, and the projector evaluation. The two partitions agree to within output is likewise left unnormalised. This allows a percentage point on age, sex and every class SIGReg to control the scale and moments of the prevalence (Section A), which is what PTB-XL’s projected distribution, which a terminal Layerstratification should deliver and what was verified Norm would otherwise fix by construction. rather than assumed. A shared projector with dimensions 256 → Recordings are read at the native 500 Hz and deci512 → 256 and a GELU nonlinearity indepenmated locally to 100 Hz by polyphase FIR resamdently maps each of the 125 contextual encoder pling, giving 1,000 samples per lead. The whole tokens to zt ∈ R256 , without mixing informarecording is retained as a single training example. tion across time. These tokens support the selfEach lead is then z-scored against per-lead corpus supervised training losses. Together, the encoder statistics fitted once over folds 1–9. and projector form the deployed representation The encoder receives the resampled 100 Hz signal, path and contain 1,232,384 parameters. For downwhereas R-peaks are detected once on the native stream evaluation, tokens with defined cardiac 500 Hz recordings with a Pan–Tompkins-style phase are mean-pooled without weighting to form detector [53]. Cardiac phase is then estimated by a record-level representation. linear interpolation between consecutive peaks, The selected transport operator instantiates the as in Equation (4). Each 80 ms token is assigned block decomposition of Equation (10b) in the the phase at the centre of its patch. 256-dimensional projected space. The invariSamples before the first detected peak and at or ant capacity is K0 = 4, and the remainafter the final detected peak have undefined phase, ing 252 dimensions carry harmonics n = and are excluded from phase-dependent analyses. 1, . . . , 10 with multiplicities (c1 , . . . , c10 ) = Section A states the full preprocessing chain and (24, 24, 20, 16, 12, 10, 8, 6, 4, 2), each unit of cn contributing one two-dimensional block, so that the fitted statistics. P 2 n cn = 252. Each coordinate pair in harmonic n undergoes a closed-form rotation by n∆. 9

(a) Representation path Input: 12-lead ECG 10 s, 12 × 1000 at 100 Hz

(b) Context block A encoder: 969,472 parameters

Patchify no parameters 125 patches of 80 ms → 125 × 96

x: 125 × 256 tokens

Patch MLP Linear 96→512, GELU, 512→256 180,992 parameters

CLN1 channel LayerNorm, 512 GELU

Context block A 2× causal Conv1D, k = 3, d = 1 394,240 parameters

Causal Conv1D 256→256, k = 3, d = 1 left-only pad 2, no bias, 196,608

Context block B 2× causal Conv1D, k = 3, d = 2 394,240 parameters

CLN2 channel LayerNorm, 512 GELU

Projector Linear 256→512, GELU, 512→256 262,912 parameters

Causal Conv1D 256→256, k = 3, d = 1 left-only pad 2, no bias, 196,608

Projected tokens 125 × 256 Deployed: 1,232,384 parameters

+ output = x + v: 125 × 256, no terminal norm

training only, discarded

2 × 196,608 + 2 × 512 = 394,240 Predictor 4× causal Transformer blocks pre-LayerNorm, width 256, 4 heads

Block B repeats this anatomy with d = 2 and left-only pad 4; together the two blocks look back 12 tokens (0.96 s).

FFN 1,024, 3,227,084 parameters

Figure 6: W INDER architecture. (a) The encoder maps 80 ms of 12-lead ECG to a 256-dimensional latent space. Fixed patchification, a patch MLP, and two causal context blocks form the 969,472-parameter encoder. The projector produces 256-dimensional latent tokens from the encoder’s output. Encoder and projector together form the 1,232,384-parameter deployed path. During self-supervised training the projected tokens also enter the 3,227,084-parameter pre-LayerNorm causal Transformer predictor, which is discarded at deployment. (b) Context block A repeats layer normalisation, GELU, and causal convolution twice under one identity skip, with no normalisation after the residual addition. Left-only padding in the convolutions ensures causality.

The operator is fixed and has zero learnable pa- ation, the mean-pooled representation is passed rameters. to a linear probe fitted separately for each task. This probe is an evaluation readout, not part of the During self-supervised training, the projected pretrained model, and its weights are excluded tokens also enter a causal Transformer predicfrom the reported parameter counts. tor comprising four pre-LayerNorm blocks, four attention heads, width 256, and feed-forward width 1,024. It uses learned relative-position bi- 4.3 Training ases. The predictor adds 3,227,084 parameters, increasing the optimised total from the deployed Self-supervised training minimises the three-term 1,232,384 parameters to 4,459,468. It is discarded objective of Equation (13), with all three terms atafter pre-training and contributes neither to de- tached to the projector output z. The live weights ployment nor to the reported diagnostic probes. are λSIG = 0.15 and λtrans = 1. The nominal The masking and prediction-loss mechanics are loss weights were selected through a coarse hyperparameter sweep. In the control condition, the described in Section 4.3. transport loss is disabled by setting λtrans = 0. Latent-geometry analyses use the pre-projection encoder output to test whether phase structure For each record, the prediction loss applies only is learned by the encoder itself. Training losses to the token after a sampled causal cutoff (Secand diagnostic probes instead use the projected tion 3.2.1). The cutoff is drawn independently tokens zt , because these are the representations per record, uniformly over admissible token indirectly constrained by the objectives and used dices, and redrawn every batch. Inputs after the for downstream evaluation. For diagnostic evalu- cutoff are replaced by a learned mask token, so the output at c + 1 depends only on the visible 10

latent prefix and is compared with zc+1 . The evaluation protocol of the SIGReg loss is given in Section C.1. The transport loss, Ltrans , is averaged over all valid token pairs within each record, and then across records. Using all valid token pairs avoids the phase-dependent sampling bias observed when only a subset of pairs was selected.

two seeds, three training steps, and two arms. Across that grid, the seed-averaged superclass gap ranges from +0.0774 to +0.0910. None of the six matched cross-arm confidence-interval pairs overlaps. The two seeds agree: the largest between-seed spread in any W INDER superclass cell is 0.0054, well inside the confidence intervals. These results provide within-study evidence from one sealed PTB-XL fold. They do not establish Training uses AdamW with a short linear generalisation to external datasets or a directly learning-rate warm-up followed by cosine decomparable leaderboard ranking. cay. The optimiser and schedule are fixed across all runs. Their complete specification, including batch size and training duration, is given in Sec- 5.1 Diagnostic accuracy. tion C.3. Each frozen representation is evaluated with the During training, each of six signal augmenta- same linear-probe protocol, with uncertainty estitions is applied independently with probability mated by resampling patients rather than records 50 %. Gaussian noise and a 25 Hz sinusoid sim- over 1,000 bootstrap replicates. Sealed fold 10 ulate broadband and narrow-band measurement holds 2,198 records, of which 2,158 carry a dinoise, respectively. Low-frequency drift models agnostic superclass. The probe scores 2,140 baseline wander, while slow amplitude modula- of these; 18 are excluded because they lack tion represents gradual changes in signal strength. a valid phase assignment after quality control. Lead dropout simulates missing channels, and At the selected cell, W INDER reaches superper-lead gain scaling varies the relative ampli- class macro-AUROC 0.8702+0.0092 −0.0095 , compared tudes between leads. Full augmentation details with 0.7826+0.0115 for the control. The cor−0.0115 are reported in Section C.2. responding 17-class scores are 0.8421+0.0105 −0.0120 Four pre-training runs were conducted: a and 0.7568+0.0144 ; the 23-class scores are −0.0161 transport-enabled arm and a control arm, each us0.8236+0.0220 and 0.7415+0.0284 −0.0197 −0.0289 . Phase transing two random seeds. The complete run design, port therefore makes diagnostic structure more checkpoint schedule, and hardware specification linearly accessible on the sealed fold. This perforare given in Section C.4. mance claim is specific to a frozen linear readout. The post-hoc heart-rate stratification reported below weakens, but does not exclude, an explana5 Results tion of this gap based on heart rate. No published result shares this protocol exactly, so Section D Published ECG representation-learning results places W INDER within the range reported by selfvary in their pre-training data, parameter scale, supervised models on PTB-XL, at a smaller painput windowing, and downstream readout. A rameter and pre-training footprint, rather than matched ablation therefore isolates the contribu- ranking it against them. tion of phase transport. The two arms of the ablation share their data, architecture, random seeds, 5.2 Mechanism validity. training schedule, and evaluation protocol. They differ only in the transport objective: W INDER For tokens at t and t′ , define the transported and sets λtrans = 1.0, whereas the control disables untransported cosine similarities as phase transport by setting λtrans = 0.0. This cR := R∆tt′ ẑt , ẑt′ , comparison evaluates the effect of phase transport (22) on downstream performance, use of the declared cI := ⟨ẑt , ẑt′ ⟩ . phase action, and any associated robustness gains or failure modes. Because phase transport acts The raw pairwise gain is gtt′ = cR − cI . Its mean only during training, both arms have identical and the reported gain fraction are, respectively, inference paths. Ett′ [cR − cI ] ḡ := Ett′ [cR − cI ], gf := . The main rows of Table 2 report seed 0 at Ett′ [1 − cI ] (23) step 5,000 on sealed fold 10. The training step was selected after the results were known: Thus, the gain fraction normalises the mean gain step 5,000 has the largest seed-averaged super- by the available improvement from the untransclass gap and the smallest seed-averaged lead- ported similarity to unit similarity. The predropout separation. The pre-registered evalu- registered G1 gate compares this quantity unation grid comprised all 12 combinations of der the true cardiac clock with the result after 11

Metric

W INDER (λtrans =1.0)

Control (λtrans =0.0)

Diagnostic accuracy Superclass macro-AUROC, 5 classes ↑ Subclass macro-AUROC, 17 classes ↑ Subclass macro-AUROC, 23 classes ↑

0.8702+0.0092 −0.0095 0.8421+0.0105 −0.0120 0.8236+0.0220 −0.0197

0.7826+0.0115 −0.0115 0.7568+0.0144 −0.0161 0.7415+0.0284 −0.0289

Mechanism validity Transport gain fraction ↑ G1 shuffled-ϕ gate Paired true − shuffled gain ↑

0.8786 Pass 0.8154+0.0159 −0.0177

−0.0744 Fail 0.0325+0.0023 −0.0024

Robustness to degradation Mean signed ∆ macro-AUROC, single-lead −0.0034 dropout ↑ Worst signed ∆ macro-AUROC across single-lead −0.0089 (aVR) dropouts ↑ Latent geometry Fundamental loop amplitude at σ = 0 ↑ 1.8941 Fundamental-loop coherence (descriptive), mean 0.9927 over the 7 non-reference bins ↑

−0.0107 −0.0218 (III)

0.3817 0.3159

Anomaly detection (strongest of six families; one fixed detector, maximum severity) Ectopic-beat AUROC, severity 2.0 ↑ 0.8662 0.6198 Excess over own zero-severity reference ↑ +0.2758 +0.1393 Localisation hit rate ↑ 0.779 0.025 Median latency among alarmed records ↓ 80 ms 720 ms Table 2: W INDER (λtrans = 1.0) vs. control (λtrans = 0.0). Phase transport improves diagnostic accuracy, latent organisation, and lead-dropout robustness at identical inference cost. 95 % CIs are patient-clustered bootstraps. Intervals are 1,000-replicate percentile bounds; remaining values are point estimates. Arrows give the preferred direction; bold marks the better arm where the comparison is non-degenerate. Provenance, protocol, and the caveats attaching to each block are given in the text.

shuffling the phase labels. It requires the paired true-minus-shuffled interval to exclude zero and the shuffled gain fraction to satisfy |gf | ≤ 0.02. W INDER achieves mean gain ḡ = 0.8182 and gain fraction gf = 0.8786, and passes G1; the control’s gain fraction is −0.0744 and it fails.

arm’s own intact score. The mean signed change is −0.0034 for W INDER and −0.0107 for the control, a 3.1× ratio at step 5,000. The worst signed changes are −0.0089 after removing aVR and −0.0218 after removing lead III, respectively. The seed-averaged mean-degradation ratio widens to 11.3× by step 30,000. These robustThe paired true-minus-shuffled gains are ness summaries are descriptive point estimates. +0.0023 0.8154+0.0159 −0.0177 for W INDER and 0.0325−0.0024 for the control. The control interval also excludes zero, so the shuffled-magnitude condition 5.4 Latent geometry. carries the arm discrimination: its shuffled Two complementary metrics quantify phase gain fraction is −0.1104, outside the registered organisation in latent space. Fundamentaltolerance. The gain fraction itself is a point harmonic Amplitude measures how strongly toestimate; the available interval brackets mean ken representations align after removing their cargain (0.8182+0.0154 −0.0173 for W INDER ) and cannot be diac phase, while Orbit-subspace Coherence meaattached to the fraction. This test establishes that sures whether phase-bin means remain within a the constrained representation uses the declared common rotational subspace. cardiac clock; it does not by itself explain the For a token t with cardiac phase ϕt , the dephased diagnostic gain. latent is defined as 5.3

zet := R−ϕt zt .

Single-lead robustness.

(24)

This counter-rotation brings phase-locked comEach lead is zeroed independently at evaluation ponents to the common reference phase ϕ = 0. while the probe remains fitted on intact data. To calculate the Fundamental-harmonic AmpliThe resulting AUROC is compared with that tude, first average the dephased latents per record. 12

WINDER, seed 0

WINDER , seed 1

Control, seed 0

Control, seed 1

10.0

Sb (harmonic nj =1)

7.5 5.0 2.5 0.0 −2.5 −5.0 −7.5

0

−5

5

0

−5

5

0

−5

5

0

−5

Cb

Cb

Cb

Cb

RMS radius 7.10

RMS radius 6.89

RMS radius 1.02

RMS radius 1.03

5

Sb / RMS radius

2 1 0 −1 −2 −2

−1

0

1

2

−2

Cb / RMS radius 0

−1

0

1

2

−2

Cb / RMS radius

−1

0

1

2

−2

Cb / RMS radius ¼

¼=2

−1

0

1

2

Cb / RMS radius 3¼=2

Cardiac phase ' at bin centre (rad), 24 bins

Figure 7: Latent cardiac-phase organisation with and without transport. Visualisation of the cosine and sine overlaps defined in Equation (27). Tokens are grouped into 24 cardiac-phase bins; each point plots (Cb , Sb ) for the mean n = 1 representation of one bin relative to the reference bin. Each column shows one run: both W INDER seeds and both control seeds. Colour encodes ϕ cyclically, so the first and last bins are neighbours in colour as well as in phase. Top row: shared axis limits, so loop size is comparable across panels. Bottom row: each panel normalised by its own RMS radius, so loop shape is comparable. To visualise the same overlap construction at finer phase resolution, tokens from folds 1–9 are grouped into 24 cardiac-phase bins; the sealed-fold coherence estimate uses eight bins.

Then, compute the mean Euclidean norm of the 24 two-dimensional n = 1 components. The resulting amplitude is 1.8941 for W INDER and 0.3817 for the control, indicating a roughly fivefold stronger phase-locked fundamental component in W INDER. Orbit-subspace coherence is computed from the original, non-dephased token representations. Tokens are grouped into eight cardiac-phase bins, and the mean n = 1 representation vb ∈ R2c1 is calculated for each bin. Let v0 u := (25) ∥v0 ∥2

If all c1 blocks undergo one common phase rotation, these overlaps vary as cos ∆b and sin ∆b . Their combined magnitude therefore defines the coherence p Cb2 + Sb2 κb := ∈ [0, 1]. (28) ∥vb ∥2 A value of one means that the phase-bin mean lies entirely within the rotational plane spanned by u and Ju; lower values indicate components outside this common plane. Mean coherence across the seven non-reference bins is 0.9927 for W INDER and 0.3159 for the control. This metric measures whether the c1 blocks rotate coherently, but not whether the phase bins occur in the correct order around the loop.

denote the normalised mean of the reference bin, and let   The trajectory in Figure 7 complements this met0 −1 J := Ic1 ⊗ (26) ric by showing that the bins also follow their 1 0 cardiac-phase order around the loop. Tokens apply a simultaneous quarter-turn to all c1 two- are grouped into 24 cardiac-phase bins, and each dimensional blocks. The cosine and sine overlaps bin’s mean representation is projected onto the two-dimensional subspace in which the rotation of bin b with the reference are acts at the fundamental harmonic nf =1; the Cb := ⟨u, vb ⟩, Sb := ⟨Ju, vb ⟩. (27) four columns are the two W INDER seeds and 13

2¼

WINDER

Control

UMAP-1

UMAP-1

3¼=2

P Wave

¼

Diastole

¼=2

T Wave

0

QRS

Cardiac phase ' (rad)

UMAP-2

UMAP-2

2¼

Figure 8: UMAP visualisation of latent cardiac-phase organisation. Token-level representations at seed 0 are shown separately for W INDER and the control. UMAP receives no cardiac-phase or diagnostic information, points are coloured post hoc by cardiac phase ϕ.

the two control seeds of Section C.4. In both W INDER runs the phase-binned trajectory walks once around a closed circle in bin order; in both control runs it collapses to a small path that neither closes nor preserves the ordering. The figure and the table quantify related but distinct properties on different splits: the RMS loop radius drawn from folds 1–9 separates the arms by roughly a factor of seven, whereas the sealed-fold amplitude separates them by a factor of five. The figure therefore locates the mechanism, while the sealed-fold quantities measure it.

reference — the same perturbation applied at zero amplitude — because these references differ between arms. For ectopic beats at severity 2.0, W INDER reaches AUROC 0.8662, compared with 0.6198 for the control. The referencecorrected excesses are +0.2758 and +0.1393, and localisation hit rates are 0.779 and 0.025. This is the battery’s strongest detection result.

Latency is interpretable only when the detector first clears its own zero-severity reference. W INDER raises an alarm on 92.0 % of ectopicbeat records, whereas the control alarms on Phase organisation is also visible without the har- 14.6 %, so the control’s 720 ms median is conmonic construction. Figure 8 embeds token-level ditioned on a small, unrepresentative subset. The representations with a general-purpose non-linear corresponding W INDER median is 80 ms. projection that receives no phase, arm or diagThat median is exactly one token. The ennosis information, and colours the result by ϕ coder partitions each recording into 125 nonafterwards. Each point is one token, an 80 ms overlapping 80 ms patches (Section 4), and the slice of one recording, so colour organisation is detector emits one score per token, so an alarm a property of the representation rather than of can only be timestamped to a patch boundary and the projection: W INDER’s tokens form a phaseno reported latency can fall below a single patch. ordered annulus and the control’s do not. Both The patching we elect therefore sets the floor on figures are encoded from folds 1–9 at the reported this measurement: 80 ms is the finest delay the evaluation cell and at the training seeds of Sectoken grid can express, not a measured bound on tion C.4, because the sealed fold did not persist how quickly the representation responds to onset. per-record latents; the sealed-fold amplitude and Resolving faster detection would require a shorter coherence values are those in Table 2. The empatch, and hence more tokens per record. A difbedding is a qualitative illustration the geometry ferent admissible detector for seed 0 on the same claim. causal clock reaches 0.916 on this family. Selecting it per cell would have added roughly +0.05 and was excluded by the registered protocol. 5.5 Anomaly detection. The step 5,000 seed-averaged results for the reOne causal detector was fixed before unblind- maining families bound the detection claim. Ting and applied unchanged across six synthetic wave inversion gives a weak reference-corrected perturbation families. Each maximum-severity excess of +0.1031 for W INDER against +0.0250 result is read against that cell’s own zero-severity 14

for the control. Amplitude attenuation and lead dropout are effectively undetected, at +0.0138 and +0.0115 for W INDER. Baseline wander reverses after reference correction, with the control responding more strongly (+0.2242 against +0.1240). Together with the ST-shift result, these measurements support no arm-discriminating claim for the other four families.

6

Discussion

W INDER uses a soft physiological prior: the transport loss encourages, but does not force, representations to vary predictably with cardiac phase. The expected phase action R∆ is fixed analytically as a set of harmonic rotations. Integer harmonics guarantee that a complete cardiac cycle returns the latent coordinates to their starting point, R2π = I. Transport gain and latentgeometry measurements then determine whether the learned representation actually follows this prescribed action.

More broadly, cardiac phase is one instance of a general construction in which a measured cyclic coordinate defines a fixed harmonic action on latent space, allowing equivariant structure to be introduced without learning the symmetry itself.

7

Conclusions

W INDER introduces a phase-equivariant objective for physiological representation learning. It represents cardiac phase through a fixed, closedform action on the latent space, whose invariant and phase-rotating harmonic subspaces are specified without learnable transport parameters. The transport objective encourages the learned representations to follow this prescribed action, and the resulting organisation can be measured through transport gain and latent-geometry diagnostics. Notably, because the construction requires only a measured cyclic coordinate, W INDER opens a broader class of phase-equivariant world models for cyclic biosignals, in which physiological structure is injected analytically rather than purchased through parameter scale.

The matched comparison isolates the effect of phase transport; the control differs only by disabling the transport loss. Phase transport improves diagnostic accessibility while strengthen- Acknowledgments and Disclosure of ing robustness to lead loss. The result is therefore Funding not merely a better readout, but a performance gain accompanied by evidence that the intended We thank Thorir Mar Ingolfsson for helpful discardiac-phase action organises the representation. cussions in the preparation of this work. The This declared geometry also provides a moni- authors acknowledge the use of resources protorable contract for a compact causal encoder. vided by the Isambard 3 Tier-2 HPC Facility. Phase organisation and transport violations can Isambard 3 is hosted by the University of Bristol be quantified directly from the latent representa- and operated by the GW4 Alliance and is funded tions using metrics defined by the fixed harmonic by UK Research and Innovation; and the Engiaction. The anomaly results show the promise of neering and Physical Sciences Research Council this view for continuous monitoring at the point [EP/X039137/1]. This work was supported by Inof acquisition, including ambulatory telemetry novate UK, part of UK Research and Innovation and bedside rhythm surveillance. These results (UKRI), under grant number 10201397. suggest that phase-aware anomaly scores may support continuous monitoring, but they were References evaluated only on synthetic perturbations and do not establish performance in a deployed clinical [1] Ary L. Goldberger, Zachary D. Goldberger, system. and Alexei Shvilkin. Goldberger’s Clinical Electrocardiography: A Simplified ApThis work adopts a single corpus and a delibproach. Elsevier, 10 edition, 2024. ISBN erately simple encoder backbone to give promi978-0-323-82475-0. nence to the method and stress-test the phasetransport mechanism in isolation. The evidence is [2] Arthur C. Guyton and John E. Hall. Textconsequently a controlled within-study comparibook of Medical Physiology. Elsevier, 2006. son under a frozen linear readout, not a commonprotocol leaderboard. Having established benefits [3] Lucas Plagwitz, Lucas Bickmann, Michael against the control, future work should evaluate Fujarski, Alexander Brenner, Warnes Gobalbroader corpora and stronger backbones, examine akrishnan, Lars Eckardt, Antonius Büscher, which physiological, morphological, and diagnosand Julian Varghese. The rlign algorithm tic features are encoded by the invariant and harfor enhanced electrocardiogram analysis monic latent subspaces, and extend single-step through heart rate–corrected ECG alignlatent prediction to multi-step forecasts. ment for explainable classification and clus15

tering. European Heart Journal – Digital Health, 7(5):ztag067, 2026. ISSN 26343916. doi: 10.1093/ehjdh/ztag067. [4] Yann LeCun. A path towards autonomous machine intelligence. OpenReview, 2022. URL https://openreview. net/forum?id=BZ5a1r-kVsf. Version 0.9.2, 2022-06-27.

arXiv:2602.01456; PMLR volume/pages not yet posted. [13] Léo Nicollier, Max Dunitz, Marc Pic, Pablo Musé, Enric Meinhardt-Llopis, and Gabriele Facciolo. SPHERE-JEPA: Spherical prediction with homogeneous embeddings, 2026. arXiv preprint arXiv:2605.26900.

[5] Mahmoud Assran, Quentin Duval, Ishan [14] Haiyu Wu, Randall Balestriero, and MorMisra, Piotr Bojanowski, Pascal Vincent, gan Levine. VISReg: Variance-invarianceMichael Rabbat, Yann LeCun, and Nicolas sketching regularization for JEPA training, Ballas. Self-supervised learning from im2026. arXiv preprint arXiv:2606.02572. ages with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF [15] Kai Zhao, Dongliang Nie, Yuchen Lin, Zhehan Luo, Yixiao Gu, Deng-Ping Fan, Conference on Computer Vision and Pattern and Dan Zeng. Sub-JEPA: Subspace Recognition, pages 15619–15629, 2023. Gaussian regularization for stable end-to[6] Sehun Kim. Learning general represenend world models, 2026. arXiv preprint tation of 12-lead electrocardiogram with arXiv:2605.09241. a joint-embedding predictive architecture, [16] Lucas Thil, Jesse Read, Rim Kaddah, and 2024. arXiv preprint arXiv:2410.08559. Guillaume Doquet. Subspace-decomposed [7] Saarang Panchavati, Uddhav Panchavati, HiJEPAs: Disentangling progression and conroki Nariai, Corey Arnold, and William tent in latent world models, 2026. arXiv Speier. Laya: A LeJEPA approach to EEG preprint arXiv:2605.31111. via latent prediction over reconstruction, [17] Chang Liu, Fei Suo, Yanzhou Jin, Yusuke 2026. arXiv preprint arXiv:2603.16281. Iwasawa, Yutaka Matsuo, and Yaonan [8] Randall Balestriero and Yann LeCun. Zhu. Temporally centered SIGReg improves LeJEPA: Provable and scalable selfmulti-task LeWorldModel learning: From supervised learning without the heuristics, analysis to method, 2026. arXiv preprint 2025. arXiv preprint arXiv:2511.08544. arXiv:2607.26924. [9] Lucas Maes, Quentin Le Lidec, Damien [18] Alexander Chemeris, Ming Jin, and Randall Scieur, Yann LeCun, and Randall Balestriero. LeNEPA: No-augmentation Balestriero. LeWorldModel: Stable next-latent prediction for time-series repend-to-end joint-embedding predictive resentation learning. In 12th Minarchitecture from pixels, 2026. arXiv ing and Learning from Time Series preprint arXiv:2603.19312. Workshop (MILETS), KDD 2026, 2026. arXiv:2607.00958. [10] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. [19] David Klindt, Yann LeCun, and Randall Richemond, Elena Buchatskaya, Carl DoerBalestriero. When does LeJEPA learn sch, Bernardo Avila Pires, Zhaohan Daniel a world model?, 2026. arXiv preprint Guo, Mohammad Gheshlaghi Azar, et al. arXiv:2605.26379. Bootstrap your own latent: A new approach to self-supervised learning. Advances in [20] Danaé Broustail, Anna Tegon, Thorir Mar Ingolfsson, Yawei Li, and Luca Benini. Neural Information Processing Systems, 33: LuMamba: Latent unified mamba for 21271–21284, 2020. electrode topology-invariant and efficient [11] T. W. Epps and L. B. Pulley. A test for norEEG modeling. In European Signal mality based on the empirical characteristic Processing Conference (EUSIPCO), 2026. function. Biometrika, 70(3):723–726, 1983. arXiv:2603.19100; EUSIPCO 2026 proceedings not yet posted. [12] Yilun Kuang, Yash Dagade, Tim G. J. Rudner, Randall Balestriero, and Yann LeCun. [21] Jonas Petersen, Gian-Alessandro LomRectified LpJEPA: Joint-embedding predicbardi, Riccardo Maggioni, Camilla Maztive architectures with sparse and maximumzoleni, Federico Martelli, and Philipp Peentropy representations. In International tersen. HEPA: A self-supervised horizonConference on Machine Learning, 2026. conditioned event predictive architecture 16

for time series. In ICML 2026 Workshop on Foundation Models for Structured Data (FMSD), Spotlight, 2026. arXiv:2605.11130.

Geng, Zhipeng Zeng, Qiming Chen, Xinyue Gu, Rong Jin, and Liang Sun. Baguan-TS: Dual in-context learning model for time series forecasting with covariates. In International Conference on Machine Learning (ICML), 2026.

[22] Jose Geraldo Fernandes, Luiz Facury, Pedro Robles Dutenhefner, and Wagner Meira. Beyond patient invariance: [31] Xinghong Fu, Yanhong Li, Georgios PaLearning cardiac dynamics via actionpaioannou, and Yoon Kim. Reverso: Efficonditioned JEPAs, 2026. arXiv preprint cient time series foundation models for zeroarXiv:2604.22618. shot forecasting. In 2nd ICML Workshop on Foundation Models for Structured Data, [23] Hamza Shafiq, Hung Manh Pham, Bin 2026. arXiv preprint arXiv:2602.17634. Zhu, Pan Zhou, Jun Hu, and Aaqib Saeed. CardioState-JEPA: Delay-aware [32] Jingru Fei, Kun Yi, Alex Xing Wang, Qingcross-modal learning of a shared carsong Wen, Xiangxiang Zhu, and Wei Fan. diac representation, 2026. arXiv preprint Olivia: Harmonizing time series foundation arXiv:2608.12944. models with power spectral density. In International Conference on Machine Learning, [24] Blaise Delaney, Salil Patel, Yuji Xing, Do2026. arXiv preprint arXiv:2605.17340. minic Dootson, Karin Sevegnani, and Chrystalina Antoniades. Sonata: A hybrid world [33] Uros Zivanovic, Serafina Di Gioia, Andre model for inertial kinematics under clinScaffidi, Martín de los Rios, Gabriella Conical data scarcity, 2026. arXiv preprint tardo, and Roberto Trotta. Rotary masked arXiv:2604.18058. autoencoders are versatile learners. In Advances in Neural Information Processing [25] Riccardo Grazzi, Julien Siems, Arber Zela, Systems, volume 38, pages 133952–133987, Jörg K. H. Franke, Frank Hutter, and Massi2025. arXiv preprint arXiv:2505.20535. miliano Pontil. Unlocking state-tracking in linear RNNs through negative eigenvalues. [34] Taco Cohen and Max Welling. Group equivIn International Conference on Learning ariant convolutional networks. In ProceedRepresentations, 2025. ings of the 33rd International Conference on Machine Learning, volume 48 of Proceed[26] Zhikang Chen, Yue Wang, Sen Cui, ings of Machine Learning Research, pages Yu Zhang, Changshui Zhang, Tianling Ren, 2990–2999, 2016. and Tingting Zhu. ECG-WM: A physiologyinformed ECG world model for clinical in- [35] Michael M. Bronstein, Joan Bruna, Taco tervention simulation, 2026. arXiv preprint Cohen, and Petar Veličković. GeometarXiv:2605.17580. ric deep learning: Grids, groups, graphs, geodesics, and gauges, 2021. arXiv preprint [27] M A Al-Masud and Nils Strodthoff. PrearXiv:2104.13478. training strategies and scaling for ECG foundation models: A systematic study, 2026. [36] David W. Romero, Erik J. Bekkers, Jakub M. arXiv preprint arXiv:2605.12241. Tomczak, and Mark Hoogendoorn. Wavelet networks: Scale-translation equivariant [28] Morad Laglil, Bertrand Pracca, Emilie Devilearning from raw time-series. Transacjver, and Eric Gaussier. Foundation models tions on Machine Learning Research, 2023. and fine-tuning: Toward a new generation arXiv:2006.05259. of models for time series forecasting, 2026. arXiv preprint arXiv:2607.23146. [37] Daniel E. Worrall and Max Welling. Deep scale-spaces: Equivariance over scale. In [29] Girish Narayanswamy, Maxwell A. Xu, Advances in Neural Information Processing A. Ali Heydari, Samy Abdel-Ghaffar, MarSystems, 2019. arXiv:1905.11697. ius Guerard, Kara Vaillancourt, Zhihan Zhang, Jake Garrison, Levi Albuquerque, [38] Lars Graf, Thomas Ortner, Stanisław Dimitris Spathis, et al. Towards a genWoźniak, and Angeliki Pantazi. Floweral intelligence and interface for wearState: Sampling-rate-equivariant timeable health data, 2026. arXiv preprint series forecasting, 2025. arXiv preprint arXiv:2605.22759. arXiv:2508.05287. [30] Linxiao Yang, Xue Jiang, Gezheng Xu, Tian [39] Andrea Morandi. Scale-equivariant genZhou, Min Yang, Zhaoyang Zhu, Linyuan erative forecasting: Weight-tied dilated 17

convolutions, wavelet scattering inputs, and spectral-consistency training for selfsimilar time series, 2026. arXiv preprint arXiv:2605.17582.

Ole Jakob Elle, Ulysse Côté-Allard, and Arian Ranjbar. Echo-POSED: Geometric selfdistillation for echocardiography guidance, 2026. arXiv preprint arXiv:2606.02634.

[40] Irina Higgins, David Amos, David Pfau, [49] Zakaria Mhammedi, Andrew Hellicar, Ashfaqur Rahman, and James Bailey. EffiSébastien Racanière, Loïc Matthey, Danilo cient orthogonal parametrisation of recurRezende, and Alexander Lerchner. Towards rent neural networks using householder rea definition of disentangled representations, flections. In Proceedings of the 34th In2018. arXiv preprint arXiv:1812.02230. ternational Conference on Machine Learning, volume 70 of Proceedings of Ma[41] Robin Winter, Marco Bertolini, Tuan Le, chine Learning Research, pages 2401–2409. Frank Noé, and Djork-Arné Clevert. UnPMLR, 2017. supervised learning of group invariant and equivariant representations. In Advances [50] Patrick Wagner, Nils Strodthoff, Ralf-Dieter in Neural Information Processing Systems, Bousseljot, Dieter Kreiseler, Fatima I Lunze, 2022. arXiv:2202.07559. Wojciech Samek, and Tobias Schaeffter. Ptb-xl, a large publicly available electrocar[42] Giovanni Luca Marchetti, Gustaf Tegnér, diography dataset. Scientific data, 7(1):154, Anastasiia Varava, and Danica Kragic. 2020. Equivariant representation learning via class-pose decomposition. In Proceedings [51] Nils Strodthoff, Patrick Wagner, Tobias of the 26th International Conference on ArSchaeffter, and Wojciech Samek. Deep tificial Intelligence and Statistics, volume learning for ECG analysis: Benchmarks and 206 of Proceedings of Machine Learning insights from PTB-XL. IEEE Journal of Research, pages 4745–4756, 2023. Biomedical and Health Informatics, 25(5): 1519–1528, 2021. doi: 10.1109/JBHI.2020. [43] Jiawen Zhu, Shuhan Liu, Shengxuan Li, 3022989. Qiming Shi, and Di Weng. POEM: Phaseaware SO(2) feature rotation for time se- [52] Patrick Wagner, Nils Strodthoff, Ralf-Dieter Bousseljot, Wojciech Samek, and Tobias ries forecasting under periodicity drift, 2026. Schaeffter. PTB-XL, a large publicly availarXiv preprint arXiv:2608.03630. able electrocardiography dataset. Phys[44] Davide Marincione, Michele Mancusi, GiorioNet, November 2022. doi: 10.13026/ gio Strano, Luca Cerovaz, Donato Crisoskfzx-aw45. URL https://doi.org/10. tomi, Roberto Ribuoli, and Emanuele 13026/kfzx-aw45. Version 1.0.3. Rodolà. PHALAR: Phasors for learned musical audio representations, 2026. arXiv [53] Jiapu Pan and Willis J. Tompkins. A realtime QRS detection algorithm. IEEE Transpreprint arXiv:2605.03929. actions on Biomedical Engineering, BME32(3):230–236, 1985. doi: 10.1109/TBME. [45] Bosong Huang, Panzhen Zhao, Zengxiang 1985.325532. Li, Patricia Lee, Wei Jin, Alan Wee-Chung Liew, Ming Jin, and Shirui Pan. LVCG: [54] Siwon Kim. Hierarchical self-supervised Learning ECG representations in the larepresentation learning framework for multent vectorcardiogram space, 2026. arXiv tivariate time series grounded in ECG analypreprint arXiv:2605.31249. sis, 2026. arXiv preprint arXiv:2607.01145. [46] Jintai Chen, Xiangshang Zheng, Hongyun [55] Kuba Weimann and Tim O. F. ConYu, Danny Z. Chen, and Jian Wu. Electrorad. Self-supervised pre-training with jointcardio panorama: Synthesizing new ECG embedding predictive architecture boosts views with self-supervision, 2021. arXiv ECG classification performance. Computers preprint arXiv:2105.06293. in Biology and Medicine, 196:110809, 2025. doi: 10.1016/j.compbiomed.2025.110809. [47] Yoshihiro Maruyama. Learning with category-equivariant architectures for hu- [56] Yi Yuan, Joseph Van Duyn, Runze Yan, Zhuoyi Huang, Sulaiman Vesal, Sergey Plis, man activity recognition, 2025. arXiv Xiao Hu, Gloria Hyunjung Kwak, Ran Xiao, preprint arXiv:2511.01139. and Alex Fedorov. Learning ECG represen[48] Elias Stenhede, Edvart Grüner Bjerke, tations via poly-window contrastive learnJoanna Sulkowska, Eivind Bjørkan Orstad, ing, 2025. arXiv preprint arXiv:2508.15225. 18

[57] Temesgen Mehari and Nils Strodthoff. Selfsupervised representation learning from 12lead ECG data. In Machine Learning for Health, 2021.

A

at the fundamental, and 0.39 rad at the highest retained harmonic, n = 7. The error grows with harmonic order because harmonic n advances n times faster in phase, so the tolerance on the token timestamp is set by the highest retained harmonic rather than by the fundamental.

Data protocol C

Training specification

This appendix states the preprocessing chain and phase-clock construction in full, the normalisa- C.1 SIGReg quadrature tion convention and the evidence for choosing it, and the composition of the pre-training pool For all reported runs, LSIG is evaluated on the against the sealed fold. raw projector tokens z, without rotation into a canonical phase frame (Section B). Each evaluaTable 3 gives the preprocessing steps. tion uses M = 256 random projection directions. All reported runs use a fixed per-lead normalisa- At each training step, the projection directions are tion: one mean and standard deviation are fitted sampled afresh from the unit sphere. on the decimated training-fold signals and applied unchanged to every recording. Unlike per-beat For each axis, the characteristic-function integral or per-recording normalisation, this preserves in Equation (2) is approximated using J = 17 between-recording amplitude variation within trapezoidal quadrature knots over [0, 3]. The each lead. The values o of the means and standard interval exploits the evenness of the integrand and therefore represents the symmetric domain deviations are given in Table 4. [−3, 3]. The Gaussian weighting assigns little Section A the demographic and diagnostic com- mass beyond |t| = 3, while retaining J = 17 position of the pre-training folds and the test fold. knots over the shorter interval gives finer resAll 19,601 fold-1–9 recordings are used for pre- olution near the origin, where the integrand training, and the normalisation statistics of Ta- varies most. This choice is also supported by ble 4 are fitted over the same 19,601. The 222 the LeJEPA ablation in Table 1a, where [−3, 3] recordings that fail phase quality control (1.0% and [−5, 5] yield comparable linear-probe perforof the corpus) are retained in the pool, since a mance [8]. The absolute scale of SIGReg depends model consuming no phase is unaffected by them on the quadrature grid. and filtering the pool on that criterion would drop Two alternative SIGReg frames were tested but recordings for an unrelated reason. not adopted. Demodulating each token to phase zero (the “canonical” frame) produced transport gains from −0.38 to −0.17 and reduced the efB Per-token phase assignment fective rank to 2.9–11.2. Reducing each record Each 80 ms token summarises an eight-sample to a canonical template (the “record-canonical” patch of the decimated signal, so a single scalar frame) instead yielded zero transport gain bephase must stand for the whole patch. Two con- cause the representations concentrated in the nonrotating invariant block. All reported runs thereventions are used, for different purposes. fore use the raw token frame, with record-level Centre convention (training). Each token carries SIGReg disabled. the phase at its patch centre, 3.5 sample intervals after the patch start. Phase tags are computed once per record offline and loaded alongside each C.2 Augmentation provenance waveform batch, so the training loop never reSignal augmentation follows precedent from rederives them. lated time-series and ECG models. Reverso inCausal convention (reporting only). Descriptors cludes augmentation in its zero-shot forecasting that must not read beyond a token’s own support recipe, although its ablation assigns a larger effect — time since the preceding R-peak, distance to to synthetic pre-training data [31]. In the ECG the nearest R-peak — instead take the phase at setting, ER-JEPA independently applies baseline the token’s last sample. This convention never wander, Gaussian noise, and power-line noise enters training. during pre-training [54]. These works motivate signal augmentation generally, but do not validate Assigning the causal boundary during training the specific six-transform stack used here. would impose a fixed 35 ms offset, which becomes a heart-rate-dependent fraction of the cy- The stack itself was selected in a preliminary cle: 0.056 rad standard deviation across records architecture-scouting experiment rather than from 19

#

Step

Operation and parameters

1

Read

2

Verify extent

3

Verify lead order

4

Decimate

5

Window

6

Normalise

7

Arrange layout

The 16-bit waveform release at 500 Hz is converted to physical millivolts in single-precision floating point as (raw − baseline)/gain, scaled by the declared unit factor of the recording header. Each signal’s stored checksum is verified. Each recording is required to contain 5,000 samples across 12 leads, every sample finite, before any filtering. A header that does not declare fs = 500 Hz is rejected. I, II, III, aVR, aVL, aVF, V1–V6, checked against each recording’s own header. A mismatch is rejected rather than corrected by reordering. 500 → 100 Hz by polyphase resampling with interpolation factor 1 and decimation factor 5: a single linear-phase Kaiserwindowed FIR filter, with group delay compensated internally, so output sample n corresponds to input time n/fout with no timing shift. The signal is extended at each endpoint by its boundary value rather than by zeros, so a non-zero baseline introduces no step discontinuity at t = 0 or t = 10 s. The whole recording: 1,000 samples across 12 leads. No cropping, no stride, no overlap — exactly one training example per recording. Per-lead z-score against corpus statistics fitted on the decimated signal over the training folds only (Table 4). Lead-major, 12 leads by 1,000 samples, in single-precision floating point. Table 3: ECG preprocessing pipeline.

Lead

µ [mV]

σ [mV]

Lead

µ [mV]

σ [mV]

Folds 1–9 Fold 10

I −0.002 0.171 V1 0.000 0.234 II −0.001 0.167 V2 −0.001 0.338 III 0.000 0.172 V3 −0.001 0.335 aVR 0.002 0.143 V4 −0.002 0.311 aVL −0.001 0.147 V5 −0.001 0.291 aVF −0.001 0.147 V6 −0.001 0.243 Table 4: Per-lead normalisation statistics. Fitted corpus normalisation statistics. µ and σ are the per-lead mean and standard deviation.

the four pre-training runs reported in Section 4.3. This experiment used one random seed, 5,000 training steps, a 6,000-record training subset, and evaluation on fold 9. The full augmentation stack achieved a macro-AUROC of 0.8589, compared with 0.8513 for the control arm. This provides preliminary support for carrying the stack forward, but does not estimate uncertainty or isolate the contribution of individual augmentations. See Table 6 for the complete specification. C.3

Recordings Patients

19,601 16,965

2,198 1,904

Age, median (years) Age, interquartile range Age sentinel (age = 300) Male, %

61 50–72 259 52.2

63 50–74 34 51.5

Normal ECG (NORM), % Myocardial infarction (MI), % ST/T change (STTC), % Conduction disturbance (CD), % Hypertrophy (HYP), % U NLABELED recordings

43.6 25.1 24.0 22.5 12.2 371

43.8 25.0 23.7 22.6 11.9 40

Beats per recording, median 12 12 Phase yield, median 0.908 0.909 Recordings failing phase QC 202 20 Table 5: Pre-training and sealed-fold cohort composition. Diagnostic prevalences are multi-label and may sum to more than 100%. UNLABELED denotes recordings without an eligible diagnostic statement. The age = 300 sentinel denotes de-identified patients over 89. Phase yield is the fraction of samples assigned a phase. Phase-QC failures remain in the self-supervised pool but are excluded from phase-dependent analyses.

Optimiser and schedule

All optimiser and schedule settings were fixed across the four pre-training runs and were not C.4 Four training runs swept; their values are given in Table 7. Training uses AdamW with a linear warm-up followed by The experiment follows a two-arm, two-seed decosine learning-rate decay. sign. The transport-enabled arm uses λtrans = 1, 20

Augmentation

Operation

Parameters

Gaussian noise

Add independent Gaussian noise to the waveform. Add a sinusoidal interference component.

Zero mean; standard deviation equal to 5% of the record RMS. Frequency: 25 Hz. Amplitude: uniform on 5%–20% of record RMS. Phase offset ξ: uniform on [0, 2π). One shared tone, identical across leads. Sum of 1–3 sinusoids, count drawn uniformly from {1, 2, 3}; each component’s frequency uniform on 0.05–0.4 Hz and amplitude uniform on 5%–20% of record RMS, with an independent phase offset ξ uniform on [0, 2π). Shared across all leads. Envelope 1+a sin(2πf t+ξ). Depth a: uniform on 2%–10%. Frequency f : uniform on 0.05–0.3 Hz. Phase offset ξ: uniform on [0, 2π). One record-global envelope, identical across leads. Number of dropped leads k: uniform on {1, 2, 3}. The k leads are a uniformly random subset of the 12, drawn without replacement, independently per record. Gain gl : drawn independently and uniformly on [0.7, 1.3] for each of the 12 leads, independently per record.

Narrow-band tone

Low-frequency drift

Add a slowly varying baseline component.

Amplitude modulation

Multiply the waveform by a slowly varying amplitude envelope.

Lead dropout

Set complete ECG leads to zero.

Per-lead gain

Scale each ECG lead independently.

Table 6: Training-time augmentation settings. Each transform is applied independently to each record with probability 0.5, gated independently per augmentation type. All additive amplitudes are a fraction of that record’s own clean-input RMS, computed once before any augmentation is applied, so a stacked dose always refers to the same physical scale.

Hyperparameter Optimiser Batch size Training steps Peak learning rate Minimum learning rate Warm-up steps AdamW β AdamW ϵ Weight decay Maximum gradient norm

Value

Arm Transport Transport Control Control

AdamW 64 30,000 3 × 10−4 1 × 10−6 5 (0.9, 0.95) 1 × 10−8 1 × 10−4 1.0

Seed

λtrans

Elapsed [min]

0 1 0 1

1.0 1.0 0.0 0.0

249.6 254.9 240.0 225.3

Table 8: Four pre-training runs. Elapsed time is the wallclock training time recorded for each run.

Training was performed on a single NVIDIA A100 40 GB GPU in two matched concurrent Table 7: Optimiser and schedule hyperparameters. Used pairs, with approximately 11.7 GB peak memory for all reported pre-training runs. per pair.

D while the control retains the same cyclic operator but assigns the transport loss zero weight. All remaining model, data, and optimisation settings are identical across the four runs. Within each seed, randomness is separated into independent streams for parameter initialisation, mask sampling, SIGReg projections, augmentation, and data order. The active stream states are stored in each checkpoint. Checkpoints are saved every 2,500 steps and at the end of training.

Positioning within PTB-XL self-supervised learning

This appendix situates W INDER (1.2M parameters) within the published self-supervised ECG representation-learning literature on PTB-XL. The comparison is restricted to methods that report PTB-XL diagnostic performance and provide sufficient information about their pre-training data, model scale, and downstream evaluation protocol. The resulting set is curated, not exhaustive: its purpose is to identify the nearest

21

methodological and empirical reference points for W INDER, not to construct a leaderboard. Despite its ubiquity in the literature, there is no shared protocol for PTB-XL evaluation.

training folds. Poly-Window [56] (3.8M3 ) most closely matches W INDER’s 100 Hz, full-record frozen-probe evaluation, although self-supervised pre-training folds are not reported.

LeNEPA (3.6M2 ) provides the nearest methodological comparison. Both methods combine causal latent prediction with SIGReg, however there are two key differences in how embedded windows are related: LeNEPA applies SIGReg to the sequence of token embeddings from each ECG, preventing collapse without giving their arrangement a cardiac-phase meaning; W INDER includes the equivariant transport loss that requires latents to follow measured cardiac-phase.

CardioState-JEPA [23] (94.3M4 ) provides a separate cross-modal reference, learning a shared representation from ECG, PPG, and PCG with cardiac phase included as an auxiliary prediction target. This differs from W INDER, which uses ECG alone and imposes phase as a fixed group action on the latent space rather than as a quantity to predict. Their ablations independently suggest that cardiac-phase supervision is useful, although phase is treated as a prediction target rather than an equivariant constraint. CardioState-JEPA is Beyond LeNEPA, the principal signal-only JEPA therefore relevant to cardiac-state representation, references are S4-JEPA (3.0M) and the modbut its multimodal pre-training prevents a direct els of Weimann and Conrad [55] and Kim signal-only comparison. [6]. Weimann JEPA is evaluated at several scales; here, we consider only the smallest, ViT- Although none of the published results is diXS (3.6M), as the closest scale comparison to rectly comparable with W INDER under a fully W INDER. It uses masked prediction, is pre- matched protocol, its diagnostic performance trained on configurations ranging from PTB-XL lies within the same broad range as the sealone to ten ECG corpora, and evaluates frozen lected self-supervised models. W INDER reaches representations through learned cross-attention this range with a 1.2M-parameter model, prepooling over 2.5 s crops. ECG-JEPA (87.2M) in- training restricted to 19,601 PTB-XL records, stead uses a substantially larger Transformer with and parameter-free mean pooling followed by cross-pattern attention, pre-trained on 174,140 ex- a linear classifier. Several reference methods ternal ECGs and evaluated through mean-pooled use more expressive learned readouts, including frozen representations. Neither method assigns Weimann’s cross-attention pooling. Table 9 therecardiac phase an explicit action in the latent fore supports a narrower, deployment-relevant space. claim: phase-transport training produces representations from which diagnostic information is Readout capacity is itself a material protocol diflinearly accessible without a learned sequence ference. Weimann’s learned cross-attention poolaggregator. ing can adaptively combine crop-level features, whereas W INDER uses parameter-free mean pooling followed by a linear classifier. Table 9 therefore does not support a head-to-head accuracy claim. The narrower result is that phase-transport training yields diagnostic information that remains accessible to a deliberately low-capacity readout, without a learned sequence aggregator. This is directly relevant to deployment because the downstream readout adds only a linear classifier. Contrastive predictive objectives provide a complementary set of self-supervised references. S4CPC (3M) of Al-Masud and Strodthoff [27] is compact but is pre-trained on 11.2 million external ECG samples, and its reported PTB-XL split is insufficiently specified for direct comparison. ECG-CPC (5.8M) is similarly compact, but reports the all-71 task and does not clearly restrict PTB-XL pre-training to the downstream 2 Italicised parameter counts are estimates rather than val-

ues explicitly reported by the source. This count is estimated from the released PTB-XL ViT-XS configuration.

3 Estimated from the canonical 1D ResNet18-512/128 architecture. 4 Estimated from the released configuration at repository commit ab242c3.

22

Method

Parameters [M]

Pre-training examples

PTB-XL PTB-XL evaluation protocol AUROC in pretraining?

Latent-prediction objectives W INDER

1.2

17k

✓

100 Hz; full 10 s record; 0.8702 frozen representation with a plain linear probe; folds 1–8 train and fold 10 test

S4-JEPA [27]

3.0

11Ma

×

240 Hz; frozen mean-pooled 0.880 linear evaluation; concrete PTB-XL fold assignment not stated

LeNEPA [18]

3.6

N.A.

✓

5,000-sample records; fixed- 0.880c horizon pre-training; frozen intermediate-layer probe; label mapping insufficiently specified

JEPA [55]

3.6

1.0M

✓

JEPA ViT-XS

3.6

0.80M

×

JEPA ViT-XS

3.6

17k

✓

500 Hz; 2.5 s crops with 0.915 record-level probability averaging; frozen encoder with learned cross-attention pooling Same protocol as the all- 0.915 corpora JEPA row Same protocol as the all- 0.912 corpora JEPA row

ECG-JEPA [6]

87.2

0.17M

×

Eight leads at 250 Hz; full 0.912 10 s record; frozen meanpooled representation with linear classifier; official folds

CardioStateJEPA [23]

94.3

> 5.9Mb

×

Cross-modal ECG–PPG– 0.891 PCG pre-training; frozen linear probe; official-style PTB-XL split

ViT-XS

Contrastive objectives S4-CPC [27]

3.0

11Ma

×

Same protocol and split limi- 0.860 tation as S4-JEPA

Poly-Window [56]

3.8

22k

✓

100 Hz; full 10 s record; 0.891 frozen encoder with linear classifier; official downstream folds

ECG-CPC [57]

5.8

55k

✓

100 Hz; frozen linear evalua- N.A. tion with 2.5 s crop aggregation; all 71 statements

a

The S4 studies report pre-training samples rather than complete ECG records. b CardioState-JEPA combines ECG records, PPG segments, PCG recordings, and paired multimodal corpora; the paired-corpus counts are not reported. c LeNEPA’s reported AUROC uses all 71 PTB-XL labels rather than the five diagnostic superclasses.

Table 9: Published self-supervised comparisons on PTB-XL. Results are reproduced in each source’s original evaluation setting and are not a common leaderboard. AUROC values refer to the five diagnostic superclasses. Green, orange, and red shading in the parameter and pre-training-example columns denotes 0.5–1.5× , > 1.5–3× , and > 3× W INDER values, respectively. Despite differences in evaluation protocols, W INDER achieves AUROC values in the same broad range as the selected models while using fewer parameters and less pre-training data.

23

Related documents

• Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning#662312
• Capturing Cardiac Cyclicity through Phase-Equivariant Self-Supervised Learning#661262
• Learning Cardiac Features: ECG Biometrics Across Time and~Exercise#1036350
• Learning Cardiac Features: ECG Biometrics Across Time and~Exercise#1006887
• BeatGraph: Self-Supervised Heartbeat Graphs for Infant ECG Representations from the Home Environment#1106245