Physiological Noise Augmentation Improves Non-Invasive Brain-to-Speech
arXiv:2607.05165v1 [cs.LG] 6 Jul 2026
Benjamin Ballyk∗ Teyun Kwon∗ Miran Özdogan
Oiwi Parker Jones
PNPL Department of Engineering Science University of Oxford {benjamin.ballyk, teyun.kwon}@eng.ox.ac.uk
Abstract Non-invasive brain-to-speech decoding aims to restore communication to patients suffering from neurodegenerative disease, without the risks of neurosurgery. Existing MEG- and EEG-based methods, while scalable, continue to suffer from high word error rates driven by relatively low signal-to-noise ratios compared to invasive recordings. We propose physiological noise augmentation (PNA), a data augmentation method that explicitly trains decoders to become invariant to task-agnostic artifacts (e.g. ocular and cardiac activity). PNA draws inspiration from automatic speech recognition systems, where environmental noise (e.g. dogs barking, city traffic) is added to clean speech to improve robustness. Analogously, we decompose brain recordings into clean data and noise artifacts using independent component analysis (ICA), before scaling and remixing to generate biophysically realistic, label-preserving training examples. We show that PNA approximates anisotropic regularization, penalizing decoder sensitivity along artifact-dominated directions. On MegNIST, a 12k-trial imagined-digit MEG dataset, PNA with 10-trial averaging improves EEGNet decoding accuracy by 4.7 percentage points (absolute) over training on real data alone. Our results suggest that artifact-aware augmentation and trial averaging are complementary tools for improving robustness in non-invasive speech BCIs.
1
Introduction
Neurodegenerative disease, stroke, and cervical spine injuries collectively affect more than 110 million patients globally, and often irreversibly impair one’s ability to articulate thoughts into speech [12, 13, 33]. For nearly four decades, brain-to-speech research has sought to restore communication for these patients, yet this goal remains a central challenge at the intersection of neuroscience and biomedical engineering [11, 3, 44]. Although recent intracranial brain-computer interfaces (BCIs) have achieved increasingly accurate decoding of intended speech from cortical activity [30, 42, 5], these invasive systems carry risks of neurosurgical complications and long-term electrode instability [23]. Consequently, there is a growing imperative to develop safer and more accessible non-invasive alternatives based on magnetoencephalography (MEG), electroencephalography (EEG), or functional magnetic resonance imaging (fMRI) [44, 7]. Non-invasive brain recordings, however, suffer from low signal-to-noise ratio (SNR), particularly in the context of imagined speech, which lacks overt articulatory and auditory feedback [34, 28, 31, 26, 27]. For several decades, this limitation has been addressed using multi-trial averaging, in which repeated recordings of the same thought are ∗ Equal contribution.
Preprint.
(A) Artifact Acquisition
(B) Component Identification Run ICA
𝑿𝒓𝒂𝒘
Isolate Artifact Components
𝑺 = 𝑾𝑿𝒓𝒂𝒘
𝝓𝒆𝒐𝒈
𝝆
𝝓𝒆𝒄𝒈
𝜏
𝑺𝒆𝒐𝒈
𝒔𝒊 Correlation Threshold
𝑺𝒆𝒄𝒈
𝒔𝒊
(C) Stochastic Remixing Return to Sensor Space
ℝ𝑁×𝑇
ℝ𝐶×𝑇
𝑟𝑖 𝑒𝑜𝑔
𝒙𝟏
…
𝑨
Construct Augmented Data
𝑿𝒂𝒖𝒈 = 𝑿𝒄𝒍𝒆𝒂𝒏 + 𝑐𝑒𝑜𝑔 𝑿𝒆𝒐𝒈 + 𝑐𝑒𝑐𝑔 𝑿𝒆𝒄𝒈
𝑟𝑖 𝑒𝑐𝑔
𝑐𝑚 = 𝑟𝑚
…
𝒔𝒊
Freq.
Sample Plausible Artifact Energy Ratios
…
𝒙𝑪
𝑨𝑺𝒄𝒍𝒆𝒂𝒏 = 𝑿𝒄𝒍𝒆𝒂𝒏
𝑟𝑚 ~
𝑿𝒎, 𝒅 𝐹 𝑿𝒄𝒍𝒆𝒂𝒏, 𝒅 𝐹
𝐷
𝑑=1
𝑟𝒊 Sampled Energy Ratio
𝑿𝒄𝒍𝒆𝒂𝒏, 𝒌 𝑿𝒎, 𝒌 Instance-wise Normalization
Figure 1: Overview of physiological noise augmentation. (A) Raw brain recordings (Xraw ) and reference channels (e.g. ocular ϕeog and cardiac ϕecg ) are collected. (B) Independent component analysis (ICA) decomposes Xraw into independent components, which are correlated with references to identify artifact components. (C) Artifact components are projected back to sensor space, stochastically scaled using their empirical amplitudes, and remixed to generate augmented samples.
averaged to enhance phase-locked signal relative to independent noise [38]. However, this approach is often slow and burdensome, as users must repeat themselves several times, which limits practical applicability. Concurrently, data collection bottlenecks have motivated augmentation strategies to expose decoding models to a broader range of realistic test-time conditions. While often modestly beneficial, existing augmentations typically operate at the input signal level rather than targeting underlying neurophysiological artifacts, leaving models exposed to structured noise that dominates imagined speech signals [25, 15]. We address this gap by introducing physiological noise augmentation (PNA), a data augmentation framework that isolates artifact-related independent components (e.g., ocular and cardiac activity) and re-injects remixed variants to generate physiologically realistic, label-preserving training examples. By exposing the decoder to a wider range of artifact realizations, PNA encourages invariance to nuisance structure and promotes reliance on task-relevant signals. Combined with multi-trial averaging to reduce residual variability, this approach reduces the number of repetitions required to achieve strong decoding performance.
Contributions. We introduce PNA, an augmentation framework for noninvasive brain-to-speech that promotes invariance to task-agnostic artifacts by remixing artifact-related independent components. PNA generates realistic sensor-space perturbations by scaling artifacts according to the empirical distribution of energy ratios. We provide theoretical support for PNA by showing that, under multi-trial averaging, the method approximates anisotropic Jacobian regularization that penalizes the decoder’s sensitivity to directions aligned with tracked artifacts. Empirically, PNA substantially improves performance on the MegNIST imagined speech dataset, increasing decoding accuracy by 4.7% (absolute) over real-data baselines and reaching 77.6% decoding accuracy with EEGNet. 2
2
Related Work
We explore three threads of related literature that define the technical setting of this paper: noninvasive neural decoding, artifact handling in M/EEG, and data augmentation for neural time series. Non-Invasive Speech Decoding. Recent progress into non-invasive decoding has focused predominantly on the task of perceived speech, and has been driven largely by MEG data scaling and self-supervised learning [19]. [10] apply contrastive learning on aligned embeddings of audio and neural responses, [7] scale across subjects, devices, and languages for transferable word-level retrieval, and [18] integrate language-model rescoring to push toward open-vocabulary brain-to-text. These methods exploit a structural advantage of perceived speech: the acoustic stimulus provides a higher SNR, largely time-locked target that anchors learning [1]. In contrast, imagined speech, typically provides weaker SNRs and less predictable temporal alignment [36]. We study this harder setting via single-subject MEG classification of imagined digits and address the noise problem directly through the artifact-aware augmentation. Artifact Handling in M/EEG. Physiological artifacts that appear in MEG and EEG signals, such as ocular, muscular or cardiac activity, are typically treated as nuisance signals to be suppressed. The dominant pipeline for artifact suppression identifies and removes physiological artifacts via independent component analysis (ICA) [2, 20, 40], with recent work automating component selection via reference-channel correlation, topographies, or learned classifiers [35, 43]. PNA inverts this convention: rather than discarding artifact components, we use the same ICA decomposition to generate physiologically realistic training perturbations, exposing the decoder to the artifact distribution it will encounter at test time. Data Augmentation for Neural Decoding. Augmenting clean inputs with realistic nuisances has a long history in automatic speech recognition (ASR), where noise corpora [37] and reverberation [21] are routinely added to clean speech to improve test-time robustness. Neural signal augmentations to date have imported the input-level transformations (e.g. time-frequency warping [41], SpecAugmentstyle masking [9, 32]), without the corresponding ASR principle of drawing perturbations from the empirical noise distribution of the target domain. PNA closes this gap: by sampling artifact realizations directly from auxiliary EOG/ECG references and the decomposed signal, it produces augmentations that match the empirical artifact statistics of the recording session.
3
Method
We model brain-to-speech decoding as a multiclass classification problem over a fixed vocabulary, V, of size V . Let X = [x1 , . . . , xT ] ∈ RC×T represent a spatiotemporal neural recording, where C is the number of channels, T is the number of time samples per word trial, and xt ∈ RC denotes the sensor measurements at time t. Our goal is to learn a decoder fθ : RC×T → ∆V −1 that maps the input signal to a vector of word probabilities in the (V − 1)-simplex, such that the v-th element, fθ (X)v , denotes the predicted probability of word v ∈ V. In practice, the observed recording is an additive mixture X = Xtask +Xartifact , where Xtask represents task-relevant neural activity and Xartifact ∈ A represents task-irrelevant physiological artifacts (e.g. cardiac or ocular activity), residing in an artifact subspace, A ⊂ RC×T . To ensure the decoder’s predictions are driven exclusively by neural intent, we seek a model that is invariant to the artifact subspace, satisfying: ∀ Xtask ∈ RC×T , Xartifact ∈ A
fθ (Xtask + Xartifact ) = fθ (Xtask ),
(1)
to ensure that the predicted distribution over V depends only on task-related neural signals. 3.1
Physiological Noise Augmentation
PNA seeks to enforce Equation (1) in three steps (Figure 1): (i) recording nuisance reference channels during data acquisition, (ii) identifying and removing artifact-related ICA components, and (iii) re-injecting scaled artifact projections into the cleaned data to generate physiologically plausible synthetic samples. 3
Frequency
400
Ocular (¹=0.090) Cardiac (¹=0.256)
300 200 100 0
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
RMS Ratio (Artifact/Clean)
® = 0 (shuffling only)
® = 0:5 (underscaled)
−20
−20
® = 1 (calibrated)
® = 2 (overscaled)
40
PC2
20 0 −20 −40 0
20
40
0
20
40
−20
0
20
40
−20
0
20
40
PC1 Clean
Raw
Augmented
Augmented centroid
Figure 2: Stochastic amplitude sampling calibrates PNA-augmented MEG embeddings to the distribution of raw data. (Top) Empirical artifact-to-clean amplitude ratio distributions (zeros removed). (Bottom) PCA fit to 15-trial-averaged raw (red) and clean (black) embeddings of imagined digits; augmented samples (blue, gold centroid) are projected at varying scalings α. Augmented embeddings interpolate between clusters for α ∈ [0, 1], match Eq. (3) at α = 1, and shift off the biological support for α > 1. Pointwise misalignment at α = 0 reflects random indexing of artifact components.
(i) Measuring artifacts. During recording experiments spanning Ttotal time steps, we collect reference waveforms for task-agnostic artifacts.2 In MegNIST data used for our experiments, electrooculography (EOG) and electrocardiography (ECG) are recorded to capture ocular and cardiac activity, respectively (Figure 1A). Let ϕp ∈ RTtotal denote the reference time series for artifact p ∈ P. (ii) Removing artifact-correlated sources. PNA assumes that sensor recordings may be separated into statistically independent source components using independent component analysis (ICA) [2], which we overview in Appendix A.1. We fit FastICA [16] on recordings using MNE-Python [14], b yielding source estimates S. Artifact-related components are identified by their correlation with reference channels. For each tracked artifact type p ∈ P, we define Sp = {i ∈ {1, . . . , N } : |ρ(ŝi , ϕp )| ≥ τp } as the set of associated nuisance component indices. Here, ŝi ∈ RT is the time-series of the i-th b ϕp denotes the reference time-series for artifact p, and ρ(·, ·) is ICA component (the i-th row of S), Pearson correlation computed over Ttotal time samples. τp are selected correlation thresholds. b We project each artifact’s components back to sensor P space as Xp = A:,Sp SSp ,: , and recover the cleaned recording by subtraction, Xclean = Xraw − p∈P Xp , following the standard ICA cleaning convention used in MNE-Python. Figure 6 in Appendix C shows a topographical and waveform view of extracted ICA components using EOG (ocular) and ECG (cardiac) reference signals, alongside ICA component correlations. 2 Recording experiments are generally divided into several sessions, with independent component analysis repeated for
each session to account for changes in sensor placement.
4
(iii) Augmenting with physiological noise. We generate augmented trials by remixing scaled artifact reconstructions back into the cleaned signal, X Xaug = Xclean + cp Xp , (2) p∈P
where each cp is a stochastically sampled scaling coefficient chosen to span the empirical range of signal-to-artifact ratios. Let ∥ · ∥F denote the Frobenius norm over sensors (of the same type) and time, and let Xp,d and Xclean,d denote the reconstructions for trial d ∈ {1, . . . , D}. We define the empirical artifact-to-clean amplitude ratios rp,d =
∥Xp,d ∥F , ∥Xclean,d ∥F + ε
with ε > 0, a small numerical-stability constant. Figure 2 (Top) shows the resulting distributions for EOG and ECG; ECG components carry roughly twice the relative energy of EOG. During each forward pass, we sample a donor trial index j ∼ Uniform{1, . . . , D} and set cp = αp · rp,j ·
∥Xclean ∥F , ∥Xp,j ∥F + ε
∀ p ∈ P,
(3)
which rescales the donor artifact Xp,j so its amplitude relative to Xclean matches the sampled ratio rp,j .For generality, a fixed per-artifact scaling parameter, αp , enables fine-grained control over the expected artifact magnitude. In our experiments, however, we set αp = 1 for all p to maximize the fidelity of the augmented data, as illustrated in Figure 2 (bottom). Augmented samples therefore span the empirical artifact-to-signal distribution, encouraging the decoder toward invariance under variable contamination at test time. We provide pseudocode in Algorithm 1. Algorithm 1 Physiological Noise Augmentation Require: Training recording Xraw partitioned into D trials {Xd }D d=1 ; references {ϕp }p∈P ; thresholds {τp }; per-artifact scales {αp }; stability constant ε // Stage 1: Offline preprocessing (run once) b ← WXraw W, A ← FastICA(Xraw ); S for p ∈ P do Sp ← {i : |ρ(ŝi , ϕp )| ≥ τp } b S ,: Xp ← A:,Sp S p end for P Xclean ← Xraw − p∈P Xp for d = 1 . . . D, p ∈ P do rp,d ← ∥Xp,d ∥F / (∥Xclean,d ∥F + ε) end for
▷ ICA decomposition ▷ Assume Sp disjoint across p ▷ Artifact-correlated components ▷ Project artifact p to sensor space ▷ Cleaned recordings ▷ Empirical artifact-to-clean ratio
// Stage 2: Online augmentation (called per forward pass) function AUGMENT(Xclean,d ) Sample j ∼ Uniform{1, . . . , D} for p ∈ P do cp ← αp · rp,j · ∥Xclean,d ∥F / (∥Xp,j ∥F + ε) end for P return Xclean,d + p∈P cp Xp,j end function
▷ Random artifact donor trial ▷ Calibrate scaling
Thus far, we have considered onlythe effect of tracked artifacts on decoding. Optionally, after applying PNA, we further apply multi-trial averaging to suppress residual untracked artifacts that are not phase-locked to internal speech onset. Multi-trial averaging has long been used to improve SNR in noninvasive brain-to-text systems [11]; a brief overview is provided in Appendix A.2. Figure 3 presents t-SNE embeddings [39] of MegNIST data before and after averaging with resampling. Without averaging, t-SNE fails to identify well-separated local clusters corresponding to 5
Single Trials
15-Trial Averaged
2
8
9
5
7
1
8
6
3
0
6 5 4
7 4
3 2 1
Imagined Digit (0-9)
9
0
Figure 3: t-SNE embeddings of single-trial (left) and 15-trial averaged (right) intra-patient MEG recordings of imagined digits (0–9). Averaging is performed with resampling to preserve dataset cardinality; t-SNE uses perplexity = 30. imagined digit classes. In contrast, clear class-specific clusters emerge after 15-trial averaging. We repeat this analysis in Figure 7 of Appendix C using UMAP [29], which provides a more faithful representation of global inter-class structure. 3.2
Theoretical Motivation
Combining PNA’s artifact injection (Section 3.1 (iii)) with K-trial P averaging yields training inputs of the form x̄ + δ, where x̄ is the trial-averaged signal and δ = p∈P cp Xp is the injected artifact mixture. By construction, δ has zero mean and covariance Σδ = (α2 /K) Σn , where Σn is the empirical artifact covariance estimated from the sampled {cp Xp } ensemble and α is the augmentation scaling parameter, whose effect is visualized with K = 15 in Figure 2. We now show that, in expectation and to leading order, this procedure is equivalent to training on clean inputs with an added penalty that suppresses decoder sensitivity along directions of high artifact variance. Proposition 1 (PNA as anisotropic regularization). Let z(x̄; θ) ∈ RV be the logits of a C 3 decoder, ℓ(z, y) a C 3 loss criterion with targets, y, and define the loss gradient g = ∇z ℓ, loss Hessian Hℓ = ∇2z ℓ, logit Jacobian Jz = ∂x̄ z, and logit Hessian Hzr = ∇2x̄ zr (r = 1, . . . , V ), all evaluated at the clean K-trial average input x̄. Suppose the augmented input xaug = x̄ + δ has E[δ | x̄, y] = 0 and Cov(δ) = (α2 /K)Σn . Then " # V 2 X α2 Eδ [ℓaug (z, y)] = ℓclean (z, y) + Tr Hℓ Jz Σn Jz⊤ + gr Tr(Hzr Σn ) + o αK . (4) 2K r=1 If, in addition, either (a) Tr(Hzr Σn ) ≈ 0 for each r (the decoder is locally linear in artifactcovariance directions), or (b) g ≈ 0 at x̄ (the model predicts near-correctly at this data point), the second-order term simplifies to a loss-curvature-weighted Jacobian penalty: Eδ [ℓaug (z, y)] ≈ ℓclean (z, y) +
α2 Tr Hℓ Jz Σn Jz⊤ . 2K
(5)
Proof. See Appendix B.1. Corollary 1 (Anisotropic Jacobian Regularization under Squared Error Loss). Let ℓ(z, y) = ∥y − z∥22 be the squared error loss. Under the assumptions of Proposition 1, and provided either (a) Tr(Hzr Σn ) ≈ 0 for each r, or (b) g = −2(y − z) ≈ 0 at x̄, applying this loss to augmented data averaged over K trials is equivalent to applying it to clean data, with an anisotropic Jacobian penalty: α2 Eδ [ℓaug (z, y)] ≈ ℓclean (z, y) + Tr Jz Σn Jz⊤ , K 2 where Σδ = αK Σn is the covariance of the injected artifact noise. Proof. See Appendix B.2.
6
PV Corollary 2 (PNA Regularization under Cross-Entropy Loss). Let ℓ(z, y) = − i=1 yi log(σ(z)i ) P be the softmax cross-entropy loss, where σ(z)i = ezi / j ezj denotes the predicted probability for class i. Under the assumptions of Proposition 1, and provided either (a) Tr(Hzr Σn ) ≈ 0 for each r, or (b) g = p − y ≈ 0 at x̄, the augmented loss reduces to minimizing the clean loss plus a Fisher-weighted anisotropic Jacobian penalty: Eδ [ℓaug (z, y)] ≈ ℓclean (z, y) +
α2 Tr Diag(p) − pp⊤ Jz Σn Jz⊤ , 2K 2
where p = σ(z) is the vector of predicted probabilities and Σδ = αK Σn is the covariance of the injected artifact noise. Proof. See Appendix B.3. Proposition 1 and its corollaries are related to the classical equivalence between Gaussian noise and Tikhonov regularization [4]. Notably, in order to ensure that the resulting regularizer is non-negative, the loss criteria must be convex with respect to the logits (a condition satisfied by both squared error and cross-entropy). The simplification of the theorem to a pure Jacobian penalty requires the second-order logit Hessian terms to vanish, which occurs if either (a) the decoder is locally linear in the directions of artifact noise, or (b) the loss gradient g is approximately zero. For a classifier trained under cross-entropy, condition (b) corresponds to the regime where predictions are well-aligned with targets (p ≈ y). Consequently, this Jacobian approximation becomes increasingly precise as the model converges, characterizing the effective regularization landscape during the later stages of training. 3.3
Preprocessing & Implementation Details
In our pipeline, we perform data augmentation before preprocessing so that augmentation preserves the original relative scales and variances of physiological signals. We follow the preprocessing strategy of [10], applying a robust scaler fitted only to the training data to avoid leakage. Similarly, ICA and augmentation are only applied to training data, and validation/test sets are recorded separately to prevent leakage. Full details are presented in Appendix D.
4
Experiments
We evaluate PNA on the task of classifying imagined digits from MEG data, and compare to a set of common brain-to-speech augmentations. Dataset. We conduct experiments on MegNIST, a single-subject MEG dataset comprising 5 hours (12,000 trials) of class-balanced imagined digits (0–9). We choose MegNIST because (i) to our knowledge, it is the only publicly available MEG dataset for internal speech, and (ii) it includes EOG and ECG reference signals alongside MEG recordings, enabling PNA. Preprocessing and data splits are detailed in Table 4 of Appendix D. Models. For each augmentation setting, we train two models: a multilayer perceptron (MLP) and EEGNet [22]. This choice enables a direct comparison between a simple architecture with flattened inputs and a modern convolutional architecture operating on 2D (unflattened) inputs. We optimize the model hyperparameters with grid search, and report final values in Appendix D. 4.1
Decoding Performance and the Effect of Averaging
Table 1 shows an ablation of PNA, using various combinations of raw data, clean data, augmented data, and 10-trial averaging. The framework does not overcome the SNR bottleneck in the single-trial, however its impact is substantially amplified when combined with multi-trial averaging. With 10-trial averaging, accuracies increase from 54.2%±1.1% to 57.4%±2.0% for MLP and from 73.0%±1.4% to 76.3% ± 1.1% for EEGNet when augmented samples are included. This result suggests that augmentation is most effective once averaging has mitigated extreme low-SNR conditions, allowing models to better exploit invariances introduced by PNA. Full results across models and training configurations are provided in Appendix E. We use a 1:1 raw-to-augmented data ratio for simplicity and as supported by Figure 5. 7
Decoding accuracy
0.6 0.5
¢ = 0:044
Raw only Raw + Augmented ¢ = 0:046
0.4
¢ = 0:041
0.3
¢ = 0:048 ¢ = 0:028
0.2
Random Chance Decoding
t-SNE
0.1
1
2
4
K (trials averaged)
8
16
Figure 4: MLP decoding accuracy uplift from PNA augmentation and associated t-SNE embeddings at various levels of averaging. The top plot uses 10k training trials for raw-only runs (red) and an additional 10k augmented trials for raw + augmented runs (blue), both over 5 seeds. The bottom plots show t-SNE embeddings of raw (red triangles) and augmented (blue dots) data at each level of averaging. Table 1: Ablation of Data Augmentation Strategies. Raw Only, Clean Only, and PNA Only each use 10k training trials. Raw + PNA applies augmentation online during training with probability p = 0.5. PNA is applied before 10-trial averaging. Random chance decoding yields an expected accuracy of 0.1. Best results over 5 seeds are shown in bold; standard errors are reported in scriptsize. MLP
EEGN ET
Clean Only + AVERAGING
0.255 ± 0.008 0.581 ± 0.015
0.321 ± 0.004 0.678 ± 0.022
Raw Only (p = 0) + AVERAGING
0.255 ± 0.008 0.542 ± 0.011
0.337 ± 0.005 0.730 ± 0.014
Raw + PNA (p = 0.5) + AVERAGING
0.262 ± 0.006 0.574 ± 0.020
0.332 ± 0.011 0.763 ± 0.011
PNA Only (p = 1) + AVERAGING
0.267 ± 0.005 0.504 ± 0.030
0.319 ± 0.017 0.626 ± 0.015
Data Strategy
4.2
Comparison to Augmentation Baselines
Table 2 compares the effects of several augmentation baselines, with and without 10-trial averaging. We find that PNA, when added to existing raw data, is able to outperform training on raw data with MLP, and on 10-trial averaged data with EEGNet, the latter of which achieves the highest decoding accuracy among all baselines, 76.3% ± 1.1%. Notably, Figure 4 provides empirical evidence that the enhancement under PNA remains relatively consistent beyond 2 trials of averaging.
5
Discussion
We present Physiological Noise Augmentation (PNA), an ICA-based framework that enforces invariance to physiological artifacts in non-invasive brain-to-speech decoding. By remixing artifact components to generate diverse, label-preserving samples, PNA improves decoding accuracy by 4.7% over real-data training alone for MEG-based imagined digit classification with EEGNet. We further demonstrate that PNA and multi-trial averaging are complementary: PNA reduces sensi8
Decoding accuracy
0.64
0.80
0.60
0.76
0.56
0.72
0.52
0 1:4 1:2
1:1
3:2
(A) MLP
4:1
0.68
9:1
0 1:4 1:2
Augmented-to-Raw Data Ratio
1:1
3:2
4:1
(B) EEGNet
9:1
Figure 5: Decoding accuracy scales with PNA samples. Models are trained on a fixed budget of 10k raw trials; the x-axis represents the ratio of augmented-to-raw data, ranging from 0 (baseline, vertical dotted line) to 90k (9 : 1 ratio). Shaded regions denote 95% confidence intervals (n = 5 independent seeds); all runs use 10-trial averaging. Table 2: Comparison against existing brain-to-speech augmentations. All augmentation methods use 10k raw trials with augmentation applied online with probability 0.5. None uses 10k raw trials without augmentation. Averaging is performed after PNA and before each baseline augmentation. Best results are shown in bold; standard errors are reported in scriptsize. MLP
EEGN ET
Augmentation Type
Single-Trial
10-Trial
Single-Trial
10-Trial
None (Raw Data)
0.255 ±0.008
0.542 ±0.011
0.337 ±0.005
0.730 ±0.014
Raw + White Noise Raw + Smooth Time Mask Raw + Frequency Shift Raw + Temporal Shift Raw + Amplitude Scaling
0.261 ±0.007 0.256 ±0.008 0.270 ±0.005 0.265 ±0.004 0.260 ±0.010
0.578 ±0.011 0.573 ±0.015 0.596 ±0.022 0.584 ±0.019 0.555 ±0.017
0.329 ±0.006 0.317 ±0.009 0.321 ±0.010 0.328 ±0.006 0.341 ±0.009
0.747 ±0.012 0.713 ±0.023 0.739 ±0.016 0.736 ±0.016 0.735 ±0.021
Raw + PNA
0.262 ±0.006
0.574 ±0.020
0.332 ±0.011
0.763 ±0.011
tivity to tracked nuisances, while averaging suppresses residual task-agnostic variability, together approximating Jacobian regularization. Limitations and Future Work. PNA currently requires artifact reference signals to be recorded during data collection. Future work will investigate classification methods to identify nuisance artifact components from ICA on MEG, eliminating the need for reference channels and substantially expanding the range of compatible datasets. In addition, MegNIST is limited to a single subject performing a constrained task. Extending PNA to multi-subject datasets and broader internal speech vocabularies will clarify the transferability of artifacts. More broadly, PNA is designed to benefit from access to a wider range of task-agnostic physiological artifacts. Beyond EOG and ECG, future work should incorporate additional high-amplitude reference modalities, such as EMG, to maximally teach the decoder to become invariant to task-irrelevant artifacts. Impact and Implications. The development of PNA represents a methodological shift from heuristic input-level perturbations toward a physiologically-grounded regularization framework. PNA provides a principled mechanism for injecting domain-specific inductive biases into neural decoders by explicitly drawing from reference channels to isolate structured nuisances. Our theoretical and empirical results demonstrate that the associated anisotropic Jacobian penalty substantially reduces the “repetition burden” required for high-accuracy decoding, directly addressing the core latency–accuracy trade-off that has long hindered the clinical viability of non-invasive BCIs. More broadly, PNA establishes a scalable template for artifact-aware machine learning in any neural recording modality where referencetracked noise typically confounds task-relevant signals. 9
References [1] Gopala K Anumanchipalli, Josh Chartier, and Edward F Chang. Speech synthesis from neural decoding of spoken sentences. Nature, 568(7753):493–498, 2019. [2] Anthony J. Bell and Terrence J. Sejnowski. An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6):1129–1159, 1995. doi: 10.1162/ neco.1995.7.6.1129. [3] N. Birbaumer, N. Ghanayim, T. Hinterberger, I. Iversen, B. Kotchoubey, A. Kübler, J. Perelmouter, E. Taub, and H. Flor. A spelling device for the paralysed. Nature, 398(6725):297–298, Mar 1999. ISSN 1476-4687. doi: 10.1038/18581. URL https://doi.org/10.1038/18581. [4] Chris M Bishop. Training with noise is equivalent to tikhonov regularization. Neural computation, 7(1):108–116, 1995. [5] Nicholas S. Card, Maitreyee Wairagkar, Carrina Iacobacci, Xianda Hou, Tyler Singer-Clark, Francis R. Willett, Erin M. Kunz, Chaofei Fan, Maryam Vahdati Nia, Darrel R. Deo, Aparna Srinivasan, Eun Young Choi, Matthew F. Glasser, Leigh R. Hochberg, Jaimie M. Henderson, Kiarash Shahlaie, Sergey D. Stavisky, and David M. Brandman. An accurate and rapidly calibrating speech neuroprosthesis. New England Journal of Medicine, 391(7):609–618, 2024. doi: 10.1056/NEJMoa2314132. [6] Pierre Comon. Independent component analysis, a new concept? Signal processing, 36(3): 287–314, 1994. [7] Stéphane d’Ascoli, Corentin Bel, Jérémy Rapin, Hubert Banville, Yohann Benchetrit, Christophe Pallier, and Jean-Rémi King. Towards decoding individual words from non-invasive brain recordings. Nature Communications, 16(1):10521, Nov 2025. ISSN 2041-1723. doi: 10.1038/ s41467-025-65499-0. [8] George D Dawson. A summation technique for the detection of small evoked potentials. Electroencephalography & clinical neurophysiology, 1954. [9] Xabier de Zuazo, Ibon Saratxaga, and Eva Navas. Megconformer: Conformer-based meg decoder for robust speech and phoneme classification. arXiv e-prints, page arXiv:2512.01443, December 2025. doi: 10.48550/arXiv.2512.01443. [10] Alexandre Défossez, Charlotte Caucheteux, Jérémy Rapin, Ori Kabeli, and Jean-Rémi King. Decoding speech perception from non-invasive brain recordings. Nature Machine Intelligence, 5(10):1097–1107, Oct 2023. ISSN 2522-5839. doi: 10.1038/s42256-023-00714-5. [11] Lawrence Ashley Farwell and Emanuel Donchin. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and clinical Neurophysiology, 70(6):510–523, 1988. [12] Valery L Feigin, Benjamin A Stark, Catherine Owens Johnson, Gregory A Roth, Catherine Bisignano, Gdiom Gebreheat Abady, Mitra Abbasifard, Mohsen Abbasi-Kangevari, Foad AbdAllah, Vida Abedi, et al. Global, regional, and national burden of stroke and its risk factors, 1990–2019: a systematic analysis for the global burden of disease study 2019. The Lancet Neurology, 20(10):795–820, 2021. [13] GBDTB. Global, regional, and national burden of traumatic brain injury and spinal cord injury, 1990-2016: a systematic analysis for the global burden of disease study 2016. Lancet Neurology, 18(1):56–87, 2019. [14] Alexandre Gramfort, Martin Luessi, Eric Larson, Denis A Engemann, Daniel Strohmeier, Christian Brodbeck, Roman Goj, Mainak Jas, Teon Brooks, Lauri Parkkonen, et al. Meg and eeg data analysis with mne-python. Frontiers in Neuroinformatics, 7:267, 2013. [15] Chao He, Jialu Liu, Yuesheng Zhu, and Wencai Du. Data augmentation for deep neural networks model in eeg classification task: a review. Frontiers in Human Neuroscience, 15:765525, 2021. 10
[16] Aapo Hyvärinen. Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks, 10(3):626–634, 1999. doi: 10.1109/72.761722. [17] Aapo Hyvärinen and Erkki Oja. Independent component analysis: algorithms and applications. Neural Networks, 13(4-5):411–430, 2000. doi: 10.1016/S0893-6080(00)00026-5. [18] Dulhan Jayalath, Gilad Landau, and Oiwi Parker Jones. Unlocking non-invasive brain-to-text. arXiv e-prints, page arXiv:2505.13446, May 2025. doi: 10.48550/arXiv.2505.13446. [19] Wei-Bang Jiang, Li-Ming Zhao, and Bao-Liang Lu. Large brain model for learning generic representations with tremendous eeg data in bci. arXiv preprint arXiv:2405.18765, 2024. [20] Tzyy-Ping Jung, Scott Makeig, Colin Humphries, Te-Won Lee, Martin J Mckeown, Vicente Iragui, and Terrence J Sejnowski. Removing electroencephalographic artifacts by blind source separation. Psychophysiology, 37(2):163–178, 2000. [21] Tom Ko, Vijayaditya Peddinti, Daniel Povey, Michael L Seltzer, and Sanjeev Khudanpur. A study on data augmentation of reverberant speech for robust speech recognition. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5220–5224. IEEE, 2017. [22] Vernon J. Lawhern, Amelia J. Solon, Nicholas R. Waytowich, Stephen M. Gordon, Chou P. Hung, and Brent J. Lance. Eegnet: a compact convolutional neural network for eeg-based brain–computer interfaces. Journal of Neural Engineering, 15(5):056013, July 2018. doi: 10.1088/1741-2552/aace8c. [23] Eric C. Leuthardt, Gerwin Schalk, Jarod Roland, Andrew Rouse, and Daniel Moran. Evolution of brain–computer interfaces: Going beyond classic motor physiology. Frontiers in Neuroscience, 15:599549, 2021. doi: 10.3389/fnins.2021.599549. [24] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019. URL https://openreview.net/ forum?id=Bkg6RiCqY7. arXiv:1711.05101. [25] Jing Luo, Yaojie Wang, Rong Xu, Guangming Liu, Xiaofan Wang, and Yijing Gong. Channel drop out: A simple way to prevent cnn from overfitting in motor imagery based bci. In International Conference of Pioneering Computer Scientists, Engineers and Educators, pages 443–452. Springer, 2021. [26] Stéphanie Martin, Peter Brunner, Chris Holdgraf, Hans-Jochen Heinze, Nathan E. Crone, Jochem Rieger, Gerwin Schalk, Robert T. Knight, and Brian N. Pasley. Decoding spectrotemporal features of overt and covert speech from the human cortex. Frontiers in Neuroengineering, 7:14, 2014. doi: 10.3389/fneng.2014.00014. [27] Stephanie Martin, Peter Brunner, Iñaki Iturrate, José del R. Millán, Gerwin Schalk, Robert T. Knight, and Brian N. Pasley. Word pair classification during imagined speech using direct brain recordings. Scientific Reports, 6:25803, 2016. doi: 10.1038/srep25803. [28] Stephanie Martin, Iñaki Iturrate, José del R. Millán, Robert T. Knight, and Brian N. Pasley. Decoding inner speech using electrocorticography: Progress and challenges toward a speech prosthesis. Frontiers in Neuroscience, Volume 12 - 2018, 2018. ISSN 1662453X. doi: 10.3389/fnins.2018.00422. URL https://www.frontiersin.org/journals/ neuroscience/articles/10.3389/fnins.2018.00422. [29] Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018. [30] David A. Moses, Sean L. Metzger, Jessie R. Liu, Gopala K. Anumanchipalli, Joseph G. Makin, Pengfei F. Sun, Josh Chartier, Maximilian E. Dougherty, Patricia M. Liu, Gary M. Abrams, Adelyn Tu-Chan, Karunesh Ganguly, and Edward F. Chang. Neuroprosthesis for decoding speech in a paralyzed person with anarthria. New England Journal of Medicine, 385(3):217–227, 2021. doi: 10.1056/NEJMoa2027540. 11
[31] Jerrin Thomas Panachakel and Angarai Ganesan Ramakrishnan. Decoding covert speech from eeg-a comprehensive review. Frontiers in Neuroscience, Volume 15 - 2021, 2021. ISSN 1662453X. doi: 10.3389/fnins.2021.642251. URL https://www.frontiersin.org/journals/ neuroscience/articles/10.3389/fnins.2021.642251. [32] Daniel S. Park, William Chan, Yu Zhang, Chung-Cheng Chiu, Barret Zoph, Ekin D. Cubuk, and Quoc V. Le. Specaugment: A simple data augmentation method for automatic speech recognition. arXiv e-prints, page arXiv:1904.08779, April 2019. doi: 10.48550/arXiv.1904. 08779. [33] Jin Park, Jee-Eun Kim, and Tae-Jin Song. The global burden of motor neuron disease: an analysis of the 2019 global burden of disease study. Frontiers in Neurology, 13:864339, 2022. [34] Xiaomei Pei, Dennis L. Barbour, Eric C. Leuthardt, and Gerwin Schalk. Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans. Journal of Neural Engineering, 8(4):046028, 2011. doi: 10.1088/1741-2560/8/4/046028. [35] Luca Pion-Tonachini, Ken Kreutz-Delgado, and Scott Makeig. Iclabel: An automated electroencephalographic independent component classifier, dataset, and website. NeuroImage, 198: 181–197, 2019. [36] Timothée Proix, Jaime Delgado Saa, Andy Christen, Stephanie Martin, Brian N Pasley, Robert T Knight, Xing Tian, David Poeppel, Werner K Doyle, Orrin Devinsky, et al. Imagined speech can be decoded from low-and cross-frequency intracranial eeg features. Nature communications, 13 (1):48, 2022. [37] David Snyder, Guoguo Chen, and Daniel Povey. Musan: A music, speech, and noise corpus. arXiv preprint arXiv:1510.08484, 2015. [38] Samuel Sutton, Margery Braren, Joseph Zubin, and E. R. John. Evoked-potential correlates of stimulus uncertainty. Science, 150(3700):1187–1188, 1965. doi: 10.1126/science.150.3700. 1187. URL https://www.science.org/doi/abs/10.1126/science.150.3700.1187. [39] Laurens Van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008. [40] Ricardo Vigário, Jaakko Sarela, Veikko Jousmiki, Matti Hamalainen, and Erkki Oja. Independent component approach to the analysis of eeg and meg recordings. IEEE transactions on biomedical engineering, 47(5):589–593, 2002. [41] Ziwei Wang, Siyang Li, Xiaoqing Chen, and Dongrui Wu. Time–frequency transform based eeg data augmentation for brain–computer interfaces. Knowledge-Based Systems, 311:113074, 2025. ISSN 0950-7051. doi: https://doi.org/10.1016/j.knosys.2025.113074. [42] Francis R. Willett, Erin M. Kunz, Chaofei Fan, Donald T. Avansino, Guy H. Wilson, Eun Young Choi, Foram Kamdar, Matthew F. Glasser, Leigh R. Hochberg, Shaul Druckmann, Krishna V. Shenoy, and Jaimie M. Henderson. A high-performance speech neuroprosthesis. Nature, 620 (7976):1031–1036, 2023. doi: 10.1038/s41586-023-06377-x. [43] Irene Winkler, Stefan Haufe, and Michael Tangermann. Automatic classification of artifactual ica-components for artifact removal in eeg signals. Behavioral and brain functions, 7(1):30, 2011. [44] Jonathan R. Wolpaw, Niels Birbaumer, Dennis J. McFarland, Gert Pfurtscheller, and Theresa M. Vaughan. Brain-computer interfaces for communication and control. Clinical Neurophysiology, 113(6):767–791, June 2002. doi: 10.1016/S1388-2457(02)00057-3.
12
A
Additional Background
A.1
Independent Component Analysis
Independent component analysis (ICA) is modeled on the assumption that a matrix of spatiotemporal sensor recordings, X, arises from instantaneous linear mixing of N independent sources, S = [s1 , . . . , sT ] ∈ RN ×T according to xt = Ast ,
for t = 1, . . . , T,
or equivalently, X = AS,
(6)
where A ∈ RC×N is an unknown mixing matrix. Independent component analysis (ICA) seeks to recover latent independent sources (rows of S), by estimating an unmixing matrix W ∈ RN ×C which yields component estimates, b = WX, S
(7)
b are as statistically independent as possible [6]. A canonical objective is to pick such that the rows of S the optimal unmixing matrix, W∗ , to minimize the mutual information of the component random variables: W⋆ ∈ arg min I(ŝ1,: , . . . , ŝN,: ) ,
(8)
W∈RN ×C
b where ŝi,: ∈ R1×T is the i-th row of S. Notably, the estimation of W was originally formulated as training a single-layer neural network for unsupervised learning, as in the Infomax ICA algorithm [2]. However, current implementations typically use the FastICA algorithm, which improves computational efficiency by using a fixed-point iteration to maximize a non-Gaussianity contrast as proxy for statistical independence [16, 17].3 A.2
Multi-Trial Averaging and Resampling
We define the K-trial average for a set of trials {X(k) }K k=1 as the spatiotemporal mean: K
X=
1 X (k) X . K
(9)
k=1
In brain-to-speech decoding, K-trial averaging at inference requires the patient to imagine the target stimulus K times. This approach assumes that each trial consists of a phase-locked neural signal plus independent stochastic noise; consequently, averaging preserves the coherent signal component while the noise variance is attenuated by a factor of K. For K condition-matched trials with covariance Σ, the √ sample average X satisfies Cov(X) = Σ/K under the assumption of independence, yielding a K scaling in Signal-to-Noise Ratio (SNR) [8]. During training, we consider two averaging strategies: 1. Averaging without resampling: Trials are partitioned into disjoint groups of size K and averaged. While simple, this reduces the effective training set size by a factor of K, limiting input variability. 2. Averaging with resampling: Subsets of size K are randomly drawn (without replacement N within each subset) from the full training set. For a dataset of size N , there are K possible subsets, a combinatorial explosion (when min(K, N − K) >> 0) that allows for massive data augmentation while preserving the original dataset’s coverage. In this work, we mainly employ averaging with resampling. To ensure a fair comparison with singletrial models, we fix the number of generated samples to match the original training set cardinality N , 3 Most ICA algorithms also approximate the mixing matrix, A, by taking an inverse or pseudoinverse of W.
13
unless otherwise stated. For robustness, however, we also also re-run certain main results to verify that outcomes also hold empirically under averaging without resampling (Figure 9 in Appendix E). Crucially, averaging implicitly infuses task awareness into output data, as phase-locked portions of the signal remain relatively unaffected, while other components destructively interfere.4
B
Proofs of Theoretical Results
B.1
Proof of Proposition 1
Proof. The proof proceeds by applying a multivariate Taylor expansion to the logits with respect to the input noise, then expanding the loss with respect to the perturbed logits. Step 1: Expansion of the logits. Let ∆z = z(x̄ + δ; θ) − z(x̄; θ). Expanding each coordinate zr to second order around x̄: ∆zr = (∇x̄ zr )⊤ δ + 12 δ ⊤ Hzr δ + O(∥δ∥3 ), Step 2: Expansion of the loss. Expanding ℓ(z + ∆z, y) around z = z(x̄; θ): ℓ(z + ∆z, y) = ℓ(z, y) + g ⊤ ∆z + 21 ∆z ⊤ Hℓ ∆z + O(∥∆z∥3 ). Since ∥∆z∥ = O(∥δ∥), the trailing term is O(∥δ∥3 ). Step 3: Substitution. Substituting the expression for ∆z: • g ⊤ ∆z = g ⊤ Jz δ + 12
PV
r=1 gr δ
⊤
Hzr δ + O(∥δ∥3 ).
• 21 ∆z ⊤ Hℓ ∆z = 12 δ ⊤ Jz⊤ Hℓ Jz δ + O(∥δ∥3 ), where the cross term (Jz δ)⊤ Hℓ q(δ) contributes at O(∥δ∥3 ) and is absorbed into the remainder. Collecting, ℓ(z + ∆z, y) = ℓ(z, y) + g ⊤ Jz δ + 21 δ ⊤ Jz⊤ Hℓ Jz δ + 21
V X
gr δ ⊤ Hzr δ + O(∥δ∥3 ).
r=1
Step 4: Expectation and final substitution. Taking expectation over δ given (x̄, y), using Eδ [δ] = 0 and Eδ [δ ⊤ Aδ] = Tr(AΣδ ): Eδ [ℓaug (z, y)] = ℓclean (z, y) + 12 Tr(Jz⊤ Hℓ Jz Σδ ) + 21
V X
gr Tr(Hzr Σδ ) + Eδ [O(∥δ∥3 )].
r=1 2
For δ with Cov(δ) = (α /K)Σn and finite third absolute moment, Eδ [∥δ∥3 ] = O((α2 /K)3/2 ), so the remainder is o(α2 /K) as α2 /K → 0. Using trace cyclicity, Tr(Jz⊤ Hℓ Jz Σδ ) = Tr(Hℓ Jz Σδ Jz⊤ ), and substituting Σδ = (α2 /K)Σn : Eδ [ℓaug (z, y)] = ℓclean (z, y) +
B.2
V 2 α2 α2 X Tr(Hℓ Jz Σn Jz⊤ ) + gr Tr(Hzr Σn ) + o αK . 2K 2K r=1
Proof of Corollary 1
Proof. Under regime (a) or (b) of Proposition 1, Eδ [ℓaug (z, y)] ≈ ℓclean (z, y) +
α2 Tr Hℓ Jz Σn Jz⊤ . 2K
(10)
4 MegNIST and similar datasets prompt participants to imagine each label at a screen-cued moment, enabling precise
temporal alignment across trials.
14
For the squared error loss ℓ(z, y) = ∥y − z∥22 =
PV
2 i=1 (yi − zi ) , the loss Hessian is constant:
Hℓ = ∇2z ℓ(z, y) = 2IV , where IV is the V × V identity matrix. Substituting into the regularization term of Equation (10): α2 α2 Tr(2IV Jz Σn Jz⊤ ) = Tr(Jz Σn Jz⊤ ), 2K K yielding Eδ [ℓaug (z, y)] ≈ ℓclean (z, y) +
α2 Tr(Jz Σn Jz⊤ ), K
as claimed. B.3
Proof of Corollary 2
Proof. Under regime (a) or (b) of Proposition 1, Eδ [ℓaug (z, y)] ≈ ℓclean (z, y) +
α2 Tr Hℓ Jz Σn Jz⊤ . 2K
(11)
PV For the softmax cross-entropy loss ℓ(z, y) = − i=1 yi log pi with p = σ(z), the gradient with respect to the j-th logit is gj = pj − yj , so the Hessian elements are ( pj (1 − pj ) if i = j, ∂gj ∂pj = = (Hℓ )ij = ∂zi ∂zi −pi pj if i ̸= j, using the standard derivative of the softmax. In closed form, Hℓ = ∇2z ℓ(z, y) = Diag(p) − pp⊤ , the predictive Fisher information at p. Substituting into Equation (11) yields Eδ [ℓaug (z, y)] ≈ ℓclean (z, y) +
α2 Tr [Diag(p) − pp⊤ ] Jz Σn Jz⊤ , 2K
as claimed.
C A
Additional Data Visualization C
B
Figure 6: ICA artifact component selection for EOG and ECG signals. The topographies captured in column A show the relative strengths of the ICA mapping from source to sensor space for a sample EOG (top) and ECG (bottom) component. The corresponding ICA component waveforms are captured in column B, where they are compared to the reference EOG and ECG signals. Finally, column C show the absolute Pearson correlations between the first 50 ICA components and the artifact waveform for EOG (top) and ECG (bottom). 15
Single Trials
15-Trial Averaged 9
1
8
2
3
7 6
7
5 9 8
5 4
6
3
Imagined Digit (0-9)
4
2
0
1 0
Figure 7: UMAP embeddings of intra-patient MEG recordings for imagined digits (0–9), shown for single trials (left) and 15-trial averages (right). Averaging improves cluster separability, and—unlike t-SNE, UMAP preserves aspects of global structure, revealing a separation between representations of digit 0 and digits 1–9
Unlike t-SNE, UMAP preserves some global structure, and the observed separation of digit 0 from digits 1–9 may reflect a systematic difference in neural representation rather than a purely local clustering artifact. This could arise from semantic differences between zero and non-zero numerals or from task-related strategies. While intgriguing, the effect is preliminary and must be validated across multiple subjects and modalities.
Averaged (5 trials)
Averaged (10 trials)
Averaged (15 trials)
t-SNE
PCA
Single Trials
Raw
Clean
Augmented
Figure 8: PCA and t-SNE embeddings of raw, augmented (ocular and cardiac), and clean data at various levels of averaging. While raw and augmented data remain closely aligned, the cluster of clean data separates with additional averaging, indicating that PNA correctly restores the corruption to artifact-cleaned data that would be seen at test time. Under t-SNE, label-wise clusters emerge for substantial averaging. 16
D
Model, Pre-processing and Baseline Augmentation Parameters Table 3: Model hyperparameters used in our experiments. Parameter
Value
Averaging # Samples Averaged (K) # Training Samples (using resampling)
10 10,000
Architectural Parameters MLP # Hidden Layers MLP Hidden Layer Width EEGNet # Feature Maps (F1 ) EEGNet # Spatial Filters per Feature Map (D) EEGNet # Pointwise Filters (F2 ) EEGNet Kernel Length
1 128 16 4 64 25
Training Batch Size Maximum # Training Epochs Early Stopping MLP Dropout Rate MLP Learning Rate MLP Weight Decay EEGNet Learning Rate EEGNet Weight Decay Optimizer
64 200 After 25 Epochs 0.35 0.0001 0.0005 0.0005 0.001 AdamW [24]
Table 4: Preprocessing and data splits used on MegNIST data. We apply channel-wise robust scaling to account for different magnitudes of gradiometer (T/m) and magnetometer (T) channels. Parameter
Value
Input MEG Data # Gradiometer Channels # Magnetometer Channels Original Sampling Rate
102 204 1000 Hz
Preprocessing Downsampled rate High-Pass Filter Low-Pass Filter Channel-wise Standardization Data Splits Train : Val : Test
250 Hz 0.1 Hz 120 Hz IQR = [-1, 1] 10 : 1 : 1
Table 5: Baseline augmentation hyperparameters, tuned to optimize single-trial validation set performance. Parameter Baseline Augmentation White Noise std Smooth Time Mask Length Frequency Shift Temporal Shift Amplitude Scaling
Value 0.1 50 0.5 Hz 1 step uniformly drawn from [0.9, 1.1]
17
Decoding accuracy
E
Additional Experimental Results 0.56
0.68
0.52
0.64
0.48
0.60
0.44
0.56
0.40
0 1:4 1:2
1:1
3:2
(A) MLP
4:1
9:1
0.52
0 1:4 1:2
Augmented-to-Raw Data Ratio
1:1
3:2
(B) EEGNet
4:1
9:1
Decoding accuracy
Figure 9: Decoding accuracy scales with PNA samples (averaging without resampling). Models are trained on a fixed budget of 10k raw trials (1k post-averaging); the x-axis represents the ratio of augmented-to-raw data, ranging from 0 (baseline, vertical dotted line) to 90k (9 : 1 ratio; 9k post-averaging). Shaded regions denote 95% confidence intervals (n = 5 independent seeds); all runs 10-trial averaging.
0.30
0.36
0.28
0.34
0.26
0.32
0.24
0.30
0.22
0 1:4 1:2
1:1
3:2
(A) MLP
4:1
9:1
0.28
0 1:4 1:2
Augmented-to-Raw Data Ratio
1:1
3:2
(B) EEGNet
4:1
9:1
Figure 10: Single-Trial Decoding accuracy under various PNA data infusions. Models are trained on a fixed budget of 10k raw trials; the x-axis represents the ratio of augmented-to-raw data, ranging from 0 (baseline, vertical dotted line) to 90k (9 : 1 ratio). Shaded regions denote 95% confidence intervals (n = 5 independent seeds). For “Raw + Aug.”, we concatenate varying amounts of augmented samples to 10,000 raw samples, to create final dataset sizes of 10,500, 11,000, 11,500, 12,000, 12,500, 15,000, 20,000, 25,000, 50,000 and 100,000.
F
Compute Resources
We used NVIDIA H100, L40S, RTX 8000, A100 and RTX A6000 GPUs with up to 96GiB of GPU memory and with up to 1TiB of RAM. We estimate that we used up to 5,000 hours of GPU compute over the course of this work, including final results (2,000 hours), hyperparameter searches (1,500 hours) and further experiments which were not included in the paper (500 hours).
18