Multimodal Pretraining for Generalizable EEG Representation Learning
arXiv:2607.21384v1 [cs.AI] 23 Jul 2026
Targol Bakhtiarvand, Jugal Kalita, Adham Atyabi Abstract— Electroencephalography (EEG) models used for epilepsy are often limited to specific datasets and tasks. This limited approach can make it challenging to apply these models across different datasets or in various situations. However, recent studies in foundation models and self-supervised learning suggest that an adaptable EEG backbone could support a range of EEG related tasks. In this study, we have developed a multimodal EEG foundation model that combines a raw signal encoder based on the Mamba architecture, a Vision Transformer (ViT)-style encoder for time-frequency data, and a lightweight encoder for text, all within a shared embedding space. The pretraining process relies on several innovative techniques, such as masked modeling, cross-view contrastive alignment, and temporal consistency losses. These methods are designed to create rich, seizure-relevant representations without requiring labeled data. To assess the efficacy and generalization of our pretrained model, we fine-tuned it on the canonical CHB-MIT seizure detection benchmark and additional seizure detection datasets, and conducted extensive experiments comparing different model variants. On the standard CHB-MIT split, our best single model achieved an AUROC of 0.874, and an ensemble variant reached 0.878 AUROC, representing state-ofthe-art performance on this benchmark. In addition to standard train-test splits, we evaluated performance under a leave-onesubject-out (LOSO) protocol, which is rarely reported in prior EEG seizure modeling work and highlights the difficulty of patient-independent seizure detection, with a mean LOSO balanced accuracy of 0.558 across 19 subjects. Across datasets and evaluation settings, our multimodal foundation model enabled robust seizure detection and straightforward adaptation to new seizure detection scenarios, while also supporting interpretable seizure localization. These findings highlight the potential of EEG-specific foundation models as versatile tools that can be reused not just for seizure detection but also for broader analyses of EEG data related to seizures.
I. INTRODUCTION Electroencephalography (EEG) recordings enable non-invasive high-temporal-resolution measurements of brain activity but are difficult to model. The signals have low signal-to-noise ratio, are prone to physiological and environmental artifacts, and differ substantially between subjects, recording montages, paradigms, and recording devices [1]. Variations in sampling rates, channel layouts, referencing schemes, and annotation quality across public epilepsy datasets impairs the generalizability and reproducibility of learned models [2]. Supervised EEG encoders also rely on costly expert annotations and often learn models that do not generalize to new tasks and datasets [3]. These challenges have motivated recent work on large-scale, self-supervised EEG foundation models Department of orado Colorado
Computer Science, Springs, Colorado
University of Springs, CO,
{tbakhtia,jkalita,aatyabi}@uccs.edu
ColUSA
that pretrain on hundreds to thousands of subjects using mask-based reconstruction and transformer backbones [4]. Seizure detection is performed on ictal (and near-ictal) windows of EEG where there is already obvious electrographic abnormality. Typical tasks therefore involve binary classification of short windows as seizure vs. non-seizure in real time [5]. Performance is measured in terms of sensitivity, false-positive rate, and detection delay. State-of-the-art clinical systems for scalp EEG achieve about 75 to 90% sensitivity and 0.1 to 5 false alarms per hour in the monitoring-unit environment [6]. However, these performance figures largely come from studies that do not use strict leave-one-subjectout (LOSO) evaluation or true across-subject generalization tests, which are known to give more realistic estimates of performance on unseen patients [7]. Detection is still imperfect. Seizures that are short, lowamplitude, focal, or obscured by artifacts may go undetected, while interictal abnormalities and artifacts account for most false positives [6]. Furthermore, model generalization to patients and realistic long-term recordings is lacking, even for recent deep learning methods [8]. Multimodal representation learning models raw EEG, time-frequency views (e.g. CWT/STFT spectrograms), and/or textual information (labels/clinical descriptions) together, allowing the detector to leverage complementary ictal signatures. Joint models that fuse raw EEG with spectrogram-like features via dual-branch or multi-stream architectures consistently achieve higher accuracy, sensitivity, and specificity compared to single-view approaches [9]. Recently, CLIP-style EEG-text models have shown that aligning EEG and text in a joint latent space produces robust, high-performing seizure detectors generalizable across datasets [10]. Together, these findings suggest multimodal learning can increase sensitivity while decreasing false alarms in EEG-based seizure detection [11]. This work introduces a domain-specific foundation model for seizure detection that jointly learns from raw EEG, CWT-based time-frequency scalograms, and associated textual information (labels/reports). Unlike prior LOSO-validated signal-only systems [12] and epilepsy foundation models that focus solely on neurophysiological inputs [13], our model: 1) employs strict LOSO cross-subject evaluation to ensure patient-independent generalization in line with current methodological recommendations and to reveal the substantial performance gap between within-subject and true across-subject seizure detection; 2) learns a shared representation over raw EEG, CWT
spectrograms, and text to capture complementary temporal, spectral, and semantic seizure cues, which we show can be used for interpretable seizure localization; and 3) is pre-trained as a foundation model, enabling efficient fine-tuning and improved performance across seizure-detection benchmarks compared with strong unimodal and non-foundational baselines, including state-of-the-art AUROC on the canonical CHB-MIT split (0.874 for the best single model and 0.878 AUROC with a simple ensemble). The remainder of this paper is organized as follows. Section II reviews related work on EEG seizure detection, self-supervised learning, and multimodal foundation models. Section III describes the unified EEG preprocessing pipeline. Section IV presents the model architecture. Section V reports seizure detection evaluation results across three protocols. Section VI presents interpretability analysis. Section VII discusses key findings, and Section VIII concludes. II. R ELATED W ORKS This section reviews EEG seizure detection, selfsupervised EEG learning, multimodal and foundation models, and RAG in biomedicine, and then highlights how the proposed model differs by jointly aligning raw, timefrequency, and text representations. A. EEG Seizure Detection Classical detectors use hand-crafted time, frequency, or time-frequency features with SVM or ensemble classifiers [14], [15], [16]. Recent deep learning approaches transform EEG into scalogram or imaged representations and report >98% accuracy on public benchmarks [17], [18], but most are fully supervised and evaluated on patient-specific or loosely patient-independent data. B. Self-Supervised Learning for EEG SSL methods exploit unlabeled EEG via temporal prediction, contrastive coding, and masked reconstruction, outperforming supervised networks in low-label regimes and transferring across clinical tasks [19], [20], [21]. C. Multimodal and foundation models Recent EEG foundation models (EFMs) pretrain large transformers on unlabeled EEG for masked reconstruction or contrastive objectives into general-purpose encoders [22]. They review existing EFMs and find that the majority of these approaches assume EEG as a unimodal sequence input (often using patches) that leverages purely raw time-series data without explicit alignment to alternative representational views such as time–frequency maps or grounding to text [23]. LEAD learns from Alzheimer’s-data subject-level detection via subject-regularized transformer and Alzheimer’s-guided contrastive pretraining, demonstrating strongsubject-independent performance at LOSO crossvalidation on two large Alzheimer’s Disease cohorts [24]. CBraMod introduces a criss-cross transformer framework that separately model spatial and temporal dependencies
across brain regions and achieves state-of-the-art performance across 10 BCI tasks and 12 datasets, but is also unimodal EEG without text [22]. BrainRVQ (and related codebook-based large brainwave models) focuses on high-fidelity raw EEG tokenization and masked modeling for general-purpose EEG representation learning, and serves as a strong unimodal foundation baseline in our experiments. Outside EEG, pretraining multimodal physiological foundation models with masked autoencoding applied to ECG, photoplethysmogram (PPG), respiration, and correlated signals find that cross-modal reconstruction and modality dropout yield better downstream task performance and robustness on health tasks [25]. Large EHR-centric foundation models are benchmarked and find that multimodal pretraining often improves predictive performance, but few models generalize across tasks [26]. Reviews of EFMs, however, highlight that much of the evaluation of EFMs are heterogeneous and optimistic. Few models use strict subject-based cross-validation splitting schemes such as LOSO or nested LOSO CV as a default standard despite strong evidence for inter-subject variability [27]. Studies using LOSO as an evaluation basis for EEG disease classification and cross-subject BCIs make similar arguments for the necessity of subject-based CV. Table I compares other studies that performed EFM. Reviews of EFMs highlight that evaluation protocols are heterogeneous and optimistic, with few models adopting strict LOSO as default [27]. Unlike prior unimodal EFMs such as CBraMod and BrainRVQ, our model jointly aligns raw EEG, time-frequency, and text within a single LOSO-validated backbone. III. U NIFIED EEG P IPELINE We validate on CHB-MIT [28], SEED-DV [29], and TUH EEG [30]. All recordings are segmented into nonoverlapping 15-second windows, resampled to 256 Hz (3,840 samples/channel), bandpass filtered (0.5–70 Hz) with optional 50/60 Hz notch filtering, and per-channel z-score normalized. CWT scalograms are computed using a Complex Morlet wavelet over 0.5–70 Hz with F = 64 logarithmically spaced frequency bins, yielding Xtf ∈ RC×F ×T . Channel names are harmonized and dataset-specific orderings enforced to enable a single model to ingest all three datasets. CHB-MIT (23 ch, 256 Hz, seizure labels), SEED-DV (62 ch, 200 Hz, affect labels), and TUH EEG (19 ch, variable Hz, unlabeled) are each segmented into non-overlapping 15second windows at 256 Hz, bandpass filtered (0.5–70 Hz), per-channel z-score normalized, and paired with CWT scalograms (F = 64, 0.5–70 Hz). IV. M ODEL A RCHITECTURE The model learns a shared representation from two informative views of each EEG window: (i) raw timedomain signals, and (ii) time–frequency scalograms generated with Continuous Wavelet Transform (CWT). The raw and time-frequency embeddings are combined to form a single window-level representation. In addition, a lightweight retrieval-based text branch provides EEG-domain context
TABLE I C OMPARISON OF EEG F OUNDATION M ODELS AND M ULTIMODAL A PPROACHES Model / Category Early EEG-FMs (surveyed)
Modalities & Alignment Raw EEG only; sequence/patch transformers LEAD Raw EEG; AD-guided contrastive learning BraMod Raw EEG; criss-cross spatial/temporal modeling Multimodal physiological FMs Multiple physiological signals; cross(non EEG-text) modal reconstruction Proposed model (ours) Raw EEG + time-frequency + text jointly aligned
Evaluation (Subject-Level) Notes Citations Mixed protocols; rarely LOSO/N- Limited and heterogeneous [22], [23] LOSO evaluation protocols LOSO evaluation at subject level Dementia-specific foundation [24] model Cross-dataset evaluation; not text- Designed for broad BCI tasks [22] aligned Patient-wise splits rather than Focus on general health time[25] LOSO EEG series modeling LOSO seizure detection evaluation First EEG foundation model – aligning raw, spectral, and text modalities
L = λm Lmask + λf Lfuture + λc Lnce + λt Ltext
(1)
A. Optimization and Training Stability
Fig. 1. Architecture of the proposed multimodal EEG foundation model. (A) Input: raw EEG windows and CWT scalograms. (B) Dual Encoder: Mamba-based raw encoder, TF Transformer encoder, and retrievalaugmented text branch, each projecting to 256-D. (C) Fusion and pretraining objectives. For downstream seizure detection, only channel adapters and classification head (7.5% of parameters) are fine-tuned.
and is aligned with the EEG embeddings via a contrastive objective. Raw windows are represented as Xraw ∈ RC×T with T = 3840. A 1D convolutional patch embedding with patch size 64 yields N = T /64 = 60 tokens per window. The resulting tokens are passed through a Mamba backbone that generates token-level features zraw ∈ RB×N ×d and a pooled embedding eraw ∈ RB×d with d = 256. Time–frequency windows are stored as Xtf ∈ RC×F ×T , computed via CWT with a Morlet wavelet over 0.5–70 Hz using F = 64 frequency bins. ViT-style Conv2D patch embedding generates time–frequency tokens that are encoded by a lightweight Transformer to produce ztf ∈ RB×Ntf ×d and etf ∈ RB×d , where B represents batch size, Ntf represents the number of time-frequency tokens, and d is the feature dimensionality. The two EEG embeddings are fused via a lightweight MLP: efused = MLP([eraw ; etf ]) ∈ RB×256 . A text branch retrieves top-k passages (k=3–5) from a FAISS-indexed EEG corpus, encodes them with a frozen sentence encoder (384D), and projects to d = 256, aligned to EEG embeddings via contrastive loss. The overall training objective is a weighted combination of four loss terms:
We optimize using AdamW (lr = 3×10−4 , weight decay) for 20,000 steps with batch size 32. A step-wise LR decay is applied at 10k steps. As shown in Fig. 2, total loss drops sharply before 5k steps and plateaus, indicating stable Training losses convergence. The masked re- Fig. 2. construction loss approaches (TUH+SEED). zero rapidly, while InfoNCE drops more slowly as raw and TF embeddings align. The future prediction loss remains consistently low throughout training. B. Cross-View Representation Alignment We monitor InfoNCE loss and cross-view retrieval accuracy during pretraining. As shown in Fig. 3, InfoNCE decreases steadily, with retrieval accuracy in both raw→TF and TF→raw directions approaching 100% by the final checkpoint. This near- Fig. 3. InfoNCE loss (raw↔TF). perfect bi-directional retrieval confirms robust cross-modal alignment. Fig. 4 summarizes alignment progression across checkpoints. After pretraining on TUH EEG and SEED-DV, the encoders are adapted to CHB-MIT for seizure detection. V. S EIZURE D ETECTION E VALUATION A. Experimental Setup All experiments are carried out using the CHB-MIT scalp EEG dataset, which includes long-term recordings from 24 pediatric patients who have intractable epilepsy.All experiments use CHB-MIT scalp EEG (24 subjects), preprocessed as described in SectionIII.
TABLE II LOSO RESULTS PER SUBJECT. E VAL BAL .ACC IS COMPUTED ON THE VALIDATION SUBJECT; T EST BAL .ACC ON THE HELD - OUT TEST SUBJECT.
Fig. 4.
Training progress across checkpoints.
Fine-tuning uses AdamW with cosine annealing and linear warmup over five epochs, batch size 8 with gradient accumulation over 4 steps (effective batch size 32), and binary focal loss (γ = 2.0) with class-weighted positive weighting to address the severe class imbalance. Only 7.5% of parameters, the subject-specific channel adapters and classification head, are updated, keeping the pretrained backbone frozen.
Subject
Eval BAcc
Test BAcc
Test TPR
Test TNR
chb01 chb02 chb03 chb04 chb05 chb06 chb07 chb08 chb09 chb10 chb11 chb12 chb13 chb14 chb15 chb16 chb17 chb18 chb19
0.801 0.642 0.577 0.519 0.592 0.745 0.718 0.736 0.722 0.802 0.730 0.697 0.784 0.771 0.808 0.770 0.781 0.784 0.780
0.495 0.511 0.432 0.414 0.494 0.484 0.488 0.487 0.462 0.642 0.504 0.565 0.635 0.556 0.620 0.647 0.468 0.786 0.817
0.216 0.260 0.152 0.396 0.445 0.699 0.718 0.366 0.640 0.571 0.713 0.562 0.712 0.613 0.568 0.615 0.935 0.908 0.946
0.774 0.762 0.711 0.431 0.542 0.347 0.138 0.729 0.213 0.803 0.125 0.568 0.935 0.433 0.806 0.727 0.000 0.663 0.657
Mean
0.729
0.558
0.570
0.538
B. Evaluation Protocol We benchmark our model across three complementary protocols with increasing amounts of data to systematically understand generalization, from zero-shot cross-subject transfer to complete within-subject adaptation. Leave-One-Subject-Out (LOSO): The strictest and most clinically realistic protocol. For each test subject, all remaining subjects form the training set, and an independent heldout subject serves as the validation set for early stopping. The model receives no data from the test subject during training. This protocol provides a direct estimate of crosssubject generalization to completely unseen patients, which serves as an appropriate benchmark for evaluating clinical deployability. BrainRVQ Fixed-Split Protocol: To ensure our findings can be directly compared with existing benchmarks from foundation models, we also evaluate using the protocol established by BrainRVQ [31]. This involves using subjects 1-19 for training, subjects 20 and 21 for validation, and subjects 22 and 23 for testing. In our results, we focus on balanced accuracy, AUC-PR, and AUROC, following the metrics commonly used in previous studies. C. Model Variants We evaluate a parameter-efficient variant (7.5% params, 1.27M/17M updated) and a 5-fold ensemble (V3) on chb22– 23. D. Quantitative Results 1) LOSO Generalization: Table II reports per-subject and mean results under the LOSO protocol. To our knowledge, this is the first evaluation of an EEG foundation model under strict leave-one-subject-out cross-validation on the CHBMIT dataset. LOSO evaluation is conducted on subjects chb01 to chb19; subjects chb20 to chb23 are withheld as the
fixed test set for the standard benchmark comparison (Section V), and are excluded from LOSO to ensure strict separation between evaluation protocols and prevent any form of data leakage. The mean test balanced accuracy of 0.558 and mean TPR of 0.570 reflect the fundamental difficulty of zero-shot cross-subject seizure generalization, consistent with known inter-subject variability in ictal morphology. The mean LOSO eval balanced accuracy of 0.729, evaluated on the held-out validation set, indicates that the model learns cross-subject transferable representations, but the gap between validation and test performance is substantial, a direct consequence of the high inter-subject variability inherent in pediatric epilepsy recordings. Notably, subjects chb18 and chb19 achieve test balanced accuracy of 0.786 and 0.817 respectively, while chb17 exhibits complete TNR collapse (TNR=0.000), illustrating the high inter-subject variability characteristic of pediatric epilepsy. These results establish an honest lower bound on clinical deployability without any subject-specific adaptation. 2) Comparison with State-of-the-Art: Table III evaluates our model versus published baselines under the BrainRVQ fixed-split protocol. Our parameter-efficient fine-tuning variant yields best balanced accuracy (0.781) and AUCPR (0.709) of all methods, improving over BrainRVQ by 10.2% balanced accuracy and 52.5% AUC-PR. The latter is a substantial improvement on the primary metric used to assess models on imbalanced clinical data: at all operating thresholds, our model finds many more seizures without equivalently raising false alarms. This is while updating only 7.5% of parameters, versus end-to-end fine-tuning across all baselines. Our AUROC of 0.863 is lower than BrainRVQ (0.928), a gap we attribute to early convergence in the efficient fine-tuning regime and the small size of the val/test subject pool in this protocol. Importantly, this AUROC comparison is not directly meaningful across protocols: the
TABLE III C OMPARISON WITH STATE - OF - THE - ART METHODS UNDER THE B RAIN RVQ FIXED - SPLIT PROTOCOL ( CHB 01–19 TRAIN , CHB 20–21 VALIDATION , CHB 22–23 TEST ). B EST RESULTS PER COLUMN ARE HIGHLIGHTED IN BOLD . Method
BAcc
AUC-PR
AUROC
Params Tuned
EEGNet [32] ST-Transformer [33] BENDR [34] BIOT [35] LaBraM [36] CBraMod [22] BrainRVQ [31]
0.566 0.592 0.561 0.707 0.708 0.740 0.709
0.191 0.142 0.307 0.328 0.329 0.369 0.465
0.805 0.824 0.863 0.876 0.868 0.889 0.928
100% 100% 100% 100% 100% 100% 100%
Ours (efficient)
0.781
0.709
0.863
7.5%
TABLE IV 5- FOLD CROSS - VALIDATION ENSEMBLE RESULTS ON CHB 22–23 TEST SUBJECTS . N OTE THAT THIS PROTOCOL DIFFERS FROM THE FIXED - SPLIT COMPARISON IN TABLE III.
Model
AUROC
AUCPR
BAcc
TPR
TNR
BIOT CBraMod BrainRVQ
0.821 0.847 0.871
– – –
– – –
– – –
– – –
Ours V3 mean Ours V3 ensemble
0.874 0.878
0.722 0.729
0.780 0.784
0.797 0.798
0.763 0.770
BrainRVQ fixed-split uses subjects 1–19 for training, giving access to far more subject-specific variation at test time, whereas our efficient variant updates only 7.5% of parameters. The AUC-PR metric, which better reflects performance on the minority seizure class, shows a 52.5% improvement over BrainRVQ, confirming the practical superiority of our approach on imbalanced clinical data. 3) Cross-Validation Ensemble Results: To further validate robustness, we trained five independent model folds on the chb22-23 test subjects using 5-fold cross-validation and report both per-fold mean and ensemble results in Table IV. The 5-fold mean achieves AUROC 0.874 ± 0.006 and AUCPR 0.722 ± 0.007, while the ensemble (probability averaging across folds) achieves AUROC 0.878 and AUC-PR 0.729, surpassing the previous state-of-the-art BrainRVQ (AUROC 0.871) under this evaluation setting. VI. INTERPRETABILITY ANALYSIS To validate that the model’s predictions are grounded in physiologically meaningful EEG features rather than spurious correlates, we present two complementary analyses: gradient-based saliency mapping and temporal seizure-onset localization. A. GradCAM Saliency Analysis Figure 5 shows a GradCAM saliency analysis for a correctly identified seizure window from the subject chb22, who has a predicted probability of 0.717. This analysis illustrates how different parts of the model contribute to its decision by computing gradients from both raw EEG data and timefrequency data separately. In the raw EEG branch (found
Fig. 5. GradCAM-based interpretability analysis for a seizure window from subject chb22 (predicted probability = 0.717). Top-left: Raw EEG signals (23 channels) with GradCAM saliency overlay (red indicates high importance), highlighting salient transient events. Top-right: Attention pooling weights over time from the raw branch, showing increasing focus toward later segments. Bottom-left: Time-frequency GradCAM map (frequency × time), with high saliency concentrated in mid-to-high frequency bands corresponding to ictal activity. Bottom-right: Attention weights from the time-frequency branch, confirming frequency-localized importance. These results demonstrate that the model focuses on physiologically meaningful EEG patterns.
in the top-left of the figure), we see that high-saliency time patches correspond with noticeable large-amplitude spikes across several channels, particularly at around 3 seconds, 9 seconds, and 14 seconds. These spikes align with what’s known as ictal spike-and-wave patterns. Looking at the attention pool weights in the top-right of the figure, we observe an interesting trend: attention gradually ramps up towards the end of the window as the seizure activity develops. On the other hand, the time-frequency branch (shown in the bottom-left) focuses its saliency in the mid-to-high frequency bands, particularly in patches 3 to 8. This aligns with the typical spectral signature associated with spike-andwave discharges. The attention pool weights for the timefrequency branch (bottom-right) reinforce this observation about frequency localization. B. Seizure Onset Localization Figure 6 illustrates the model’s prospective detection capability on subject chb22. We stitch a seizure-free recording (chb22 19.edf, 0-3600s) with a seizure recording (chb22 20.edf, 3600-7200s) and plot the model’s predicted P(seizure) as a smoothed trajectory over the full two-hour span. The classification threshold (0.39) is calibrated persubject using Youden’s J statistic on all available chb22 windows. The model’s first detection occurs 45 seconds before the annotated seizure onset, demonstrating prospective seizure anticipation consistent with detection of pre-ictal EEG changes that precede the clinically annotated onset. The elevated interictal baseline reflects pathological background activity common in drug-resistant focal epilepsy, supporting the model’s sensitivity to pre-ictal spectro-temporal changes for early warning applications.
work will extend subject-specific adaptation to all 24 subjects and incorporate meta-learning for improved few-shot calibration. R EFERENCES
Fig. 6. Seizure onset localization for subject chb22. The predicted seizure probability shows that the model detects seizure activity approximately 45 seconds before the annotated onset (vertical dashed line), demonstrating early detection capability. The bottom panel shows ground-truth labels.
VII. DISCUSSION We benchmark a multimodal EEG foundation model across three protocols to guide future work in seizure detection. A strict LOSO evaluation across 19 subjects demonstrates that cross-subject seizure generalization remains challenging (mean BAcc 0.558, mean TPR 0.570), confirming high inter-subject variability in pediatric epilepsy; we therefore advocate subject-level evaluation over fixed cross-subject splits. On five subjects (chb01–chb05), withinsubject fine-tuning using 60% of subject data raises mean balanced accuracy to 0.643 (+0.174 over LOSO), confirming that subject-specific adaptation substantially improves performance. Few-shot calibration with 5–40% of subject data yields only marginal gains (mean BAcc 0.490–0.506), suggesting that small calibration windows are insufficient to capture seizure morphology diversity and that substantial labeled data is required for effective adaptation. Parameterefficient tuning of only 7.5% of parameters achieves stateof-the-art performance, suggesting the pretrained representations transfer well to EEG seizure detection; aggressive fine-tuning risks catastrophic forgetting. Finally, GradCAM and onset localization analyses confirm that predictions are grounded in known ictal EEG biomarkers, and the model’s 45-second early detection on subject chb22 shows promise for seizure anticipation, though this must be validated across more subjects. VIII. CONCLUSION We present a multimodal EEG foundation model that jointly learns from raw EEG, CWT scalograms, and text, achieving state-of-the-art balanced accuracy (0.781) and AUC-PR (0.709) under the BrainRVQ fixed-split protocol with only 7.5% of parameters fine-tuned, a 52.5% AUCPR improvement over BrainRVQ. Our LOSO evaluation across 19 CHB-MIT subjects constitutes the first crosssubject lower bound reported for an EEG foundation model on this benchmark (mean BAcc 0.558, mean TPR 0.570). Limitations include: within-subject and few-shot experiments covering only chb01 to chb05; reliance on general EEG knowledge rather than patient-specific clinical reports; and early detection validated on a single subject (chb22), requiring broader evaluation for real-time deployment. Future
[1] Y. W. et al., “A review of machine learning and deep learning trends in eeg-based epileptic seizure prediction,” IEEE Access, vol. 13, pp. 159 812–159 842, 2025. [2] S. W. et al., “Eeg datasets for seizure detection and prediction: A review,” Epilepsia Open, vol. 8, pp. 252–267, 2023. [3] J. L. et al., “A simple review of eeg foundation models,” ArXiv, 2025. [4] Y. Y. et al., “Foundation models for eeg decoding,” J. Neural Eng., vol. 22, 2025. [5] P. B. et al., “A review of feature extraction in epileptic seizure detection using eeg,” Biomed. Signal Process. Control., vol. 57, 2019. [6] C. Baumgartner and J. Koren, “Seizure detection using scalp-eeg,” Epilepsia, vol. 59, pp. 14–22, 2018. [7] S. K. et al., “The necessity of loso cross validation for eeg disease diagnosis,” in Int. Conf. Brain Informatics. Springer, 2021, pp. 558– 567. [8] J. X. et al., “Eeg-based epileptic seizure detection using deep learning: A survey,” Neurocomputing, vol. 610, p. 128644, 2024. [9] A. H. A. et al., “Detection of epileptic seizure using eeg signals and deep learning,” Chaos Solitons Fractals, 2024. [10] Z. W. et al., “Distilclip-eeg: Enhancing epileptic seizure detection through multi-modal learning,” IEEE J. Biomed. Health Inform., vol. PP, 2025. [11] B. W. et al., “Detection of epileptic seizures using multimodal dualstream networks,” Sensors, vol. 24, 2024. [12] I. T. et al., “Epilepsy detection in 121 patient populations using hypercube pattern from eeg,” Inf. Fusion, vol. 96, pp. 252–268, 2023. [13] Z. L. et al., “Epilepsyfm: A domain-specific foundation model for epileptic eeg,” Neural Networks, vol. 193, p. 108060, 2025. [14] L. Vidyaratne and K. Iftekharuddin, “Real-time epileptic seizure detection using eeg,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 25, pp. 2146–2156, 2017. [15] M. Amiri, H. Aghaeinia, and H. Amindavar, “Automatic epileptic seizure detection using sparse csp and synchrosqueezing transform,” Biomed. Signal Process. Control., vol. 79, p. 104022, 2023. [16] M. S. et al., “An eeg real-time epilepsy detection approach using wavelet transform and machine learning,” Biomed. Signal Process. Control., vol. 77, p. 103820, 2022. [17] J. W. L. et al., “A rhythmic encoding approach based on eeg timefrequency image for seizure detection,” Biomed. Signal Process. Control., vol. 99, p. 106824, 2025. [18] S. U. Khan, S. Jan, and I. Koo, “Robust epileptic seizure detection using lstm and time-frequency eeg images,” Sensors, vol. 23, 2023. [19] H. J. B. et al., “Uncovering the structure of clinical eeg signals with self-supervised learning,” J. Neural Eng., vol. 18, 2020. [20] M. R. et al., “Self-supervised learning for electroencephalography,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, pp. 1457–1471, 2022. [21] W. W. et al., “Self-supervised learning for eeg: A systematic survey,” ACM Comput. Surv., vol. 57, pp. 1–38, 2024. [22] J. W. et al., “Cbramod: A criss-cross brain foundation model for eeg decoding,” ArXiv, 2024. [23] G. Kuruppu, N. Wagh, and Y. Varatharajah, “Eeg foundation models: A critical review,” ArXiv, 2025. [24] Y. W. et al., “Lead: Large foundation model for eeg-based alzheimer’s detection,” ArXiv, 2025. [25] C. F. et al., “Promoting cross-modal representations for multimodal foundation models,” ArXiv, 2024. [26] K. Y. et al., “Benchmarking foundation models with multimodal ehr,” IEEE J. Biomed. Health Inform., 2025. [27] F. D. P. et al., “The role of data partitioning on eeg-based deep learning models,” Comput. Biol. Med., vol. 196, p. 110608, 2025. [28] A. L. G. et al., “Physiobank, physiotoolkit, and physionet,” Circulation, vol. 101, 2000. [29] X. W. et al., “Investigating eeg-based functional connectivity for multimodal emotion recognition,” J. Neural Eng., vol. 19, p. 016012, 2022. [30] I. Obeid and J. Picone, “The tuh eeg data corpus,” Front. Neurosci., vol. 10, p. 196, 2016. [31] M. C. et al., “Brainrvq: A high-fidelity eeg foundation model,” arXiv:2602.16951, 2026.
[32] V. J. L. et al., “Eegnet: a compact cnn for eeg-based bcis,” J. Neural Eng., vol. 15, 2016. [33] W. Zheng and B. Pan, “A spatiotemporal symmetrical transformer for eeg emotion recognition,” Biomed. Signal Process. Control., 2024. [34] D. Kostas, S. T. Aroca-Ouellette, and F. Rudzicz, “Bendr: Contrastive self-supervised learning from massive eeg data,” Front. Hum. Neurosci., 2021. [35] C. Yang, M. B. Westover, and J. Sun, “Biot: Biosignal transformer for cross-data learning,” arXiv:2305.06718, 2023. [36] W.-B. Jiang, L.-M. Zhao, and B.-L. Lu, “Large brain model for generic eeg representations in bci,” arXiv preprint, 2024.