arXiv CSopen access

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

machine learning, deep learning, neural networks

Runhe Zhou 1 * Shanglin Li 2 3 * Guanxiang Huang 4 Xinliang Zhou 1 Qibin Zhao 3 Motoaki Kawanabe 2 Yi Ding 1 † Cuntai Guan 1 †

Abstract

arXiv:2604.12579v1 [cs.LG] 14 Apr 2026

Electroencephalography (EEG)-based multimodal learning integrates brain signals with complementary modalities to improve mental state assessment, providing great clinical potential. The effectiveness of such paradigms largely depends on the representation learning on heterogeneous modalities. For EEG-based paradigms, one promising approach is to leverage their hierarchical structures, as recent studies have shown that both EEG and associated modalities (e.g., facial expressions) exhibit hierarchical structures reflecting complex cognitive processes. However, Euclidean embeddings struggle to represent these hierarchical structures due to their flat geometry, while hyperbolic spaces, with their exponential growth property, are naturally suited for them. In this work, we propose EEGMoCE, a novel hyperbolic mixture-of-curvature experts framework designed for multimodal neurotechnology. EEG-MoCE assigns each modality to an expert in a learnable-curvature hyperbolic space, enabling adaptive modeling of its intrinsic geometry. A curvature-aware fusion strategy then dynamically weights experts, emphasizing modalities with richer hierarchical information. Extensive experiments on benchmark datasets demonstrate that EEG-MoCE achieves state-of-the-art performance, including emotion recognition, sleep staging, and cognitive assessment. Code will be released.

Figure 1. Euclidean vs. hyperbolic geometry for hierarchical data. Euclidean space is flat and tends to under-represent hierarchical branching; hyperbolic space exhibits exponential volume growth and better preserves tree-like separation. Hyperbolic geometry is informative for multimodal learning, where modalities may differ in how strongly hierarchical their underlying structure is.

tive processes (Bell & Cuevas, 2012). EEG-based neurotechnology aims to extract meaningful patterns to support applications such as sleep stage classification (Aboalayon et al., 2016), cognitive assessment (Shin et al., 2018), and emotion recognition (Suhaimi et al., 2020). However, EEG signals are highly susceptible to external artifacts, and their inherent complexity makes accurately inferring mental states a significant challenge (Lotte et al., 2018; Li et al., 2025). To overcome these limitations, EEG-based multimodal learning frameworks (Sharma & Meena, 2024), leveraging the strengths of other complementary modalities, have the potential to improve robustness and performance in mental state assessment (Lee et al., 2025). For instance, emotion recognition tasks often benefit from the integration of EEG, facial video, and speech data (Pillalamarri & Shanmugam, 2025). The complementary yet heterogeneous nature stems from their distinct information sources: facial expressions and speech capture observable behavioral cues, while EEG directly measures internal brain activity (Lee et al., 2024).

1. Introduction Electroencephalography (EEG) records multi-channel electrical activity of the brain (Niedermeyer & Lopes da Silva, 2005) and provides valuable insights into underlying cogni-

While integrating these modalities offers a broader perspective, a fundamental challenge remains in effectively representing heterogeneous modalities (Baltrušaitis et al., 2019). For EEG-based multimodal learning, one promising approach is to leverage their hierarchical structures, as recent studies suggest that the brain’s hierarchical cognitive pro-

Equal contribution 1 Nanyang Technological University, Singapore 2 ATR, Kyoto, Japan 3 Riken AIP, Tokyo, Japan 4 University of Cambridge, Cambridge, UK. Correspondence to: Yi Ding <[email protected]>, Cuntai Guan <[email protected]>. Preprint. April 15, 2026.

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

Figure 2. Architecture of EEG-MoCE on EAV dataset. Other datasets use the same overall architecture, with only modality encoders adapted. (a) Modality-specific hyperbolic experts: each modality (e.g., EEG, audio, video) is encoded by an expert that embeds inputs in its own learnable-curvature hyperbolic space. (b) Curvature-oriented fusion: expert representations are aggregated by a curvature-aware scheme, combining curvature-scaled temperature and a curvature prior bias, that favors modalities with richer hierarchical structures.

• Extensive cross-subject experiments showing strong gains and SotA performance on three public EEGbased multimodal datasets.

cesses can be encoded in EEG (Sun et al., 2023; Turner et al., 2023) and associated modalities (e.g., facial expressions) (Mettes et al., 2024). While Euclidean embeddings currently dominate EEG decoding approaches, they struggle to represent the exponential growth of states in hierarchical structures due to their flat geometry, where distances and areas scale only linearly or quadratically with radius (Peng et al., 2022). In contrast, hyperbolic spaces with negative curvature exhibit exponential growth, making them naturally suited to model such processes. Leveraging this representational advantage, Li et al. (Li et al., 2026) introduced hyperbolic geometry for EEG classification and achieved stateof-the-art (SotA) performance in cross-subject decoding. Beyond EEG, many modalities employed in multimodal neurotechnology, such as facial expression analysis (Mettes et al., 2024) and physiological signal (Búzás et al., 2024), also exhibit inherent hierarchical structures. This shared characteristic motivates the exploration of hyperbolic geometry as a unified embedding framework for EEG-based multimodal learning.

Additionally, EEG-MoCE employs recent advances in hyperbolic alignment methods (Li et al., 2026) to mitigate distribution shifts, focusing on the most challenging crosssubject evaluation setting. We organize the paper by quantifying hierarchical structure across modalities, presenting the EEG-MoCE methodology, and reporting cross-subject benchmarks together with analyses and ablations.

2. Preliminaries 2.1. Related Work Cognitive hierarchy. The human brain exhibits hierarchical organization across multiple cognitive domains. In emotion regulation as an example, initial affective responses originate from subcortical structures, are modulated by limbic circuits, and are ultimately refined by neocortical regions into complex emotional experiences (Panksepp, 2011). Sun et al. (Sun et al., 2023) demonstrated hierarchical emotion ambiguity processing, with distinct EEG patterns revealing the progression from early subcortical responses to later cortical regulation. Hierarchical structure is also implicit in the mechanisms of other modalities (Section C.1). Accordingly, hyperbolic geometry supports embeddings of hierarchical data (Krioukov et al., 2010; Peng et al., 2022), motivating a unified geometric view of all modalities in EEG-based multimodal learning.

In this work, we introduce EEG-MoCE, a hyperbolic mixture-of-curvature framework for EEG-based multimodal learning (Figure 2), with the following contributions: • To our knowledge, the first systematic hierarchical analysis and hyperbolic framework for multimodal physiological signals. • Per-modality experts with learnable curvatures that adapt to intrinsic modality differences.

EEG-based multimodal learning. Integrating EEG with complementary modalities has shown promise across emotion recognition (Lee et al., 2024), sleep staging (Jia et al.,

• Curvature-guided fusion leveraging learned curvatures as proxies for hierarchical structure and modality importance. 2

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

where ⟨p, q⟩L = p⊤ s qs − pt qt denotes the Lorentzian inner product. Each point p ∈ LnK is decomposed as ⊤ p = [pt , p⊤ s ] , where pt > 0 is the time component and n ps ∈ R is the space component, following the convention from special relativity (Ratcliffe, 2006). The geodesic distance between two points p, q ∈ LnK is defined as:

2021; Mostafaei et al., 2024), and cognitive assessment (Liu et al., 2025; Shi et al., 2025). However, due to modality characteristics that are fundamentally diverse (Pillalamarri & Shanmugam, 2025), learning unified representations across heterogeneous modalities remains a fundamental challenge (Baltrušaitis et al., 2019). For example, EEG captures internal neural dynamics with inherent variability and noise, while behavioral modalities like facial expressions and speech capture observable and distinct temporal and spatial properties (Lee et al., 2024). To address these challenges, various Euclidean architectures have been proposed, as detailed in Section C.2, whereas hyperbolic multimodal learning methods have not yet been explored for EEG-based multimodal learning.

dL (p, q) = √

M X µ = wFMη {pi }M ηi d2L (q, pi ). (3) i=1 = arg min n q∈LK

At each point p ∈ LnK , the tangent space Tp LnK consists of all vectors v ∈ Rn+1 satisfying ⟨p, v⟩L = 0. The exn n ponential map expK p : Tp LK → LK projects vectors from the tangent space onto the manifold, while the logarithmic n n map logK p : LK → Tp LK performs the inverse projection. The parallel transport operation PTp→q (v) maps tangent vectors from Tp LnK to Tq LnK while preserving geometric properties (see Section A for closed-form expressions). Hyperbolic neural networks. Hyperbolic neural networks operate directly on the Lorentz manifold (Bdeir et al., 2024). The Lorentz fully-connected layer transforms features while maintaining the manifold constraint. Given in⊤ n put p = [pt , p⊤ s ] ∈ LK , the transformation follows (Yang et al., 2024): ! p ∥p̃s ∥2 − 1/K fL (p) = , where p̃s = ψ(Wp+b), p̃s (4) ⊤ d′ ′ producing p̃ = [p̃t , p̃⊤ ] ∈ L , where d denotes the outs K ′ put dimension, W ∈ Rd ×(n+1) acts on the full Lorentz ′ vector, b ∈ Rd , and ψ is an optional activation function. Activation functions such as Lorentz ELU operate on the space components and are combined with the time component to maintain the manifold structure. In hyperbolic attention mechanisms, the standard dot-product similarity is replaced with negative squared geodesic distance to compute attention weights (Yang et al., 2024): exp −d2L (qi , kj )/τ αij = P , (5) 2 ′ j ′ exp (−dL (qi , kj )/τ )

The hyperbolic space is a Riemannian manifold of constant negative curvature K < 0. The Lorentz hyperboloid model is employed in this work owing to its numerical stability during gradient-based optimization (Mishne et al., 2023). The n-dimensional Lorentz model represents the upper sheet of a two-sheeted hyperboloid embedded in Minkowski space, defined as: p ∈ Rn+1 ⟨p, p⟩L =

1 , pt > 0 , K

i=1

With uniform weights, this reduces to the standard Fréchet mean. The Fréchet variance ν 2 corresponds to the minimum value achieved at the Fréchet mean.

2.2. Hyperbolic Geometry

(2)

n M Given a collection of points P {pi ∈ LK }i=1 with nonM negative weights {ηi }i=1 ( i ηi > 0), the weighted Fréchet mean generalizes the notion of centroid to Riemannian manifolds by minimizing the weighted sum of squared geodesic distances:

Hyperbolic and mixed-curvature learning. Hyperbolic neural networks (Shimizu et al., 2021) have gained significant attention in natural language processing (Ganea et al., 2018) and computer vision (Peng et al., 2022; Mettes et al., 2024) for hierarchical data. In the EEG literature, (Li et al., 2026) demonstrated improved cross-subject generalization by embedding the representation in hyperbolic space. Chang et al. (Chang et al., 2025) performed contrastive pretraining for emotion recognition in hyperbolic space. These methods for neuroscience applications focus on single-modality EEG with fixed curvature. Recent work has shown that for a single textual modality, different components or tasks may exhibit varying degrees of hierarchical structure that are better captured by mixture-of-curvature models (He et al., 2025). Prior work discusses mixed-curvature learning for graph embeddings (Gu et al., 2019) and weighted manifolds for heterogeneous graphs (Nguyen-Van et al., 2023). In this work, we generalize mixture-of-curvature modeling to EEG-based multimodal learning, given that different modalities may exhibit varying degrees of hierarchical structure (Section 4.1). Larger curvature magnitude |K| enables embedding deeper hierarchies with lower distortion without increasing dimensionality (Alvarado et al., 2023), implying that learned curvature can correlate with structural complexity and thus fusion contribution (Section 4.2). We therefore use the learned curvatures to guide cross-modal fusion (Section 3.1).

LnK :=

1 cosh−1 (K⟨p, q⟩L ) . −K

where qi , kj ∈ LnK denote query and key vectors, and τ > 0 is a temperature parameter. The weighted Fréchet

(1) 3

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

mean (Equation (3)) is used for aggregating value vectors to maintain geometric consistency. The Lorentz multinomial logistic regression (MLR) classifier extends the Euclidean MLR by measuring distances from data points to decision hyperplanes in hyperbolic space (see Section A for details).

other modalities with a curvature-scaled temperature: τ (m) = p

τ0 |K (m) |

(6)

where τ0 > 0 is a base temperature hyperparameter. Higher |K| yields lower temperature, producing sharper attention distributions. This enables modalities with richer hierarchical structure to be more selective in aggregating cross-modal information.

δ-hyperbolicity. Khrulkov et al. (Khrulkov et al., 2020) introduced δ-hyperbolicity as a metric to assess the tree-like structure inherent in embeddings. This measure determines the minimal value of δ such that the four-point condition, expressed through the Gromov product, is satisfied (mathematical formulation in Section A.5). The Gromov productbased characterization of hyperbolic spaces implies that metric relationships among any four points approximate those in a tree structure, with deviations bounded by an additive constant δ. Smaller values of δ ≥ 0 indicate embeddings that are more closely aligned with hyperbolic geometry. To facilitate comparison across modalities, the diameter-normalized δrel ∈ [0, 1] is computed for each modality (Section A.5).

Furthermore, we incorporate a learnable curvature-based prior into the attention computation to explicitly weight contributions from hierarchically-rich modalities: α̃m→j ∝ exp

−d2L (q(m) , k(j) ) (j) + λ · ϕ(K ) , (7) τ (m)

where q(m) , k(j) ∈ LdKf are query and key vectors after projection to the fusion manifold, d2L denotes the squared Lorentzian distance (Equation (2)), ϕ(K) = log(|K| + ϵ) is a log-curvature transform with small constant ϵ > 0 for numerical stability, and λ > 0 is a learnable scalar controlling the strength of the curvature prior, constrained to be positive via softplus reparameterization λ = log(1 + eλ̃ ) with unconstrained λ̃ ∈ R. The attended features are aggregated via the weighted Fréchet mean (Equation (3)) on the fusion manifold, preserving hyperbolic geometry throughout the fusion process.

3. Methods 3.1. Curvature-Oriented Cross-Modal Fusion Let M denote the set of modalities and d denote the feature dimension. In our mixture-of-curvature framework, each modality m ∈ M is embedded into its own Lorentz manifold LdK (m) with learnable curvature K (m) < 0. This design is motivated by the observation that mixed-curvature representations can better capture heterogeneous data structures (Gu et al., 2019), and extends naturally to multimodal settings where each modality may possess distinct geometric properties, as we demonstrate empirically in Section 4.1.

Following (He et al., 2025; Yang et al., 2024), we project each modality into a shared fusion manifold LdKf with curvature-dependent scaling (Equation (8)). The fusion curvature is computed mean of modality curvaP as the(m) 1 tures: Kf = |M| K . For a representation m∈M

Curvature as information indicator. The central idea of our fusion strategy is that curvature magnitude serves as a learned geometric indicator of hierarchical complexity. Theoretically, hyperbolic spaces with larger |K| can embed deeper hierarchies with lower distortion without increasing embedding dimensionality (Sala et al., 2018; Alvarado et al., 2023). Since different modalities may exhibit varying degrees of hierarchical structure (as shown in Section 4.1), when curvatures are learned end-to-end, modalities exhibiting richer hierarchical structure will naturally converge to larger |K| values to minimize embedding distortion (Nguyen-Van et al., 2023; Gu et al., 2019). This provides a data-driven signal for modality importance: larger curvature magnitudes correspond to modalities carrying richer information, which can guide cross-modal fusion.

z(m) ∈ LdK (m) , the projection proceeds via logarithmic and exponential maps with a curvature-dependent scaling factor: s ! K (m) Kf (m) K (m) (m) zf = expo · logo (z ) . (8) Kf This is a well-defined map between Lorentz models under which the projected point satisfies the hyperboloid constraint on the shared fusion manifold LdKf . The scaling factor p K (m) /Kf preserves the pairwise ordering induced by Lorentz geodesic distances (Yang et al., 2024), and the corresponding reverse projection is likewise well defined (He et al., 2025).

Curvature-guided attention mechanism. Based on this, we design a cross-modal attention mechanism where curvature explicitly guides information aggregation. Our attention mechanism extends Equation (5) using negative squared hyperbolic distance as similarity. Each modality m attends to

3.2. EEG-MoCE framework Following the principle of compositional design (Li et al., 2026) that incorporates the advantages of both Euclidean and hyperbolic spaces, we structure EEG-MoCE 4

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

as a composition of four modules: modality-specific Eu(m) clidean encoders {eθ }m∈M , mixture-of-curvature experts (m) {Eϕ }m∈M , a cross-modal fusion module Fω , and a hyperbolic classifier gψ . The complete model is parameterized as: ! M (m) (m) hΘ = gψ ◦ Fω ◦ Eϕ ◦ eθ , (9)

cross-attention layer is followed by hyperbolic layer normalization, and multi-head attention outputs within each layer are aggregated via Fréchet mean (Equation (3)). The final fused representation is obtained by aggregating all modality features via Fréchet mean followed by a hyperbolic linear projection (Equation (4)). Classification is performed using hyperbolic multinomial logistic regression (HMLR) (Ganea et al., 2018; Shimizu et al., 2021), which defines decision boundaries as geodesic hyperplanes on the manifold (Section A.3.1).

m∈M

where Θ = {θ, ϕ, ω, ψ, {K (m) }m∈M } includes all learnable parameters. Figure 2 illustrates the overall architecture. Modality-specific Euclidean encoders. Each modality m is first processed by a domain-appropriate Eu(m) clidean encoder eθ . For EEG signals, we adopt EEGNet (Lawhern et al., 2018) with temporal convolution for frequency-specific filtering, depthwise spatial convolution for electrode-wise patterns, and separable temporal convolution for temporal summarization. For other peripheral physiological signals like EMG and electrooculography (EOG), we use EEGNet-like CNN backbone with similar temporal convolution and spatial convolution for feature extraction. For video (facial expressions), we use a lightweight CNN backbone and apply a temporal transformer for long-range temporal modeling. For audio, we employ a 1D convolutional network operating on mel-spectrograms with a similar temporal transformer architecture as for video. These encoders output feature vectors x(m) ∈ Rd for subsequent hyperbolic processing.

Pipeline overview. From a pipeline perspective, the principal equations of EEG-MoCE define a sequential transformation: Equation (10) first embeds features into per-modality manifolds; Equation (8) then projects them onto a shared fusion manifold to enable cross-modal attention. Equation (6) and Equation (7) jointly parameterize the attention mechanism, using learned curvatures to control the weighting and sharpness of information integration. Following this, Equation (3) aggregates the attended features, and Equation (4) performs the final linear map in hyperbolic space before HMLR (Equation (27)). A detailed mapping of these equations to implementation steps and training updates is provided in Section A.4.

4. Experiments

(m)

EEG-based multimodal technology has great application in real-world scenarios, such as emotion recognition, sleep staging classification, and cognitive assessment. Emotion recognition enables affective human-computer interaction and continuous mental health monitoring, supporting applications in personalized therapy (Houssein et al., 2022) Sleep staging reduces the burden of manual polysomnographic scoring in clinical diagnosis, enabling large-scale screening of sleep disorders (Siddiqui et al., 2013) Cognitive assessment supports objective monitoring of cognitive load and fatigue in educational and occupational settings, facilitating adaptive workload scheduling and personalized training (Wallace et al., 2017).

Mixture-of-curvature experts. Each modality expert Eϕ operates in its own Lorentz manifold LdK (m) with learnable curvature K (m) < 0. Unlike prior works using fixed curvatures (Ganea et al., 2018; Chami et al., 2019), learnable curvatures allow the model to adaptively discover the optimal embedding space for each modality (Bdeir et al., 2024; Tan et al., 2024). The expert first projects Euclidean features onto the manifold via the exponential map at the origin: h(m) = expK o

(m)

(x(m) ),

(10)

p where o = [ −1/K (m) , 0]⊤ is the Lorentz origin. The expert then applies hyperbolic batch normalization with moments alignment (Li et al., 2026) for cross-subject generalization, followed by Lorentz activation and pooling (Section A), producing the modality-specific hyperbolic representation z(m) ∈ LdK (m) .

We evaluate EEG-MoCE using cross-subject evaluation on three public multimodal datasets corresponding to these applications. All datasets provide multimodal recordings for cross-subject evaluation, with preprocessing details provided in Section B. All baselines are detailed in Section C.2.

Fusion and classification. The fusion module Fω implements the curvature-oriented cross-modal fusion described in Section 3.1. All modality representations are projected to the unified fusion manifold LdKf via Equation (8). Multiple curvature-guided cross-attention layers (Equation (7)) are then stacked. The curvature prior bias term λ · ϕ(K (j) ) in Equation (7) is applied only in the first layer to keep the modality order information intact before mixing. Each

EAV (Lee et al., 2024): Multimodal emotion recognition with EEG, audio, and video from 42 participants across 5 emotion classes (neutral, anger, happiness, sadness, calmness). Task-specific baselines for EAV include: MMDFN (Hu et al., 2022), GA2MIF (Li et al., 2024), AGFIB (Shou et al., 2024), CMERC (Tu et al., 2024), HyperMML (Kang et al., 2025), HEEGNet (Li et al., 2026). 5

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts Table 1. Hyperbolicity δrel measured for raw data and encoded features following (Khrulkov et al., 2020). Lower values indicate stronger hierarchical tree-like structure and better suitability for hyperbolic embedding. All datasets’ modalities show hyperbolic structure with δrel < 0.3. Modality-wise differences motivate modality-specific learnable curvature.

ISRUC (Khalighi et al., 2016): Sleep staging with EEG, EMG, and EOG from 10 subjects, scored into 5 stages (Wake, N1, N2, N3, REM). Our evaluation protocol is equivalent to leave-one-subject-out cross-validation on this dataset. Task-specific baselines for ISRUC include: SSNet (Jia et al., 2021), CMST (Mostafaei et al., 2024), CFSNet (Cao et al., 2026), MMNet (Lin et al., 2024), XSleepFusion (Hu et al., 2025). Cognitive (Shin et al., 2018): N-back working memory task with EEG, EOG, and near-infrared spectroscopy (NIRS) from 26 participants across 3 difficulty levels (0/2/3-back). Task-specific baselines for Cognitive include: STA-Net (Liu et al., 2025), ST2A (Shi et al., 2025), EFDFNet (Xu et al., 2025), TSMMF (Si et al., 2025), EF-Net (Arif et al., 2024). We additionally consider several general multimodal frameworks across all tasks: LMF (Liu et al., 2018), CTMWA (Zhang et al., 2024), MMML (Wu et al., 2024). We treat individual sessions as distinct domains. For model fitting and evaluation, we use either a leave-one-group-out approach (when the number of source domains is smaller than 10) or a 10-fold leave-groups-out cross-validation scheme. We consider the cross-subject adaptation within each dataset and treat the subject as the grouping variable. EEG-MoCE is implemented in PyTorch and trained on NVIDIA RTX 4090 GPUs. Models are trained for 100 epochs using the Adam optimizer for Euclidean parameters and the Riemannian Adam optimizer for hyperbolic parameters, with a learning rate of 10−3 and early stopping (patience=20).

Dataset

Modality

Raw

Encoded

EAV (n = 42)

EEG Audio Video

0.097 ± 0.021 0.217 ± 0.023 0.280 ± 0.019

0.160 ± 0.013 0.293 ± 0.028 0.278 ± 0.011

ISRUC (n = 10)

EEG EMG EOG

0.107 ± 0.010 0.072 ± 0.007 0.137 ± 0.013

0.202 ± 0.019 0.202 ± 0.008 0.253 ± 0.028

Cognitive (n = 26)

EEG EOG NIRS

0.125 ± 0.009 0.220 ± 0.013 0.299 ± 0.025

0.183 ± 0.008 0.262 ± 0.011 0.292 ± 0.031

4.2. Learned Curvatures and Modal Contributions To verify that learned curvatures correlate with modality contributions, we analyze the learned curvature magnitudes |K| for each modality and compute their modal contributions in the fusion process. Modal contribution quantifies the amount of attention each modality receives from others in the cross-attention mechanism. Specifically, for each modality j, we compute the sumPof attention weights received from all other modalities: m̸=j αm→j , where αm→j denotes the attention weight from modality m to modality j. These values are normalized across all modalities to obtain percentage contributions. We conduct this analysis using a model with learnable curvatures but without curvatureoriented fusion (Section 3.1) to isolate the effect of learned curvatures.

4.1. Hierarchical Structure of Multimodal Data To further justify hyperbolic embedding, we measure the inherent tree-likeness of each modality using δ-hyperbolicity (Section A.5) for all modalities in each dataset following (Khrulkov et al., 2020). Lower δrel ∈ [0, 1] indicates a stronger hierarchical structure and better suitability for hyperbolic embedding.

Table 2 presents the results on the EAV dataset. EEG learns the highest curvature magnitude |K| = 2.34, reflecting its complex frequency-band hierarchy, followed by Vision (|K| = 2.29) and Audio (|K| = 1.91). The curvature magnitudes order is consistent with the hierarchical structure of the data revealed by δrel . Moreover, results demonstrate that EEG contributes the most (36.0%), consistent with its highest curvature magnitude. The positive correlation between curvature magnitude |K| and contribution confirms that modalities with higher curvature magnitudes receive more attention in the fusion process, validating our curvatureoriented fusion design hypothesis.

Table 1 shows the δrel across modalities for both raw data and encoded features. All modalities exhibit hierarchical structure (δrel < 0.3), confirming their suitability for hyperbolic embedding. Furthermore, each modality’s δrel exhibits different values from others, demonstrating the rationality of modality-specific learnable curvatures. In particular, EEG consistently exhibits the lowest δrel across all datasets, ranging from 0.097 to 0.183, reflecting its hierarchy and justifying its highest learned curvature. Raw features exhibit lower δrel than encoded features, suggesting that Euclidean encoders can disrupt the inherent hierarchical structure and introduce geometric distortion, which motivates our hyperbolic processing after initial encoding.

Another experimental verification of our curvature-oriented fusion design is the behavior of the parameter curvature prior λ in (Equation (7)), which also adapts during training. Table 3 shows that λ consistently increases from initialization across all datasets, indicating that the model learns to rely on curvature information for attention weighting. 6

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts Table 2. Encoded δrel , learned curvature values K and attentionbased modality contributions on EAV dataset, evaluated using a model with learnable curvatures but without curvature-oriented fusion. A negative association between δrel and |K| and a positive correlation between |K| and attention contribution support the curvature-oriented fusion hypothesis.

Modality

Encoded δrel

Contribution

EEG Vision Audio

0.160 0.278 0.293

-2.34 -2.29 -1.91

36.0 33.6 30.5

Table 5. Performance Comparison with State-of-the-Art Methods on ISRUC-S3 Sleep Staging Dataset. We evaluate all methods using balanced accuracy (Acc) and macro-averaged F1-score (F1) under leave-one-subject-out cross-validation protocol (n = 10 subjects).

Table 3. Learned curvature prior λ, initialized at 0.30 on full model with curvature-oriented fusion. Values are mean ± std across crossvalidation folds. λ consistently increases during training across datasets, indicating growing reliance on curvature information for attention weighting.

Dataset

λinit

λlearned

∆λ

EAV ISRUC Cognitive

0.300 0.300 0.300

0.464 ± 0.036 0.533 ± 0.034 0.332 ± 0.009

+0.164 +0.233 +0.032

Tables 4 to 6 present the comparison results on three datasets: EAV (Lee et al., 2024), ISRUC (Khalighi et al., 2016), and Cognitive (Shin et al., 2018), respectively. The evaluation metrics are balanced accuracy and F1 macro, defined in Section D.1. Table 4. Performance Comparison with State-of-the-Art Methods on EAV Emotion Recognition Dataset. We evaluate all methods using balanced accuracy (Acc) and macro-averaged F1-score (F1) under cross-subject 10-fold cross-validation protocol (n = 42 subjects).

Acc (%)

F1 (%)

MM-DFN CTMWA AGF-IB LMF CMERC MMML Hyper-MML GA2MIF HEEGNet

51.13 ± 6.81 52.86 ± 5.63 54.59 ± 12.45 57.29 ± 12.37 58.57 ± 11.95 59.60 ± 11.83 60.76 ± 12.15 61.10 ± 12.51 61.74 ± 8.33

50.17 ± 6.95 49.92 ± 6.33 54.20 ± 15.03 56.80 ± 14.16 57.70 ± 14.30 56.55 ± 14.31 57.76 ± 13.89 58.46 ± 14.63 60.48 ± 8.24

Ours

75.88 ± 8.31

75.47 ± 8.66

Acc (%)

F1 (%)

CMST MMML CFSNet SSNet MMNet LMF CTMWA XSleepFusion

66.68 ± 8.31 73.33 ± 3.93 74.30 ± 4.82 74.56 ± 4.10 74.73 ± 4.02 75.02 ± 4.54 75.11 ± 3.96 75.19 ± 5.20

64.06 ± 10.32 72.80 ± 4.55 72.71 ± 5.31 74.13 ± 4.83 72.90 ± 5.45 74.31 ± 5.63 74.20 ± 4.46 73.62 ± 5.80

Ours

78.53 ± 2.95

75.38 ± 4.05

Table 6. Performance Comparison with State-of-the-Art Methods on Cognitive N-back Task. We evaluate all methods using balanced accuracy (Acc) and macro-averaged F1-score (F1) under crosssubject 10-fold cross-validation protocol (n = 26 subjects).

4.3. Main Experiments

Method

Acc (%)

F1 (%)

MMML STA-Net EFDFNet CTMWA ST2A LMF TSMMF EF-Net

45.32 ± 9.37 46.63 ± 11.67 49.15 ± 12.85 49.86 ± 11.72 51.28 ± 11.74 52.71 ± 11.77 53.84 ± 13.62 54.41 ± 14.03

42.67 ± 11.81 43.12 ± 15.32 45.00 ± 15.13 45.71 ± 12.23 46.08 ± 13.17 48.70 ± 13.04 50.06 ± 16.24 51.47 ± 15.18

Ours

62.39 ± 13.07

59.67 ± 13.08

nition. On the ISRUC dataset, our method achieves a 3.34% accuracy improvement over XSleepFusion, with notably lower variance, indicating more stable performance across subjects. On the Cognitive dataset, we achieve a 7.98% accuracy improvement over EF-Net, the best baseline, suggesting that our approach is effective for cognitive assessment tasks. The consistent improvements across diverse tasks and modalities validate the generalizability of our framework. 4.4. Ablation Study We conduct comprehensive ablation studies on the EAV dataset to investigate the effectiveness of our proposed components.

The comparison results show that our method achieves significant improvements across all datasets. On the EAV dataset, we outperform the all baselines with significance (p ≪ 0.01) by at least 14.14% in accuracy and 14.99% in F1-score, demonstrating the effectiveness of hyperbolic embedding and curvature-oriented fusion for emotion recog-

Euclidean vs Hyperbolic Architecture. We first examine the impact of hyperbolic geometry in both the encoder and fusion stages. Table 7 compares four architectural variants by independently enabling hyperbolic operations in the encoder and fusion modules. 7

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts Table 7. Architecture ablation on EAV comparing Euclidean and hyperbolic choices for encoder and fusion. Four variants are shown: neither hyperbolic; hyperbolic fusion only; hyperbolic encoder only; both encoder and fusion. The combination of hyperbolic encoder and hyperbolic fusion produces the best result, indicating complementary benefits.

Encoder

Fusion

Acc (%)

F1 (%)

– – ✓ ✓

– ✓ – ✓ (Ours)

60.33 ± 11.68 61.48 ± 12.63 74.17 ± 9.83 75.88 ± 8.31

57.24 ± 14.18 58.79 ± 14.47 73.41 ± 10.27 75.47 ± 8.66

learnable curvatures with initial curvature set to K = −2. w/ COMF (Ours) w/o COMF Fixed Curvature Acc (%) F1 (%)

Mean pooling 70

Score (%)

Figure 4. Ablation of hyperbolic components on the EAV dataset, focusing on learnable curvature and curvature-oriented multimodal fusion (COMF). All fusion variants use learnable curvatures to isolate the effect of the fusion mechanism. Our proposed method achieves the best performance, demonstrating the complementary benefits of learnable curvatures and COMF.

(a) Euclidean (Subject 17)

The results in Figure 4 show that both learnable curvatures and COMF contribute to performance improvements. Compared to fixed curvature, learnable curvatures provide a 2.14% accuracy gain, suggesting that modality-specific geometry optimization is beneficial. COMF contributes a 1.38% accuracy improvement over attention without curvature prior, indicating that incorporating curvature information into attention computation can enhance fusion quality. The combination of learnable curvatures and COMF yields the best performance, suggesting their complementary roles in the fusion process.

(b) Hyperbolic (Subject 17)

Single-Modality Ablation. We evaluate the complementary relationship of our multimodal framework by training with single modalities. Table 8 shows that multimodal fusion significantly outperforms any single modality. EEG achieves the best single-modality performance (62.74%), consistent with its highest curvature magnitude and largest fusion contribution, suggesting that it carries relatively more discriminative hierarchical information. Multimodal fusion (75.88%) exceeds the best single modality by 13.14%, indicating complementary relationships between modalities. The improvement over individual modality performances suggests that our fusion mechanism may capture crossmodal synergies beyond simple aggregation of independent signals.

(d) Hyperbolic (Subject 21)

Figure 3. t-SNE of fused features on EAV dataset. Hyperbolic encoder and fusion variant produces more compact and betterseparated emotion clusters than the Euclidean baseline variant, illustrating improved class separability by hyperbolic geometry.

The results demonstrate that hyperbolic geometry in both encoder and fusion stages is essential for optimal performance, with each component contributing complementary benefits. As visualized in Figure 3, the Euclidean encoder and fusion baseline produces severely overlapping emotion clusters, while our hyperbolic encoder and fusion architecture achieves clear class separation with compact, well-defined clusters for each emotion category.

Table 8. Ablation study on single-modality performance on EAV dataset.

Hyperbolic Component Ablation. Within the hyperbolic framework, we ablate key design choices: curvature learning strategy and fusion mechanism. Figure 4 shows the contribution of learnable curvatures and curvature-oriented multimodal fusion (COMF). The fixed curvature baseline is set to K = −2 using hyperbolic cross-attention mechanism with COMF. All the variants in fusion comparison utilize the 8

Modality

Acc (%)

F1 (%)

Video only Audio only EEG only

53.75 ± 10.74 60.52 ± 8.14 62.74 ± 9.04

53.46 ± 11.02 60.23 ± 8.42 62.13 ± 9.42

All modalities

75.88 ± 8.31

75.47 ± 8.66

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts Table 11. Ablation of fusion-curvature parameterization in the shared manifold. Mean uses the arithmetic mean of modality curvatures, while Learnable treats the fusion curvature as an extra trainable parameter. Mean curvature performs comparably or slightly better across all datasets, supporting the stable and effective design choice.

4.5. Additional Analysis Multi-seed robustness. To examine initialization sensitivity, we additionally run 5 random seeds and report seedlevel statistics while keeping the same data split protocol used in the main experiments. As shown in Table 9, EEGMoCE consistently outperforms the strongest task-specific baselines on all three datasets, and the seed-level standard deviations are small. This indicates that the reported improvements are not driven by favorable initialization and remain stable across repeated runs. Table 9. Seed-level robustness over 5 random seeds under the same cross-subject protocol as the main experiments. Values are mean ± std over seeds. Across EAV, ISRUC, and Cognitive, EEG-MoCE consistently exceeds the strongest baseline with low seed-level variance, supporting initialization robustness.

Dataset

Method

Acc (%)

F1 (%)

EAV

Ours HEEGNet

75.79 ± 0.78 61.54 ± 0.76

75.25 ± 0.76 60.37 ± 0.77

ISRUC

Ours XSleepFusion

78.42 ± 0.31 75.02 ± 0.67

75.10 ± 0.29 73.30 ± 0.71

Cognitive

Ours EF-Net

61.86 ± 1.75 54.28 ± 2.41

59.23 ± 1.64 51.36 ± 2.35

MMML Hyper-MML GA2MIF Ours

Test msec/trial

10.86 12.95 8.56 26.07

1.17 1.24 0.86 2.71

Acc (%)

F1 (%)

EAV

Mean Learnable

75.88 ± 8.31 75.55 ± 8.57

75.47 ± 8.66 75.10 ± 9.13

ISRUC

Mean Learnable

78.53 ± 2.95 78.08 ± 3.01

75.38 ± 4.05 74.97 ± 3.96

Cognitive

Mean Learnable

62.39 ± 13.07 60.96 ± 13.48

59.67 ± 13.08 57.68 ± 13.61

In this work, we present EEG-MoCE, the first hyperbolic framework for EEG-based multimodal learning that uses modality-specific learnable curvatures to capture varying hierarchical structure across modalities. Experiments on emotion recognition, sleep staging, and cognitive assessment show state-of-the-art performance. Analysis of learned curvatures demonstrates that they correlate with modality contributions, together with ablation studies validating our geometry-aware design. The consistent gains over prior methods, including hyperbolic baselines, underscore the value of cross-modal synergies in neurotechnology. While our framework shows promising results, several limitations and future directions warrant discussion. The current approach focuses mainly on hyperbolic geometry and modalities with hierarchical structures; future work could explore other modalities with different structural priors. The computational stability and overhead of hyperbolic operations may suggest potential directions for future research. Although the unified manifold strategy for fusion is empirically effective, richer fusion and transport mechanisms may be explored in future work. Finally, investigating the interpretability of learned representations could provide deeper insights into the underlying cognitive mechanisms.

Table 10. Computational cost on EAV measured on the same RTX 4090 GPU. Train reports seconds per epoch and Test reports milliseconds per trial. EEG-MoCE is slower due to hyperbolic operations, but its absolute inference latency remains low for realtime deployment settings.

Train sec/epoch

Fusion

5. Discussion

Computational cost. We report training and inference latency on EAV with the same RTX 4090 setup for all methods. Table 10 shows that hyperbolic operations increase computational cost relative to Euclidean baselines, but inference remains lightweight in absolute terms (2.71 ms per 20 s trial), which is still negligible for online BCI scenarios.

Method

Dataset

6. Acknowledgements This work is supported by the Ministry of Education, Singapore, under its Academic Research Fund Tier 2 (Grant No. MOE-T2EP20124-0001).

Fusion curvature design. We compare the default mean fusion curvature with a learnable fusion curvature. Table 11 shows that the simple mean is consistently comparable or slightly better across all datasets in both accuracy and F1. These results support using mean fusion curvature as a stable and effective default without introducing additional optimization variables.

7. Impact statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here. 9

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

References

Chang, J., Zhang, Z., Qian, Y., and Lin, P. Multi-scale hyperbolic contrastive learning for cross-subject EEG emotion recognition. IEEE Transactions on Affective Computing, 16(3):1716–1731, 2025.

Aboalayon, K. A. I., Faezipour, M., Almuhammadi, W. S., and Moslehpour, S. Sleep stage classification using eeg signal analysis: a comprehensive survey and new investigation. Entropy, 18(9):272, 2016.

Chen, W., Han, X., Lin, Y., Zhao, H., Liu, Z., Li, P., Sun, M., and Zhou, J. Fully hyperbolic neural networks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 5672–5686, 2022.

Alvarado, N., Lobel, H., and Petrache, M. Curvaturedimension tradeoff for generalization in hyperbolic space. In Mathematics of Modern Machine Learning Workshop at NeurIPS, 2023.

Chen, X. et al. Hyperbolic self-paced multi-expert network for cross-domain few-shot facial expression recognition. IEEE Transactions on Image Processing, 2025a.

Arif, A., Wang, Y., Yin, R., Zhang, X., and Helmy, A. EFNet: Mental state recognition by analyzing multimodal EEG-fNIRS via CNN. Sensors, 24(6):1889, 2024. doi: 10.3390/s24061889.

Chen, Z., Song, Y., Wu, X.-J., and Sebe, N. Gyrogroup batch normalization. In International Conference on Learning Representations, 2025b. Collins, E., Robinson, A. K., and Behrmann, M. Distinct neural processes for the perception of familiar versus unfamiliar faces along the visual hierarchy revealed by EEG. NeuroImage, 181:120–131, 2018. doi: 10.1016/j. neuroimage.2018.06.080.

Baltrusaitis, T., Robinson, P., and Morency, L.-P. OpenFace: An open source facial behavior analysis toolkit. In IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–10. IEEE, 2016. Baltrušaitis, T., Ahuja, C., and Morency, L.-P. Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41 (2):423–443, 2019. doi: 10.1109/TPAMI.2018.2798607.

Fréchet, M. Les éléments aléatoires de nature quelconque dans un espace distancié. Annales de l’Institut Henri Poincaré, 10(4):215–310, 1948. Ganea, O., Bécigneul, G., and Hofmann, T. Hyperbolic neural networks. In Advances in Neural Information Processing Systems, volume 31, 2018.

Bdeir, A., Schwethelm, K., and Landwehr, N. Fully hyperbolic convolutional neural networks for computer vision. In International Conference on Learning Representations, 2024.

Gromov, M. Hyperbolic groups. Springer, New York, 1987. Gu, A., Sala, F., Gunel, B., and Ré, C. Learning mixedcurvature representations in product spaces. In International Conference on Learning Representations, 2019.

Bell, M. A. and Cuevas, K. Using eeg to study cognitive development: Issues and practices. Journal of cognition and development, 13(3):281–294, 2012. doi: 10.1080/ 15248372.2012.691143.

Gulcehre, C., Denil, M., Malinowski, M., Razavi, A., Pascanu, R., Hermann, K. M., Battaglia, P., Bapst, V., Raposo, D., Santoro, A., and de Freitas, N. Hyperbolic attention networks. In International Conference on Learning Representations, 2019.

Bonnabel, S. Stochastic gradient descent on Riemannian manifolds. IEEE Transactions on Automatic Control, 58 (9):2217–2229, 2013. doi: 10.1109/TAC.2013.2254619. Búzás, A., Makai, A., Groma, G. I., Dancsházy, Z., Szendi, I., Kish, L. B., Santa-Maria, A. R., and Dér, A. Hierarchical organization of human physical activity. Scientific Reports, 14(1):5981, 2024.

He, N., Anand, R., Madhu, H., Maatouk, A., Krishnaswamy, S., Tassiulas, L., Yang, M., and Ying, R. HELM: Hyperbolic large language models via mixture-of-curvature experts. In Advances in Neural Information Processing Systems, volume 38, 2025.

Cao, Y., Xiang, W., Wei, J., Cao, S., Tian, X., Zhong, J., Fang, X., Luo, B., Lyu, H., and Li, X. CrossFusionSleepNet: A multimodal deep learning model for automatic sleep stage classification. Biomedical Signal Processing and Control, 112:108538, 2026. doi: 10.1016/j.bspc.2025.108538.

Hochstein, S. and Ahissar, M. View from the top: Hierarchies and reverse hierarchies in the visual system. Neuron, 36(5):791–804, 2002. doi: 10.1016/S0896-6273(02) 01091-7. Houssein, E. H., Hammad, A., and Ali, A. A. Human emotion recognition from eeg-based brain–computer interface using machine learning: a comprehensive review. Neural Computing and Applications, 34(15):12527–12557, 2022.

Chami, I., Ying, Z., Ré, C., and Leskovec, J. Hyperbolic graph convolutional neural networks. In Advances in Neural Information Processing Systems, volume 32, 2019. 10

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

Hu, D., Hou, X., Wei, L., Jiang, L.-X., and Mo, Y. MMDFN: Multimodal dynamic fusion network for emotion recognition in conversations. In ICASSP, pp. 7037–7041. IEEE, 2022.

Lee, M. H., Shomanov, A., Begim, B., Kabidenova, Z., Nyssanbay, A., Yazici, A., and Lee, S.-W. EAV: EEG-audiovideo dataset for emotion recognition in conversational contexts. Scientific Data, 11(1):1026, 2024.

Hu, S., Wang, Y., Liu, J., and Yang, C. XSleepFusion: A dual-stage information bottleneck fusion framework for interpretable multimodal sleep analysis. Information Fusion, 123:103275, 2025. doi: 10.1016/j.inffus.2025. 103275.

Li, J., Wang, X., Lv, G., and Zeng, Z. GA2MIF: Graph and attention based two-stage multi-source information fusion for conversational emotion detection. IEEE Transactions on Affective Computing, 15(1):130–143, 2024. doi: 10. 1109/TAFFC.2023.3261279.

Ioffe, S. and Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning, pp. 448–456. PMLR, 2015.

Li, S., Kawanabe, M., and Kobler, R. J. Spdim: Source-free unsupervised conditional and label shift adaptation in eeg. In The Thirteenth International Conference on Learning Representations, 2025.

Jia, Z., Lin, Y., Wang, J., Wang, X., Xie, P., and Zhang, Y. SalientSleepNet: Multimodal salient wave detection network for sleep staging. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21), pp. 2614–2620, 2021.

Li, S., Chu, S., Koç, O., Ding, Y., Zhao, Q., Kawanabe, M., and Chen, Z. Heegnet: Hyperbolic embeddings for eeg. In International Conference on Learning Representations, 2026.

Kang, Z., Li, Y., Gong, S., Zeng, W., Yan, H., Bian, L., Zhang, Z., Siok, W. T., and Wang, N. Hypergraph multimodal learning for EEG-based emotion recognition in conversation. arXiv preprint arXiv:2502.21154, 2025.

Lin, Y., Wang, M., Hu, F., Cheng, X., and Xu, J. Multimodal polysomnography-based automatic sleep stage classification via multiview fusion network. IEEE Transactions on Instrumentation and Measurement, 73:2504112, 2024. doi: 10.1109/TIM.2023.3343781.

Khalighi, S., Sousa, T., Santos, J. M., and Nunes, U. ISRUC-Sleep: A comprehensive public dataset for sleep researchers. Computer Methods and Programs in Biomedicine, 124:180–192, 2016.

Liu, M., Yang, B., Meng, L., Zhang, Y., Gao, S., Zan, P., and Xia, X. STA-Net: Spatial–temporal alignment network for hybrid EEG-fNIRS decoding. Information Fusion, 119:103023, 2025. doi: 10.1016/j.inffus.2025.103023.

Khrulkov, V., Mirvakhabova, L., Ustinova, E., Oseledets, I., and Lempitsky, V. Hyperbolic image embeddings. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6418–6428, 2020.

Liu, Z., Shen, Y., Lakshminarasimhan, V. B., Liang, P. P., Zadeh, A., and Morency, L.-P. Efficient low-rank multimodal fusion with modality-specific factors. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, pp. 2247–2256, 2018.

Kobler, R. J., Hirayama, J.-i., Zhao, Q., and Kawanabe, M. SPD domain-specific batch normalization to crack interpretable unsupervised domain adaptation in EEG. In Advances in Neural Information Processing Systems, volume 35, pp. 6219–6235, 2022.

Lopez, S., Chiarantano, F., et al. Hierarchical hypercomplex network for multimodal emotion recognition. arXiv preprint arXiv:2409.09194, 2024. Lotte, F., Bougrain, L., Cichocki, A., Clerc, M., Congedo, M., Rakotomamonjy, A., and Yger, F. A review of classification algorithms for eeg-based brain–computer interfaces: a 10 year update. Journal of neural engineering, 15(3):031005, 2018.

Krioukov, D., Papadopoulos, F., Kitsak, M., Vahdat, A., and Boguná, M. Hyperbolic geometry of complex networks. Physical Review E, 82(3):036106, 2010. Lawhern, V. J., Solon, A. J., Waytowich, N. R., Gordon, S. M., Hung, C. P., and Lance, B. J. EEGNet: A compact convolutional neural network for EEG-based braincomputer interfaces. Journal of Neural Engineering, 15 (5):056013, 2018.

Mettes, P., Ghadimi Atigh, M., Keller-Ressel, M., Gu, J., and Yeung, S. Hyperbolic deep learning in computer vision: A survey. International Journal of Computer Vision, 132(9):3484–3508, 2024. doi: 10.1007/ s11263-024-02043-5.

Lee, H.-T., Shim, M., Liu, X., Cheon, H.-R., Kim, S.G., Han, C.-H., and Hwang, H.-J. A review of hybrid eeg-based multimodal human–computer interfaces using deep learning: applications, advances, and challenges. Biomedical Engineering Letters, 15(4):587–618, 2025. doi: 10.1007/s13534-025-00469-5.

Mishne, G., Wan, Z., Wang, Y., and Yang, S. The numerical stability of hyperbolic representation learning. In International Conference on Machine Learning, pp. 24925–24949. PMLR, 2023. 11

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

Mostafaei, S. H., Tanha, J., and Sharafkhaneh, A. A novel deep learning model based on transformer and cross modality attention for classification of sleep stages. Journal of Biomedical Informatics, 157:104689, 2024. doi: 10.1016/j.jbi.2024.104689.

Shin, J., von Lühmann, A., Kim, D.-W., Mehnert, J., Hwang, H.-J., and Müller, K.-R. Simultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset. Scientific Data, 5(1):180003, 2018. Shou, Y., Meng, T., Ai, W., Zhang, F., Yin, N., and Li, K. Adversarial alignment and graph fusion via information bottleneck for multimodal emotion recognition in conversations. Information Fusion, 112:102590, 2024.

Nguyen-Van, T., Le, D. D., and Ta, T.-A. Improving heterogeneous graph learning with weighted mixed-curvature product manifold. arXiv preprint arXiv:2307.04514, 2023. Niedermeyer, E. and Lopes da Silva, F. H. (eds.). Electroencephalography: basic principles, clinical applications, and related fields. Lippincott Williams & Wilkins, 5th edition, 2005. ISBN 978-0190228484.

Si, X., Zhang, S., Yang, Z., Yu, J., and Ming, D. A bidirectional cross-modal transformer representation learning model for EEG-fNIRS multimodal affective BCI. Expert Systems with Applications, 266:126081, 2025. doi: 10.1016/j.eswa.2024.126081.

Panksepp, J. Cross-species affective neuroscience decoding of the primal affective experiences of humans and related animals. PLoS One, 6(9):e21236, 2011. doi: 10.1371/ journal.pone.0021236.

Siddiqui, M. M., Rahman, S., Saeed, S. H., and Banodia, A. Eeg signals play major role to diagnose sleep disorder. International Journal of Electronics and Computer Science Engineering (IJECSE), 2(2):503–505, 2013.

Peng, W., Varanka, T., Mostafa, A., Shi, H., and Zhao, G. Hyperbolic deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44 (12):10023–10044, 2022.

Suhaimi, N. S., Mountstephens, J., and Teo, J. Eeg-based emotion recognition: a state-of-the-art review of current trends and opportunities. Computational intelligence and neuroscience, 2020(1):8875426, 2020. doi: 10.1155/ 2020/8875426.

Pillalamarri, R. and Shanmugam, U. A review on eeg-based multimodal learning for emotion recognition. Artificial Intelligence Review, 58(5):131, 2025.

Sun, S., Yu, H., Yu, R., and Wang, S. Functional connectivity between the amygdala and prefrontal cortex underlies processing of emotion ambiguity. Translational psychiatry, 13(1):334, 2023. doi: 10.1038/s41398-023-02625-w.

Qu, E. and Zou, D. Lorentz direct concatenation for stable training in hyperbolic neural networks. In NeurIPS 2022 Workshop on Symmetry and Geometry in Neural Representations, 2022.

Tan, X., Lian, Q., Zhu, J., Zhang, J., Wang, Y., and Qi, Y. Effective phoneme decoding with hyperbolic neural networks for high-performance speech BCIs. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 32:3432–3441, 2024.

Ratcliffe, J. G. Foundations of Hyperbolic Manifolds. Springer, 2006. Sala, F., De Sa, C., Gu, A., and Re, C. Representation tradeoffs for hyperbolic embeddings. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 4460–4469. PMLR, 2018.

Tu, G., Xiong, F., Liang, B., Wang, H., Zeng, X., and Xu, R. Multimodal emotion recognition calibration in conversations. In Proceedings of the 32nd ACM International Conference on Multimedia, 2024.

Sharma, R. and Meena, H. K. Emerging trends in eeg signal processing: A systematic review. SN Computer Science, 5(4):415, 2024.

Turner, W., Blom, T., and Hogendoorn, H. Visual information is predictively encoded in occipital alpha/low-beta oscillations. Journal of Neuroscience, 43(30):5537–5545, 2023. doi: 10.1523/JNEUROSCI.0135-23.2023.

Shi, X., Wang, H., Li, B., Qin, Y., Peng, C., and Lu, Y. Fusion analysis of EEG-fNIRS multimodal brain signals: A multitask classification algorithm incorporating spatialtemporal convolution and dual attention mechanisms. IEEE Transactions on Instrumentation and Measurement, 74:2506312, 2025. doi: 10.1109/TIM.2025.3538086.

Ungar, A. A. Analytic Hyperbolic Geometry and Albert Einstein’s Special Theory of Relativity. World Scientific, 2nd edition, 2022. Wallace, B., Knoefel, F., Goubran, R., Zunini, R. A. L., Ren, Z., and Maccosham, A. Eeg/erp: Within episodic assessment framework for cognition. IEEE Transactions on Instrumentation and Measurement, 66(10):2525–2534, 2017.

Shimizu, R., Mukuta, Y., and Harada, T. Hyperbolic neural networks++. In International Conference on Learning Representations, 2021. 12

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

Wu, W., Fan, X., Wu, Y., Gao, Z., Li, P., Jia, Y., and Harandi, M. Modality alignment across trees on heterogeneous hyperbolic manifolds. In International Conference on Learning Representations, 2026. Wu, Z., Gong, Z., Koo, J., and Hirschberg, J. Multimodal multi-loss fusion network for sentiment analysis. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 3588–3602, Mexico City, Mexico, 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024. naacl-long.197. URL https://aclanthology. org/2024.naacl-long.197/. Xu, H., She, Q., Meng, M., Gao, Y., and Zhang, Y. EFDFNet: A multimodal deep fusion network based on feature disentanglement for attention state classification. Biomedical Signal Processing and Control, 109:108042, 2025. doi: 10.1016/j.bspc.2025.108042. Yang, M., Verma, H., Zhang, D. C., Liu, J., King, I., and Ying, R. Hypformer: Exploring efficient transformer fully in hyperbolic space. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3770–3781. ACM, 2024. doi: 10.1145/ 3637528.3672039. Zhang, B., Yuan, Z., Xu, H., and Gao, K. Crossmodal translation based meta weight adaption for robust image-text sentiment analysis. IEEE Transactions on Multimedia, 26:9949–9961, 2024. doi: 10.1109/TMM.2024.3405662.

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

A. Mathematical Foundations of Hyperbolic Neural Networks This section provides detailed mathematical foundations for hyperbolic neural network operations used in EEG-MoCE. We organize these operations into three categories: (1) fundamental Riemannian operators that enable mappings between Euclidean and hyperbolic spaces, (2) manifold-preserving neural layers for feature transformation, and (3) task-specific modules for classification and cross-modal fusion. All formulations follow established conventions in hyperbolic deep learning (Ganea et al., 2018; Chami et al., 2019). A.1. Fundamental Riemannian Operators We first establish the core geometric operations that bridge Euclidean preprocessing and hyperbolic representation learning. Tangent spaces and their role. Each point p on the Lorentz manifold LnK possesses an associated tangent space—a local Euclidean approximation where standard linear algebra applies. Formally, this tangent space is the orthogonal complement under the Lorentzian metric: Tp LnK = v ∈ Rn+1 ⟨p, v⟩L = 0 . (11) Exponential and logarithmic maps. These maps provide bidirectional transport between the manifold and tangent n n spaces (Ratcliffe, 2006). The exponential map expK p : Tp LK → LK projects tangent vectors onto the manifold along geodesics: √ v (12) expK where α = −K∥v∥L . p (v) = cosh(α)p + sinh(α) , α n n Its inverse, the logarithmic map logK p : LK → Tp LK , retrieves tangent representations:

cosh−1 (β) logK · (q − βp), p (q) = p 2 β −1

where β = K⟨p, q⟩L .

For projecting Euclidean features x ∈ Rn onto LnK , we use the distinguished origin o = [ exponential map simplifies to: expK o (x) =

√ √ ⊤ cosh( −K∥x∥) sinh( −K∥x∥) √ √ , ·x . −K −K∥x∥

(13) p

−1/K, 0⊤ ]⊤ , where the

(14)

Parallel transport. When displacing tangent vectors between different base points, parallel transport preserves geometric relationships. For v ∈ Tp LnK transported to Tq LnK (Ganea et al., 2018): PTp→q (v) = v −

K⟨q, v⟩L (p + q). 1 + K⟨p, q⟩L

(15)

A.2. Manifold-Preserving Neural Layers Standard neural network operations must be adapted to respect manifold constraints. We describe the key modifications organized by their functional role. A.2.1. N ONLINEAR ACTIVATION F UNCTIONS Applying nonlinearities directly to Lorentz vectors would violate manifold membership. Following the principle of operating on unconstrained coordinates (Shimizu et al., 2021; Bdeir et al., 2024), we apply activations exclusively to the spatial (spacelike) components ps ∈ Rn , then reconstruct the temporal coordinate to restore the Lorentz constraint ⟨p, p⟩L = 1/K: ! p ∥σ(ps )∥2 − 1/K σL (p) = , (16) σ(ps ) where σ denotes any standard activation (e.g., ELU, ReLU). This design ensures the output remains on the hyperboloid regardless of the activation choice. 14

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

A.2.2. F EATURE AGGREGATION AND P OOLING Arithmetic averaging is ill-defined on curved manifolds. Instead, spatial pooling computes the Fréchet mean (Fréchet, 1948) over features within each receptive field. Given features {pi }M i=1 in a pooling window, the aggregated output minimizes total squared geodesic distance (Equation (3) in main text). A.2.3. H YPERBOLIC BATCH N ORMALIZATION Batch normalization (Ioffe & Szegedy, 2015) is essential for stable training but requires adaptation to respect manifold geometry. Following the gyrovector formalism (Chen et al., 2025b; Li et al., 2026), we define hyperbolic batch normalization (HBN) using Lorentz gyrovector operations that generalize Euclidean centering and scaling. Gyrovector operations. The Lorentz model admits a gyrovector structure where standard vector operations are replaced by their hyperbolic analogues (Ungar, 2022). Specifically, gyroaddition ⊕, gyroinverse ⊖, and gyroscalar multiplication ⊙ are defined via exponential and logarithmic maps: p ⊕ q = expo (PTo→p (logo (q))) , ⊤

⊖p = [pt , −ps ] ,

(17) (18)

t ⊙ p = expo (t · logo (p)),

(19)

where PT denotes parallel transport (Section A.1). Batch normalization in hyperbolic space. Given a batch of activations {pi ∈ LnK }M i=1 , HBN computes the Fréchet mean PM 2 1 µ and Fréchet variance ν 2 = M (µ, p ) as batch statistics. The normalization then centers and scales each point d i i=1 L using gyrovector operations: γ HBN(pi ) = √ ⊙ (⊖µ ⊕ pi ) , ∀i ≤ M, (20) ν2 + ϵ where γ ∈ R is a learnable scaling parameter and ϵ > 0 ensures numerical stability. The centering operation ⊖µ ⊕ pi translates each point by the inverse of the batch mean, analogous to subtracting the mean in Euclidean BN. The scaling operation adjusts the dispersion via gyroscalar multiplication. Domain-specific momentum estimation. For cross-subject generalization in EEG, we adopt domain-specific momentum batch normalization (Kobler et al., 2022; Li et al., 2026). Each domain d ∈ D maintains separate running estimates (µ̃(d) , ν̃ 2(d) ) updated via geodesic interpolation: (d)

(d)

µ̃k = Geodesic(µ̃k−1 , µk ; η),

(21)

2(d) 2(d) 2(d) ν̃k = (1 − η)ν̃k−1 + η · νk ,

(22)

where Geodesic(p, q; t) returns the point at fraction t ∈ [0, 1] along the geodesic from p to q, and η is a momentum parameter following an exponential decay schedule during training. At test time, a fixed momentum ηtest adapts to unseen target domains in a source-free manner. Curvature-adaptive running statistics. When curvature K is learnable, running statistics must be transformed to preserve geometric relationships. For the running mean µ̃, we apply log-scale-exp projection: ! r Kold Kold Knew µ̃new = expo · logo (µ̃old ) . (23) Knew For variance, since ν 2 ∝ 1/|K| (geodesic distance scales with 1/ 2 2 ν̃new = ν̃old ·

|K|), we apply:

Kold . Knew

(24)

These transformations ensure that normalized features maintain consistent geometric properties as the embedding space curvature evolves during training. 15

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

A.2.4. D IMENSION T RANSFORMATION AND F EATURE P ROJECTION Linear transformations in hyperbolic space follow the Hyperbolic Transformation with Curvatures (HTC) framework (Yang ′ ⊤ n d′ ×(n+1) et al., 2024). For input p = [pt , p⊤ and b ∈ Rd : s ] ∈ LK with parameters W ∈ R ! p ∥p̃s ∥2 − 1/K fL (p) = , where p̃s = ψ(Wp + b), (25) p̃s ′

with optional activation ψ, producing output p̃ ∈ LdK where d′ denotes the output dimension. This transformation applies to the full Lorentz vector, enabling both Lorentz rotations and boosts (Chen et al., 2022). Curvature changes in our framework are handled separately via log-exp map projection (see Equation (8) in the main text). Hyperbolic concatenation. For spatially-structured inputs, we extend this to convolutions by: (1) gathering features within each receptive field, (2) performing hyperbolic concatenation of their spatial components while preserving manifold structure, and (3) applying the linear transformation. Specifically, for features {ph,w } in a window, the output is fL (HCat({ph,w })), where hyperbolic concatenation (Lorentz direct concatenation (Qu & Zou, 2022)) combines N vectors as: HCat({pi }N i=1 ) =

qP N

2 i=1 pit + (N − 1)/K ⊤ p⊤ 1s , . . . , pN s

!⊤ ∈ LnN K .

(26)

A.3. Classification and Cross-Modal Fusion A.3.1. H YPERBOLIC C LASSIFICATION VIA G EODESIC H YPERPLANES Classification in hyperbolic space replaces Euclidean linear boundaries with geodesic hyperplanes (Ganea et al., 2018; Shimizu et al., 2021). Each class c ∈ {1, . . . , C} is parameterized by a scalar ac ∈ R controlling hyperplane position and a direction vector zc ∈ Rn . The logit for class c measures the signed distance from input p ∈ LnK to the corresponding hyperplane: √ −K αc sign(αc ) · βc −1 √ ℓc (p) = sinh , (27) βc −K with intermediate quantities: √ √ αc = cosh( −Kac )⟨zc , ps ⟩ − sinh( −Kac )∥zc ∥ · pt , r 2 √ √ βc = ∥ cosh( −Kac )zc ∥2 − sinh( −Kac )∥zc ∥ .

(28) (29)

Points on the positive side of the hyperplane yield positive logits, enabling standard softmax classification. A.3.2. H YPERBOLIC C ROSS -M ODAL ATTENTION Our cross-attention mechanism enables information exchange between modalities while respecting hyperbolic geometry. Unlike self-attention, each modality m attends only to other modalities M \ {m}, ensuring pure cross-modal information flow. Attention computation. Our attention mechanism extends Hypformer’s approach (Yang et al., 2024) by using explicit hyperbolic distance computation with curvature-adaptive temperature and curvature-based priors, while maintaining the core idea of using negative squared hyperbolic distance as similarity. After projecting all modalities to the unified fusion manifold LdKf , we compute attention weights using negative squared geodesic distance as similarity (Gulcehre et al., 2019; Yang et al., 2024): exp −d2L (q(m) , k(j) )/τ (m) , αm→j = P (30) 2 (m) , k(j ′ ) )/τ (m) j ′ ∈M\{m} exp −dL (q where query q(m) and keys k(j) are obtained via hyperbolic linear projections (Equation (25)), and τ (m) > 0 is the temperature. The squared distance admits the closed form d2L (p, q) = −2/|Kf | − 2⟨p, q⟩L , ensuring closer points receive higher attention. 16

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

Algorithm 1 EEG-MoCE pipeline Require: Minibatch inputs; y if training; {K (m) }, τ0 , L. Ensure: ŷ; updated Θ if training. 1: for each modality m do (m) 2: x(m) ← eθ (·) {Euclidean encoding (Sec. 3.2)} (m) 3: h from Equation (10) {lift to per-modality LdK (m) } 4: Equation (20) {BN}; Equation (16) {activation}; Equation (26) {concatenation} → z(m) 5: end for P 6: Kf ← |M|−1 m K (m) 7: for each m do (m) 8: zf via Equation (8) {align all modalities to LdKf } 9: end for 10: for ℓ = 1, . . . , L do Equation (6) {curvature-scaled temperature}; Equation (7) with dL from Equation (2) {hyperbolic attention}; add 11: λϕ(K (j) ) term of Equation (7) iff ℓ=1 12: merge with Equation (3) {Fréchet mean} 13: end for 14: Equation (4) {linear}; ŷ ← Equation (27) {MLR} 15: if training then 16: L ← CE(ŷ, y); update Θ {RSGD (Bonnabel, 2013) where needed} 17: end if

Multi-head aggregation. Each attention head produces an output via weighted Fréchet mean over attended values. Outputs across H heads are then combined via another Fréchet mean, followed by hyperbolic layer normalization (Yang et al., 2024): n oH (m) (m) zcross = LNL wFM Headh . (31) h=1

Layer normalization applies standard normalization to spatial coordinates, then reconstructs the temporal component (analogous to Section A.2.1). A.4. EEG-MoCE algorithm pipeline For every expression referenced in Algorithm 1, we P state its functional role, linking mathematical notation to implementation logic. The scalar fusion curvature Kf = |M|−1 m∈M K (m) (Sec. 3.1) fixes the Lorentz scale of LdKf before Equation (8); it is not assigned a separate equation number. Training uses cross-entropy and parameter updates (including Riemannian steps where needed) without additional displayed objects beyond Algorithm 1. (m)

Modality expert. Equation (10) lifts eθ (·) onto the per-modality Lorentz manifold LdK (m) . Equation (20) performs gyrovector batch normalization on LdK (m) . Equation (16) applies σ to the spacelike components and reconstructs the time coordinate so the point remains on the hyperboloid. Equation (26) implements the Lorentz direct concatenation. Cross-modal fusion. Equation (8) maps each expert output from LdK (m) to the shared LdKf so that the interaction between queries and keys is well defined on a unified Lorentz manifold. Equation (6) sets the per-modality softmax temperature τ (m) . Equation (2) defines the geodesic distance dL ; Equation (7) builds logits from squared hyperbolic distance d2L /τ (m) and adds the curvature prior λϕ(K (j) ). Equation (3) fuses attended values inside each fusion layer and performs the final modality-level pool on LdKf . Classification head. Equation (4) applies the Lorentz fully connected map to the fused embedding on LdKf . Equation (27) maps that representation to class logits via the hyperbolic multinomial logistic regression described in Section A.3.1. A.5. Quantifying Hierarchical Structure To validate that target modalities benefit from hyperbolic embedding, we measure their intrinsic tree-likeness via Gromov’s δ-hyperbolicity (Gromov, 1987). 17

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

Gromov product. configuration:

For a metric space (X, d), the Gromov product quantifies how closely three points approximate a tripod

1 (d(x, y) + d(x, z) − d(y, z)) . 2 Geometrically, larger values indicate x lies further from the geodesic between y and z. (y, z)x =

Four-point condition.

(32)

A space is δ-hyperbolic if all quadruples satisfy: (x, z)w ≥ min {(x, y)w , (y, z)w } − δ.

(33)

Trees satisfy this exactly with δ = 0; the parameter δ measures deviation from perfect tree geometry. Scale-invariant metric.

For cross-dataset comparison, we normalize by diameter (Khrulkov et al., 2020): δrel (X) =

2δ(X) ∈ [0, 1]. diam(X)

(34)

Values near zero indicate strong hierarchical structure amenable to low-distortion hyperbolic embedding.

B. Dataset Details We provide detailed descriptions and preprocessing procedures for the three datasets used in our experiments. EAV (Lee et al., 2024) is a multimodal emotion recognition benchmark in conversational contexts, comprising 30-channel EEG, audio recordings of speech, and video recordings of facial expressions from 42 participants. The experiment elicited five emotions (neutral, anger, happiness, sadness, and calmness) through cue-based conversation scenarios, with each participant contributing 100 trials with simultaneous recording of all three modalities. Following the official procedure, we preprocess the dataset by downsampling the EEG signal to 100 Hz and bandpass filtering at 0.5–45 Hz. Audio is resampled to 16 kHz and converted to mel-spectrogram, while visual features are extracted using OpenFace (Baltrusaitis et al., 2016) at 5 fps. ISRUC (Khalighi et al., 2016) is a sleep dataset containing polysomnographic (PSG) recordings, including EEG, EMG, and EOG from 10 healthy subjects. Each recording was scored by two human experts according to standard sleep staging criteria (Wake, N1, N2, N3, and REM). All PSG signals are resampled to 100 Hz and bandpass filtered (EEG: 0.3–35 Hz, EOG: 0.3–30 Hz, EMG: 10–49 Hz), followed by z-score normalization. Each 30-second epoch is organized into sequences of 20 consecutive epochs for sleep staging. Cognitive (Shin et al., 2018) provides simultaneous EEG, EOG, and NIRS recordings from 26 participants performing cognitive tasks. The N-back cognitive task is a working memory task with three difficulty levels (0-, 2-, 3-back), where participants decide whether each presented digit matches the one shown n trials earlier. For preprocessing, EEG and EOG data are bandpass filtered at 0.5–50 Hz and resampled to 200 Hz across 28 channels. NIRS signals are lowpass filtered at 0.2 Hz and baseline-corrected using the −5 s to −2 s window.

C. Supplementary Related Work C.1. Hierarchical structures In this paragraph, we provide additional evidence for the claim that the modalities used in our experiments also exhibit hierarchical structure. In visual processing, elementary features are first extracted by lower cortical regions, then integrated into increasingly complex representations by higher cortical areas (Hochstein & Ahissar, 2002). For scalp EEG, Collins et al. (Collins et al., 2018) used different stimulus frequencies to reveal distinct neural processes at different levels of the visual hierarchy. Chen et al. (Chen et al., 2025a) demonstrated that facial expressions exhibit rich hierarchical relationships, where basic expressions form the foundation for more complex compound expressions. For physiological signals, Búzás et al. (Búzás et al., 2024) revealed hierarchical levels in the temporal organization of human physical activity, where activity bursts at the minute scale are organized into superstructures at longer scales (Lopez et al., 2024). Together with our empirical results on hierarchical structure, this evidence suggests that hyperbolic embeddings are a promising framework for multimodal learning. 18

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

C.2. Baseline methods We provide detailed descriptions of baseline methods used in our experiments. General multimodal fusion methods are applicable across all three tasks. LMF (Liu et al., 2018) performs efficient crossmodal integration through low-rank tensor decomposition. CTMWA (Zhang et al., 2024) employs crossmodal translation network with meta weight adaption for robust multimodal fusion. MMML (Wu et al., 2024) introduces multi-loss training strategies to jointly optimize modality-specific and fusion objectives. For these methods, we keep the fusion module intact and only adapt the encoder to support input modalities. Multimodal emotion recognition methods for EAV dataset. MM-DFN (Hu et al., 2022) employs a graph-based dynamic fusion architecture that addresses information redundancy by modeling contextual relationships across multiple semantic representations to improve inter-modal complementarity. GA2MIF (Li et al., 2024) develops two-stage multi-source information fusion using Multi-head Directed Graph Attention networks (MDGATs) for contextual modeling and Multi-head Pairwise Cross-modal Attention networks (MPCATs) for cross-modal modeling. AGF-IB (Shou et al., 2024) introduces adversarial alignment and graph fusion via information bottleneck for cross-modal learning. Hyper-MML (Kang et al., 2025) proposes a hypergraph multi-modal learning framework that integrates EEG with audio and video information, featuring an Adaptive Brain Encoder with Mutual-cross Attention (ABEMA) module for EEG processing and an Adaptive Hypergraph Fusion Module (AHFM) to model higher-order relationships among multi-modal signals. CMERC (Tu et al., 2024) presents a calibration framework that addresses prediction uncertainty in conversational emotion recognition through progressive curriculum learning, hybrid contrastive representation refinement, and confidence-based regularization. HEEGNet (Li et al., 2026) introduces hybrid Euclidean-hyperbolic neural networks for single modality EEG representation learning with improved cross-subject generalization. Among these methods, Hyper-MML and HEEGNet are the only methods designed to support EEG input. For the other methods, following (Kang et al., 2025), we replace text modality with EEG modality for fair comparison. Multimodal sleep staging methods for ISRUC dataset. SalientSleepNet (SSNet) (Jia et al., 2021) employs a temporal fully convolutional network architecture with multi-scale extraction and multimodal attention mechanisms to identify salient wave patterns in sleep physiological signals. CrossModalSleepTransformer (CMST) (Mostafaei et al., 2024) employs transformer encoder-decoder architecture with cross-modality attention for integrating multiple physiological channels. CrossFusionSleepNet (CFSNet) (Cao et al., 2026) presents a parallel architecture that processes multimodal features from both temporal and spectral domains, utilizing cross-attention to model interdependencies between time and frequency representations. MMNet (Lin et al., 2024) develops multiview fusion networks with multiscale local feature extraction and cross-linked fusion for sleep analysis. XSleepFusion (Hu et al., 2025) proposes a dual-stage information bottleneck framework with evolutionary attention Transformer for interpretable multimodal sleep analysis. Hybrid EEG-fNIRS learning methods for Cognitive dataset. STA-Net (Liu et al., 2025) proposes spatial-temporal alignment networks with functional near-infrared spectroscopy (fNIRS)-guided spatial alignment and EEG-guided temporal alignment to synchronize EEG and fNIRS signals. ST2A (Shi et al., 2025) introduces spatial-temporal convolution with dual attention mechanisms for EEG-fNIRS classification. EFDFNet (Xu et al., 2025) employs fNIRS features to enhance EEG feature disentanglement and uses a deep fusion strategy for effective multimodal feature integration, combining EMCNet (an attention state classification network for EEG that combines Mamba and Transformer) with fNIRS processing. TSMMF (Si et al., 2025) employs a bidirectional cross-modal transformer architecture for temporal-spatial fusion in affective BCI applications, building unified representations while maintaining separate modality branches to retain distinctive characteristics of EEG and fNIRS signals. EF-Net (Arif et al., 2024) is a CNN-based multimodal deep-learning model designed for subject-independent mental state recognition from EEG-fNIRS signals.

D. Additional Experimental Details D.1. Definition of Evaluation Metrics The evaluation metrics are balanced accuracy and F1 macro, defined as:

Balanced Accuracy =

1X T Pi + T Ni n i=1 T Pi + T Ni + F Pi + F Ni 19

(35)

EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts n

F1 Macro =

1X 2 · T Pi n i=1 2 · T Pi + F Pi + F Ni

(36)

where n is the number of classes, T Pi is the true positive count for class i, T Ni is the true negative count for class i, F Pi is the false positive count for class i, and F Ni is the false negative count for class i. D.2. Optimized fusion curvature To further examine the mean-curvature design, we compare it with an optimization-based fusion curvature computed from pretrained per-modality curvatures. The optimization-based reference follows the perspective introduced in concurrent work on modality alignment across hyperbolic manifolds (Wu et al., 2026). Table 12 shows that the two estimates are statistically close on all datasets. We perform paired two one-sided tests (TOST) on fold-wise paired means with equivalence margin ±0.005. The test supports that the mean curvature is an stable approximation for practical use. Table 12. Mean fusion curvature vs. optimization-based fusion curvature (Optimized K) derived using the objective in (Wu et al., 2026). Values are mean ± std across folds. The close agreement across datasets supports the practical use of mean fusion curvature.

Dataset

Mean K

Optimized K

EAV ISRUC Cognitive

−2.2544 ± 0.1020 −1.3511 ± 0.0777 −1.9834 ± 0.0630

−2.2509 ± 0.1013 −1.3547 ± 0.0760 −1.9821 ± 0.0623

D.3. Additional Experimental Results on Cognitive Dataset We report additional results on the Word Generation task of the Cognitive dataset. In this task, participants were instructed to think of words beginning with the given letter as fast as possible without repeating the same word; in Baseline, they relaxed and gazed at the fixation cross for low cognitive load. Table 13 compares our method with state-of-the-art approaches under the same cross-subject evaluation protocol. Table 13. Performance comparison with state-of-the-art methods on the Cognitive dataset, Word Generation task (n = 26). We report balanced accuracy (Acc) and macro-averaged F1 (F1) under cross-subject 10-fold cross-validation.

Method

Acc (%)

F1 (%)

MMML (Wu et al., 2024) EFDFNet (Xu et al., 2025) LMF (Liu et al., 2018) CTMWA (Zhang et al., 2024) STA-Net (Liu et al., 2025) TSMMF (Si et al., 2025) EF-Net (Arif et al., 2024) ST2A (Shi et al., 2025)

56.35 ± 8.22 58.85 ± 7.63 59.74 ± 6.18 59.87 ± 8.15 62.50 ± 8.72 63.82 ± 9.47 64.53 ± 11.08 67.27 ± 10.45

51.76 ± 12.07 53.83 ± 11.13 59.16 ± 6.38 58.68 ± 8.63 58.08 ± 12.65 62.87 ± 10.36 63.43 ± 12.33 66.64 ± 12.51

Ours

73.65 ± 6.93

73.36 ± 7.09

Record · ID 13113 · SHA-256 f4199b77aa4698b0
Conceptio Open Knowledge Archive — every document is proof-bundled with source, license, and retrieval metadata.