arXiv:2605.22775v1 [cs.LG] 21 May 2026
MambaGaze: Bidirectional Mamba with Explicit Missing Data Modeling for Cognitive Load Assessment from Eye-Gaze Tracking Data Amir Mousavi1 , Mohammad Sadegh Sirjani1 , Erfan Nourbakhsh1 , Mimi Xie1 and Rocky Slavin1 , Leslie Neely3 , John Davis2 , John Quarles1 1 Department of Computer Science, College of AI, Cyber and Computing, The University of Texas at San Antonio 2 Department of Educational Psychology, College of Education and Human Development, The University of Texas at San Antonio 3 Department of Neuroscience, Developmental and Regenerative Biology, College of Sciences, The University of Texas at San Antonio {seyedamir.mousavi, mohammadsadegh.sirjani, erfan.nourbakhsh, mimi.xie, rocky.slavin, john.quarles}@utsa.edu, {leslie.neely, john.davis2}@utsa.edu Abstract Real-time cognitive load assessment from eyetracking signals could potentially enable adaptive human-centered-AI such as safety-critical applications such as driver vigilance monitoring or automated flight deck assistance, yet two challenges persist: handling frequent data missingness from blinks and tracking failures, and efficiently modeling long-range temporal dependencies. We propose MambaGaze, a framework that addresses these challenges through 1) XMD encoding, which augments raw features with observation masks and time-deltas to explicitly model data uncertainty, and 2) bidirectional Mamba-2, which captures temporal dependencies with linear computational complexity. Experiments on CLARE and CL-Drive datasets under leave-one-subject-out evaluation show that MambaGaze achieves 76.8% and 73.1% accuracy, respectively, outperforming CNN, Transformer, ResNet, and VGG baselines by 4–12 percentage points. Edge deployment benchmarks on NVIDIA Jetson platforms demonstrate real-time inference at 43–68 FPS with power consumption below 7.5W, confirming feasibility for wearable cognitive load monitoring.
1
Introduction
Cognitive load assessment from physiological signals has emerged as a critical research direction with applications in adaptive interfaces, educational technology, and safetycritical systems [Appel et al., 2018; Wang et al., 2013]. Among various physiological indicators, eye-tracking signals, such as pupil diameter, gaze dynamics, fixations, and saccades, provide rich temporal cues that correlate with cognitive states [Jin et al., 2025; Ekin et al., 2025]. However, effective assessment from such signals via deep learning-based classification requires addressing two key challenges: (1) rep-
Figure 1: Key Challenges in Real-Time Cognitive Load Assessment from Eye-Tracking Data.
resenting incomplete observations arising from blinks, occlusions, and tracking failures, and (2) capturing long-range temporal dependencies efficiently under strict latency constraints for edge deployment (see Figure 1). Existing deep learning approaches typically treat missing data as a preprocessing problem, applying interpolation, forward-fill, or deletion before model input [Sarkar and Etemad, 2023; Little and Rubin, 2020]. Additionally, window-based aggregation over fixed intervals [Bhatti and others, 2024] reduces temporal granularity. These strategies distort important fine-grained dynamics for cognitive state inference or discard "informative missingness"—
patterns where the extended gaps of data itself may indicate specific gaze behaviors or disengagement. Moreover, conventional approaches lose valuable information about observation reliability and introduce systematic bias, or become heavily reliant on manual feature engineering. while these methods report strong offline validation performance, they have not been validated for edge deployment or real-time processing constraints that practical cognitive monitoring systems require. For temporal modeling, recurrent architectures suffer from vanishing gradients, while Transformer-based approaches have quadratic complexity that limits scalability [Bozkir et al., 2019] for high-frequency physiological streams and prevents efficient edge implementation. Recent state space models such as Mamba-2 [Dao and Gu, 2024], achieve linear complexity while maintaining strong long-range modeling capabilities. However, adapting these models for classification tasks with incomplete data remains unexplored. In this paper, we propose a novel architecture combining Timeseries-Masks-Deltas (XMD) input encoding [Che et al., 2018] with bidirectional Mamba-2 model to enhance cognitive load classification for practical deployment with low-latency requirements. We address the challenge of asynchronous sensor and processing sampling by constructing a tensor that augments resampled feature values (X) with binary masks (M) indicating the arrival of fresh data observation, and time deltas (D) that quantify the temporal gap since the last observed value. This representation is processed by a bidirectional Mamba-2 model, which aggregates temporal context from both forward and reverse directions. This framework allows the model to distinguish between genuine signal updates and artifacts, maintain the linear complexity of State Space Models (SSMs), and utilize future context to stabilize physiological state inference, even when input data is sparse or irregular. Our main contributions can be summarized as follows: • We present an integration of explicit missingness representations (XMD) with selective State Space Models for eye-tracking data. By embedding observation masks and time intervals directly into the input, we enable the model to learn from missingness patterns rather than treating them as noise to be removed. • We introduce a bidirectional formulation of Mamba2 that captures global temporal context—incorporating both past history and retrospective future information—without the quadratic cost of Transformers. This provides the computational efficiency required for processing extended recordings on edge devices. • Extensive experiments demonstrate that our approach achieves 76.8% accuracy on the CLARE dataset and 73.1% on CL-Drive using Leave-one-Subject-out cross validation for personalization and to avoid relying on optimal performance that K-fold may produce. The model outperforms baseline CNN and Transformer architectures while maintaining real-time inference latencies of 14.8–23.8 ms
2
Related Works
2.1
Eye-Tracking Signals for Cognitive Load Assessment
Eye-tracking measurements provide temporal indicators of cognitive demand across multiple timescales. Relevant biomarkers include pupil diameter dynamics and eye movement behaviors including fixation patterns and saccadic characteristics [Jin et al., 2025; Ekin et al., 2025]. While these signals enable non-invasive cognitive state monitoring, they are vulnerable to environmental and technical artifacts. Specifically, pupil measurements demonstrate high sensitivity to surrounding light variations and calibration errors, whereas gaze sequences frequently contain gaps resulting from physiological events (blinks) and tracking failures (occlusions, signal dropout) [Little and Rubin, 2020]. Conventional cognitive load pipelines address these challenges through extensive preprocessing and feature aggregation procedures, which necessarily suppress rapid temporal dynamics and systematically exclude missing segments as measurement noise. The present work adopts an alternative approach: direct classification from raw eye-tracking sequences while explicitly preserving observation reliability metadata, thereby enabling the model to leverage missingness patterns as informative signals rather than artifacts requiring elimination.
2.2
Deep Learning Architectures in Cognitive Load Benchmarks
Contemporary benchmark datasets have established deep neural networks as standard baselines for cognitive load recognition from physiological and eye-tracking measurements. The CLARE dataset employs convolutional and Transformer-based architectures as representative sequence modeling baselines, and demonstrates competitive classification performance under cross-validation evaluation protocols [Bhatti and others, 2024]. Similarly, the CL-Drive benchmark evaluates visual recognition backbones including VGG and ResNet architectures for modeling cognitive load in vehicular driving tasks [Angkan et al., 2024b]. Despite demonstrating effective representation learning capabilities, these investigations share a common limitation in missing data handling. Specifically, existing approaches employ: 1. Complete-case deletion, removing time windows containing gaps 2. Deterministic imputation methods (linear interpolation, forward-fill propagation) 3. Preprocessing pipelines that execute prior to model input [Sarkar and Etemad, 2023; Little and Rubin, 2020] This design paradigm prevent distinction between observed measurements and imputed estimates, so the learning system may not be able to identify associations between cognitive states and observation gap patterns. Our methodology maintains explicit differentiation through XMD encoding and validates performance under leave-one-subject-out(LOSO) in additon to K-fold cross-validation to also reflect deployment scenarios where held-out subjects exhibit distinct personalized characteristics.
2.3
Missingness-Aware Modeling and Efficient Long-Range Sequence Processing
Parallel research developments have investigated learning frameworks for incomplete temporal sequences. GRU-D and related recurrent architectures incorporate binary masking indicators and temporal decay functions (time-since-lastobservation) into hidden state updates which understand differentiation between observed values and model-generated imputations [Che et al., 2018]. However, standard recurrent neural networks exhibit fundamental limitations in longcontext modeling and computational parallelization [Goodfellow et al., 2016]. Transformer architectures address these constraints through self-attention mechanisms, though their quadratic computational complexity (O(T 2 )) limits scalability for high-frequency data streams [Vaswani et al., 2017]. Structured state space models (SSMs) provide an alternative framework with linear time complexity. Recent SSM families including S4 and Mamba have demonstrated effective long-range dependency modeling while maintaining computational efficiency [Gu et al., 2022; Gu and Dao, 2023]. The Mamba-2 architecture further advances selective state-space parameterization, enabling input-dependent filtering operations at scale [Dao and Gu, 2024]. Despite these methodological advances, existing cognitive load classification frameworks have not integrated: (1) explicit missingness representations encompassing observed values, binary indicators, and temporal decay signals with (2) bidirectional structured state space architectures for eye-tracking sequence processing. While GRUD [Che et al., 2018] and related approaches employ maskdelta formulations with fixed exponential decay, our integration with Mamba-2’s selective parameterization enables input-dependent filtering that adapts to the heterogeneous temporal dynamics of eye-tracking signals.
3
Methodology
3.1
Overview
This section presents MambaGaze for cognitive load classification from eye-tracking data. The framework processes eyetracking sequences through four stages: • Data Processing Pipeline: Signal preprocessing, temporal resampling, segmentation, and baseline normalization that preserves observation reliability information (Figure 2, Section 3.2); • XMD Input Encoding: Augments raw features with observation masks and time-deltas for explicit missing data modeling (Figure 3a, Section 3.2); • Bidirectional Mamba-2 Processing: Enables efficient long-range temporal modeling with linear complexity using separate forward and backward state space models (Figure 3b, Section 3.3); • Attention-based Pooling: Adaptively aggregates temporal information through learned attention weights for sequence classification (Figure 3c, Section 3.4).
3.2
Problem Formulation
We frame cognitive load assessment as a time-series classification task with inherent missing data challenges. Let X ∈ RF ×T denote eye-tracking signals with F features and length T , and Y = {0, 1} represent the binary label space for low or high cognitive load. Missing Data Challenge. Unlike clinical signals with controlled acquisition, eye-tracking data exhibits frequent missingness due to blinks, gaze aversion, and tracking failures. Standard approaches—deletion or simple imputation— discard valuable reliability information. The missing pattern itself carries signal: frequent blinks may indicate fatigue, while tracking losses correlate with rapid eye movements during high cognitive load. Solution Strategy. Our goal is to learn a mapping Φ : RF ×T → Y that explicitly models observation uncertainty. We address this through: (1) a data processing pipeline that preserves missingness information rather than discarding it, (2) XMD encoding that augments features with observation masks and time-deltas, and (3) a bidirectional state space model that efficiently captures long-range temporal dependencies. Data Processing Pipeline The data processing pipeline transforms raw asynchronous eye-tracking recordings into fixed-length, normalized windows suitable for sequence modeling while preserving observation reliability information. Signal Preprocessing. Raw eye-tracking data arrives asynchronously, with multiple sensor channels updating at different times. We extract F = 10 canonical features: pupil diameter (left/right), gaze coordinates (x, y), velocity, acceleration, fixation indicator, saccade indicator, blink events, and distance. Duplicate timestamps are coalesced per-column using the last valid observation, preventing incorrect merging of asynchronous channel updates. Temporal Resampling. We resample all signals to a uniform grid at fs = 50 Hz using forward-fill interpolation, yielding T = 500 timesteps per 10-second window. Critically, we compute observation masks from the original source timestamps via per-channel change detection before resampling—this preserves the true observation pattern rather than the interpolated values. Temporal Segmentation and Label Assignment. We segment continuous recordings into fixed-length windows xk ∈ RF ×T of duration w = 10 seconds. Cognitive load labels are provided at ∆ = 10-second intervals (NASATLX scores 1–9, binarized at threshold 5). For each window Wk = [tk , tk + w), we assign the label corresponding to the interval covering the window’s midpoint: tk + w/2 y(Wk ) = yi⋆ , i⋆ = (1) ∆ This midpoint rule ensures unambiguous label assignments at interval boundaries. Baseline Normalization. Within each participant, we base compute baseline statistics (µbase f , σf ) from a dedicated resting-state recording. All features are z-score normalized
Figure 2: Data processing pipeline. Raw eye-tracking data from experiment, baseline, and label CSVs undergoes time synchronization, deduplication, and canonical mapping during ingestion. The signal path performs 50Hz grid resampling and forward-backward fill imputation, while structural metadata tracks observation masks (M) and time-deltas (D) via source change detection. Z-score normalization uses baseline statistics (µ, σ), and windowing produces fixed-length tensors (Ximputed , Mgrid , Dgrid ) with aligned binary labels for participant-based KFold/LOSO splitting.
relative to these baseline statistics: x̃t,f =
xt,f − µbase f σfbase + ϵ
(2)
This participant-specific normalization accounts for interindividual variability in pupil size, gaze patterns, and sensor calibration. XMD Input Encoding To explicitly model observation uncertainty, we adopt ValuesMasks-Deltas (XMD) encoding [Che et al., 2018] that represents missing data within the model input, enabling the network to learn different behaviors for observed versus imputed values. Values. For each feature f , missing values are imputed using forward-fill with baseline initialization: if mt,f = 1 xt,f x̃t,f = x̃t−1,f if mt,f = 0 and t > 1 (3) base µf if mt,f = 0 and t = 1 Masks. Binary observation masks M ∈ {0, 1}T ×F indicate data reliability. The mask is computed via change detection on the original (pre-resampling) data: mt,f = 1 when feature f receives a genuinely new observation at time t, distinguishing true measurements from interpolated values. Deltas. Time deltas D ∈ RT≥0×F encode elapsed time since
the last valid observation: 0 δt,f = δt−1,f + ∆t
if mt,f = 1 if mt,f = 0
(4)
where ∆t = 1/fs . We apply log-scaling δ̄t,f = log(1 + δt,f ) to compress the dynamic range. The complete XMD representation concatenates these components: Zt = [x̃t ∥mt ∥δ̄ t ] ∈ R3F (5) This tripling of input dimensionality (F ′ = 3F = 30) provides explicit uncertainty quantification without architectural modifications.
3.3
Architecture
We employ Mamba-2 [Dao and Gu, 2024], a selective state space model achieving linear complexity O(T ) compared to the quadratic complexity O(T 2 ) of Transformer-based attention. For eye-tracking sequences spanning hundreds of timesteps (T = 500 at 50 Hz), this efficiency is essential. Mamba-2 introduces selective state spaces where transition parameters depend on the input content, computing inputdependent parameters Bt , Ct , and step size ∆t from each input xt . This selectivity enables content-aware filtering, allowing the model to learn to retain or forget information based on input relevance. For eye-tracking data, transient events like saccades and blinks require different temporal dynamics than sustained states like fixations (see Appendix C for SSM background).
Figure 3: MambaGaze architecture. (a) XMD encoding augments eye-tracking features with observation masks and time-deltas for explicit missing data modeling. (b) Bidirectional Mamba-2 processes sequences in both temporal directions with separate parameter sets. (c) Attention pooling aggregates temporal information for binary cognitive load classification.
Bidirectional Mamba-2 Processing Cognitive load assessment benefits from bidirectional context, as pupil dilation may reflect anticipation of upcoming cognitive demands or reaction to recent mental effort. Standard Mamba processes sequences causally (left-to-right), limiting access to future context. We extend Mamba-2 to bidirectional processing using two separate branches with independent parameters. Forward Mamba. Processes the sequence in causal order:
End-to-End Computation For each XMD-encoded window Zk ∈ RT ×3F , the complete forward pass proceeds as: fwd Hfwd ) k = Mamba(Zk ; θ
Hbwd = flip(Mamba(flip(Zk ); θ bwd )) k fwd D cfwd k = AttnPool(Hk ) ∈ R D cbwd = AttnPool(Hbwd k k )∈R bwd 2D ck = [cfwd k ∥ck ] ∈ R
Hfwd = Mamba(Z1:T ; θ fwd ) (6) Backward Mamba. Processes the time-reversed sequence: Hbwd = flip(Mamba(flip(Z1:T ); θ bwd )) (7) where flip(·) reverses the temporal dimension. Each branch consists of L stacked Mamba-2 layers with pre-normalization and residual connections: hl = hl−1 + Dropout(Mamba(LayerNorm(hl−1 ))) (8) An input projection layer maps the XMD features to the model dimension: h0 = Win Z + bin , where Win ∈ RD×3F . Unlike bidirectional RNNs that concatenate hidden states at each timestep, we maintain separate forward and backward representations throughout processing. This design allows each branch to specialize in its temporal direction—the forward Mamba learns to anticipate future patterns while the backward Mamba captures historical context. predictions require complete 10-second segments, matching the labeling granularity where ground-truth annotations reflect integrated cognitive state rather than instantaneous measurements.
ŷk = σ(gψ (ck ))
(9) (10) (11) (12) (13) (14)
where AttnPool(·) denotes the attention pooling mechanism (Section 3.4) and gψ is the classification head.
3.4
Attention Pooling and Classification
Sequence classification requires aggregating temporal representations into a fixed-size vector. Rather than using the final hidden state (which biases toward recent information) or simple averaging (which weights all timesteps equally), we employ learned attention pooling that adaptively weights timesteps based on their relevance to the classification task.
Attention Pooling For each direction, we compute attention scores using an additive attention mechanism with a learned query: et = w⊤ tanh(Wa ht + ba ) exp(et ) αt = softmaxt (et ) = PT ′ t′ =1 exp(et ) cdir =
T X
αt hdir t
Dataset
Model
LOSO Acc F1Macro
K-fold Acc F1Macro
CLARE
CNN [Bhatti and others, 2024] Transformer [Bhatti and others, 2024] MambaGaze (Ours)
70.3 65.0 76.8
55.5 52.1 55.4
66.9 60.4 73.3
56.0 50.8 54.4
CL-Drive
ResNet [Angkan et al., 2024a] VGG [Angkan et al., 2024a] MambaGaze (Ours)
69.0 68.5 73.1
60.6 62.8 47.8
64.9 64.2 68.8
57.1 59.7 47.0
(15) (16)
(17)
Table 1: Comparison with baseline methods (%). MambaGaze achieves higher accuracy on both datasets under both evaluation protocols.
t=1
where Wa ∈ RD×D , ba ∈ RD , and w ∈ RD are learnable parameters. Separate attention modules are used for forward and backward directions, yielding context vectors cfwd and cbwd that are concatenated to form the final sequence representation c ∈ R2D . The attention weights αt provide interpretability by identifying which temporal regions drive the classification decision. Classification Head Architecture The classification head gψ is a lightweight module that maps the pooled representation to a binary prediction: ŷ = σ(wc⊤ LayerNorm(c) + bc )
(18)
where σ(·) is the sigmoid function, wc ∈ R2D , and bc ∈ R are learnable parameters. The layer normalization stabilizes the activations before the final projection, improving training stability.
3.5
Training Objective
Loss Function with Automatic Class Rebalancing Let yk ∈ {0, 1} and ŷk ∈ [0, 1] denote the window label and predicted probability. To address class imbalance in cognitive load distributions, we minimize a weighted binary crossentropy loss: N
L=−
1 X [w+ yk log(ŷk ) + (1 − yk ) log(1 − ŷk )] (19) N k=1
tasks) [Bhatti and others, 2024] and CL-Drive (15 subjects in simulated driving) [Angkan et al., 2024a]. Both provide eye-tracking data with self-reported cognitive load labels at 10-second intervals (1-9 scale), binarized into low (1-4) versus high (5-9) classes. Preprocessing. Raw eye-tracking data is resampled to 50 Hz and segmented into non-overlapping windows of T = 500 timesteps (10 seconds), aligned with labeling intervals. We extract F = 10 features including pupil diameter, gaze position, velocity, acceleration, fixation/saccade/blink indicators, and screen distance. With XMD encoding, each feature is augmented with observation mask and time-delta channels, yielding F ′ = 30 input dimensions. Details are in Appendix D. Evaluation Protocol. We use two cross-validation strategies: (i) Leave-One-Subject-Out (LOSO) to assess generalization to unseen individuals, and (ii) participant-based Kfold cross-validation. To address class imbalance, we apply dynamic positive class weighting, threshold optimization, and post-hoc calibration (see Appendix E for details). Baseline Methods. We compare against dataset-specific baselines: CNN and Transformer models for CLARE [Bhatti and others, 2024], and ResNet and VGG architectures for CL-Drive [Angkan et al., 2024a]. All baselines use identical preprocessing and evaluation protocols to isolate architectural differences.
4.2
Experiment Analysis
4
Experiments
To validate MambaGaze’s effectiveness, we compare it against CNN and Transformer baselines trained from scratch, as shown in Table 1. Notably, MambaGaze substantially outperforms all baselines on subject-independent LOSO evaluation, achieving accuracy improvements of +6.5pp on CLARE and +4.1pp on CL-Drive over the best baselines. On CLARE, MambaGaze reaches 76.8% accuracy, outperforming CNN (70.3%) and Transformer (65.0%) by 6.5 and 11.8 percentage points, respectively. On CL-Drive, MambaGaze achieves 73.1% accuracy, surpassing ResNet (69.0%) and VGG (68.5%). These results demonstrate that Mamba’s selective state space modeling effectively addresses the temporal dynamics in eyetracking signals, with consistent improvements across both datasets and evaluation protocols.
4.1
Experimental Setup
4.3
Automatic Class Weighting. The positive class weight w+ is computed automatically per fold from training label counts (n0 , n1 ): n+ w+ = (20) n− This inverse-frequency weighting gives the minority class higher effective weight. We monitor macro-F1 on the validation set to detect majority-class collapse and for early stopping. Training uses AdamW [Loshchilov and Hutter, 2019] optimization with early stopping based on validation AUC. See Appendix E for complete hyperparameter settings.
Datasets. datasets:
We evaluate on two public cognitive load CLARE (20 subjects performing cognitive
Ablation Study
Class Imbalance Optimization Analysis. We analyze the contribution of each optimization technique for han-
dling class imbalance, as shown in Figure 4. We denote: MambaGaze-Raw as the baseline without optimization (standard BCE loss, threshold 0.5, no calibration), and MambaGaze-P, MambaGaze-T, MambaGaze-C as variants with positive class weighting, threshold optimization, and post-hoc calibration respectively. The ablation reveals a clear performance progression across configurations. MambaGaze-T (threshold optimization) provides the largest individual gain, improving accuracy by +10.7pp on CLARE and +16.7pp on CL-Drive over the raw baseline. This indicates that the default 0.5 threshold is suboptimal for imbalanced cognitive load distributions. Threshold selection uses validation data within each fold to avoid test leakage. The accuracy gains with stable AUC confirm that improvements stem from calibrated decision boundaries rather than enhanced separability alone. MambaGazePTC combines all three techniques to achieve the best overall performance: 77.9% accuracy on CLARE-LOSO and 77.8% on CL-Drive-LOSO, with F1 improvements of +7.6pp and +15.5pp, respectively, over the raw baseline.
Model
Latency
LOSO FPS
Power Latency
K-fold FPS
Power
AGX Orin CNN/ResNet 4.2/5.0 241/201 3.7/3.7 4.5/4.9 226/203 3.9/3.7 Trans./VGG 4.2/4.1 236/247 3.7/3.8 4.3/4.0 231/248 3.7/3.8 MambaGaze 14.8/14.7 67/68 3.8/3.8 14.8/14.8 68/68 3.8/3.8 Orin NX CNN/ResNet 5.2/6.6 193/155 6.5/6.2 6.1/6.4 169/156 6.5/6.1 Trans./VGG 7.5/5.3 144/188 6.7/6.4 6.0/5.4 172/187 6.1/6.7 MambaGaze 23.8/19.3 43/52 7.5/6.5 20.3/19.5 50/51 6.7/6.4 Orin Nano CNN/ResNet 4.6/5.6 215/177 6.1/5.9 4.8/5.8 221/173 6.2/5.9 Trans./VGG 4.9/4.3 204/233 5.7/6.4 4.8/4.3 209/234 5.7/6.3 MambaGaze 18.2/18.1 55/55 6.4/6.4 18.2/18.2 55/55 6.4/6.4
Table 2: Edge deployment benchmarks across NVIDIA Jetson platforms using MambaGaze-PTC. Latency is reported in milliseconds (ms) and power in watts (W). Values are shown as CLARE / CLDrive.
Edge Deployment Analysis. To evaluate real-time applicability, we benchmark trained models on three NVIDIA Jetson platforms, as shown in Table 2. All models achieve real-time performance across all Jetson platforms, with consistent results between LOSO and K-fold trained models. MambaGaze operates at 43–68 FPS, sufficient for 10-second prediction windows in cognitive load assessment. While CNN/Transformer baselines achieve higher throughput (140–248 FPS), MambaGaze’s superior accuracy justifies the latency trade-off. Power consumption remains within 3.7–7.5W, enabling deployment in power-constrained environments such as vehicles or wearable systems. Even on the resource-constrained Orin Nano (8GB RAM, 40 TOPS), MambaGaze maintains 55 FPS, well above the 0.1 Hz sampling requirement for real-time cognitive load monitoring.
5
Conclusion
We propose MambaGaze, a framework for cognitive load classification from eye-tracking data with inherent missing observations. MambaGaze addresses two key challenges through XMD encoding, which augments raw features with observation masks and time-deltas to explicitly model data uncertainty, and bidirectional Mamba-2, which captures longrange temporal dependencies with linear computational complexity. Experiments on CLARE and CL-Drive datasets under leave-one-subject-out evaluation show that MambaGaze substantially outperforms CNN, Transformer, ResNet, and VGG baselines, achieving accuracies of 76.8% and 73.1%, respectively. Our results demonstrate that explicit missing data modeling combined with efficient state space architectures can effectively address the observation uncertainty and temporal complexity challenges in wearable cognitive load assessment. Edge deployment benchmarks confirm realtime feasibility at 55–68 FPS with power consumption below 7.5W. Future work will explore extending this approach to multimodal physiological signals and investigating online adaptation for personalized cognitive load thresholds.
Figure 4: Ablation study on class imbalance optimization techniques across CLARE (top) and CL-Drive (bottom) datasets. Raw: baseline without optimization; P: positive class weighting; T: threshold optimization; C: post-hoc calibration; PTC: all three techniques combined. Each subplot shows performance for a specific metric under LOSO and K-fold evaluation protocols. MambaGaze-PTC consistently achieves the best performance, with threshold optimization (T) providing the largest individual improvement.
References
[Angkan et al., 2024a] Prithila Angkan, Behnam Behinaein, Zunayed Mahmud, Anubhav Bhatti, Dirk Rodenburg, Paul Hungler, and Ali Etemad. Multimodal brain–computer interface for in-vehicle driver cognitive load measurement: Dataset and baselines. IEEE Transactions on Intelligent Transportation Systems, 25(6):5949–5964, 2024.
[Angkan et al., 2024b] Prithila Angkan, Behnam Behinaein, Zunayed Mahmud, Anubhav Bhatti, Dirk Rodenburg, Paul Hungler, and Ali Etemad. Multimodal brain-computer interface for in-vehicle driver cognitive load measurement: Dataset and baselines. IEEE Transactions on Intelligent Transportation Systems, 25:5949–5964, 2024.
[Appel et al., 2018] Tobias Appel, Christian Scharinger, Peter Gerjets, and Enkelejda Kasneci. Cross-subject workload classification using pupil-related measures. In Proceedings of the 2018 ACM Symposium on Eye Tracking Research & Applications, ETRA ’18, New York, NY, USA, 2018. Association for Computing Machinery.
[Bhatti and others, 2024] Umer Asgher Bhatti et al. Clare: A dataset for cognitive load assessment in realtime. arXiv preprint, 2024.
[Bozkir et al., 2019] Efe Bozkir, David Geisler, and Enkelejda Kasneci. Person independent, privacy preserving, and real time assessment of cognitive load using eye tracking in a virtual reality setup. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pages 1834– 1837, 2019.
[Che et al., 2018] Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu. Recurrent neural networks for multivariate time series with missing values. Scientific Reports, 8(1):6085, 2018. Nature Scientific Reports - GRU-D.
[Dao and Gu, 2024] Tri Dao and Albert Gu. Transformers are ssms: Generalized models and efficient algorithms through structured state space duality. In International Conference on Machine Learning, 2024. ICML 2024. [Ekin et al., 2025] Merve Ekin, Krzysztof Krejtz, Carlos Duarte, Andrew Duchowski, and Izabela Krejtz. Prediction of intrinsic and extraneous cognitive load with oculometric and biometric indicators. Scientific Reports, 15:89336, 2025. Nature Scientific Reports. [Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016. [Gu and Dao, 2023] Albert Gu and Tri Dao. Mamba: Lineartime sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752, 2023. ICLR 2024 submission. [Gu et al., 2022] Albert Gu, Ankit Gupta, Karan Goel, and Christopher Ré. On the parameterization and initialization of diagonal state space models. Advances in Neural Information Processing Systems, 35:35971–35983, 2022. NeurIPS 2022. [Jin et al., 2025] Xiaofu Jin, Yunpeng Bai, Lina Xu, Shuai Ma, Danqing Shi, Luwen Yu, and Mingming Fan. Decoding cognitive load: eye-tracking insights into working memory and visual attention. In Proceedings of the 2025 Symposium on Eye Tracking Research and Applications. ACM, 2025. ACM ETRA - Premier eye tracking venue. [Little and Rubin, 2020] Roderick JA Little and Donald B Rubin. Statistical analysis with missing data. Wiley, 3rd edition, 2020. [Loshchilov and Hutter, 2019] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations (ICLR 2019), 2019. [Sarkar and Etemad, 2023] Platon Sarkar and Ali Etemad. Deep imputation of missing values in time series health data: A review with benchmarking. Journal of Biomedical Informatics, 144:104440, 2023. [Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017. NeurIPS 2017 - Foundational Transformer paper. [Wang et al., 2013] Weihong Wang, Zhidong Li, Yang Wang, and Fang Chen. Indexing cognitive workload based on pupillary response under luminance and emotional changes. In Proceedings of the 2013 International Conference on Intelligent User Interfaces, IUI ’13, page 247–256, New York, NY, USA, 2013. Association for Computing Machinery.
Appendix A
Baseline Methods
We compare MambaGaze against baseline methods for cognitive load classification from eye-tracking signals: CNN [Bhatti and others, 2024] represents the family of convolutional architectures for end-to-end eye-tracking analysis. This baseline uses a shallow CNN with multiple convolutional and max-pooling layers, followed by fully connected layers for binary classification. The model learns features directly from aggregated eye-tracking statistics without explicit missing data handling. Transformer [Bhatti and others, 2024] uses multi-head self-attention mechanisms to model temporal dependencies in eye-tracking sequences. The model applies positional encoding to input windows and processes them through stacked transformer encoder blocks. This architecture has quadratic complexity O(T 2 ) in sequence length. ResNet [Angkan et al., 2024a] applies residual network architectures originally designed for image classification to eye-tracking time series. Features are extracted from raw signals using 1D convolutions with skip connections. VGG [Angkan et al., 2024a] uses VGG-style deep convolutional networks adapted for time-series classification. Multiple convolutional blocks with small kernels extract hierarchical features. All baselines are evaluated using identical LOSO crossvalidation protocols to ensure fair comparison of subjectindependent generalization performance.
B
B.2
Datasets Used in This Work
We evaluate on two publicly available cognitive load datasets that provide fine-grained, time-aligned labels suitable for real-time assessment. CLARE (Cognitive Load Assessment in REaltime). CLARE contains multimodal physiological signals (ECG, EDA, EEG, and eye gaze) from 24 participants performing MATB-II multitasking protocols [Bhatti and others, 2024]. Participants provide in-situ verbal ratings of perceived cognitive load every 10 seconds on a 9-point Likert scale (1-9). Each participant completed four 9-minute sessions with nine complexity levels. The public release contains 20 subjects with eye-tracking data recorded at varying rates, resampled to 50 Hz. The MATB-II (Multi-Attribute Task Battery II) protocol requires participants to simultaneously manage four subtasks: system monitoring (detecting indicator changes), tracking (controlling a cursor), communications (responding to radio calls), and resource management (maintaining fuel levels). Task difficulty is manipulated by varying subtask demands across blocks.
Missing Data Characteristics
Eye-tracking data exhibits frequent missingness due to blinks, gaze aversion, and tracking failures. Analysis of the CLARE dataset reveals: • Basic features (pupil diameter, gaze position): ∼80% completeness • Derived features (saccades, fixations): ∼16% completeness This motivates our XMD encoding approach, which preserves observation reliability information rather than discarding it through simple imputation.
B.3
Dataset Access
Both datasets are publicly available: • CLARE: https://borealisdata.ca/dataset.xhtml? persistentId=doi:10.5683/SP3/H0AELT • CL-Drive: https://borealisdata.ca/dataset.xhtml? persistentId=doi:10.5683/SP3/JJ2YZZ
Dataset Characteristics
This section provides comprehensive analysis of the cognitive load datasets used for evaluation.
B.1
CL-Drive (Cognitive Load in Driving). CL-Drive captures multimodal signals (EEG, ECG, EDA, and eye tracking) from 21 participants in a driving simulator under scenarios of varying complexity [Angkan et al., 2024a]. Drivers verbalize their perceived workload at 10-second intervals using a 9point PAAS scale. Each participant performed driving tasks across 9 complexity levels, each lasting 3 minutes. Driving scenarios range from simple highway cruising (low load) to complex urban navigation with pedestrians, traffic lights, and lane changes (high load).
C
State Space Model Background
State space models (SSMs) parameterize sequence-tosequence transformations through continuous-time linear dynamical systems. The standard formulation maps an input signal u(t) ∈ R to output y(t) ∈ R via latent state h(t) ∈ RN : h′ (t) = Ah(t) + Bu(t) y(t) = Ch(t) + Du(t)
(21) (22)
where A ∈ RN ×N , B ∈ RN ×1 , C ∈ R1×N , and D ∈ R are learnable parameters.
C.1
Discretization
For discrete sequences sampled at intervals ∆, these continuous dynamics are discretized using the zero-order hold (ZOH) method: Ā = exp(∆A) −1
B̄ = (∆A)
(exp(∆A) − I) · ∆B
(23) (24)
This yields the discrete recurrence relation: ht = Āht−1 + B̄ut yt = Cht
(25) (26)
C.2
Selective State Spaces (Mamba)
6. Fixation flag: Binary indicator during fixation events
Standard SSMs use fixed parameters independent of input content. Mamba [Gu and Dao, 2023] introduces selective state spaces where parameters depend on the input: Bt = LinearB (xt ) Ct = LinearC (xt ) ∆t = softplus(Linear∆ (xt ))
(27) (28) (29)
This input-dependent selectivity enables content-aware filtering, crucial for eye-tracking where transient events (saccades, blinks) require different temporal dynamics than sustained states (fixations).
C.3
Structured State Space Duality (Mamba-2)
Mamba-2 [Dao and Gu, 2024] establishes a theoretical connection between selective SSMs and linear attention, enabling 2-8× speedup over Mamba-1 on modern GPUs through optimized kernels.
7. Saccade flag: Binary indicator during saccade events 8. Distance: Eye-to-screen distance
D.3
Baseline Normalization
Baseline pupil diameter varies significantly between participants. We estimate robust baselines using percentile filtering: p̃base = median({pt : pt ∈ [Q10 , Q90 ]})
(30)
Features are z-scored relative to participant-specific baselines: xt,f − µbase f x̂t,f = (31) base σf + ϵ
D.4
XMD Encoding Details
For each feature f and timestep t:
D
Preprocessing Pipeline
D.1
Preprocessing Hyperparameters
• Mask mt,f = 1 if a new observation arrived at time t; 0 otherwise • Delta δt,f : Seconds since last valid observation:
Parameter
Value
δt,f =
Resampling Target sampling rate Window duration Samples per window Window stride
50 Hz 10.0 s 500 Non-overlapping
Normalization Baseline source Normalization method Missing value fill
Per-participant baseline session Z-score (per feature) Forward-fill, then baseline
Labeling Label source Label granularity Binarization threshold
NASA-TLX self-report 10-second bins ≥ 5 (high load)
XMD Output Tensors Values shape Masks shape Deltas shape XMD concatenated
(N, 500, 10) (N, 500, 10) (N, 500, 10) (N, 500, 30)
Table 3: Preprocessing hyperparameters.
D.2
Feature Extraction
We extract F = 10 features from raw eye-tracking data: 1. Pupil diameter (left and right): Direct measure of cognitive load 2. Gaze position (x, y): Screen coordinates in normalized units 3. Gaze velocity: Computed from position derivatives 4. Gaze acceleration: Second derivative of position 5. Blink indicator: Binary flag for detected blinks
0 δt−1,f + ∆t
if mt,f = 1 otherwise
(32)
• Log-scaling: δ̄t,f = log(1 + δt,f ) compresses dynamic range
E
Implementation Details
E.1
Model Architecture
Parameter
Value
Input Configuration Input features (XMD: 3 × F ) Sequence length T
30 500
Mamba-2 Block Hidden dimension (d_model) SSM state dimension (d_state) Convolution kernel (d_conv) Expansion factor (expand)
128 16 4 2
Architecture Layers per direction Bidirectional Total Mamba layers Attention pooling Dropout
4 Yes 8 Additive (learned query) 0.1
Output Pooled dimension Classification head
2 × 128 = 256 LayerNorm → Linear(1)
Table 4: Model architecture hyperparameters.
E.2
Training Configuration Parameter
Value
Optimizer Learning rate Weight decay Batch size Max epochs Gradient clipping Dropout
AdamW 1 × 10−4 0.0 128 100 0.5 0.0
Early stopping patience Early stopping metric Validation split
15 epochs Validation ACC 5% of training data
Loss function Positive class weight Mixed precision (AMP)
BCEWithLogitsLoss w+ = n− /n+ (dynamic) Enabled
Figure 5: NVIDIA Jetson Orin edge devices used for deployment benchmarks. From left to right: AGX Orin (high-performance), Orin NX (mid-range), and Orin Nano (entry-level).
Table 5: Training hyperparameters.
E.3
AGX Orin
Orin NX
Orin Nano
GPU Cores Memory AI Performance Max TDP
2048 CUDA 64 GB 275 TOPS 60W
1024 CUDA 16 GB 100 TOPS 25W
1024 CUDA 8 GB 40 TOPS 15W
Class Imbalance Handling
To address class imbalance, we apply three optimization techniques: Dynamic Positive Class Weighting. The positive class weight is computed per fold from training label counts: n+ w+ = (33) n− Threshold Optimization. We search for the optimal classification threshold by maximizing accuracy on the validation set, rather than using the default 0.5 threshold. Post-hoc Calibration. For folds where the model predicts inversely (majority class predictions flipped), we apply calibration to correct the decision boundary.
F
Spec
Table 6: NVIDIA Jetson edge device specifications.
G.2
Accuracy (ACC). TP + TN TP + TN + FP + FN Area Under ROC Curve (AUC). Z 1 AUC = TPR(t) dFPR(t)
G.3 (34)
(35)
0
F1 Score. F1 =
(36)
1 (F1low + F1high ) 2
(37)
F1macro =
G
Edge Deployment Details
G.1
Target Devices
Value
Batch size Sequence length Input dimensions
1 (real-time constraint) 500 timesteps 30 (XMD encoding)
Warmup iterations Benchmark iterations Power monitoring interval
20 100 50 ms
Memory Optimization
For memory-constrained devices (Orin Nano, 8 GB), we implement: • Inference Mode: torch.inference_mode() disables gradient tracking • Cache Management: Explicit GPU cache clearing between iterations
2 · Precision · Recall Precision + Recall
Macro F1.
Parameter
Table 7: Edge benchmark configuration.
Evaluation Metrics ACC =
Benchmark Protocol
We evaluate on three NVIDIA Jetson Orin platforms spanning high-performance to entry-level tiers, as shown in Figure 5. All devices utilize a Unified Memory Architecture (UMA) where CPU and GPU share a single LPDDR5 memory pool, making efficient memory management critical for deployment.
• Memory Pinning: Page-locked memory for CPU-GPU transfers
G.4
Real-time Feasibility
All models achieve real-time performance across all Jetson platforms. MambaGaze operates at 55–68 FPS, well above the 0.1 Hz requirement for 10-second cognitive load prediction windows. Power consumption remains within 3.7–7.5W, enabling deployment in power-constrained systems.