PARAMETER - EFFICIENT P ROMPT T UNING OF V ISION F OUNDATION M ODEL W ITH A DAPTIVE F OCAL L OSS FOR I NTERPRETABLE MCI S CREENING
1
Javad Khoramdel∗1 , Farhad Hoseyni1 , Amirhossein Nikoofard1 APAC Research Group, Faculty of Electrical Engineering, K. N. Toosi University of Technology, Tehran, Iran
arXiv:2607.15047v1 [cs.CV] 16 Jul 2026
A BSTRACT Mild Cognitive Impairment is a critical early stage of cognitive decline that frequently precedes Alzheimer’s disease, yet its automated detection from neuropsychological drawing tests remains fundamentally constrained by data scarcity, class imbalance, and diagnostic ambiguity near clinical boundaries. Existing methodologies attempt to bypass these constraints using computationally expensive, fully fine-tuned hybrid architectures that relegate spatial explainability to a post-hoc approximation rather than an intrinsic model property. We propose a parameter-efficient framework utilizing frozen DINOv2-Small model adapted via three modality-specific learnable prompt tokens while Operating with 1.19 million trainable parameters, each token serves as a query in a shared cross-attention layer over the source image patch tokens. Crucially, spatial explainability is achieved directly through these attention maps; as a structural consequence of the architecture. Then taskconditioned embeddings fused via an attention module to quantify modality-level importance per subject. To handle boundary ambiguity, a MoCA-adapted focal loss introduced that integrates continuous cognitive scores into the training target, loss modulation, and adaptive sample weighting, strictly generalizing standard soft-label approaches. Under stratified five-fold cross-validation, the proposed architecture yields an MCI-class F1 of 0.641 ± 0.026 and an AUC of 0.795 ± 0.024, outperforming the computationally heavier ResViT baseline by 0.110 in MCI-class F1. Keywords Vision Foundation Models, Parameter-Efficient Fine-Tuning, Interpretable Deep Learning, Mild Cognitive Impairment, Neuropsychological Assessment, Medical Image Analysis
1
Introduction
Mild Cognitive Impairment (MCI) occupies a clinically critical window between normal ageing and dementia: subjects scoring below 25 on the Montreal Cognitive Assessment (MoCA) [1] are classified as MCI, yet scores cluster densely around this cutoff, making the boundary inherently uncertain and the diagnostic task sensitive to both labelling strategy and evaluation design. Ruengchaijatuporn et al. [2] address this by jointly processing clock drawing test (CDT), cubecopying, and trail-making images through three fully fine-tuned VGG16 [3] backbones and introducing MoCA-derived soft labels to relax the hard threshold, demonstrating that multi-modal inputs and score-informed labelling substantially improve detection. Interpretability is recovered post-hoc via attention rollout [4], and evaluation relies on random splits that do not guarantee balanced class representation on a 2.4:1 skewed dataset. Sirshar et al. [5] subsequently propose ResViT, a parallel hybrid of a ResNet50 [6] and ViT-B/16 [7], arguing that combining local convolutional and global transformer features is necessary for this task. Yet their 32-million-parameter model is evaluated on a single held-out partition without cross-validation, and performance is reported only in aggregate, obscuring MCI-class sensitivity under class imbalance. We argue that such architectural complexity is unnecessary. An architecture including a self-supervised vision foundation model already encodes rich local and global representations, and the domain gap can be closed far more efficiently through lightweight, task-specific adaptation rather than by stacking two large pretrained networks. Furthermore, MoCA scores carry richer supervisory information than a single soft-label transformation can exploit, ∗
[email protected], [email protected], [email protected]
Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening Frozen DINOv2 ViT-S/14_reg Input tokens (sequence)
🔒
PatchTokenCrossAttention (shared parameters) applied to each image independently
[CLS] token (1, D) Transformer Encoder
Register tokens (R, D) Patch tokens (16, D) Learnable token (clock) (1, D)
Frozen DINOv2 ViT-S/14_reg Input tokens (sequence)
Layer (one block) × layers
[CLS] token (1, D)
Patch tokens (16, D)
Layer (one block) × layers
Input tokens (sequence) [CLS] token (1, D)
x_norm_clock (B, 1, D)
LayerNorm
K, V
LayerNorm
Multi-Head Attention (Q from query, K,V from patches)
Patch tokens (16, D)
Layer (one block) × layers
Squeeze
v_clock (B, D)
x_norm_regtokens x_norm_patchtokens
Query token (cube)
LayerNorm
K, V
LayerNorm
Multi-Head Attention (Q from query, K,V from patches)
x_norm_cube (B, 1, D)
Frozen (not trainable)
Cube trainables
v_trail ↓
Trainable (parameters)
Clock trainables
Intermediate
Trail trainables
T (tokens) (B, 3, D)
🔥
Learnable pool query (B, 1, D) Squeeze LayerNorm
v_cube(B, D)
🔥
LayerNorm
Multi-Head Attention (Q from pool query, K,V from tokens)
Pooled patient vector (B, D)
x_norm_clstoken (B, D) x_norm_regtokens
Legend
v_cube
Attention pooling over 3 tokens
🔥
x_norm_clstoken (B, D)
PatchTokenCrossAttention (shared parameters) applied to each image independently
Transformer Encoder
Learnable token (trail) (1, D)
x_norm_patchtokens
🔒
Register tokens (R, D)
Stack per-image vectors v_clock
Query token (clock)
PatchTokenCrossAttention (shared parameters) applied to each image independently
Transformer Encoder
Learnable token (cube) (1, D)
Frozen DINOv2 ViT-S/14_reg
x_norm_regtokens
🔒
Register tokens (R, D)
🔥
x_norm_clstoken (B, D)
Classifier head (trainable) Query token (trail)
LayerNorm
K, V
LayerNorm
x_norm_patchtokens x_norm_trail (B, 1, D)
Multi-Head Attention (Q from query, K,V from patches)
Squeeze Linear (D, 2) v_trail (B, D)
Binary prediction Class 0
Class 1
Figure 1: Overview of the proposed framework. Each drawing (clock, cube, trail) is processed by a shared frozen DINOv2-Small backbone with a modality-specific learnable token, which is then refined via a shared cross-attention over the backbone’s patch tokens. The 3 resulting embeddings are aggregated by a learnable query attention pooling module into a single vector, which is passed to a linear classifier. Frozen components are shown in blue; trainable components in orange. and robust evaluation on a skewed dataset requires stratified cross-validation with MCI-class sensitivity as the primary optimisation target. 1. Parameter-efficient, interpretable MCI classification from neuropsychological drawing triplets. Modalityspecific prompt tokens adapt a frozen DINOv2-Small [8] backbone with under 6% of parameters trainable, yielding spatial attention maps and modality importance weights as direct byproducts of inference. 2. MoCA-aware focal loss. A unified loss incorporates the continuous MoCA score into a soft target, a MoCA-probability modulator, and MoCA-bin adaptive weighting, strictly generalising [2]. 3. Targeted augmentation strategies. Image inversion, class-balanced sampling, type-preserving Mixup, and MoCA-neighbour drawing swap address the photometric domain gap and increase triplet diversity.
2
Related Works
Recent approaches to automated cognitive screening primarily apply deep convolutional networks (CNNs) and Vision Transformers (ViTs) to the CDT. Chen et al. [9] combined manifold learning with DenseNet-121 for binary dementia screening, demonstrating high classification accuracy. However, their evaluation relies on a skewed dataset with a high prevalence of pathological cases, complicating cross-study generalization. Similarly, Raksasat et al. [10] introduced API-Net, a contrastive learning architecture, while Liu et al. [11] proposed a self-supervised contrastive framework utilizing MoCo. Both studies struggle with class imbalance. Raksasat et al. collapsed moderate and severe impairment stages into a single monolithic class, destroying fine-grained staging capability. Liu et al.’s framework suffers from catastrophic misclassifications in minority categories, resulting in severely degraded macro-averaged performance of 0.554. Hu et al. [12] extended this paradigm by applying an ordinal loss function to a ViT backbone for severity classification on a large cohort. While demonstrating strong binary discrimination, their reliance on the computationally expensive full fine-tuning of massive pre-trained networks lacks parameter efficiency. Generative architectures have also been leveraged to extract disentangled features for CDT assessment. Bandyopadhyay et al. [13] utilized a Relevance Factor Variational Autoencoder (RF-VAE) to differentiate dementia from cognitively normal patients with robust predictive performance. Addressing demographic biases inherent in such models, Zhang et al. [14] subsequently applied static sample reweighting to mitigate elevated Type I error rates observed in patients with limited formal education. Both approaches, however, collapse the continuous spectrum of cognitive decline into rigid binary classifications. Transitioning beyond unimodal assessment, Yang et al. [15] proposed a multimodal graph neural network (GNN) combining cube-copying images, demographics, and cognitive scores to achieve high diagnostic sensitivity. However, their reliance on explicit line-simplification algorithms to convert images into geometric graphs is problematic for 2
Parameter-efficient Prompt Tuning of Vision Foundation Model With Adaptive Focal Loss for Interpretable MCI Screening