How Reliable Is Individualized Functional-Network Attribution in Resting-State fMRI Cognitive Prediction in ADNI?

How Reliable Is Individualized Functional-Network Attribution in Resting-State fMRI Cognitive Prediction in ADNI? A Scan–Rescan Evaluation

arXiv:2610.37472v1 [cs.CV] 27 Oct 2026

Sheethal Bhat1 , Mahfuzur Rahman Chowdhury1 , Imam Ahmed Shaik2 , Rose Dawn Bharat1 , Siming Bayer1 , Andreas Maier1 1 Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU),

Martensstr. 3, 91058 Erlangen, Germany 2 Siemens Healthineers, Henkestr. 127, 91052 Erlangen, Germany

sheethal.bhat@fau.de

Abstract. Individualized attribution estimates which functional networks drive a model prediction for a given subject and is often proposed as an interpretable resting-state fMRI (rs-fMRI) biomarker, but its test–retest reliability is rarely quantified. We evaluate per-subject network-ablation attribution for a frozen resting-state functional connectivity (FC) predictor (Schaefer-200 FC predicting MMSE/ADAS-Cog13) trained on 𝑁=875 subjects from the Alzheimer’s Disease Neuroimaging Initiative (ADNI), using (i) within-session split-half, (ii) scan–rescan repeat visits (74 subjects, 205 baseline–repeat pairs; median inter-visit interval 24.4 months), and (iii) a subject-level permutation null. Split-half reliability is near-ceiling (Spearman 𝜌=0.948/0.950) but drops on scan–rescan (𝜌=0.609/0.616). Under the mismatched-subject null, correlations remain substantial (𝜌≈0.46–0.47), indicating strong population-shared structure, yet matched pairs exceed this null (𝑧≈6.6–6.9, 𝑝 ≤ 10−4 ). Thus, in ADNI, individualized network attribution contains a modest but non-zero person-specific component atop a strong population-shared pattern. We recommend reporting scan–rescan reliability together with a permutation-null control, rather than within-session proxies alone, to avoid overstating individual specificity.

1

Introduction

Models that map resting-state functional connectivity (FC) to cognition and clinical phenotypes are increasingly used in Alzheimer’s disease (AD) neuroimaging [1, 2]. Beyond group-level accuracy, a common objective is individualized attribution: for a given subject, estimating which functional networks drive that subject’s prediction. Such per-subject attribution is only meaningful if it is reproducible. Reliability is a central constraint. Large-scale analyses suggest that stable brain– behavior effects often require much larger samples than historically used [3], and

2

Bhat et al.

FC test–retest reliability is frequently modest (pooled mean edge-level Intraclass Correlation Coefficient, ICC ≈ 0.29) [4]. The ICC describes how reproducibly a single connectivity edge is measured. Prior work on subject-specific parcellations reports reproducibility across short-interval repeat sessions [5, 6], but does not quantify test–retest reliability of model-derived attribution signatures over the multiyear intervals typical of aging cohorts. An additional aspect is attribution stability: feature-importance maps can vary substantially under resampling, noise, and other perturbations even when predictive accuracy is similar [7, 8]. In FC prediction, however, attribution work has largely emphasized group-averaged importance. For connectomes rather than attributions, differential identifiability formalizes the person-specific, test–retest-identifiable component [9], and permutation-null models are standard for separating population-shared from person-specific structure [10, 11]. Neither has been applied to per-subject attribution signatures. To the best of our knowledge, no prior work combines persubject attribution of an FC-based cognitive predictor with a quantitative test–retest evaluation on independent repeat scans of the same individuals. We address this gap using a frozen FC-based multilayer perceptron (FC-MLP) trained to jointly predict MMSE and ADAS-Cog13 from Schaefer-200 FC on 𝑁=875 ADNI subjects (Sect. 2). Instead of edge-level ICC, we assess test–retest consistency of each subject’s network-attribution ranking. We compute per-subject networkablation signatures and evaluate reproducibility with three checks: within-session split-half, scan–rescan across independent repeat visits, and a cluster-aware subjectlevel permutation null. The main contributions in this study are: 1. Quantify how split-half overestimates scan–rescan reliability of per-subject network attribution (74 subjects; 205 session pairs). 2. Use a subject-level permutation null to estimate the mismatched-subject baseline and the residual person-specific component. 3. Provide reporting recommendations to avoid overstating individual specificity.

2

Methods

To attribute a prediction to functional networks for a single subject, let 𝑓 denote the frozen FC-MLP mapping a subject’s connectivity vector 𝑥 to a two-dimensional prediction ŷ = 𝑓 (𝑥) with targets 𝑡 ∈ {MMSE, ADAS-Cog13}. For a given target 𝑡, we write the scalar prediction as 𝑦ˆ (𝑡 ) = [ 𝑓 (𝑥)] 𝑡 . For each of the seven Yeo networks [12] 𝑛 ∈ {Vis, SomMot, DorsAttn, SalVentAttn, Limbic, Cont, Default} (see Fig. 1), we ablate the input by zeroing all FC edges incident to nodes in 𝑛. Because the model input concatenates an FC vector and an edge-coverage mask, 𝑥 = [FC; mask], we apply the ablation to both operands: the ablated edges are set to zero in the FC vector and their corresponding mask entries are also set to zero. This yields 𝑥\𝑛 and 𝑦ˆ 𝑛(𝑡 ) = [ 𝑓 (𝑥\𝑛 )] 𝑡 . The per-target network importance is then 𝐼𝑛(𝑡 ) = 𝑦ˆ (𝑡 ) − 𝑦ˆ 𝑛(𝑡 ) , and the subject-specific importance signature for target 𝑡 is I (𝑡 ) = (𝐼1(𝑡 ) , . . . , 𝐼7(𝑡 ) ).

Reliability of Individualized rs-fMRI Attribution

3

Fig. 1. Top 6 of the 7 Yeo networks by group-level importance, visualized on anatomical MRI. Group-level importance is quantified by the drop in test-set explained variance when ablating 2 , computed on the held-out test set and averaged over three each network, Δ𝑅 2 = 𝑅 2 − 𝑅\𝑛 independently trained runs (shown separately for MMSE and ADAS-Cog13). Δ𝑅 2 can exceed 2 < 0. The displayed “combined MMSE+ADAS-Cog13 rank” orders the total 𝑅 2 when 𝑅\𝑛 networks by the summed Δ𝑅 2 across the two targets. The 7th (lowest-ranked, Somatomotor) network is omitted for layout only; all 7 are used in every analysis.

This implementation reproduces the group-level ablation results (Fig. 1) for all 7 × 2 network–target combinations to six decimal places. Attribution reliability between two occasions 𝑎, 𝑏 for the same subject is the Spearman rank correlation 𝜌(I𝑎 , I𝑏 ), i.e., the agreement in the within-subject network ranking. Fig. 2 summarizes three reliability checks. (i) Split-half: for each test subject (𝑛=132), we split the denoised rs-fMRI time series after scrubbing into odd/even surviving volumes, recompute FC and attribution per half, and compute 𝜌(Ihalf-A , Ihalf-B ) (minimum 30 volumes per half; 0 exclusions). We additionally test whether split-half 𝜌 increases with per-half scan duration. (ii) Scan–rescan: we attribute independent repeat-visit rs-fMRI and compute 𝜌(Ibaseline , Irepeat ) for baseline × all follow-ups, yielding 205 pairs from 74 subjects with uneven repeats (𝑘 𝑖 ∈ [1, 12]; 30 subjects contribute 1 pair, 18 contribute 2, 13 contribute 3; tail to 12). (iii) Permutation null: for 𝐵=10,000 permutations, we draw a derangement 𝜋 of the 74 subjects, pair each subject 𝑖’s baseline signature with the repeat signatures of subject 𝜋(𝑖) (repeats kept

4

Bhat et al. (b) Scan–rescan

(a) Split-half rs-fMRI session (after scrubbing)

subject i

Repeat 2

Baseline odd volumes

…

even volumes

Half A

Half B

IA

IB

ρ(IA , IB )

(c) Permutation null Repeat 1

subject j

Baseline

kj = 1

Ibaseline

ki pairs

Baseline

Repeat ki

Baseline

Repeat 1

Baseline

Repeat Repeat Repeat Repeat

Irepeat

Repeat Repeat

no self-match; repeats kept as block Tobs

z, p ρ(Ibaseline , Irepeat ) per pair 205 pairs, 74 subjects; interval median 24.4 mo (IQR 12.2–48.4) null: 10,000 subject-level derangements

Fig. 2. Overview of the three progressively stricter reliability checks. (a) Within-session splithalf: odd and even volumes of one session are each attributed and compared by Spearman 𝜌. (b) Scan–rescan: each subject’s baseline is compared with all repeat visits (median 24.4 months apart). (c) Permutation null: 10,000 subject-level derangements match each baseline with another subject’s repeat visits (kept as a block, no self-match); the observed mean 𝑇obs is compared with this null.

as a block), and recompute the subject-level mean reliability. With 𝑇obs the observed subject-level mean and 𝑏 the number of permutations with statistic ≥ 𝑇obs , we report 𝑝 = (𝑏 + 1)/(𝐵 + 1) and 𝑧 = (𝑇obs − 𝜇null )/𝜎null [11]. Artifact checks. Edge-count: let 𝑐 ∈ R7 be the fixed vector of ablated edge counts (unique edges zeroed when ablating each network). For each subject we compute the across-network Spearman correlation 𝜌(I (𝑡 ) , 𝑐) (7 items) and summarize it across subjects by its mean, separately per target; with only 7 items, Spearman 𝜌 is highly discrete, so this check is treated as descriptive. For scan–rescan, we first average each subject’s per-pair 𝜌(I (𝑡 ) , 𝑐) across that subject’s repeat visits and then compute the across-subject mean. Scan duration: we correlate (Spearman, across the 132 test subjects) each subject’s per-half scan duration (seconds; half A, equivalent to half B up to one volume) with split-half 𝜌. Aggregation and inference. All Spearman correlations are computed separately within each of the three trained runs and averaged across runs. Spearman 𝜌 uses average ranks for ties; no ties occurred in any importance signature (verified for all 1,674 scan–rescan signatures, i.e., 279 sessions × 2 targets × 3 runs, and all split-half signatures). As subjects contribute unequal numbers of repeat pairs, scan– rescan is summarized at the subject level by first averaging each subject’s per-pair 𝜌 values and then summarizing across subjects; we also report the pair-pooled mean for comparability. Uncertainty on headline scan–rescan and split-half means is a 95 % percentile bootstrap CI over 10,000 resamples of subjects (resampling whole subjects). We test the split-half versus scan–rescan gap with a Wilcoxon signed-rank test across the 74 subjects present in both analyses.

Reliability of Individualized rs-fMRI Attribution

5

Tab. 1. Attribution reliability (Spearman 𝜌 of the per-subject 7-network importance signature). The subject is the unit of analysis (split-half: 𝑛=132; scan–rescan: 𝑛=74, with pairs averaged within subject). CIs are subject-level bootstrap intervals; “run SD” is the SD across 3 runs; “–” indicates not computed. Permutation-null rows use 10,000 subject-level derangements.

Statistic Mean 95 % CI Median Std 10–90% Run SD (𝑛=3) Null mean ± SD Null range 𝑧-score 𝑝-value Pair-pooled mean

Split-half MMSE ADAS-Cog13 0.948 0.950 0.929–0.964 0.934–0.964 0.988 0.988 0.101 0.085 0.87–1.00 0.85–1.00 0.005 0.007 – – – – –

3

Experimental details

3.1

Data description

Scan–rescan MMSE ADAS-Cog13 0.609 0.616 0.553–0.663 0.571–0.660 0.625 0.661 0.242 0.198 0.34–0.90 0.33–0.85 0.033 0.020 0.463 ± 0.021 0.472 ± 0.022 0.379–0.549 0.393–0.562 6.89 6.63 ≤ 10−4 ≤ 10−4 0.593 0.588

ADNI is a multi-site, longitudinal study of aging and Alzheimer’s disease with multimodal neuroimaging [13]. We analyzed data from 𝑁=875 unique ADNI participants who had both T1-weighted MRI and rs-fMRI scans. Within the 𝑁=875 cohort, diagnoses are CN (𝑛=464), MCI (𝑛=315), Dementia (𝑛=94), and unknown (𝑛=2). Age is 72.6 ± 7.9 years (range 50–97); 489 subjects are female and 386 male. Subjects were enrolled in ADNI1 (𝑛=37), ADNI-GO (𝑛=38), ADNI2 (𝑛=256), ADNI3 (𝑛=474), and ADNI4 (𝑛=68); 2 unknown. rs-fMRI scans were acquired at 3.0 T across 53 sites with heterogeneous protocols (TR 0.607–6.02 s; 96–972 volumes after removal of initial volumes). For the upstream prediction model, the cohort is split once at the subject level into fixed train/validation/test partitions of 611/132/132 stratified by diagnosis. We evaluate attribution and reliability only on the held-out 132-subject test split; scan–rescan further restricts to test subjects with repeat-visit rs-fMRI. 3.2

Implementation details

We analyze a trained FC-MLP and keep its weights fixed for all attribution and reliability analyses. Input features. rs-fMRI preprocessing includes motion correction, HD-BET brain extraction [14], rigid rs-fMRI→T1 registration, detrending, and nuisance regression of mean white-matter (WM) and cerebrospinal-fluid (CSF) signals plus 5 anatomical CompCor (aCompCor) components [15] from a combined WM/CSF mask, without global signal regression. Time series are band-pass filtered (0.01–0.1 Hz, using each

6

Bhat et al.

subject’s TR), 𝑧-scored, and scrubbed at framewise displacement (FD) > 0.5 mm after filtering; FD was computed from translation magnitude only rather than the full six-parameter formulation [16], as rotation parameters were not retained upstream. No spatial smoothing or slice-timing correction is applied. We compute Schaefer200 [17] FC as a Fisher-𝑧 transformed 200 × 200 Pearson correlation matrix and vectorize the upper triangle (19,900 edges). An edge-coverage mask is concatenated, yielding a 39,800-dimensional input. Model and training (frozen). The FC-MLP encoder maps 39,800 → 256 → 128 with LayerNorm/GELU/dropout (0.3), and the decoder maps 128 → 64 → 2 with GELU/dropout (0.3). The model is trained with Huber loss (𝛿=1.0) using Adam (lr = 10−4 , weight decay 10−5 ; no schedule/warmup), batch size 16, and up to 100 epochs. Early stopping uses patience 15 on validation loss; we keep the single best checkpoint (lowest validation loss). During training only, an augmentation zeros 5 randomly chosen parcels per sample. Targets are 𝑧-scored on train only and inverse-transformed for reporting. Three independent runs with different random initializations share the same fixed split. ComBat harmonization [18] uses the TR group as batch variable (a proxy for acquisition protocol) and protects diagnosis, whose composition differs across TR groups (𝜒2 =35.66, 𝑝 < 0.001); no other covariates are included. It is fit on the training split only, never refit, and applied to validation/test subjects and split-half halves, but not to repeat-visit sessions (both sides of scan–rescan use pre-ComBat features). 3.3

Results

The frozen predictor is weak: on the 𝑛=113 test subjects with both targets, 𝑅 2 =0.053± 0.034 for MMSE and 0.087 ± 0.037 for ADAS-Cog13 (mean ± SD over runs). As a diagnostic check, the predicted ADAS-Cog13 score separates CN from Dementia with AUC 0.780 (95 % bootstrap CI 0.611–0.911; 𝑛 = 72 of the 113 subjects with both targets: 58 CN, 14 Dementia). Within-session split-half reliability is near-ceiling for both targets (mean 𝜌=0.948/0.950, median ≈ 0.99; Tab. 1). On independent repeat visits (205 session pairs from 74 subjects), subject-level scan–rescan reliability drops to 𝜌=0.609 (MMSE) and 0.616 (ADAS-Cog13), with large between-subject variation (10th–90th percentiles 0.34–0.90 and 0.33–0.85). Pair-pooled means are slightly lower (0.593/0.588) because subjects with more follow-ups receive more weight. Restricted to the same 74 subjects, split-half reliability is 0.933/0.939, a paired difference of 0.324/0.323 (Wilcoxon 𝑝 < 10−12 ), indicating that split-half substantially overestimates test–retest reliability. Under the subject-level permutation null, mismatched subjects still correlate strongly (𝜌=0.463/0.472), but the matched statistic exceeds the null (𝑧=6.89/6.63; 𝑝 ≤ 10−4 ), corresponding to a modest person-specific excess of ≈ 0.15 (MMSE) and ≈ 0.14 (ADAS-Cog13). The artifact checks do not indicate a network-size effect: mean edge-count correlations 𝜌(I (𝑡 ) , 𝑐) are weakly negative for split-half (MMSE −0.120; ADAS-Cog13 −0.163) and repeat visits (MMSE −0.112; ADAS-Cog13

Reliability of Individualized rs-fMRI Attribution

7

−0.158). Split-half reliability increases with per-half scan duration (𝜌=0.199/0.277), consistent with sampling-noise limits, although duration is confounded with acquisition protocol in ADNI.

4

Discussion

Per-subject attribution reliability depends strongly on protocol: split-half is nearceiling (𝜌≈0.95), but drops on scan–rescan across independent repeat visits (𝜌≈0.61– 0.62), consistent with additional between-visit variability beyond within-session sampling noise. The permutation null suggests that much of this structure is population-shared, with only a modest person-specific excess; robustness to other cohorts, preprocessing, parcellations, attribution operators, and models remains untested. The coexistence of shared and person-specific structure mirrors FC itself, where connectomes are identifiable across sessions [9, 10] yet test–retest reliability is modest [4]. The strong shared profile likely has two sources. First, the ablations overlap by construction: every between-network edge (𝑖, 𝑗) with 𝑖 ∈ 𝑛, 𝑗 ∈ 𝑚 is zeroed (𝑡 ) in both 𝑥\𝑛 and 𝑥 \𝑚 , coupling 𝐼𝑛(𝑡 ) and 𝐼𝑚 ; within-network-only ablation would remove this overlap. Second, the weak predictor may respond mainly to group-level FC structure. Because 𝜌 is bounded above, a high shared baseline leaves little room for an individual excess, so the person-specific component may be underestimated. Limitations include the modest repeat-visit sample (74 subjects; 205 session pairs) and the weak, run-variable predictor (Sect. 3). Repeat-visit sessions were not ComBat-harmonized, so part of the split-half to scan–rescan gap may reflect this mismatch. Inter-visit intervals are long and variable (median 24.4 months, IQR 12.2–48.4) and are not modeled. We therefore recommend treating within-session proxies as expected upper bounds, reporting scan–rescan estimates when repeat visits exist, and including a permutation-null (or equivalent) control; these recommendations should be validated in further aging/AD cohorts such as OASIS-3 and AIBL. In summary, per-subject network attributions are far less reproducible across visits than within-session estimates suggest, and most of their apparent stability reflects a pattern shared across the cohort, so individual specificity should be demonstrated against an explicit population baseline before attributions are interpreted as personal biomarkers. Acknowledgement. We acknowledge HPC support from XXXXXX and partial hardware funding from the XXXXX. Data were obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI; adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design of ADNI but did not participate in this report. Code, derived features, and the subject-level split file are available upon reasonable request, subject to ADNI DUA constraints.

8

Bhat et al.

References 1. Dadi K, Rahim M, Abraham A et al. Benchmarking functional connectome-based predictive models for resting-state fMRI. NeuroImage. 2019;192:115–34. 2. Svaldi DO, Joshi AA, Shattuck DW et al. Optimizing differential identifiability improves connectome predictive modeling of cognitive deficits from functional connectivity in Alzheimer’s disease. Human Brain Mapping. 2021;42(11):3500–16. 3. Marek S, Tervo-Clemmens B, Calabro FJ et al. Reproducible brain-wide association studies require thousands of individuals. Nature. 2022;603:654–60. 4. Noble S, Scheinost D, Constable RT. A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis. NeuroImage. 2019;203:116157. 5. Wang D, Buckner RL, Fox MD et al. Parcellating cortical functional networks in individuals. Nature Neuroscience. 2015;18:1853–60. 6. Kong R, Li J, Orban C et al. Spatial topography of individual-specific cortical networks predicts human cognition, personality, and emotion. Cerebral Cortex. 2019;29(6):2533– 51. 7. Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B. Sanity Checks for Saliency Maps. Advances in Neural Information Processing Systems (NeurIPS). 2018:9525–36. 8. Ghorbani A, Abid A, Zou J. Interpretation of Neural Networks Is Fragile. Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. (01). 2019:3681–8. 9. Amico E, Goñi J. The quest for identifiability in human functional connectomes. Scientific Reports. 2018;8(1):8254. 10. Finn ES, Shen X, Scheinost D et al. Functional connectome fingerprinting: identifying individuals using patterns of brain connectivity. Nature Neuroscience. 2015;18:1664–71. 11. Nichols TE, Holmes AP. Nonparametric permutation tests for functional neuroimaging: a primer with examples. Human Brain Mapping. 2002;15(1):1–25. 12. Yeo BTT, Krienen FM, Sepulcre J, Sabuncu MR, Lashkari D, Hollinshead M et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. Journal of Neurophysiology. 2011;106(3):1125–65. 13. Jack CR, Bernstein MA, Fox NC, Thompson P, Alexander G, Harvey D et al. The Alzheimer’s Disease Neuroimaging Initiative (ADNI): MRI methods. Journal of Magnetic Resonance Imaging. 2008;27(4):685–91. 14. Isensee F, Schell M, Pflueger I, Brugnara G, Bonekamp D, Neuberger U et al. Automated brain extraction of multisequence MRI using artificial neural networks. Human Brain Mapping. 2019;40(17):4952–64. 15. Behzadi Y, Restom K, Liau J, Liu TT. A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. NeuroImage. 2007;37(1):90–101. 16. Power JD, Barnes KA, Snyder AZ, Schlaggar BL, Petersen SE. Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion. NeuroImage. 2012;59(3):2142–54. 17. Schaefer A, Kong R, Gordon EM, Laumann TO, Zuo XN, Holmes AJ et al. Local-Global Parcellation of the Human Cerebral Cortex from Intrinsic Functional Connectivity MRI. Cerebral Cortex. 2018;28(9):3095–114. 18. Fortin JP, Parker D, Tunç B, Watanabe T, Elliott MA, Ruparel K et al. Harmonization of multi-site diffusion tensor imaging data. NeuroImage. 2017;161:149–70.

Related documents

• How Reliable Is Individualized Functional-Network Attribution in Resting-State fMRI Cognitive Prediction in ADNI?#1469075
• Neural Representation of Exogenous and Endogenous Temporal Expectations Based on fMRI#127548
• MultiFlow: Vision-Driven Multimodal fMRI Encoding via Stimulus-Informed Source Flow Matching#1326589
• Comparative multivariate decoding adjudicates theories of semantic representation in the anterior temporal lobes and the rest of the cortex#1429119
• Calibration-Free Estimation of Cerebrovascular Reactivity From Resting-State BOLD fMRI Using Reconstructed Respiratory Fluctuations.#1147257