Are Natural-Domain Foundation Models Effective for Accelerated Cardiac MRI Reconstruction? Anam Hashmi1 , Mayug Maniparambil1 , Julia Dietlmeier1 , Kathleen M. Curran2 , Noel E. O’Connor1 1 Dublin City University, Ireland, 2 University College Dublin, Ireland [email protected], [email protected], [email protected]
arXiv:2604.22557v1 [eess.IV] 24 Apr 2026
[email protected], [email protected]
Abstract The emergence of large-scale pretrained foundation models has transformed computer vision, enabling strong performance across diverse downstream tasks. However, their potential for physics-based inverse problems, such as accelerated cardiac MRI reconstruction, remains largely underexplored. In this work, we investigate whether natural-domain foundation models can serve as effective image priors for accelerated cardiac MRI reconstruction, and compare the performance obtained against domain-specific counterparts such as BiomedCLIP. We propose an unrolled reconstruction framework that incorporates pretrained, frozen visual encoders, such as CLIP, DINOv2, and BiomedCLIP, within each cascade to guide the reconstruction process. Through extensive experiments, we show that while task-specific state-of-the-art reconstruction models such as E2E-VarNet achieve superior performance in standard in-distribution settings, foundation-model-based approaches remain competitive. More importantly, in challenging cross-domain scenarios, where models are trained on cardiac MRI and evaluated on anatomically distinct knee and brain datasets–foundation models exhibit improved robustness, particularly under high acceleration factors and limited low-frequency sampling. We further observe that natural-image-pretrained models, such as CLIP, learn highly transferable structural representations, while domain-specific pretraining (BiomedCLIP) provides modest additional gains in more ill-posed regimes. Overall, our results suggest that pretrained foundation models offer a promising source of transferable priors, enabling improved robustness and generalization in accelerated MRI reconstruction. Code: github.com/Hashmi360/CMR-Recon.
1. Introduction Cardiovascular disease continues to be the leading cause of mortality worldwide, underscoring the importance of accu-
rate and non-invasive imaging for early diagnosis and monitoring [38]. Cardiac magnetic resonance (CMR) imaging has emerged as a powerful tool due to its ability to provide high-resolution assessment of cardiac anatomy, function, and tissue properties without the use of ionizing radiation [11, 21]. Despite its clinical benefits, MRI acquisition is inherently slow because k-space measurements must be collected sequentially during the scan. Long acquisition times can lead to patient discomfort and motion artifacts [39]. Accelerated MRI addresses this challenge by undersampling k-space to shorten scan duration; however, reconstructing images from such incomplete measurements is an ill-posed inverse problem (i.e. there are fewer measurements than unknowns) [12] that requires strong prior information. In recent years, deep learning–based reconstruction approaches have emerged as a powerful solution to this challenge [12, 26]. In particular, unrolled networks have achieved state-of-the-art performance [38] by combining physics-based data consistency with learned image priors, effectively integrating the MRI acquisition model with datadriven representations. However, these conventional deep learning models are typically designed for specific tasks and often require substantial retraining or fine-tuning when deployed in new settings [30]. Moreover, their clinical applicability is frequently challenged by domain shifts arising from variations in acquisition protocols, imaging settings, scanner hardware, and anatomical differences across patient populations [8]. These limitations highlight a key challenge: how to develop reconstruction methods that generalize reliably across diverse settings. More recently, the broader deep learning community has increasingly shifted toward large-scale foundation models, which are pretrained on extensive datasets and exhibit strong generalization across tasks and domains [3, 15]. Their success has been particularly transformative in natural language processing [4, 18, 28], and has recently been extended to computer vision and medical imaging [5, 17, 22, 27, 29, 36, 41]. By learning rich, transferable representa-
tions from large-scale data, these models enable strong performance even in low-data or zero-shot regimes, reducing the need for task-specific supervision. This paradigm shift suggests a promising alternative to highly specialized architectures: leveraging pretrained models as general-purpose priors. However, applying foundation models to medical imaging, and in particular to physics-based inverse problems such as MRI reconstruction–remains largely unexplored. A key challenge lies in the domain gap between natural images used for large-scale pretraining and medical imaging data. While medical imaging is often constrained by limited annotated data due to privacy and acquisition costs [15, 23, 24], making it well-suited for transfer learning, it is unclear whether representations learned from natural images can effectively transfer to this fundamentally different domain. These challenges motivate a shift in perspective: rather than designing increasingly task-specific reconstruction architectures, we investigate whether the transferable representations learned by foundation models can serve as effective priors for accelerated MRI reconstruction. Building on this perspective, we investigate whether frozen vision foundation models can serve as effective priors for physics-based MRI reconstruction, and compare natural-image-pretrained models with domain-specific counterparts such as BiomedCLIP [41]. We propose an unrolled reconstruction framework that integrates pretrained visual encoders, such as CLIP [29], DINOv2 [27], and BiomedCLIP [41], within each cascade of the reconstruction pipeline. The foundation models remain frozen during training and provide feature representations that guide iterative refinement, while data-consistency operations enforce fidelity to the acquired k-space measurements. Through extensive experiments on the CMRxRecon 2023 dataset [34], we observe that task-specific reconstruction models, such as E2E-VarNet [32], achieve superior performance in standard in-distribution settings, as expected due to their end-to-end optimization on domainspecific MRI data. However, a markedly different trend emerges under more challenging conditions. In crossdomain evaluations and at higher acceleration factors, corresponding to increasingly ill-posed reconstruction problems, the performance gap between the two approaches narrows substantially. In these regimes, foundation-modelbased reconstructions become competitive and, in some cases, surpass task-specific baseline. We further observe that natural-image-pretrained models, such as CLIP, learn highly transferable structural representations, while domain-specific pretraining (BiomedCLIP) provides modest additional gains in more ill-posed regimes. These findings reveal a regime-dependent trade-off: while fully supervised reconstruction network excel when
sufficient domain-specific data is available, pretrained foundation models provide more robust and transferable priors under distribution shifts. Overall, this suggests that largescale visual pretraining captures structural representations that generalize beyond natural images and can be effectively leveraged for medical image reconstruction, particularly in challenging, cross-domain settings. In summary, our main contributions are: • To the best of our knowledge, we present one of the first studies investigating frozen vision foundation models as priors for physics-based MRI reconstruction. We introduce an unrolled reconstruction framework that integrates pretrained encoders (CLIP, DINOv2, BiomedCLIP) within each cascade, and compare domain-specific (BiomedCLIP) and natural-image-pretrained models. • We evaluate performance across both in-distribution and challenging cross-domain settings, revealing a regimedependent behavior in which task-specific model dominate under standard conditions, while foundation-modelbased approaches become increasingly competitive as reconstruction difficulty and domain shift increase. • We show that large-scale pretrained visual representations, despite being learned from natural images, capture transferable structural priors that can generalize to MRI reconstruction across anatomically distinct domains.
2. Background and Related Work Inverse Problem Modeling of Accelerated MRI Reconstruction. Magnetic resonance imaging (MRI) acquires measurements of the underlying anatomy in the frequency domain, known as k-space. During acquisition, multiple receiver coils are used, each with a distinct spatial sensitivity profile [12]. Let x∗ ∈ Cn denote the target image. The measurement from the i-th coil is given by k_i = \mathcal {F}(S_i x^\ast ) + z_i, \quad i = 1, \dots , N,
(1)
where Si denotes the coil sensitivity map, F the Fourier transform, and zi measurement noise. Collectively, measurements across all coils are denoted by k = (k1 , . . . , kN ). To accelerate acquisition, only a subset of k-space is sampled using an undersampling mask, yielding \tilde {k}_i = M k_i, \quad i = 1, \dots , N,
(2)
where M is a binary sampling operator. The forward model can thus be written compactly as k̃ = A(x∗ ), where A(·) represents the undersampled multi-coil acquisition process. Recovering x∗ from k̃ is an ill-posed inverse problem due to insufficient measurements. Classical approaches address this by incorporating prior knowledge, such as sparsity in a transform domain, leading to compressed sensing formulations [6, 10, 20]: \hat {x} = \arg \min _x \; \|\mathcal {A}(x) - \tilde {k}\|^2 + \mathcal {R}(x),
(3)
where R(·) encodes prior information. In modern approaches, this prior is learned directly from data using deep neural networks, forming the basis of learning-based MRI reconstruction methods. Deep Learning for Accelerated MRI Reconstruction. Deep learning has become the dominant paradigm for accelerated MRI reconstruction, with models learning datadriven image priors from large-scale datasets. Among these, unrolled networks achieve state-of-the-art performance by modeling reconstruction as a sequence of cascades, each corresponding to an iteration of an optimization algorithm [12, 32, 38]. This formulation enables tight integration of MRI acquisition physics, particularly k-space undersampling, with learned image-domain priors, yielding iterative refinement schemes that progressively improve reconstruction quality. A prominent and widely adopted example is the End-to-End Variational Network (E2E-VarNet) [32], which has established itself as a strong state-of-the-art baseline on benchmarks such as fastMRI [12, 40]. Owing to its effectiveness and generality, E2E-VarNet has become a foundational architecture in MRI reconstruction, with many subsequent methods building upon its design. Foundation Models. Foundation models have significantly advanced transfer learning, enabling pretrained models to generalize across diverse tasks [15]. In computer vision, CLIP [29] has gained widespread attention for learning aligned image–text representations via contrastive learning, supporting strong performance across a range of downstream tasks [42]. Recent work has explored its use in medical imaging, showing that pretrained visual features can rival or even surpass domain-specific models in certain clinical settings [1, 25, 36]. To bridge the domain gap, BiomedCLIP [41] extends CLIP to the biomedical domain using curated image–text pairs, achieving state-of-the-art results on multiple medical benchmarks. Similarly, selfsupervised vision transformers such as DINOv2 [27] learn robust and transferable representations without manual annotations, achieving strong performance across diverse vision tasks [3, 31]. Despite these advances, existing work primarily focuses on high-level vision tasks. The role of foundation models in physics-based inverse problems, such as MRI reconstruction, remains largely unexplored.
3. Methodology The primary objective of this work is to evaluate the effectiveness of vision foundation models in the context of MRI reconstruction. In particular, we aim to assess whether features learned from large-scale pretraining, primarily on natural images–can serve as transferable priors for reconstructing undersampled MRI data. To this end, we pro-
pose an unrolled reconstruction framework that integrates pretrained vision foundation models within a physics-based MRI reconstruction pipeline. Given undersampled multicoil k-space measurements, the objective is to recover an image that is both consistent with the acquired data and guided by informative structural priors. The reconstruction is modeled as a sequence of cascades, each comprising a data-consistency step and a learned refinement module. Unlike conventional approaches that learn priors solely from task-specific MRI data, our method incorporates frozen visual encoders–such as CLIP [29], DINOv2 [27], and BiomedCLIP [41], within each cascade. These encoders extract feature representations from intermediate reconstructions, which are used to guide iterative refinement. This design enables the model to leverage large-scale pretrained knowledge while maintaining fidelity to the underlying acquisition physics. Foundation Model Encoders. All encoders are transformer-based models pretrained on large-scale datasets using self-supervised or multimodal objectives. We select representative models spanning both naturalimage and medical-domain pretraining to study the impact of pretraining data on reconstruction performance. For all models, we use the ViT-B backbone for consistency across experiments. • CLIP (Contrastive Language-Image Pre-training) [29] is a vision–language model trained via contrastive learning on 400 million image–text pairs [29, 37], to learn aligned multimodal embeddings. This objective enables CLIP to capture rich semantic and structural features that transfer effectively across diverse visual tasks [15]. • DINOv2 (DIstillation with NO labels) [27] is a stateof-the-art self-supervised Vision Transformer trained on a curated dataset of 142M natural images using a selfdistillation objective. It learns robust and transferable visual representations without manual annotations. • BiomedCLIP [41] is a domain-specific extension of CLIP trained on PMC-15M, a large-scale biomedical dataset of 15M image–text pairs from PubMed Central. It captures fine-grained alignments between medical imagery and clinical context, enabling direct comparison between natural-domain and medical-domain pretrained representations.
3.1. Network Architecture Our architecture follows an unrolled reconstruction paradigm, where a sequence of cascades iteratively refines the image. Each cascade consists of a data-consistency step in k-space and an image-domain refinement module. The data-consistency operation enforces agreement with acquired measurements, while the refinement module incorporates learned priors to improve reconstruction quality.
In contrast to conventional approaches that rely on taskspecific CNN priors, we introduce a foundation-modelguided denoiser, which leverages pretrained visual representations. This module integrates a frozen vision transformer encoder with a lightweight decoder, enabling the use of transferable features within each cascade. We first describe the architecture of the proposed foundation-modelbased denoiser, followed by its integration within the unrolled reconstruction framework.
by projecting the original complex-valued reconstruction through a shallow convolutional layer and injecting it at the final decoding stage. The final output is produced via a lightweight convolutional head, yielding a two-channel reconstruction corresponding to the real and imaginary components. Overall, the proposed denoiser combines frozen foundation model features, multi-layer transformer fusion, and a lightweight hierarchical decoder to provide transferable yet task-adapted priors for MRI reconstruction.
3.2. Iterative Unrolled Reconstruction Foundation Model Denoiser. Given an intermediate reconstruction, the complex-valued MRI image is converted to a magnitude image and normalized using percentilebased scaling [7] to reduce outliers. The image is then replicated across three channels, resized to 224 × 224, and standardized using ImageNet [9] statistics to match the input distribution of pretrained encoders. The normalized image is passed through a frozen Vision Transformer encoder (CLIP, DINOv2, or BiomedCLIP), whose parameters remain fixed throughout training to ensure that reconstruction is guided by pretrained representations rather than domain-specific adaptation. We refer to the resulting models as CMR-CLIP, CMR-DINOv2, and CMR-BiomedCLIP, respectively. Instead of relying solely on the final transformer layer, we extract intermediate features from the first six layers, which capture low-level and structural information beneficial for reconstruction [15]. These features are fused via a learnable mechanism: each layer is first aligned using LayerNorm [2], then combined using softmax-normalized weights constrained to sum to one [16], enabling adaptive integration of multi-level representations. The fused patch tokens are reshaped into spatial feature maps and processed by a UNETR-style [14] hierarchical decoder with multiscale skip connections from intermediate encoder layers (layers 5, 4, and 3). Each skip feature is reshaped, resized via bilinear interpolation, and projected using 1 × 1 convolutions before fusion. Each decoding stage consists of bilinear upsampling followed by convolutional refinement. In contrast to standard UNETR [14] designs that employ full convolutions, we use depthwise separable convolutions (a 3 × 3 depthwise convolution followed by a 1 × 1 pointwise convolution) to improve parameter efficiency [13], along with instance normalization [33] and ReLU activation. At each stage, the upsampled features are concatenated with the corresponding multi-scale skip features from the encoder, as illustrated in Fig. 1(a), and further refined using a 3 × 3 convolution, instance normalization, and non-linearity. This process is repeated across multiple stages, progressively recovering spatial resolution and integrating both high-level and low-level information. To preserve fine-grained details, we additionally incorporate an input-level skip connection
Architectures based on unrolled optimization strategies have shown strong effectiveness in addressing inverse problems, particularly in accelerated MRI reconstruction [12]. These methods model reconstruction as a sequence of cascades, where each stage iteratively refines the estimate by enforcing data consistency and incorporating learned image priors. Building on the E2E-VarNet framework [32], we adopt an unrolled reconstruction approach in the k-space domain, where the solution to the regularized inverse problem \hat {x} = \arg \min _{x} \; \left \| \mathcal {A}(x) - \tilde {k} \right \|_2^2 + \mathcal {R}(x) is approximated by unfolding the optimization into a sequence of T cascades. Each cascade corresponds to an update of the form: \hat {k}^{t+1} = \hat {k}^{t} - \mu ^t M (\hat {k}^{t} - \tilde {k}) + G(\hat {k}^{t}),
(4)
where k̂ t denotes the current estimate in k-space at cascade t, µt is a learnable step size, and G(·) represents a learned regularization term [12]. The second term enforces data consistency (DC) by ensuring that the reconstructed k-space remains aligned with the acquired measurements at sampled locations. The regularization is applied in the image domain through our proposed foundation-model-guided denoiser. Specifically, the mapping G(·) can be expressed as: G(k) = \mathcal {F} \big ( \mathcal {E} \big ( \mathcal {D} \big ( \mathcal {R} (\mathcal {F}^{-1}(k)) \big ) \big ) \big ),
(5)
where D denotes the proposed denoiser, R(x1 , . . . , xN ) = PN ∗ i=1 Si xi is the reduction operator that combines multicoil images using the corresponding sensitivity maps, and E(x) = (S1 x, . . . , SN x) is the expansion operator that maps the combined image back to individual coil images. This formulation enables the integration of learned imagedomain priors within the iterative reconstruction process while maintaining consistency with the underlying acquisition model. We estimate coil sensitivity maps using a standard sensitivity estimation module [32], where maps are derived from the low-frequency (ACS) region of the undersampled k-space data. These maps enable transformations
(a) Proposed denoiser architecture. Multi-level features from the first six layers of the frozen vision encoder are fused using LayerNorm and learnable softmax weights. Selected intermediate layers (z5 , z4 , z3 ) provide spatially aligned skip connections, which are concatenated with decoder features at corresponding resolutions. A lightweight hierarchical decoder progressively upsamples and refines these features to generate the final reconstruction. Adapted from [14]
(b) Overview of the unrolled model architecture. Reconstruction is performed through T cascades, each consisting of a data-consistency (DC) step in k-space and an imagedomain refinement using the proposed denoiser. A sensitivity map estimator (SME) computes coil sensitivity maps, enabling transformations between image and multi-coil domains. Adapted from [12]
Figure 1. Overview of the proposed architecture.
between image space and multi-coil representations during reconstruction. Starting from the masked input k-space, the model applies a sequence of cascades, each performing data consistency in k-space followed by image-domain refinement using the proposed denoiser. After the final cascade, the reconstructed image is obtained via an inverse Fourier transform and combined across coils using root-sum-ofsquares (RSS) [32]. This iterative formulation allows the reconstruction to be progressively refined while jointly leveraging acquisition physics and transferable pretrained visual priors.
report results on the CMRxRecon [34] dataset, where models are trained and tested on cardiac MRI data under varying acceleration factors (×4, ×8, and ×10). Reconstruction quality is assessed using standard metrics, including SSIM, PSNR, and NMSE. In particular, we emphasize the structural similarity index measure (SSIM) [35], which is widely regarded as the primary evaluation metric in medical image reconstruction [12]. To further evaluate generalization, we consider challenging cross-domain settings by training models on CMRxRecon [34] and testing them on anatomically distinct datasets from fastMRI [40], including knee and brain MRI. This setup enables us to assess the robustness of the proposed approach under significant domain shifts.
4.1. CMRxRecon Dataset The CMRxRecon dataset [34] consists of 120 cardiac MRI cases acquired on 3T scanners, providing fully sampled dynamic cine and multi-contrast raw k-space data. The cine sequences include multiple standard cardiac views, such as short-axis (SAX) and long-axis views (2-chamber, 3chamber, and 4-chamber), while the multi-contrast acquisitions comprise T1-weighted and T2-weighted images. For our experiments, we use the provided fully sampled data to simulate undersampling with acceleration factors of ×4, ×8, and ×10 using uniform sampling patterns, along with corresponding masks and 24 auto-calibration (ACS) lines [34]. The dataset is split into training, validation, and test sets using a 70%/10%/20% partition, resulting in 17,916 training samples, 2,664 validation samples, and 5,112 test samples. All models are trained and evaluated on this split for in-distribution experiments.
4.2. fastMRI Dataset To evaluate cross-domain generalization, we test models trained on CMRxRecon [34] on the fastMRI dataset [32, 40], which consists of fully sampled knee and brain MRI scans acquired on 1.5T and 3T scanners. The knee subset includes coronal proton density-weighted images with and without fat suppression, while the brain subset contains axial T1-weighted, T2-weighted, and FLAIR sequences. For evaluation, we use a subset of 1,767 knee images and 3,168 brain images. Undersampling is simulated using equispaced sampling masks with center fractions of 0.04 and 0.08 [32], corresponding to different amounts of low-frequency information. We consider acceleration factors of ×4, ×8, and ×10 to assess performance under varying reconstruction difficulty in cross-domain settings.
4. Experiments
4.3. Implementation Details
In this section, we present the experimental setup and evaluate the performance of the proposed method on both indistribution and cross-domain reconstruction tasks. We first
Our framework is implemented following the standard unrolled reconstruction setting. All input images are resized to 224 × 224 to match the input requirements of the pretrained
vision encoders. We compare our method against the stateof-the-art E2E-VarNet [32], which we reimplement using the same architecture and hyperparameters for a fair comparison. To ensure consistency, we adopt the same sensitivity map estimation network as E2E-VarNet across all models. All reconstruction models are trained with 12 cascades (unrolled iterations). The baseline E2E-VarNet contains approximately 29.9M trainable parameters, while our foundation-model-based approach has approximately 56M trainable parameters due to the additional decoder and fusion components, with the foundation model encoder kept frozen. We train separate models for acceleration factors of ×4, ×8, and ×10. The models are optimized using the Adam optimizer with an initial learning rate of 1 × 10−3 , which is decayed by a factor of 0.1 after 40 epochs. We use no weight decay. Training is performed for 50 epochs with early stopping based on validation performance (patience of 5 epochs). We minimize the SSIM loss [32, 35] between the reconstructed and target images. For the sensitivity estimation network, we use 4 pooling layers and 8 base channels, consistent with prior work [32]. All experiments were conducted on NVIDIA RTX 4090 and RTX 3090 GPUs.
Table 1. Reconstruction performance on the CMRxRecon test set under different acceleration factors. Best results in bold, secondbest underlined.
Acc.
Method
SSIM ↑
PSNR ↑
NMSE ↓
×4
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.9676 0.9585 0.9548 0.9557
41.29 40.12 39.57 39.67
0.0154 0.0226 0.0249 0.0252
×8
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.9502 0.9359 0.9340 0.9358
38.21 36.83 36.65 36.79
0.0227 0.0340 0.0340 0.0343
×10
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.9417 0.9215 0.9223 0.9240
37.32 35.51 35.54 35.73
0.0262 0.0418 0.0419 0.0401
5. Results 5.1. In-Distribution Reconstruction Results We first evaluate the proposed method on the CMRxRecon dataset [34], where models are trained and tested on cardiac MRI data under varying acceleration factors. Table 1 summarizes reconstruction performance on the CMRxRecon test set across acceleration factors of ×4, ×8, and ×10. Qualitative comparisons are shown in Fig. 2. As expected, the task-specific E2E-VarNet [32] consistently achieves the best results across all metrics, benefiting from end-to-end supervision tailored to MRI reconstruction. Foundation-model-based approaches, while using frozen encoders pretrained on natural or biomedical data, remain competitive. At ×4, CLIP achieves an SSIM of 0.9585 compared to 0.9676 for E2E-VarNet, and similar trends persist at ×8 and ×10, with the performance gap widening as acceleration increases. This reflects the growing difficulty of the reconstruction problem, where task-specific model retain an advantage in in-distribution settings. Among foundation models, performance differences are relatively small. CMR-CLIP performs best at lower acceleration factors (×4 and ×8), indicating that natural-image pretraining provides strong general-purpose representations. At higher acceleration (×10), CMR-BiomedCLIP becomes the strongest variant, achieving the highest SSIM (0.9240), suggesting that domain-specific pretraining offers advantages in more ill-posed regimes. Overall, while task-specific architecture remain superior in standard settings, foundation models achieve competitive
Figure 2. Qualitative results at ×8 acceleration. From top to bottom: E2E-VarNet, CMR-CLIP, CMR-DINOv2, and CMRBiomedCLIP. Columns show the target image, reconstruction, error map, and zero-filled input, respectively.
performance without MRI-specific training, demonstrating their ability to learn transferable structural priors.
5.2. Cross-Domain Generalization Results We evaluate the generalization capability of the proposed approach under challenging cross-domain settings by training on the CMRxRecon dataset [34] and testing on the fastMRI knee and brain datasets [40]. These datasets differ substantially in anatomical structure, image contrast, and acquisition characteristics, making this a stringent test of robustness. In addition to the domain shift in anatomy, we introduce a further mismatch in the sampling patterns. During training, models are exposed to equispaced undersampling masks with fixed auto-calibration signal (ACS) lines (24 central k-space lines). At test time, however, we use
Figure 3. Out-of-distribution qualitative results. Visual comparison on fastMRI knee (left) and brain (right) datasets at ×10 acceleration and center fraction 0.08. From top to bottom: E2E-VarNet, CMR-CLIP, and CMR-BiomedCLIP. Columns show the target image, reconstruction, error map, and zero-filled input, respectively. Red boxes highlight regions with notable differences in structural fidelity.
equispaced masks defined by center fractions of 0.04 and 0.08 [32], where the center fraction specifies the proportion of low-frequency k-space that is fully sampled. This results in a different distribution of sampling patterns and reduces the amount of low-frequency information available, making reconstruction significantly more challenging. Together, these shifts in anatomy, contrast, and sampling strategy create a realistic and demanding evaluation setting for assessing the robustness and transferability of learned reconstruction priors. Quantitative results are reported in Tables 2 and 3, with corresponding qualitative comparisons shown in Fig. 3.
Analysis and Discussion. The cross-domain results on fastMRI knee and brain datasets reveal several important trends regarding the behavior of foundation-modelbased priors under distribution shift. At lower acceleration (×4), E2E-VarNet consistently achieves the best performance across both knee and brain datasets. This is expected, as fully supervised reconstruction models are optimized for the training distribution and can effectively exploit dataset-specific priors when sufficient measurements are available. However, as reconstruction becomes more challenging, through higher acceleration (×8, ×10) and reduced center fraction (0.04), a clear shift in behavior emerges. The performance of E2E-VarNet degrades more rapidly, while foundation-model-based approaches exhibit more stable performance and consistently match or outperform E2E-VarNet. This trend is consistent across both anatomical domains. On the knee dataset, CMR-BiomedCLIP achieves the best performance at ×8, while CMR-CLIP and CMRBiomedCLIP outperform E2E-VarNet at ×10, particularly under reduced center fraction. Similarly, on the brain
dataset, foundation models surpass E2E-VarNet at ×8 and ×10, with CMR-BiomedCLIP achieving the strongest overall performance at the highest acceleration. These gains are most pronounced under center fraction 0.04, where limited low-frequency information increases reconstruction ambiguity and places greater reliance on learned priors. Comparing foundation models, CMR-CLIP performs strongly at moderate acceleration, indicating that naturalimage pretraining captures highly transferable structural representations. CMR-BiomedCLIP provides additional gains in more ill-posed regimes, suggesting that domainspecific pretraining becomes beneficial as reconstruction difficulty increases. In contrast, CMR-DINOv2 consistently underperforms across settings, particularly under severe domain shift, likely due to the absence of cross-modal alignment in its pretraining objective [19], which may limit its ability to encode higher-level structural priors necessary for robust MRI reconstruction. Overall, these results suggest a regime-dependent behavior: task-specific model excel in in-distribution and wellconditioned settings, while pretrained foundation models provide more robust and transferable priors under severe undersampling and domain shift. This highlights the potential of leveraging large-scale pretrained representations as a complementary alternative to conventional task-specific reconstruction pipelines, particularly in scenarios where training data is mismatched.
6. Conclusion We present the first study investigating frozen vision foundation models as priors for physics-based MRI reconstruction, introducing an unrolled framework that integrates pretrained visual encoders within each cascade. While taskspecific models such as E2E-VarNet achieve superior per-
Table 2. OOD reconstruction performance on the FastMRI Knee dataset under different acceleration factors. Best results in bold, secondbest underlined.
Acc
Method
Center frac = 0.08
Center frac = 0.04
SSIM ↑
PSNR ↑
NMSE ↓
SSIM ↑
PSNR ↑
NMSE ↓
4x
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.8061 0.7864 0.7412 0.7848
31.06 30.88 27.50 30.66
0.0280 0.0288 0.0647 0.0302
0.7769 0.7387 0.6873 0.7310
29.21 27.92 21.91 27.60
0.0412 0.0548 0.2781 0.0591
8x
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.7161 0.7232 0.7137 0.7285
28.59 28.72 27.33 28.84
0.0479 0.0470 0.0646 0.0457
0.6552 0.6668 0.6566 0.6712
25.88 26.17 25.61 26.26
0.0876 0.0820 0.0932 0.0804
10x
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.6956 0.7150 0.6960 0.7133
28.08 28.36 27.03 28.55
0.0542 0.0509 0.0684 0.0491
0.6287 0.6576 0.6244 0.6545
25.26 26.08 24.73 25.91
0.1014 0.0839 0.1148 0.0873
Table 3. OOD reconstruction performance on the FastMRI Brain dataset under different acceleration factors. Best results in bold, secondbest underlined.
Acc
Method
Center frac = 0.08
Center frac = 0.04
SSIM ↑
PSNR ↑
NMSE ↓
SSIM ↑
PSNR ↑
NMSE ↓
4x
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.8209 0.7962 0.7321 0.7985
30.29 29.69 25.65 29.52
0.0241 0.0281 0.0808 0.0290
0.7670 0.7208 0.6531 0.7163
27.23 25.79 20.59 25.43
0.0483 0.0682 0.3796 0.0747
8x
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.7363 0.7391 0.7203 0.7447
27.25 27.36 25.28 27.14
0.0473 0.0462 0.0753 0.0483
0.6416 0.6505 0.6300 0.6514
23.74 23.99 22.86 23.88
0.1069 0.1006 0.1309 0.1035
10x
E2E-VarNet CMR-CLIP CMR-DINOv2 CMR-BiomedCLIP
0.7234 0.7261 0.7142 0.7326
26.84 26.22 25.42 26.87
0.0520 0.0594 0.0730 0.0515
0.6209 0.6312 0.6135 0.6374
23.33 23.32 22.63 23.47
0.1172 0.1166 0.1407 0.1134
formance in standard in-distribution settings, foundationmodel-based approaches demonstrate greater robustness under cross-domain shifts and severe undersampling, where the reconstruction problem becomes increasingly ill-posed. Notably, as reconstruction difficulty increases, these models provide more stable and transferable priors, narrowing and in some cases surpassing the performance gap with the supervised baseline. Furthermore, we find that naturalimage-pretrained models such as CLIP already capture highly transferable structural representations, with domain-
specific pretraining BiomedCLIP offering modest additional gains in challenging regimes. Overall, our findings highlight the potential of large-scale pretrained visual representations as a complementary source of priors for improving robustness and generalization in MRI reconstruction and, more broadly, in physics-based inverse problems.
Acknowledgements This work was supported by Taighde Éireann–Research Ireland under Grant numbers 18/CRT/6183 & 12/RC/2289 P2.
References [1] Deepa Anand, Vanika Singhal, Dattesh D Shanbhag, Shriram KS, Uday Patil, Chitresh Bhushan, Kavitha Manickam, Dawei Gui, Rakesh Mullick, Avinash Gopal, et al. One-shot localization and segmentation of medical images with foundation models. arXiv preprint arXiv:2310.18642, 2023. 3 [2] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016. 4 [3] Mohammed Baharoon, Waseem Qureshi, Jiahong Ouyang, Yanwu Xu, Abdulrhman Aljouie, and Wei Peng. Evaluating general purpose vision foundation models for medical image analysis: An experimental study of dinov2 on radiology benchmarks. arXiv preprint arXiv:2312.02366, 2023. 1, 3 [4] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021. 1 [5] Victor Ion Butoi, Jose Javier Gonzalez Ortiz, Tianyu Ma, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Universeg: Universal medical image segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 21438–21451, 2023. 1 [6] Emmanuel J Candes, Justin K Romberg, and Terence Tao. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 59(8):1207–1223, 2006. 2 [7] Kerem Cekmeceli, Meva Himmetoglu, Guney I Tombak, Anna Susmelj, Ertunc Erdil, and Ender Konukoglu. Do vision foundation models enhance domain generalization in medical image segmentation? In European Conference on Computer Vision, pages 185–200. Springer, 2024. 4 [8] Soumitri Chattopadhyay, Basar Demir, and Marc Niethammer. Zero-shot domain generalization of foundational models for 3d medical image segmentation: an experimental study. arXiv preprint arXiv:2503.22862, 2025. 1 [9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 4 [10] David L Donoho. Compressed sensing. IEEE Transactions on information theory, 52(4):1289–1306, 2006. 2 [11] Katerina Eyre, Katherine Lindsay, Saad Razzaq, Michael Chetrit, and Matthias Friedrich. Simultaneous multiparametric acquisition and reconstruction techniques in cardiac magnetic resonance imaging: basic concepts and status of clinical development. Frontiers in cardiovascular medicine, 9:953823, 2022. 1 [12] Zalan Fabian, Berk Tinaz, and Mahdi Soltanolkotabi. Humus-net: Hybrid unrolled multi-scale network architecture for accelerated mri reconstruction. Advances in Neural Information Processing Systems, 35:25306–25319, 2022. 1, 2, 3, 4, 5 [13] Yunhui Guo, Yandong Li, Liqiang Wang, and Tajana Rosing. Depthwise convolution is all you need for learning multiple
visual domains. In Proceedings of the AAAI conference on artificial intelligence, pages 8368–8375, 2019. 4 [14] Ali Hatamizadeh, Yucheng Tang, Vishwesh Nath, Dong Yang, Andriy Myronenko, Bennett Landman, Holger R Roth, and Daguang Xu. Unetr: Transformers for 3d medical image segmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 574–584, 2022. 4, 5 [15] JP Huix, AR Ganeshan, JF Haslum, M Söderberg, C Matsoukas, and K Smith. Are natural domain foundation models useful for medical image classification? in 2024 ieee/cvf winter conference on applications of computer vision (wacv). waikoloa, hi, usa: Ieee;[cited 2024 aug 27]. 7619–7628, 2024. 1, 2, 3, 4 [16] Dongsheng Jiang, Yuchen Liu, Songlin Liu, Jin’e Zhao, Hao Zhang, Zhen Gao, Xiaopeng Zhang, Jin Li, and Hongkai Xiong. From clip to dino: Visual encoders shout in multi-modal large language models. arXiv preprint arXiv:2310.08825, 2023. 4 [17] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023. 1 [18] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-tosequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 7871–7880, 2020. 1 [19] Yiming Liu, Yuhui Zhang, Dhruba Ghosh, Ludwig Schmidt, and Serena Yeung-Levy. Data or language supervision: What makes clip better than dino? arXiv preprint arXiv:2510.11835, 2025. 7 [20] Michael Lustig, David L Donoho, Juan M Santos, and John M Pauly. Compressed sensing mri. IEEE signal processing magazine, 25(2):72–82, 2008. 2 [21] Jun Lyu, Chen Qin, Shuo Wang, Fanwen Wang, Yan Li, Zi Wang, Kunyuan Guo, Cheng Ouyang, Michael Tänzer, Meng Liu, et al. The state-of-the-art in cardiac mri reconstruction: Results of the cmrxrecon challenge in miccai 2023. Medical Image Analysis, 101:103485, 2025. 1 [22] Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature communications, 15(1):654, 2024. 1 [23] Christos Matsoukas, Johan Fredin Haslum, Moein Sorkhei, Magnus Söderberg, and Kevin Smith. What makes transfer learning work for medical images: Feature reuse & other factors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9225–9234, 2022. 2 [24] Christos Matsoukas, Johan Fredin Haslum, Moein Sorkhei, Magnus Söderberg, and Kevin Smith. Pretrained vits yield versatile representations for medical images. arXiv preprint arXiv:2303.07034, 2023. 2 [25] Philip Müller, Georgios Kaissis, Congyu Zou, and Daniel Rueckert. Radiological reports improve pre-training for lo-
calized imaging tasks on chest x-rays. In International Conference on Medical Image Computing and ComputerAssisted Intervention, pages 647–657. Springer, 2022. 3 [26] Gregory Ongie, Ajil Jalal, Christopher A Metzler, Richard G Baraniuk, Alexandros G Dimakis, and Rebecca Willett. Deep learning techniques for inverse problems in imaging. IEEE Journal on Selected Areas in Information Theory, 1 (1):39–56, 2020. 1 [27] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 2, 3 [28] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018. 1 [29] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PmLR, 2021. 1, 2, 3 [30] Praveenbalaji Rajendran, Mojtaba Safari, Wenfeng He, Mingzhe Hu, Shansong Wang, Jun Zhou, and Xiaofeng Yang. Foundation models in medical image analysis: A systematic review and meta-analysis. arXiv preprint arXiv:2510.16973, 2025. 1 [31] Xinrui Song, Xuanang Xu, and Pingkun Yan. General purpose image encoder dinov2 for medical image registration. arXiv preprint arXiv:2402.15687, 2024. 3 [32] Anuroop Sriram, Jure Zbontar, Tullie Murrell, Aaron Defazio, C Lawrence Zitnick, Nafissa Yakubova, Florian Knoll, and Patricia Johnson. End-to-end variational networks for accelerated mri reconstruction. In International conference on medical image computing and computer-assisted intervention, pages 64–73. Springer, 2020. 2, 3, 4, 5, 6, 7 [33] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022, 2016. 4 [34] Chengyan Wang, Jun Lyu, Shuo Wang, Chen Qin, Kunyuan Guo, Xinyu Zhang, Xiaotong Yu, Yan Li, Fanwen Wang, Jianhua Jin, et al. Cmrxrecon: an open cardiac mri dataset for the competition of accelerated image reconstruction. arXiv preprint arXiv:2309.10836, 2023. 2, 5, 6 [35] Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for image quality assessment. In The thrity-seventh asilomar conference on signals, systems & computers, 2003, pages 1398–1402. Ieee, 2003. 5, 6 [36] Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3876–3887, 2022. 1, 3 [37] Stefano Woerner and Christian F Baumgartner. Navigating data scarcity using foundation models: A benchmark of fewshot and zero-shot learning approaches in medical imaging. In International Workshop on Foundation Models for General Medical AI, pages 30–39. Springer, 2024. 3
[38] Bingyu Xin, Meng Ye, Leon Axel, and Dimitris N Metaxas. Fill the k-space and refine the image: Prompting for dynamic and multi-contrast mri reconstruction. In International Workshop on Statistical Atlases and Computational Models of the Heart, pages 261–273. Springer, 2023. 1, 3 [39] George Yiasemis, Jan-Jakob Sonke, Clarisa Sánchez, and Jonas Teuwen. Recurrent variational network: a deep learning inverse problem solver applied to the task of accelerated mri reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 732–741, 2022. 1 [40] Jure Zbontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Matthew J Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, et al. fastmri: An open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.08839, 2018. 3, 5, 6 [41] Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915, 2023. 1, 2, 3 [42] Zihao Zhao, Yuxiao Liu, Han Wu, Mei Wang, Yonghao Li, Sheng Wang, Lin Teng, Disheng Liu, Zhiming Cui, Qian Wang, et al. Clip in medical imaging: A survey. Medical Image Analysis, 102:103551, 2025. 3