How Far from Clinical Deployment? Evaluating the Complete Unsupervised Domain Adaptation Pipeline in Medical Imaging

How Far from Clinical Deployment? Evaluating the Complete Unsupervised Domain Adaptation Pipeline in Medical Imaging Yiheng Xiong1

Luisa Gallée1 1

Daniel Santak Wolf1,2

Michael Götz1

Section of Experimental Radiology, Ulm University Medical Center 2

Visual Computing Group, Ulm University

⚠

Abstract

Deep learning models excel at medical image-based diagnosis, but their performance often degrades substantially under domain shifts arising from differences in imaging modalities, acquisition protocols, and patient populations [23]. Annotating training data from a new domain is costly, requiring scarce medical expertise. Unsupervised domain adaptation (UDA) addresses both challenges by transferring knowledge from a labeled source domain to an unlabeled target domain, with recent algorithms adapting robustly across clinically relevant shifts, such as crossmodality [50], cross-site [27], and cross-age [11]. However, deploying UDA in clinical routine requires a practitioner to

Labeled SLO Images

UDA Algorithms

Hyperparameters

DANN

Config. 1

CDAN

Config. 2

Trained Models

…

1. Introduction

Adaptation Unlabeled OCT Images

…

Deploying unsupervised domain adaptation (UDA) in clinical practice requires choosing which algorithm to use and which of its trained models to ship. However, the deployment (target) domain is unlabeled, so models cannot be evaluated directly on it, leaving it unclear which to select. We address this by evaluating the complete UDA pipeline, considering both adaptation and label-free selection together. Our study covers eleven clinically relevant crossdomain scenarios from nine medical imaging datasets, with ten UDA algorithms and 13 label-free selection methods (validators), evaluating over 80,000 trained models in total. By this, we find that a capable adapted model usually exists, but identifying it without target labels is difficult: the validator-selected models leave a large and structural target performance gap to the best available one, with no evaluated validator consistently reliable. Towards closing it, we explore two strategies, ensembling and a small targetlabeling budget; both narrow this gap but do not close it entirely. Overall, deployable UDA depends on the complete pipeline; addressing the less explored selection step could bring much of current UDA closer to clinical use. Our code is open sourced at Complete UDA Pipeline.

Which one to select for deployment?

…

arXiv:2608.12035v1 [cs.CV] 12 Aug 2026

Heiko Hillenhagen1

Figure 1. The complete UDA pipeline at deployment. Adaptation trains models on a labeled source domain (here SLO) and an unlabeled target domain (here OCT), producing many candidate models that vary in UDA algorithm and hyperparameter configuration. At deployment, one model needs to be selected, yet without target labels they cannot be evaluated directly to decide which.

decide which algorithm to use and which of its checkpoints (a model snapshot at a training iteration) to ship, as shown in Figure 1. Thus, model selection is an unavoidable part of the complete UDA pipeline. Furthermore, without target labels, candidate models cannot be evaluated directly on the target domain, leaving it unclear which one to select for deployment: the standard approach of comparing them on a held-out labeled validation set is unavailable [34]. Yet, existing medical UDA surveys [5, 14, 23, 25, 39, 42, 51] and evaluations [3, 8, 36, 41, 44] primarily focus on adaptation in isolation, leaving the selection step largely unexamined. As a result, it remains unclear how the complete UDA pipeline behaves under clinical deployment conditions, and thus how far current UDA truly is from clinical deployment. To address this, we evaluate the complete UDA pipeline in medical imaging, considering both adaptation and labelfree selection that clinical deployment requires. For adaptation, we include ten UDA algorithms spanning multiple paradigms, with some tailored to medical imaging [15, 54]. For selection, we adopt 13 established label-free selection approaches (a.k.a. validators), such as IWCV [43] and DEV [53]. Each validator assigns a scalar validation score to each checkpoint without using target labels, and

the checkpoint with the best score is selected. Our evaluation covers brain MRI, chest X-ray (CXR), and retinal imaging, with eleven clinically relevant cross-domain scenarios in total. Brain MRI and CXR use four datasets each, with within-modality shifts arising from differences in hospitals, scanners, or patient populations, while retinal imaging involves across-modality shifts between SLO and OCT. Altogether, this amounts to roughly 16,500 checkpoint configurations (each set by the algorithm, its hyperparameters, and training iteration), or over 80,000 trained checkpoints once repeated across folds or random seeds. Our study points to a consistent picture across different cross-domain scenarios. Adaptation itself is often effective: a capable adapted model usually exists among the candidates. What proves difficult is identifying it without target labels: the models selected by the evaluated validators leave a large target performance gap to the best available model. We further trace the origin of this gap and find it largely structural: no single evaluated validator works consistently well across all scenarios; one that succeeds in some fails in others, so no default choice can be recommended in advance. In the absence of a consistently reliable validator across all scenarios, towards closing the gap, we turn to two strategies. Instead of relying on one validator, the first ensembles predictions across validator-selected checkpoints from every algorithm; instead of remaining fully label-free, the second spends a small target-labeling budget. Both narrow the gap, though neither closes it entirely and each carries its own limitations. Our contributions are summarized as follows: • We present, to our knowledge, the first study to evaluate the complete adaptation and label-free selection UDA pipeline in medical imaging under clinical deployment conditions, spanning ten algorithms, 13 validators, and nine datasets across eleven cross-domain scenarios. • By considering adaptation and selection as a whole, we find that a capable adapted model usually exists, yet the remaining distance to clinical deployment lies largely in label-free selection: the selection gap is large and structural, with no evaluated validator consistently reliable. • Towards closing the gap, we point to two strategies, ensembling and a small target-labeling budget; both narrow this gap and open directions for more reliable selection.

2. Related Work 2.1. Surveys and Benchmarks on Medical UDA UDA in medical imaging has been reviewed from several angles. Choudhary et al. [5] give an early overview of deep domain adaptation, Guan and Liu [14] provide a widely used taxonomy across modalities, and Kumari and Singh [23] survey more recent deep UDA methods and datasets. More recent surveys cover emerging diffusion-

and foundation-model approaches [51], broader reviews of distribution shift also include UDA [42], and others focus on specific settings such as functional brain data [39] and segmentation [25]. Alongside these surveys, several benchmarks evaluate adaptation under realistic shifts: M3DA [41] for 3D MRI/CT segmentation, CrossMoDA [8] for crossmodality segmentation, M3-UDA [36] for multi-hospital fetal ultrasound detection, and Chamarthi et al. [3] and Sultana et al. [44] for skin lesion classification. These efforts have established rigorous comparisons of adaptation methods on clinically relevant data, but the selection step remains largely underexplored. Our study complements them by evaluating the complete pipeline that clinical deployment requires, adaptation and label-free selection together.

2.2. Label-Free Model Selection in UDA Selecting a model without target labels has been studied mainly in general-vision UDA, where a range of validators have been proposed [12, 17, 35, 38, 43, 48, 52, 53] and evaluated. Musgrave et al. [34] provide one of the first UDA model selection benchmarks covering many algorithms, though over a relatively small set of three label-free validators. Subsequent evaluations [10, 18, 24] strengthen evaluation practice further with more validators, with SKADA-Bench [24] additionally covering broader modalities, though with an emphasis on shallow adaptation methods. These works, however, primarily focus on selecting checkpoints within individual algorithms. In practice, deployment also requires choosing which algorithm to use, so a more realistic setting also calls for selecting across algorithms. Beyond this, medical images pose distinct challenges, exhibiting more pronounced and varied shifts than natural images [23], under which the behavior of these validators remains unclear. To address these, our study examines selection also across algorithms, over a broad set of 13 validators and ten deep UDA algorithms, including medical-specific ones, and, in particular, under the shifts that clinical routine faces, such as cross-modality, crossinstitution, and cross-cohort, assessing how the complete UDA pipeline behaves under deployment conditions.

3. Datasets and Methods We study the complete UDA pipeline in medical imaging under the conditions a clinical deployment would face, focusing on classification tasks such as disease diagnosis. The pipeline comprises two stages. In the adaptation stage, UDA algorithms share a common structure, a backbone, a classification head, and an adaptation module, and optimize a joint objective L = Lcls + λ Ladapt ,

(1)

where Lcls supervises the model with source labels, Ladapt aligns feature distributions across domains, and λ controls

Table 1. Overview of the datasets, UDA algorithms, and validators used in our study. (a) Datasets across brain MRI, CXR, and retinal imaging (SLO / OCT), with the number of positive (Pos.) and negative (Neg.) samples per dataset. Positive and negative denote, respectively, Alzheimer’s disease and cognitively normal for brain MRI, pneumonia and non-pneumonia for CXR, and glaucoma and non-glaucoma for retinal imaging. (b) UDA algorithms spanning multiple paradigms. (c) Validators grouped into source-guided and target-based criteria. (a) Datasets Organ Modality Dataset ADNI-1 [19] ADNI-2 [19] Brain MRI ADNI-3 [19] AIBL [9]

Chest

Eye

X-Ray

SLO OCT

Pos. 200 159 85 78

Neg. 221 232 431 477

RSNA [49] Child CXR [22] LDD [2] CRD [1]

6,012 20,672 4,273 1,583 5,776 3,919 9,237 10,319

FairDomain [45]

4,453 4,453

5,547 5,547

(b) UDA Algorithms Pseudo Lab.: ATDOC [26] Cls. Disc.: MCD [37] Adv. Align: DANN [13], CDAN [31], DALN [4] Medical-Spec.: AD2A (Brain MRI) [15], CoUDA (CXR) [54] Feat. Dist.: MMD [30] Info. Max: MCC [20] SVD Loss: BNM [6]

Source-Guided: Target-Based:

the adaptation strength. Running an algorithm with a given λ produces a training run, during which we save a sequence of checkpoints. Each checkpoint θ is thus uniquely defined by its algorithm, its adaptation strength λ, and its training iteration, and every checkpoint is a candidate for deployment. In the selection stage, we replicate the constraint of clinical deployment: target labels are unavailable, so the model to deploy must be chosen without them. The role of a validator is to provide a label-free validation score for the target performance. Formally, a validator is a scoring function V : θ 7→ s ∈ R,

(2)

that maps a checkpoint θ to a scalar score s = V (θ) without using target labels. The selected model for deployment is the checkpoint with the best validation score, θ⋆ = arg max V (θ), θ∈Θ

(3)

where Θ is the candidate checkpoint pool and scores are oriented so that higher is better. Datasets. To construct medical UDA scenarios, four widely adopted brain MRI datasets are used: ADNI-1, ADNI-2, ADNI-3 [19], and AIBL [9]. Subjects appearing in multiple ADNI datasets are kept in only one to prevent data leakage. In addition, four publicly available CXR datasets are used: RSNA [49], Child CXR [22], LDD [2], and CRD [1]. For retinal imaging, we use FairDomain dataset [45], which provides paired SLO and OCT acquisitions of the same cohort. For brain MRI and CXR, each dataset is treated as a separate domain, and transfer is performed across datasets within each modality; for retinal data, transfer is performed between SLO and OCT modalities. Brain MRI datasets contain Alzheimer’s disease and cognitively normal subjects, CXR datasets contain pneumonia and non-pneumonia subjects, and retinal datasets contain glaucoma and nonglaucoma subjects. All modalities undergo standard prepro-

(c) Validators Source-Risk [12], IWCV [43], DEV [53], DEV-N [35] Entropy [33], InfoMax [34], Corr-C [48], BNM (V) [35], MCC (V) [20], SND [38], ClassAMI [35], MixVal [17], TransScore [52]

cessing following prior work [15, 45, 54]. Dataset statistics are summarized in Table 1 (a). Methods. Since we study the complete pipeline as a whole, we include a representative set of UDA algorithms and, crucially, a diverse set of label-free validators. As shown in Table 1 (b) and (c), the ten algorithms span multiple paradigms, including feature-distance minimization [30], adversarial alignment [4, 13, 31], information maximization [20], SVD loss [6], pseudo-labeling [26], classifier discrepancy [37], and medical-specific techniques [15, 54]. The 13 validators cover both source-guided criteria [12, 35, 43, 53] and target-based ones [17, 20, 33, 34, 38, 48, 52]. Experimental Setup. Following Guan et al. [15], five UDA scenarios (source→target) are constructed for brain MRI: ADNI-1→ADNI-2, ADNI-1→ADNI-3, ADNI-2→ADNI1, ADNI-2→ADNI-3, and ADNI-1+2→AIBL. Following Feng et al. [11] and Liu et al. [28], two CXR transfers are used, each in both directions: RSNA↔Child CXR and LDD↔CRD. For retinal imaging, the two cross-modality directions SLO→OCT and OCT→SLO are evaluated. Both source and target data are split into training and validation sets, and target performance is measured on the target validation set using balanced accuracy. For datasets without predefined splits, stratified five-fold cross-validation is performed; for datasets with explicit train/test splits, results are averaged over three random seeds. In both cases the mean ± standard deviation is reported; median values with 95% confidence intervals are additionally provided in the supplementary material. Implementation Details. For all experiments, the classification head is a three-layer MLP with a dropout rate of 0.5. For brain MRI, a 3D ResNet-50 [16] trained from scratch is used as the backbone, with a batch size of eight per domain. For CXR and retinal imaging, a 2D ResNet-50 pretrained on ImageNet [7] is used, with a batch size of 48 per domain. All algorithms are trained with AdamW (weight decay 1e4) and a one-cycle learning rate schedule with warm-up and

Table 2. Complete UDA results on ADNI-1→ADNI-2 (target accuracy, %; eight best validators by Across-Algo accuracy shown). Each cell reports the target accuracy of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle DEV-N Source-Risk InfoMax BNM (V) ClassAMI SND MCC (V) TransScore

SourceOnly 88.2±2.0 80.9±5.8 80.9±5.8 84.2±2.1 84.7±2.1 81.3±5.8 73.9±12 84.2±2.7 81.3±1.0

MMD 89.3±4.0 81.7±2.8 82.1±3.1 84.8±5.7 84.8±5.7 71.5±13 83.4±4.1 84.4±6.4 77.5±11

DANN 90.0±4.0 84.9±2.0 81.5±3.7 87.3±4.4 87.5±4.0 74.8±8.7 80.0±2.6 84.9±3.3 85.1±4.5

CDAN 89.9±3.0 81.8±8.4 81.1±8.0 86.0±1.5 84.9±3.8 83.0±4.0 79.2±5.8 84.5±4.0 86.7±2.3

DALN 91.0±3.5 80.8±9.1 80.8±9.1 88.2±3.8 87.2±2.2 84.2±1.6 85.5±4.9 86.4±3.7 85.2±5.9

MCC 91.0±1.7 83.5±3.3 83.3±3.2 81.8±9.6 81.8±9.6 81.7±7.5 78.2±5.9 80.4±8.3 80.4±9.9

a peak learning rate of 1e-3, for 10k iterations on brain MRI and retinal imaging and 30k iterations on CXR. The adaptation strength is varied over λ ∈ {0.1, 0.5, 1.0}, giving three runs per algorithm. After warm-up, checkpoints are saved at uniform intervals, yielding 50 checkpoints per run and 150 checkpoints per algorithm. Lcls is binary cross-entropy with class-balanced weighting derived from the source labels. All runs are conducted on an A6000 GPU.

4. Results and Discussion We use ADNI-1→ADNI-2 as an illustrative example for the complete UDA pipeline in Table 2, which reports the target accuracy of the checkpoint each validator selects. This selection is compared against the Oracle, which selects directly with target labels. We define the selection gap as the difference in target accuracy between the Oracle-selected model and the model selected by a label-free validator. The per-algorithm columns report selection within a single algorithm, and Avg. is their mean, while Across-Algo pools all algorithms’ checkpoints and selects across them. Due to space constraints, we display the eight best validators by their Across-Algo accuracy. Full results for all scenarios can be found in the supplementary material. We report SourceOnly (no adaptation) and TargetOnly (trained on labeled target data) as references.

4.1. Adaptation Works in Principle As shown in Table 2, under Oracle selection, ADNI1→ADNI-2 scenario reaches 93.2% accuracy when pooling all checkpoints (Across-Algo), exceeding both SourceOnly (88.2%) and TargetOnly (90.9%). The gain is broad: the Oracle-selected checkpoint beats SourceOnly for seven of the nine algorithms, with a per-algorithm average of 89.5% (Avg.), and surpasses TargetOnly for three. Figure 2 extends Table 2 to all eleven UDA scenarios, reporting the same four quantities under Oracle selection. At the per-algorithm level, the average (Avg.) exceeds SourceOnly in eight of the eleven scenarios, and in two it

BNM 89.9±4.4 82.9±3.4 84.1±4.1 83.5±6.9 83.5±6.9 79.6±4.4 78.5±8.6 85.9±3.6 76.5±11

ATDOC 85.7±9.8 76.0±12 73.2±12 78.9±10 78.9±10 78.3±10 75.8±12 80.7±13 72.7±10

MCD 87.4±4.3 81.0±6.2 81.5±6.2 83.3±5.6 83.3±5.6 63.7±19 82.2±5.6 83.1±3.7 82.4±3.7

AD2A Avg. Across-Algo TargetOnly 92.0±0.93 89.5±2.0 93.2±2.0 90.9±2.4 87.6±1.9 85.1±1.3 87.5±2.3 83.5±2.9 86.3±2.2 83.2±7.2 86.3±2.2 81.8±9.6 72.7±16 80.8±3.6 80.7±8.7 80.7±8.7 87.1±3.7 80.4±2.3 86.3±2.8 79.9±10 -

even surpasses TargetOnly. Pooling checkpoints across algorithms strengthens this further: Across-Algo lies above SourceOnly in every scenario, and exceeds TargetOnly in five. Averaged over all scenarios (Mean), the ordering is consistent: SourceOnly < Avg. < Across-Algo, with Across-Algo reaching 84.8% and approaching TargetOnly (87.5%). Adaptation is therefore often effective across these clinical UDA scenarios; in particular, a capable adapted model exists in most scenarios (evidenced by Across-Algo). Nevertheless, this model is only useful if it can be identified without target labels, as deployment requires.

4.2. Large Selection Gap from Validators The capable model established above can only be deployed if a validator selects it. Return to ADNI-1→ADNI-2 scenario as shown in Table 2, under Across-Algo, the Oracle attains 93.2%. Over the same across-algorithm pool, the model selected by the best validator (DEV-N) reaches 85.1%, and that selected by the best algorithm–validator pair (DALN with InfoMax) reaches 88.2%. Even these best cases fall 5.0 to 8.1 points short of the Oracle. Figure 3 extends Table 2 to all eleven UDA scenarios, reporting the target accuracy of the model selected by the best validator over the across-algorithm pool (Best Validator), by the best algorithm–validator pair (Best Pair), and by the across-algorithm Oracle. In every scenario, both Best Validator and Best Pair fall below the Oracle: the gap reaches up to 10.5 points (RSNA→Child CXR) and averages 6.1 points across scenarios (Mean). Crucially, Best Validator and Best Pair differ from scenario to scenario and are themselves identified using target labels; at deployment, where no labels are available to choose them in advance, the realized gap can only be larger. A capable model exists in the pool, but selecting it without labels leaves a substantial portion of the achievable performance unrealized. The Selection Gap Persists Across Architectures. To test whether our findings depend on the backbone, we repeat RSNA→Child CXR with three additional architec-

Across-Algo

TargetOnly

80.4 81.7 84.8 87.5

Target Accuracy (\%)

Avg.

SourceOnly

100 90 80 70 60 50 40 A

I-1 DN

2 A

I-1 DN

3 A

I-2 DN

1 A

I-2 DN

3 AD

2 1+ NI-

AIB

L NA RS

ild Ch

ild Ch

NA RS

L

DD

D CR

D CR

LD

D SLO

OC

T O

CT

SLO

Me

an

Figure 2. Target accuracy under Oracle selection across UDA scenarios. Bars show SourceOnly (no adaptation), the per-algorithm average (Avg.), Across-Algo (selection over all algorithms’ pooled checkpoints), and TargetOnly (labeled-target). Across-Algo exceeds SourceOnly in all scenarios and approaches TargetOnly, indicating a capable adapted model usually exists. Error bars: standard deviation.

Best Validator

Target Accuracy (\%)

100

Best Pair

Oracle

=10.5%

90

=6.1%

80 70 60 50 40 A

I-1 DN

2 A

I-1 DN

3 A

I-2 DN

1 A

I-2 DN

3 AD

2 1+ NI-

AIB

L NA RS

ild Ch

ild Ch

NA RS

L

DD

D CR

D CR

LD

D SLO

OC

T O

CT

SLO

an Me

Figure 3. Target accuracy across UDA scenarios under three selections (%). Bars show the accuracy of the model selected by the best validator over the across-algorithm pool (Best Validator), by the best algorithm–validator combination (Best Pair), and by the acrossalgorithm Oracle. The first two fall below the Oracle in every scenario, with the gap up to 10.5 points and averaging 6.1 (Mean). Note that Best Validator and Best Pair differ from scenario to scenario and are identified using target labels, so they cannot be chosen in advance; the gap realized at deployment can only be larger. Error bars: standard deviation.

Table 3. Target accuracy (%) on RSNA→Child CXR for four backbones. Best Val. is the accuracy of the model selected by the best validator under the across-algorithm pool, and Oracle that of the actual best model. The gap persists across all four backbones. Backbone ResNet-50 ResMLP ConvNeXt DeiT

Best Val. 79.0 70.9 82.6 80.4

Oracle 89.5 79.2 90.2 86.5

∆Oracle 10.5 8.3 7.6 6.1

tures spanning distinct families: ConvNeXt [29] (a modern CNN), ResMLP [47] (an MLP-based model), and DeiT [46] (a vision transformer). As shown in Table 3, the model selected by the best validator leaves a noticeable gap to the Oracle for every architecture. This suggests the selection gap is not specific to a particular backbone.

4.3. Structural Selection Gap from Validators Having established the magnitude of the selection gap, we now investigate its origin. The gap may arise either from a suboptimal choice of validator, which a more suitable one would remedy, or from a more structural limitation that no evaluated validator overcomes. Since selection is fundamentally a ranking problem (a validator succeeds by ranking checkpoints so that the best one scores highest), we trace the selection gap to validator reliability, measured by the Spearman rank correlation ρ between validation scores and true target accuracy. For validators where a lower score indicates a better checkpoint, the sign is flipped so that a positive correlation always denotes the intended direction. Within-Algorithm Selection. Figure 4 (a) and (b) fix the scenario OCT→SLO and validator BNM (V) and vary the algorithm: BNM (V) ranks DALN’s checkpoints well and

Target accuracy

= 0.88 p < 0.001

0.62 0.60 0.58 0.56 0.54

Target accuracy

0.57

0.64

0.52 22

24

26

28

BNM (V) score

0.55 0.53 0.47 0.45 0.42 0.40 28.5

30

(a) OCT→SLO, DALN

29.5

30.0

30.5

BNM (V) score

31.0

31.5

0.80 0.70 0.60 0.50 0.40

0.62 0.60

Target accuracy

Target accuracy

29.0

(b) OCT→SLO, ATDOC

= 0.59 p < 0.001

0.90

= -0.42 p < 0.001

0.50

0.58 0.55

= -0.51 p < 0.001

0.53 0.50 0.48 0.45

points that the validator scores highest: this retains the competitive checkpoints worth selecting among, while ten per algorithm provides a sufficient sample to estimate the rank correlation. On this pooled set, the correlation collapses to near zero (ρ = −0.04, p = 0.0.678). Figure 7 reports the same across-algorithm correlation across all validators and scenarios: most validators are unreliable or contradict their intended direction, with correlations weak or negative. For within-algorithm selection, a suitable validator may exist among evaluated ones, but which one differs by algorithm and scenario, so it cannot be chosen in advance. For across-algorithm selection, even this fails: most evaluated validators correlate weakly or in reverse. The gap therefore appears largely structural rather than a matter of choosing a better validator among those evaluated ones.

0.43 0.0

0.2

0.4

0.6

ClassAMI score

0.8

(c) MMD, ADNI-2→ADNI-1

0.0

0.2

0.4

0.6

ClassAMI score

0.8

(d) MMD, SLO→OCT

Figure 4. Within-algorithm Spearman correlation (ρ) between validation score and target accuracy. Each point is a checkpoint. The same validator can flip from reliable to reversed with a change of algorithm (a, b) or scenario (c, d) alone.

in the designed direction (ρ = 0.88, p < 0.001), yet ranks ATDOC’s checkpoints in reverse (ρ = −0.42, p < 0.001). Figure 4 (c) and (d) fix the algorithm (MMD) and validator (ClassAMI) and vary the scenario: the correlation is positive as intended for ADNI-2→ADNI-1 (ρ = 0.59, p < 0.001) but reversed for SLO→OCT (ρ = −0.51, p < 0.001). Further individual analyses can be found in the supplementary material. Figure 5 aggregates the withinalgorithm Spearman correlation between validation scores and target accuracy two ways: (a) averaged over algorithms for a per-scenario view, and (b) averaged over scenarios for a per-algorithm view; in both, blue denotes correlation in the intended direction (darker is stronger) and warm red the opposite. On average, validators are moderately informative, but their reliability is uneven: no validator attains consistent correlation across all scenarios in (a) or all algorithms in (b), and the strongest validator differs from column to column. DEV-N, for example, is the most reliable validator for OCT→SLO (ρ = 0.88) yet drops to near zero for RSNA→Child CXR; likewise, InfoMax is strongest for AD2A (ρ = 0.79) but falls to ρ = 0.15 for ATDOC. Across-Algorithm Selection. Figure 6 (a) shows SourceRisk on ADNI-1→ADNI-2 for a single algorithm (BNM): within this algorithm, it ranks checkpoints reasonably well (ρ = 0.57, p < 0.001). Figure 6 (b) instead pools checkpoints across all algorithms for the same scenario and validator. To form this pool while keeping the comparison meaningful, we take, for each algorithm, the ten check-

4.4. Strategies to Narrow the Selection Gap The structural selection gap leaves a large distance between what adaptation achieves and what label-free selection realizes at deployment. In the absence of a validator consistently reliable across scenarios, we explore two practical strategies and show that each recovers part of this gap. Ensembling. Since no single evaluated validator is consistently reliable, we avoid committing to one and instead ensemble the checkpoints selected by different validators, similar to Hu et al. [18] but additionally ensembling across algorithms. In the across-algorithm setting, where checkpoints from all algorithms are pooled, we select one checkpoint per algorithm with every validator and ensemble these selections by averaging their predictions. As shown in Table 4, ensembling matches or exceeds Best Val. in most scenarios. Again Best Val. is the model selected by the best validator for each scenario, which differs across scenarios and cannot be known in advance. The remaining gap to the Oracle is small in several cases (e.g., 0.6 for OCT→SLO and 2.6 for LDD→CRD), showing that it recovers much of the achievable performance. It nonetheless remains far from the Oracle in other cases such as RSNA→Child CXR. Small Target Labeling Budget. Since fully label-free selection cannot be trusted under the evaluated validators, we explore whether spending a small budget on target labeling can identify a better checkpoint. We label a small subset of the target validation set and select the checkpoint with the highest accuracy on it. The budget is varied over five levels (B1 to B5 ): 5, 10, . . . , 25 labeled samples for brain MRI, whose target validation set is around 100 samples, and 1% to 5% for the others. Each budget is repeated over ten random trials, and we report the mean. As shown in Table 4, a small budget improves selection steadily, surpassing both Best Val. (78.7) and ensembling (78.4) from B4 (20 labeled samples for brain MRI, 4% for the other scenarios) on average, though a gap to the Oracle still remains.

0.75 0.50 0.25 0

Source-Risk 0.79 0.22 0.35 0.39 0.17 0.48 0.47 0.35 0.45 0.40 IWCV 0.38 0.07 0.04 0.08 0.07 -0.07 0.15 0.13 0.27 0.24 DEV 0.41 0.07 0.09 0.14 0.04 0.05 0.19 0.15 0.24 0.21 DEV-N 0.77 0.39 0.44 0.39 0.38 0.51 0.47 0.43 0.48 0.44 Entropy 0.60 0.06 0.23 0.22 0.03 0.38 0.31 0.21 0.45 0.18 InfoMax 0.79 0.15 0.42 0.62 0.21 0.56 0.60 0.50 0.52 0.48 Corr-C 0.66 0.15 0.34 0.61 0.36 0.51 0.56 0.42 0.52 0.54 MCC (V) 0.76 0.18 0.35 0.40 0.09 0.52 0.46 0.35 0.49 0.34 BNM (V) 0.79 0.09 0.42 0.61 0.19 0.57 0.59 0.50 0.50 0.46 SND 0.48 0.14 0.23 0.38 0.38 0.21 0.31 0.30 0.43 0.47 ClassAMI 0.64 0.24 0.38 0.43 0.02 0.35 0.44 0.36 0.25 0.03 MixVal 0.30 0.10 0.08 0.34 0.15 0.37 0.33 0.19 0.27 0.21 TransScore 0.42 0.12 0.37 0.27 0.11 0.32 0.30 0.43 0.41 0.24

2 3 1 3 IBL hild NA RD DD CT LO S S C O L A C 1 1 2 2 NI- DNI- DNI- DNI- +2 A ld R DD RD LO CT D A A A A I-1 RSN Chi S O L C N AD (a) Per-scenario view (averaged over algorithms)

1.0 0.75 0.50 0.25 0

Spearman's

1.0

Spearman's

Source-Risk 0.73 0.54 0.64 0.68 0.63 0.23 0.06 0.41 0.19 0.13 0.10 IWCV 0.23 0.18 -0.01 0.25 0.23 0.15 0.12 -0.04 0.13 -0.02 0.14 DEV 0.35 0.27 0.07 0.42 0.21 0.05 0.09 0.05 0.12 0.03 -0.04 DEV-N 0.72 0.52 0.63 0.61 0.63 0.08 0.11 0.38 0.37 0.11 0.88 Entropy 0.56 0.35 0.48 0.28 0.52 -0.03 0.04 0.37 0.10 0.08 0.10 InfoMax 0.81 0.49 0.67 0.64 0.54 0.33 0.49 0.76 0.15 0.15 0.26 Corr-C 0.79 0.35 0.74 0.49 0.14 0.54 0.56 0.75 0.19 0.16 0.35 MCC (V) 0.76 0.51 0.69 0.62 0.59 0.14 0.11 0.47 0.13 0.10 0.13 BNM (V) 0.80 0.50 0.71 0.59 0.47 0.31 0.47 0.74 0.14 0.15 0.25 SND 0.59 0.28 0.42 0.40 0.02 0.52 0.51 0.40 0.11 0.05 0.18 ClassAMI 0.61 0.43 0.55 0.48 0.34 0.17 0.16 0.37 0.10 0.05 0.15 MixVal 0.31 0.22 0.32 0.29 0.21 0.27 0.11 0.40 0.10 0.17 0.18 TransScore 0.52 0.27 0.52 0.41 0.46 0.24 0.15 0.49 0.10 0.05 0.11

2A DOC BNM DAN UDA ALN ANN MCC MCD MMD D C Co D AT

AD

(b) Per-algorithm view (averaged over scenarios)

Figure 5. Within-algorithm Spearman correlation (ρ) between validation scores and target accuracy. Blue denotes correlation in the intended direction (darker is stronger), warm red the opposite. No validator is consistently reliable across all scenarios or algorithms, and the most reliable one differs from column to column.

0.80 0.70 0.60 0.50 0.40 0.30

0.80 0.75 0.70 0.65

0.5

0.6

0.7

Source-Risk score

0.8

(a) Within BNM algorithm

= -0.04 p = 0.678

0.85 AD2A ATDOC BNM CDAN DALN DANN MCC MCD MMD

0.80 0.81 0.82 0.83 0.84 0.85 0.86 0.87

Source-Risk score

(b) Across Algorithms

Figure 6. Spearman correlation (ρ) between Source-Risk scores and target accuracy on ADNI-1→ADNI-2. Each point is a checkpoint. (a) Within BNM algorithm, the correlation is moderate. (b) Across all algorithms, it collapses to near zero.

4.5. Discussion Our results show that the main barrier between current UDA and clinical deployment lies less in producing a capable adapted model, which usually exists, than in selecting it without target labels. Under Oracle selection, such a model exists in the Across-Algo pool in most scenarios, often recovering much of the gap between SourceOnly and TargetOnly and at times exceeding both, confirming that adaptation is usually effective. The evaluated validators, however, do not reliably identify this model, leaving a large and structural gap to the best available one. For within-algorithm selection, a reliable validator may exist, but which one is best shifts from algorithm to algorithm and from scenario to scenario, so no fixed choice can be made in advance. For across-algorithm selection, most validators correlate only weakly with target performance, and many rank checkpoints opposite to their intended direction. Note that we do not optimize TargetOnly extensively; in several scenar-

Source-Risk 0.29 0.01 -0.18 0.05 0.19 -0.12-0.02-0.14 0.06 0.40 0.27 IWCV 0.13 -0.07-0.13 0.11 0.12 0.12 0.05 -0.25 0.16 -0.55-0.16 DEV 0.16 0.14 0.47 -0.05 0.23 -0.24-0.27 0.29 -0.30-0.04-0.23 DEV-N 0.34 0.11 -0.23 0.07 0.07 -0.09-0.06-0.22-0.05 0.40 0.95 Entropy -0.27-0.26-0.03-0.37-0.05-0.25 0.12 -0.18 0.31 -0.45-0.65 InfoMax 0.16 -0.31 0.49 0.02 -0.38-0.11 0.47 0.56 0.23 -0.50-0.59 Corr-C -0.07-0.37-0.13-0.35-0.47-0.22 0.33 0.26 -0.04-0.53-0.66 MCC (V) 0.21 -0.31-0.27-0.15-0.30-0.28 0.08 -0.03 0.01 -0.46-0.62 BNM (V) 0.15 -0.29-0.40-0.51-0.58-0.16 0.48 0.58 0.20 -0.51-0.59 SND 0.12 -0.07 0.21 -0.19-0.17 0.23 0.14 -0.23 0.18 0.32 0.27 ClassAMI 0.24 0.28 0.43 0.13 0.25 0.07 0.11 0.28 0.28 -0.13-0.27 MixVal -0.05-0.03-0.01-0.01-0.09-0.27 0.41 0.16 0.23 -0.14-0.49 TransScore 0.01 -0.34 0.22 -0.17-0.37-0.25 0.53 0.40 -0.12-0.58-0.78 AD

1.0 0.75 0.50 0.25 0

Spearman's

0.90

= 0.57 p < 0.001

Target accuracy

Target accuracy

0.90

2 3 1 3 IBL hild NA RD DD CT LO S S C O L A C 1 1 2 2 NI- DNI- DNI- DNI- +2 A ld R DD RD LO CT A A A I-1 RSN Chi S O L C N AD

Figure 7. Across-algorithm Spearman correlation (ρ) between validation scores and target accuracy. Blue denotes correlation in the intended direction (darker is stronger), warm red the opposite. Most cells are weak or negative, showing that most validators are unreliable in this across-algorithm setting.

ios it falls below the per-algorithm Oracle average (Avg.) or even SourceOnly, which may stem from training on the target domain alone, with less labeled data than the combined source and target signal exploited by adaptation, and from target class imbalance (e.g., AIBL). The unreliability of evaluated validators also explains an observation that might otherwise question the value of adaptation: under many validators, the selected adapted model performs worse than the SourceOnly baseline (e.g., MCC with InfoMax in Table 2). We interpret this as a selection problem, not adaptation alone: a capable adapted model often exists (the Oracle confirms it), but an unreliable validator may fail to find it and sometimes picks one worse than

Table 4. Two strategies for narrowing the selection gap (target accuracy, %). Ens. ensembles the checkpoints selected by all validators across all algorithms. Small Labeling Budget selects the best checkpoint on a small labeled target subset, at five levels (B1 to B5 ; 5–25 samples for brain MRI, 1–5% otherwise). On average, ensembling matches Best Val. (the best validator under the across-algorithm pool), and the labeling budget surpasses both from around B4 , though a gap to the Oracle remains for both. Ens. Scenario ADNI-1→ADNI-2 ADNI-1→ADNI-3 ADNI-2→ADNI-1 ADNI-2→ADNI-3 ADNI-1+2→AIBL RSNA→Child CXR Child CXR→RSNA LDD→CRD CRD→LDD SLO→OCT OCT→SLO Mean

87.6 87.6 89.0 87.0 88.8 80.7 74.2 81.0 57.1 62.9 66.8 78.4

Small Labeling Budget B1 78.6 78.7 82.8 79.5 81.6 76.6 74.3 80.2 75.2 58.9 60.0 75.1

B2 83.0 80.8 84.1 79.9 82.9 78.4 75.8 81.6 77.9 61.2 60.9 76.9

B3 84.0 81.1 84.6 83.5 82.9 81.5 76.3 82.0 80.2 60.8 63.5 78.2

B4 86.5 81.2 87.4 81.3 85.1 83.1 76.9 82.2 79.8 61.3 64.5 79.0

B5 87.5 83.3 87.7 83.6 86.0 85.0 76.9 82.1 80.4 62.6 64.4 80.0

Best Val. Oracle

85.1 84.5 86.4 85.0 89.0 79.0 74.9 81.9 72.5 60.1 67.3 78.7

93.2 94.2 92.5 92.1 93.8 89.5 78.3 83.6 82.7 65.6 67.4 84.8

not adapting. On the other hand, in Table 2, under Oracle selection ATDOC and MCD fall below SourceOnly, so the limitation here lies also in adaptation. This reflects either our relatively small hyperparameter set (λ and the training iteration) or the unsuitability of these algorithms for this transfer. It nonetheless reinforces the need for reliable validators: at the across-algorithm selection scale, a good validator should recognize and avoid such negative transfer. We also notice that validator reliability, measured by Spearman correlation ρ, and the target accuracy of the selected model do not always align. As shown in Figure 7 and the Across-Algo column of Table 2, for ADNI1→ADNI-2, ClassAMI has a higher correlation than InfoMax (ρ = 0.24 vs. 0.16) yet selects a less accurate model (80.8% vs. 83.2%). This is because the two measure different things: ρ captures how well a validator orders all checkpoints, whereas selection depends on its single top-scored checkpoint, so a validator with lower ρ can still place a good checkpoint at the top. This suggests two complementary needs: a reliable validator to produce a good ranking, and potentially a further step to identify the truly best one. How Far from Clinical Deployment? The answer is twosided. Under Oracle selection, a capable model exists in the checkpoint pool in most scenarios, so adaptation itself is often effective. What is missing is the ability to identify that model without target labels. Deployment therefore depends on the complete pipeline, not adaptation alone: selection remains a largely unsolved step, and closing it would bring much of current UDA closer to clinical use. In some scenarios the achievable accuracy itself remains low, and for some algorithms even the Oracle-selected model falls

below SourceOnly, so better adaptation algorithms are still needed. Yet even there, and for any future algorithm, a capable model still has to be identified. Improving algorithms raises what is achievable, while reliable selection determines what is actually reached at deployment. The two strategies in Section 4.4 recover part of this selection gap but do not close it, and both carry practical costs. Ensembling requires training all algorithms across their hyperparameters, since which suits a given transfer is not known in advance, and multiple forward passes per image at inference; it is computationally costly and can even harm performance under naive aggregation when poor checkpoints are included (e.g., CRD→LDD). The labeling budget, though small in relative terms (5 to 25 volumes for brain MRI, 1% to 5% otherwise), carries a real annotation cost: even a single brain MRI volume is time-consuming, and 1% of a large dataset can amount to hundreds of images (e.g., over 260 for RSNA), more so for richer tasks such as segmentation or detection. Active selection [21, 32, 40] may match these gains with fewer labels. Overall, reliable label-free selection remains the central open problem for clinically deployable UDA. Limitations and Future Work. Our study covers binary classification; extending it to multi-class, multi-label, segmentation, or detection tasks, where the prediction structure differs and selection criteria may need adapting, is a valuable future direction. As classification lies at the core of these more complex tasks, label-free selection may not become easier there, and the gap could even widen. We evaluate by balanced accuracy, while other metrics such as sensitivity and specificity also matter clinically, and whether validators rank checkpoints well under them remains open.

5. Conclusion In this work, we evaluated the complete UDA pipeline in medical imaging, adaptation and label-free selection together, under clinical deployment conditions. Spanning eleven clinically relevant cross-domain scenarios from nine datasets across brain MRI, CXR, and retinal imaging, ten UDA algorithms, and 13 validators, our evaluation covers over 80,000 trained checkpoints. By considering adaptation and selection as a whole, we find that a capable adapted model often exists, but selecting it without target labels is difficult: the validator-selected models leave a large and structural gap to the best available one, as no evaluated validator is consistently reliable, for either within-algorithm or across-algorithm selection. Two strategies, ensembling and a small target-labeling budget, narrow this gap but do not close it entirely, and both have limitations. Overall, deployable UDA depends on both adaptation and selection working together; addressing the less explored selection step could bring much of current UDA closer to clinical use.

Acknowledgments This study was funded by the German Research Foundation DFG (Project: KEMAI, GRK 3012 – 520750254) and by the German Federal Ministry of Research, Technology and Space BMFTR as part of the University Medicine Network 3.0 (Project: RACOON, 01KX2524).

References [1] Covid-19 radiography database. https : / / www . kaggle . com / datasets / tawsifurrahman / covid19-radiography-database. 3 [2] Lungs disease dataset (4 types). https : / / www . kaggle . com / datasets / omkarmanohardalvi / lungs-disease-dataset-4-types. 3 [3] Sireesha Chamarthi, Katharina Fogelberg, Titus J Brinker, et al. Mitigating the influence of domain shift in skin lesion classification: A benchmark study of unsupervised domain adaptation methods. Informatics in Medicine Unlocked, 44: 101430, 2024. 1, 2 [4] Lin Chen, Huaian Chen, Zhixiang Wei, et al. Reusing the task-specific classifier as a discriminator: Discriminator-free adversarial domain adaptation. In CVPR, pages 7181–7190, 2022. 3 [5] Anirudh Choudhary, Li Tong, Yuanda Zhu, et al. Advancing medical imaging informatics by deep learning-based domain adaptation. Yearbook of medical informatics, 29(01):129– 138, 2020. 1, 2 [6] Shuhao Cui, Shuhui Wang, Junbao Zhuo, et al. Towards discriminability and diversity: Batch nuclear-norm maximization under label insufficient situations. In CVPR, pages 3941–3950, 2020. 3 [7] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 3 [8] Reuben Dorent, Aaron Kujawa, Marina Ivory, Spyridon Bakas, Nicola Rieke, Samuel Joutard, Ben Glocker, Jorge Cardoso, Marc Modat, Kayhan Batmanghelich, et al. Crossmoda 2021 challenge: Benchmark of cross-modality domain adaptation techniques for vestibular schwannoma and cochlea segmentation. Medical Image Analysis, 83:102628, 2023. 1, 2 [9] Kathryn A Ellis, Ashley I Bush, David Darby, et al. The australian imaging, biomarkers and lifestyle (aibl) study of aging: methodology and baseline characteristics of 1112 individuals recruited for a longitudinal study of alzheimer’s disease. International psychogeriatrics, 21(4):672–687, 2009. 3 [10] Linus Ericsson, Da Li, and Timothy Hospedales. Better practices for domain adaptation. In International Conference on Automated Machine Learning, pages 4–1. PMLR, 2023. 2 [11] Yangqin Feng, Zizhou Wang, Xinxing Xu, Yan Wang, Huazhu Fu, Shaohua Li, Liangli Zhen, Xiaofeng Lei, Yingnan Cui, Jordan Sim Zheng Ting, et al. Contrastive domain adaptation with consistency match for automated pneumonia diagnosis. Medical Image Analysis, 83:102664, 2023. 1, 3

[12] Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In ICML, pages 1180–1189. PMLR, 2015. 2, 3 [13] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, et al. Domain-adversarial training of neural networks. Journal of machine learning research, 17(59):1–35, 2016. 3 [14] Hao Guan and Mingxia Liu. Domain adaptation for medical image analysis: a survey. IEEE Transactions on Biomedical Engineering, 69(3):1173–1185, 2021. 1, 2 [15] Hao Guan, Yunbi Liu, Erkun Yang, et al. Multi-site mri harmonization via attention-guided deep domain adaptation for brain disorder identification. Medical image analysis, 71: 102076, 2021. 1, 3 [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, et al. Deep residual learning for image recognition. In CVPR, pages 770–778, 2016. 3 [17] Dapeng Hu, Jian Liang, Jun Hao Liew, Chuhui Xue, Song Bai, and Xinchao Wang. Mixed samples as probes for unsupervised model selection in domain adaptation. Advances in Neural Information Processing Systems, 36:37923–37941, 2023. 2, 3 [18] Dapeng Hu, Romy Luo, Jian Liang, et al. Towards reliable model selection for unsupervised domain adaptation: An empirical study and a certified baseline. NeurIPS, 37: 135883–135903, 2024. 2, 6 [19] Clifford R Jack Jr, Matt A Bernstein, Nick C Fox, et al. The alzheimer’s disease neuroimaging initiative (adni): Mri methods. Journal of Magnetic Resonance Imaging: An Official Journal of the International Society for Magnetic Resonance in Medicine, 27(4):685–691, 2008. 3 [20] Ying Jin, Ximei Wang, Mingsheng Long, et al. Minimum class confusion for versatile domain adaptation. In ECCV, pages 464–480. Springer, 2020. 3 [21] Justin Kay, Grant Van Horn, Subhransu Maji, et al. Consensus-driven active model selection. In ICCV, pages 4594–4604, 2025. 8 [22] Daniel S Kermany, Michael Goldbaum, Wenjia Cai, et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. cell, 172(5):1122–1131, 2018. 3 [23] Suruchi Kumari and Pravendra Singh. Deep learning for unsupervised domain adaptation in medical imaging: Recent advancements and future perspectives. Computers in Biology and Medicine, 170:107912, 2024. 1, 2 [24] Yanis Lalou, Théo Gnassounou, Antoine Collas, et al. Skada-bench: Benchmarking unsupervised domain adaptation methods with realistic validation on diverse modalities. arXiv preprint arXiv:2407.11676, 2024. 2 [25] Yuemeng Li and Yong Fan. Medical image segmentation with domain adaptation: A survey. arXiv preprint arXiv:2311.01702, 2023. 1, 2 [26] Jian Liang, Dapeng Hu, and Jiashi Feng. Domain adaptation with auxiliary target domain-oriented classifier. In CVPR, pages 16632–16642, 2021. 3 [27] Jingsong Liu, Han Li, Chen Yang, Michael Deutges, Ario Sadafi, Xin You, Katharina Breininger, Nassir Navab, and Peter J Schüffler. Hasd: hierarchical adaption for pathology slide-level domain-shift. In International Conference on

Medical Image Computing and Computer-Assisted Intervention, pages 332–342. Springer, 2025. 1 [28] Wentao Liu, Zhiwei Ni, Qian Chen, and Liping Ni. Attention-guided partial domain adaptation for automated pneumonia diagnosis from chest x-ray images. IEEE Journal of Biomedical and Health Informatics, 27(12):5848–5859, 2023. 3 [29] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, et al. A convnet for the 2020s. CVPR, 2022. 5 [30] Mingsheng Long, Yue Cao, Jianmin Wang, et al. Learning transferable features with deep adaptation networks. In ICML, pages 97–105. PMLR, 2015. 3 [31] Mingsheng Long, Zhangjie Cao, Jianmin Wang, et al. Conditional adversarial domain adaptation. NeurIPS, 31, 2018. 3 [32] Mitsuru Matsuura and Satoshi Hara. Active model selection: A variance minimization approach. In NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World, 2023. 8 [33] Pietro Morerio, Jacopo Cavazza, and Vittorio Murino. Minimal-entropy correlation alignment for unsupervised deep domain adaptation. arXiv preprint arXiv:1711.10288, 2017. 3 [34] Kevin Musgrave, Serge Belongie, and Ser-Nam Lim. Unsupervised domain adaptation: A reality check. arXiv preprint arXiv:2111.15672, 2021. 1, 2, 3 [35] Kevin Musgrave, Serge Belongie, and Ser-Nam Lim. Three new validators and a large-scale benchmark ranking for unsupervised domain adaptation. arXiv preprint arXiv:2208.07360, 2022. 2, 3 [36] Bin Pu, Liwen Wang, Jiewen Yang, et al. M3-uda: A new benchmark for unsupervised domain adaptive fetal cardiac structure detection. In CVPR, pages 11621–11630, 2024. 1, 2 [37] Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, et al. Maximum classifier discrepancy for unsupervised domain adaptation. In CVPR, pages 3723–3732, 2018. 3 [38] Kuniaki Saito, Donghyun Kim, Piotr Teterwak, et al. Tune it the right way: Unsupervised validation of domain adaptation via soft neighborhood density. In ICCV, pages 9184–9193, 2021. 2, 3 [39] Gita Sarafraz, Armin Behnamnia, Mehran Hosseinzadeh, et al. Domain adaptation and generalization on functional medical images: A systematic survey. arXiv preprint arXiv:2212.03176, 2022. 1, 2 [40] Christoph Sawade, Niels Landwehr, and Tobias Scheffer. Active comparison of prediction models. Advances in neural information processing systems, 25, 2012. 8 [41] Boris Shirokikh, Anvar Kurmukov, Mariia Donskova, et al. M3da: Benchmark for unsupervised domain adaptation in 3d medical image segmentation. arXiv preprint arXiv:2502.17029, 2025. 1, 2 [42] Zixian Su, Jingwei Guo, Xi Yang, Qiufeng Wang, Frans Coenen, Amir Hussain, and Kaizhu Huang. Navigating distribution shifts in medical image analysis: A survey. arXiv preprint arXiv:2411.05824, 2024. 1, 2

[43] Masashi Sugiyama, Matthias Krauledat, and Klaus-Robert Müller. Covariate shift adaptation by importance weighted cross validation. JMLR, 8(5), 2007. 1, 2, 3 [44] Nurjahan Sultana, Wenqi Lu, Xinqi Fan, et al. Domain adaptation for skin lesion: Evaluating real-world generalisation. In CVPRW, pages 3433–3443, 2025. 1, 2 [45] Yu Tian, Congcong Wen, Min Shi, Muhammad Muneeb Afzal, Hao Huang, Muhammad Osama Khan, Yan Luo, Yi Fang, and Mengyu Wang. Fairdomain: Achieving fairness in cross-domain medical image segmentation and classification. In European Conference on Computer Vision, pages 251–271. Springer, 2024. 3 [46] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In International conference on machine learning, pages 10347–10357. PMLR, 2021. 5 [47] Hugo Touvron, Piotr Bojanowski, Mathilde Caron, et al. Resmlp: Feedforward networks for image classification with data-efficient training. IEEE TPAMI, 45(4):5314–5321, 2022. 5 [48] Weijie Tu, Weijian Deng, Tom Gedeon, et al. Assessing model out-of-distribution generalization with softmax prediction probability baselines and a correlation method. 2, 3 [49] Xiaosong Wang, Yifan Peng, Le Lu, et al. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In CVPR, pages 2097–2106, 2017. 3 [50] Qingsong Xie, Yuexiang Li, Nanjun He, Munan Ning, Kai Ma, Guoxing Wang, Yong Lian, and Yefeng Zheng. Unsupervised domain adaptation for medical image segmentation by disentanglement learning and self-training. IEEE Transactions on Medical Imaging, 43(1):4–14, 2022. 1 [51] Hancun Yang, Henjun Guo, Guanlin Liu, Linkuan Zhou, Junlin Xu, Leyi Wei, Ran Su, and Qiangguo Jin. A survey on unsupervised domain adaptation in medical imaging: Methods, dataset, and future outlook. Applied Soft Computing, page 115314, 2026. 1, 2 [52] Jianfei Yang, Hanjie Qian, Yuecong Xu, Kai Wang, and Lihua Xie. Can we evaluate domain adaptation models without target-domain labels? In International Conference on Learning Representations, pages 35061–35081, 2024. 2, 3 [53] Kaichao You, Ximei Wang, Mingsheng Long, et al. Towards accurate model selection in deep unsupervised domain adaptation. In ICML, pages 7124–7133. PMLR, 2019. 1, 2, 3 [54] Yifan Zhang, Ying Wei, Qingyao Wu, et al. Collaborative unsupervised domain adaptation for medical image diagnosis. IEEE TIP, 29:7834–7844, 2020. 1, 3

6. Supplementary Material In this supplementary material, we provide the full results of our complete unsupervised domain adaptation (UDA) pipeline for each evaluated clinically relevant cross-domain scenario in Section 6.1. In addition, we provide the full within-algorithm validator reliability analyses, reported per algorithm and per scenario, in Section 6.2.

6.1. Full Results on the Complete UDA Pipeline This section provides the full per-scenario complete UDA results. Each table reports, for one cross-domain scenario, the target accuracy of the checkpoint selected by every validator (rows) for every algorithm (columns), along with Oracle, SourceOnly, and TargetOnly references. For each scenario, the first table reports mean±std and the immediately following table reports median with 95% CI (e.g., Table 5 is mean±std and Table 6 is median with 95% CI, and analogously for the remaining ten scenarios). Tables 5–25 cover the eleven clinically relevant transfer scenarios. Brain MRI: ADNI-1→ADNI-2 (Table 5), ADNI-1→ADNI-3 (Table 7), ADNI-2→ADNI-1 (Table 9), ADNI-2→ADNI-3 (Table 11), ADNI-1+2→AIBL (Table 13). Chest X-ray: RSNA→Child CXR (Table 15), Child CXR→RSNA (Table 17), LDD→CRD (Table 19), CRD→LDD (Table 21). Retinal: OCT→SLO (Table 23), SLO→OCT (Table 25). We also assess whether adaptation works in principle and the size of the selection gap using the median with 95% confidence intervals, shown in Figures 8 and 9. The conclusions match those drawn from the mean ± standard deviation in the main paper.

6.2. Full Validator Reliability Analyses for WithinAlgorithm Selection This section provides the full per-scenario validator reliability analyses. Each table reports, for one cross-domain scenario, the within-algorithm Spearman correlation (ρ) between each validation score and target accuracy, computed over the checkpoints of each algorithm. Positive values always indicate ranking in the intended direction. The ρ values here are means across folds or random seeds, whereas the example scatter plots of the main paper show a single fold or random seed for readability, so individual cells may not match exactly. Tables 27–37 cover the eleven clinically relevant transfer scenarios. Brain MRI: ADNI-1→ADNI-2 (Table 27), ADNI-1→ADNI-3 (Table 28), ADNI-2→ADNI-1 (Table 29), ADNI-2→ADNI3 (Table 30), ADNI-1+2→AIBL (Table 31). Chest X-ray: RSNA→Child CXR (Table 32), Child CXR→RSNA (Table 33), LDD→CRD (Table 34), CRD→LDD (Table 35). Retinal: OCT→SLO (Table 36), SLO→OCT (Table 37).

Across-Algo

TargetOnly

80.2 81.7 84.8 88.0

Target Accuracy (\%)

Avg.

SourceOnly

100 90 80 70 60 50 40 AD

1 NI-

2 AD

1 NI-

3 AD

2 NI-

1 AD

2 NI-

3 A

+2 I-1 N D

AIB

L NA RS

ild Ch

ild Ch

NA RS

LD

D

D CR

D CR

LD

D SLO

OC

T OC

T

SLO

Me

an

Figure 8. Target accuracy under Oracle selection across UDA scenarios, reported as the median with 95% confidence intervals. Bars show SourceOnly (no adaptation), the per-algorithm average (Avg.), Across-Algo (selection over all algorithms’ pooled checkpoints), and TargetOnly (labeled-target). Across-Algo exceeds SourceOnly in all scenarios and approaches TargetOnly, indicating a capable adapted model usually exists. Error bars: 95% confidence interval.

Target Accuracy (\%)

100

Best Validator

=9.9%

Best Pair

Oracle

90

=5.1%

80 70 60 50 40 A

I-1 DN

2 A

I-1 DN

3 A

I-2 DN

1 A

I-2 DN

3 AD

2 1+ NI-

AIB

L NA RS

ild Ch

ild Ch

NA RS

L

DD

D CR

D CR

LD

D SLO

OC

T O

CT

SLO

an

Me

Figure 9. Target accuracy across UDA scenarios under three selections (%), reported as the median with 95% confidence intervals. Bars show the accuracy of the model selected by the best validator over the across-algorithm pool (Best Validator), by the best algorithm– validator combination (Best Pair), and by the across-algorithm Oracle. The first two fall below the Oracle in every scenario, with the gap up to 9.9 points and averaging 5.1 (Mean). Best Validator and Best Pair differ from scenario to scenario and are identified using target labels, so they cannot be chosen in advance; the gap realized at deployment can only be larger. Error bars: 95% confidence interval. Table 5. Complete UDA results on ADNI-1→ADNI-2 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 88.2±2.0 80.9±5.8 76.8±9.7 78.1±11 80.9±5.8 60.9±15 84.2±2.1 84.6±4.1 84.2±2.7 84.7±2.1 81.3±5.8 73.9±12 75.9±12 81.3±1.0

MMD 89.3±4.0 82.1±3.1 80.3±5.3 83.1±2.0 81.7±2.8 58.8±20 84.8±5.7 80.4±4.3 84.4±6.4 84.8±5.7 71.5±13 83.4±4.1 57.4±6.4 77.5±11

DANN 90.0±4.0 81.5±3.7 74.6±12 83.9±5.2 84.9±2.0 56.7±15 87.3±4.4 83.4±3.6 84.9±3.3 87.5±4.0 74.8±8.7 80.0±2.6 77.9±8.7 85.1±4.5

CDAN 89.9±3.0 81.1±8.0 81.1±6.4 82.0±6.7 81.8±8.4 57.0±16 86.0±1.5 83.1±3.6 84.5±4.0 84.9±3.8 83.0±4.0 79.2±5.8 69.8±13 86.7±2.3

DALN 91.0±3.5 80.8±9.1 80.1±8.4 85.2±6.2 80.8±9.1 50.0±15 88.2±3.8 84.2±6.7 86.4±3.7 87.2±2.2 84.2±1.6 85.5±4.9 66.8±16 85.2±5.9

MCC 91.0±1.7 83.3±3.2 76.0±10 76.9±11 83.5±3.3 60.2±15 81.8±9.6 77.8±5.3 80.4±8.3 81.8±9.6 81.7±7.5 78.2±5.9 82.1±8.7 80.4±9.9

BNM 89.9±4.4 84.1±4.1 76.4±3.8 75.9±7.3 82.9±3.4 57.5±16 83.5±6.9 80.4±3.4 85.9±3.6 83.5±6.9 79.6±4.4 78.5±8.6 71.3±16 76.5±11

ATDOC 85.7±9.8 73.2±12 72.3±11 68.0±12 76.0±12 57.9±13 78.9±10 78.3±8.8 80.7±13 78.9±10 78.3±10 75.8±12 70.2±15 72.7±10

MCD 87.4±4.3 81.5±6.2 80.1±11 74.7±14 81.0±6.2 70.1±19 83.3±5.6 84.1±4.4 83.1±3.7 83.3±5.6 63.7±19 82.2±5.6 71.8±16 82.4±3.7

AD2A Avg. Across-Algo TargetOnly 92.0±0.93 89.5±2.0 93.2±2.0 90.9±2.4 87.5±2.3 83.5±2.9 84.3±2.8 76.7±4.4 82.5±8.6 79.4±13 87.6±1.9 85.1±1.3 56.9±15 50.0±10 86.3±2.2 83.2±7.2 85.6±5.1 79.6±3.0 87.1±3.7 80.4±2.3 86.3±2.2 81.8±9.6 72.7±16 80.8±3.6 80.7±8.7 80.7±8.7 60.0±16 61.1±18 86.3±2.8 79.9±10 -

Table 6. Complete UDA results on ADNI-1→ADNI-2 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 5. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 89.3±2.4 80.7±7.9 75.8±13 83.3±13 53.2±17 85.0±2.5 85.0±4.5 84.8±3.7 85.7±2.5 79.0±14 83.3±7.2 80.7±7.9 80.3±15 81.3±0.93

MMD 88.6±5.4 82.2±3.8 80.7±7.0 82.2±3.8 50.0±0.85 82.3±7.2 82.5±4.9 84.2±8.6 82.3±7.2 83.3±4.8 75.7±18 82.2±3.4 55.7±8.5 77.3±14

DANN 89.1±5.2 82.0±4.5 70.7±14 83.7±6.7 50.0±17 86.5±6.0 82.3±4.3 84.5±4.3 86.5±5.2 79.7±3.6 77.7±10 85.2±2.8 78.0±11 86.2±5.5

CDAN 89.8±3.7 82.7±10.0 80.7±8.4 83.7±8.4 50.0±18 86.8±1.9 82.7±4.7 85.6±4.9 86.8±4.6 75.9±6.8 81.3±4.7 82.7±10.0 71.3±16 86.4±3.2

DALN 91.7±4.8 82.0±12 83.7±10 87.5±6.4 50.0±0.0 86.7±4.7 87.3±7.7 86.7±4.6 86.7±2.4 84.5±6.3 84.2±2.2 82.0±12 65.2±18 83.8±6.9

MCC 91.3±1.9 83.8±4.1 79.2±12 80.8±13 50.0±17 83.3±12 79.3±6.5 83.3±11 83.3±12 78.8±8.0 84.1±9.5 84.4±4.1 83.2±10 83.3±11

BNM 90.3±4.8 81.4±4.1 76.5±4.8 79.4±8.9 50.0±18 84.3±8.5 82.0±4.2 86.7±4.5 84.3±8.5 79.6±11 79.6±5.0 81.4±4.1 77.7±18 81.8±13

AD2A 92.3±1.2 87.3±3.0 86.9±7.5 86.9±9.2 50.0±17 85.7±2.7 87.9±6.1 87.5±4.7 85.7±2.7 82.8±5.4 73.8±20 86.9±2.4 68.8±20 86.0±3.2

ATDOC 89.3±12 74.6±14 74.6±14 66.0±15 50.0±19 84.5±12 81.7±11 85.2±17 84.5±12 81.7±14 84.4±10 74.6±14 67.6±19 74.5±13

MCD Avg. Across-Algo TargetOnly 87.7±5.4 90.0±1.5 92.8±2.6 91.7±3.0 81.1±7.3 83.8±4.0 85.8±13 79.2±5.2 77.7±18 81.6±17 77.3±19 50.0±0.0 81.1±7.4 83.3±9.0 81.4±5.0 79.3±3.8 81.4±4.7 83.3±11 81.1±7.4 83.3±12 83.4±7.6 79.7±10 51.0±19 81.3±4.9 78.8±7.1 85.0±1.7 79.8±20 53.0±19 81.1±5.0 83.3±11 -

Table 7. Complete UDA results on ADNI-1→ADNI-3 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 90.2±3.7 82.0±9.5 70.0±10 73.9±12 83.3±7.1 67.8±13 79.0±5.9 79.7±6.7 79.1±7.9 78.2±7.3 79.5±6.2 73.8±6.0 78.2±11 81.9±6.1

MMD 87.3±4.5 79.0±8.9 76.7±11 69.2±15 79.2±9.7 66.8±15 72.0±14 70.8±5.1 74.9±14 75.0±15 55.1±19 71.6±5.0 64.9±18 78.5±8.2

DANN 90.7±3.1 83.8±5.3 79.6±4.9 84.2±3.3 83.8±5.3 57.2±15 79.7±5.0 76.8±1.5 84.6±2.3 79.7±5.0 80.2±3.6 77.5±4.5 73.3±9.1 83.1±3.1

CDAN 93.5±3.5 82.7±8.8 84.1±5.9 85.0±6.1 82.7±8.8 57.3±14 78.2±4.1 75.9±3.7 84.5±4.0 78.2±4.1 76.0±6.5 75.4±6.0 78.1±12 84.6±7.2

DALN 90.8±4.1 82.8±7.6 74.7±15 81.3±7.6 82.4±5.0 63.2±18 79.4±6.8 74.5±4.3 79.9±6.3 79.4±6.8 82.8±6.9 76.8±4.3 66.7±14 79.8±2.6

MCC 90.8±3.3 76.8±5.1 74.0±12 80.4±6.9 77.3±5.7 66.1±17 70.6±7.9 71.3±8.3 75.9±9.8 70.6±7.9 84.6±5.1 73.2±5.3 82.3±4.0 69.6±7.1

BNM 89.0±5.6 78.0±7.1 56.5±9.1 72.7±15 76.1±8.5 64.0±11 72.7±6.5 69.9±5.0 72.4±4.2 72.7±6.5 79.6±8.9 71.2±9.0 72.6±14 58.3±19

ATDOC 86.9±5.8 77.9±10 69.3±11 74.5±12 76.6±12 64.1±18 72.4±12 67.1±13 73.8±18 74.3±13 68.6±19 69.4±8.8 72.7±11 79.2±4.0

MCD 88.6±3.5 82.9±5.1 79.2±6.7 71.5±12 75.2±9.8 62.5±17 80.2±5.8 76.1±4.8 85.1±3.9 80.9±4.9 59.9±15 77.4±3.2 53.4±6.0 78.8±3.9

AD2A Avg. Across-Algo TargetOnly 91.1±4.3 89.9±2.0 94.2±3.1 90.2±4.6 82.4±7.6 82.9±5.2 75.3±12 64.9±15 77.4±14 84.2±7.8 83.2±4.4 84.5±5.0 50.3±0.72 50.0±0.0 80.2±5.7 70.6±7.9 72.1±11 68.7±8.5 83.5±7.6 75.9±9.8 80.2±5.7 70.6±7.9 75.7±19 79.2±9.7 74.5±2.0 76.2±3.9 59.8±12 64.7±13 77.0±12 71.1±8.0 -

Table 8. Complete UDA results on ADNI-1→ADNI-3 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 7. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 88.5±4.5 82.8±12 64.3±11 73.1±14 68.4±17 77.0±7.3 79.0±8.0 77.0±10 75.5±8.9 74.2±7.9 78.0±7.6 82.8±7.8 80.9±14 83.4±7.8

MMD 86.2±5.1 79.2±10 78.9±15 73.6±17 64.3±19 73.3±18 72.6±6.3 80.7±18 82.4±18 72.6±6.2 53.2±24 77.6±11 57.7±18 76.5±10

DANN 90.0±3.5 85.4±6.7 81.5±5.3 83.8±4.2 50.0±17 80.8±6.6 76.7±1.8 84.2±3.1 80.8±6.6 78.7±5.5 81.5±4.3 85.4±6.7 70.8±11 82.6±4.1

CDAN 93.8±4.7 80.1±11 86.1±7.6 83.7±7.1 50.0±16 79.7±5.0 76.7±4.0 86.0±4.5 79.7±5.0 74.3±7.5 74.9±8.8 80.1±11 80.7±17 88.4±8.6

DALN 91.6±5.5 86.1±9.8 77.8±19 79.5±10 51.2±20 77.4±8.7 76.0±5.2 76.8±7.4 77.4±8.7 77.1±5.4 83.8±8.8 80.8±6.4 69.4±16 80.1±3.2

MCC 90.7±4.4 78.5±6.2 79.4±15 84.1±8.2 64.5±19 72.4±11 72.4±11 72.2±12 72.4±11 74.9±6.7 84.1±5.9 78.5±7.6 81.6±4.9 71.9±9.2

BNM 87.8±6.9 79.1±8.7 56.0±11 74.9±20 69.6±13 75.0±8.2 70.7±6.3 72.8±5.3 75.0±8.2 69.3±11 78.2±10 78.0±11 75.4±17 63.1±20

AD2A 90.8±5.0 84.7±9.6 75.5±17 81.0±18 50.0±0.81 79.3±6.8 74.6±14 79.3±7.9 79.3±6.8 74.0±2.3 80.0±24 84.7±4.5 54.8±15 79.3±15

ATDOC 86.0±7.4 80.5±13 69.8±14 80.4±14 66.7±23 76.5±14 72.6±14 78.1±23 78.1±17 72.8±11 73.7±24 80.5±16 74.8±15 78.7±5.1

MCD Avg. Across-Algo TargetOnly 87.6±4.6 89.4±2.6 94.8±3.8 90.7±6.5 84.9±6.3 81.4±6.4 76.1±7.6 60.5±16 73.3±16 84.1±10 51.0±17 50.0±0.0 78.4±6.9 72.4±11 77.9±6.1 71.6±11 86.6±4.9 72.2±12 78.4±5.2 72.4±11 77.9±4.1 77.1±5.2 55.6±18 80.0±11 77.9±11 84.9±6.4 56.1±7.2 65.3±16 80.1±4.3 72.4±10 -

Table 9. Complete UDA results on ADNI-2→ADNI-1 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 89.3±4.0 79.5±8.1 81.2±6.3 82.8±2.9 80.7±8.3 63.2±19 86.0±5.1 84.5±5.6 85.5±5.3 86.4±4.9 84.4±5.1 78.5±8.0 83.2±6.5 82.4±5.7

MMD 89.8±4.2 82.6±5.1 80.6±2.2 79.3±2.0 81.4±3.6 60.8±15 83.1±6.4 81.5±6.5 85.8±3.1 85.2±4.4 77.5±8.3 80.1±3.5 73.9±12 87.3±3.3

DANN 91.3±2.2 85.7±3.9 80.6±3.6 80.5±6.9 84.7±3.7 62.5±17 87.2±3.3 84.9±3.9 85.2±4.7 87.2±3.3 86.2±5.0 81.4±4.6 71.0±13 85.1±6.2

CDAN 90.8±2.7 84.8±5.9 81.8±6.0 85.3±3.3 84.8±5.7 63.8±19 85.9±3.7 85.5±5.5 84.7±3.0 86.2±3.8 85.4±3.9 82.5±3.1 82.3±4.0 83.7±3.9

DALN 91.3±2.9 84.2±3.6 76.8±9.1 86.7±3.0 83.3±2.9 57.9±14 86.0±2.9 82.2±6.0 85.9±3.7 85.8±2.7 82.9±3.8 81.5±5.0 72.7±17 84.3±4.0

MCC 91.3±2.8 86.3±5.6 67.3±15 82.5±4.6 85.8±3.7 64.5±13 85.4±3.0 86.0±5.1 75.3±11 85.4±3.0 82.6±2.1 82.4±4.1 73.4±14 86.4±3.5

BNM 91.1±2.3 83.7±4.5 79.7±3.0 85.2±1.9 82.5±2.5 69.8±18 86.4±5.1 81.7±3.0 80.9±7.4 85.5±4.3 82.3±4.4 76.8±6.3 78.9±7.8 85.8±3.5

ATDOC 85.0±4.1 77.0±4.5 75.3±7.7 74.8±5.2 77.0±4.5 66.0±15 64.9±5.4 80.4±5.2 80.1±6.3 65.7±7.2 79.5±0.83 75.0±3.1 75.0±13 79.5±3.2

MCD 85.2±5.3 77.7±8.2 70.0±9.8 68.0±6.6 76.8±9.0 70.0±14 63.6±12 77.0±8.4 77.4±7.2 63.7±6.2 63.7±15 74.6±3.6 60.8±16 76.3±7.9

AD2A Avg. Across-Algo TargetOnly 91.0±1.9 89.6±2.5 92.5±1.8 91.0±4.2 88.9±3.6 79.5±6.3 77.7±15 75.3±7.7 70.8±19 84.9±3.3 87.9±5.3 79.5±6.3 65.0±20 55.8±13 87.6±3.8 85.7±2.9 86.6±2.9 82.4±5.5 85.5±3.9 80.1±6.3 86.5±3.5 65.1±7.0 80.3±6.2 81.8±3.0 80.6±2.6 78.8±6.6 68.8±14 63.3±11 83.9±1.8 86.4±3.5 -

Table 10. Complete UDA results on ADNI-2→ADNI-1 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 9. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 91.3±4.8 80.8±10 83.3±7.9 82.6±3.8 50.0±21 87.8±6.4 84.7±7.1 86.9±6.4 87.8±6.4 81.3±10 86.9±6.4 82.6±10 85.3±8.1 81.0±7.5

MMD 91.6±5.4 81.1±6.2 80.8±2.5 79.5±2.7 50.0±15 85.2±7.7 85.2±6.8 87.7±3.3 85.2±5.1 80.7±4.6 79.2±11 81.1±4.6 71.3±15 88.3±4.2

DANN 91.7±2.9 85.3±5.1 80.6±4.6 78.1±9.1 50.0±18 88.7±3.9 85.0±5.2 85.7±5.9 88.7±3.9 82.3±5.8 87.2±6.4 84.1±5.1 75.0±13 83.2±6.6

CDAN 90.5±3.2 84.8±7.3 80.3±7.1 86.1±3.9 50.0±20 86.2±5.1 87.0±7.2 85.2±3.8 87.2±5.1 81.4±3.8 85.5±4.9 85.7±7.3 81.1±5.0 85.0±4.8

DALN 92.0±3.5 85.0±4.4 79.2±12 87.5±3.0 53.1±16 85.0±3.4 83.0±7.4 86.0±4.7 85.0±3.2 83.7±6.1 82.7±4.7 84.1±3.8 82.2±17 85.8±4.7

MCC 92.4±3.4 87.5±6.7 65.0±20 84.3±5.1 60.2±17 85.5±3.6 85.8±6.2 78.6±13 85.5±3.6 81.1±5.2 83.0±2.6 84.3±4.6 79.2±15 86.8±4.8

BNM 91.6±3.0 83.7±6.0 80.2±4.2 85.3±2.6 79.2±19 87.4±6.5 82.0±3.5 83.7±9.5 86.7±5.7 78.5±7.6 80.5±5.0 83.7±3.0 81.1±10 84.0±3.8

AD2A 90.9±2.6 89.8±4.3 82.0±20 82.4±18 50.0±19 88.9±4.4 86.6±3.4 86.4±4.6 84.8±4.2 79.7±3.1 81.4±8.0 89.8±6.9 70.0±14 84.0±2.5

ATDOC 84.1±5.2 78.8±5.3 74.3±8.9 74.9±6.4 73.8±14 64.7±6.9 78.6±6.3 78.8±8.3 66.5±8.3 75.2±4.0 79.6±1.0 78.8±5.3 78.8±17 80.9±3.9

MCD Avg. Across-Algo TargetOnly 82.0±5.7 89.7±3.8 93.1±2.3 92.8±4.6 72.6±9.2 79.8±8.1 66.9±13 74.3±8.9 66.5±8.8 86.0±4.2 70.6±19 50.0±14 60.4±13 85.7±3.6 76.2±12 79.8±6.2 75.3±9.3 78.8±8.3 65.0±7.7 65.0±8.3 75.1±4.7 81.2±8.3 59.5±19 79.8±3.2 70.8±9.5 79.8±8.1 53.7±20 58.1±13 73.8±9.7 86.8±4.8 -

Table 11. Complete UDA results on ADNI-2→ADNI-3 (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 88.1±3.8 82.0±7.2 81.0±2.3 81.6±7.7 82.0±7.2 57.0±16 81.5±4.8 74.8±10 83.8±6.7 81.2±5.2 79.5±9.9 76.0±5.1 83.8±6.1 82.1±6.5

MMD 89.3±3.8 81.2±5.7 79.6±2.2 80.5±5.4 80.7±3.1 56.6±15 83.2±5.4 77.5±2.8 82.7±5.3 83.2±5.4 82.2±4.8 77.7±2.8 63.9±13 80.3±7.2

DANN 91.1±3.3 80.8±6.8 82.0±5.8 84.3±3.2 81.1±7.3 65.8±22 83.5±4.9 74.0±4.7 86.7±6.4 85.3±1.3 79.8±4.7 74.2±4.4 66.5±16 83.0±3.4

CDAN 91.2±3.0 84.9±5.9 84.4±6.2 81.0±6.8 84.1±6.5 58.4±19 82.7±4.7 76.7±1.5 85.6±5.8 82.7±4.7 81.9±1.9 71.7±3.9 75.2±13 80.8±4.7

DALN 90.7±2.9 83.0±2.6 84.0±7.0 82.4±8.2 83.7±3.4 49.9±0.26 78.9±6.3 72.3±6.6 84.6±6.8 78.9±6.3 83.7±7.8 74.9±8.2 70.6±16 82.2±3.8

MCC 90.1±4.1 82.4±6.5 81.2±2.9 80.6±6.7 83.2±5.8 50.0±0.0 81.7±3.8 76.0±4.9 84.7±6.3 81.7±3.8 82.0±6.2 74.8±2.0 75.6±13 79.7±5.4

BNM 90.1±2.9 79.2±7.4 78.8±8.0 85.3±6.2 80.7±5.7 64.2±20 80.1±3.0 76.9±2.7 82.4±6.4 80.1±3.0 84.4±6.2 75.4±2.2 65.9±18 79.2±1.7

ATDOC 89.2±3.1 81.3±5.4 77.4±7.6 77.6±3.5 80.2±6.8 69.0±18 82.6±7.0 72.4±2.6 78.6±7.2 54.4±7.5 82.5±7.0 76.5±4.5 75.8±15 73.9±7.8

MCD 90.0±3.0 84.0±6.5 82.0±5.8 83.6±6.5 82.8±5.7 67.7±16 78.5±8.7 79.2±7.0 79.9±7.6 60.9±17 69.0±13 78.5±5.5 65.9±17 81.8±7.0

AD2A Avg. Across-Algo TargetOnly 91.2±2.8 90.3±0.74 92.1±2.4 90.2±4.6 85.2±6.4 82.3±7.7 85.2±5.5 84.2±6.8 83.0±8.9 80.3±7.8 85.5±5.9 78.3±9.7 51.9±4.3 50.0±0.0 84.8±6.5 80.6±3.2 79.7±5.4 76.9±2.7 74.4±19 82.8±3.9 83.9±6.0 54.4±7.5 86.5±3.8 85.0±3.7 75.3±9.9 71.2±5.7 71.0±18 59.0±12 85.9±4.5 79.7±2.2 -

Table 12. Complete UDA results on ADNI-2→ADNI-3 (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 11. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 85.9±4.5 84.7±9.5 81.1±2.9 80.9±11 50.0±17 80.2±6.1 77.6±13 81.8±7.9 80.2±6.9 76.1±7.0 80.4±13 84.7±9.5 82.1±7.7 79.0±7.9

MMD 89.3±5.0 80.8±7.0 80.8±2.7 77.5±6.6 50.0±17 84.8±6.7 79.0±3.4 82.6±6.8 84.8±6.7 77.5±3.8 81.0±6.1 80.8±4.3 56.9±15 78.8±9.1

DANN 92.0±3.9 77.8±7.7 79.6±6.1 84.2±4.3 50.0±22 85.6±6.0 75.6±5.7 88.7±8.2 85.6±1.5 76.7±5.2 77.1±4.7 77.8±8.7 59.5±19 83.5±4.7

CDAN 90.8±4.1 84.2±8.3 84.3±7.7 79.3±9.0 50.0±21 81.9±5.9 76.5±1.8 83.0±7.0 81.9±5.9 71.1±5.1 82.0±2.5 84.1±8.3 77.4±17 79.7±6.5

DALN 90.3±3.7 83.4±3.0 80.1±8.1 78.4±10 50.0±0.29 77.8±7.2 73.5±9.0 84.4±9.2 77.8±7.2 76.9±11 83.5±11 84.8±3.9 73.1±19 82.2±5.4

MCC 90.3±5.6 79.6±8.0 81.9±3.8 80.7±7.8 50.0±0.0 82.2±4.2 75.5±6.1 84.2±8.3 82.2±4.2 75.0±2.3 79.5±7.5 81.9±7.1 79.5±16 81.0±7.1

BNM 88.9±3.7 79.5±10 77.2±10 82.6±7.4 50.0±20 79.5±3.7 77.3±3.0 81.4±7.9 79.5±3.7 76.6±2.6 83.7±8.2 79.5±7.3 58.8±18 79.5±2.0

AD2A 90.7±3.4 88.0±7.4 88.0±6.0 85.3±12 50.0±4.8 84.2±7.5 81.4±6.8 80.2±25 84.2±7.5 80.0±13 87.5±4.7 88.0±6.8 69.8±20 84.2±5.5

ATDOC 87.5±3.6 83.9±6.5 80.7±9.0 77.5±4.8 63.4±20 83.4±9.3 72.4±3.5 79.8±9.7 53.6±10 77.3±5.9 84.1±9.4 82.8±8.4 78.9±19 73.0±9.1

MCD Avg. Across-Algo TargetOnly 89.1±3.5 89.9±1.3 92.1±3.1 90.7±6.5 87.5±7.5 80.8±11 80.6±6.9 81.9±8.9 82.7±8.7 78.3±10 71.3±19 50.0±0.0 74.7±8.8 81.4±3.7 81.0±8.5 77.3±3.0 76.8±8.4 81.4±4.3 65.7±22 53.6±10 80.0±7.3 72.8±7.5 69.5±18 85.2±4.6 83.8±6.9 77.2±13 56.7±20 56.7±15 84.7±7.4 80.0±2.4 -

Table 13. Complete UDA results on ADNI-1+2→AIBL (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 92.2±2.3 88.4±3.0 86.8±3.3 86.3±5.3 87.6±4.2 88.3±3.3 84.7±6.7 80.4±6.8 88.3±3.8 88.2±4.1 86.6±7.0 74.0±1.8 88.8±4.0 87.8±2.0

MMD 91.6±2.0 85.0±3.5 83.2±4.6 84.1±5.0 85.0±3.5 84.5±5.4 82.4±4.9 77.3±4.7 86.0±3.0 84.7±3.0 68.5±19 75.2±3.8 59.4±12 89.5±2.4

DANN 92.8±1.1 86.6±4.5 83.8±4.7 85.3±2.2 85.0±5.0 88.6±4.8 78.9±3.2 75.4±3.0 84.7±5.5 80.2±1.1 83.9±6.1 76.0±5.6 82.0±7.5 85.9±5.3

CDAN 93.4±1.7 87.8±3.9 84.3±6.0 86.9±7.8 88.9±3.5 86.9±8.0 84.2±5.0 77.5±3.6 89.7±3.4 83.9±5.2 84.6±4.6 77.9±3.6 84.1±8.7 86.6±3.8

DALN 93.1±2.0 85.6±8.2 84.2±8.0 89.2±1.7 85.6±8.2 66.0±20 79.5±7.2 77.6±5.8 86.3±4.6 82.1±8.4 85.3±6.1 76.6±6.8 77.8±12 85.4±3.3

MCC 91.8±1.6 87.6±2.7 86.4±4.5 82.3±5.0 88.2±2.3 66.7±15 79.4±5.5 74.3±4.7 80.6±11 79.4±5.5 85.6±2.8 75.2±4.7 76.5±16 80.7±6.0

BNM 91.4±1.9 84.0±3.2 83.3±3.2 83.3±3.2 84.0±3.2 79.0±16 81.6±5.0 75.0±3.4 83.8±2.9 81.6±5.0 85.9±2.0 74.1±5.7 81.2±6.1 83.7±2.9

ATDOC 88.0±6.0 79.9±4.9 82.3±4.2 82.0±3.6 79.9±4.9 80.7±7.8 82.2±6.4 77.3±3.2 81.3±6.2 65.7±8.2 84.5±6.2 74.8±2.0 79.4±5.5 79.9±5.5

MCD 91.9±1.7 86.2±4.6 83.2±3.9 84.1±4.7 84.6±4.8 79.1±16 85.8±4.2 77.4±3.6 84.8±3.1 74.6±13 80.5±7.2 78.3±4.4 76.9±14 82.8±4.1

AD2A Avg. Across-Algo TargetOnly 93.1±2.6 91.9±1.5 93.8±2.1 81.9±3.3 86.5±2.1 84.0±5.1 87.8±1.9 84.4±3.3 83.3±2.5 89.0±1.7 88.8±3.7 82.7±5.0 80.5±18 64.8±16 82.9±2.9 81.6±5.0 75.9±3.9 75.1±3.2 88.8±3.8 84.1±4.8 86.7±6.3 65.7±8.2 88.0±3.0 88.0±3.0 73.4±4.4 76.9±5.2 67.9±13 70.9±16 86.5±2.7 81.3±6.4 -

Table 14. Complete UDA results on ADNI-1+2→AIBL (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 13. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 91.5±2.9 86.9±3.6 86.9±4.1 88.9±6.4 89.9±3.4 85.8±9.3 76.6±7.4 87.9±4.3 86.3±4.6 74.4±2.1 87.2±9.1 86.9±5.6 90.6±4.9 87.4±2.7

MMD 91.8±2.1 86.5±3.5 84.0±5.9 85.3±6.7 86.4±6.7 84.4±6.2 78.9±6.2 86.8±3.5 83.5±3.4 76.0±5.1 78.9±20 86.5±3.5 57.7±14 89.1±3.3

DANN 92.5±1.4 87.2±5.5 85.1±6.3 85.1±2.7 90.0±5.5 79.7±4.2 76.6±3.6 84.3±6.9 79.9±1.4 76.3±7.1 82.8±7.4 82.8±6.1 79.7±8.2 85.8±6.5

CDAN 92.5±1.9 88.6±5.3 85.2±7.6 87.7±11 88.7±10 83.8±6.6 77.1±4.0 88.4±4.2 83.8±6.6 77.8±4.5 85.6±5.5 88.7±4.3 86.6±11 86.5±4.7

DALN 92.0±1.9 86.6±9.7 85.1±11 89.1±2.3 53.9±22 80.5±9.0 79.1±7.4 87.7±5.8 84.9±11 73.0±8.0 81.8±6.7 86.6±9.7 82.5±14 86.6±4.1

MCC 91.5±1.9 87.4±3.0 84.9±6.0 84.8±6.2 61.5±18 81.6±6.7 74.0±6.0 85.6±13 81.6±6.7 74.0±6.3 85.1±3.1 87.7±3.0 82.8±19 83.5±7.1

BNM 91.3±2.5 85.3±3.9 84.2±3.8 84.2±3.8 85.3±20 83.5±5.7 75.5±4.7 84.8±3.8 83.5±5.7 73.0±7.1 86.1±2.5 85.3±3.9 79.9±7.6 84.8±3.8

AD2A 92.0±3.2 86.5±2.4 88.5±2.4 81.8±2.8 85.1±22 82.3±3.7 75.3±5.3 88.2±4.9 86.9±7.0 73.2±5.8 87.2±3.9 87.7±4.8 61.5±15 86.9±3.4

ATDOC 89.6±7.9 78.7±6.0 83.3±5.5 83.3±4.1 83.7±10 84.3±7.2 77.8±3.2 77.8±6.4 62.1±9.4 75.0±2.1 87.5±7.8 78.7±6.0 77.2±6.6 78.6±7.0

MCD Avg. Across-Algo TargetOnly 92.3±2.1 91.7±0.91 92.5±2.4 80.3±4.7 87.1±5.5 85.1±6.6 82.8±5.5 84.9±4.2 83.4±5.5 88.5±2.3 86.8±19 61.5±18 86.8±5.2 83.5±5.7 78.7±4.5 76.0±4.1 85.4±3.8 85.6±6.2 79.7±16 62.1±9.4 78.8±5.8 75.5±6.4 82.0±9.7 87.2±3.9 83.4±5.7 81.8±6.6 81.1±18 75.5±19 84.2±4.8 84.0±7.7 -

Table 15. Complete UDA results on RSNA→Child CXR (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 82.3±2.1 73.7±4.2 73.3±4.3 73.3±4.9 71.8±6.0 65.6±12 71.8±3.6 72.0±3.3 70.0±7.1 71.9±3.4 55.5±2.9 73.3±6.3 72.7±12 68.9±8.7

MMD 82.1±1.9 72.2±1.2 72.3±4.6 73.1±9.2 70.8±3.5 58.0±9.9 74.5±3.5 76.2±5.2 71.9±1.1 74.5±3.5 65.7±10 75.4±4.2 75.4±9.4 72.2±4.7

DANN 82.8±2.0 73.4±2.2 74.1±0.93 72.1±4.4 71.7±2.2 56.9±6.1 78.6±2.1 77.4±7.0 74.5±1.2 76.8±2.8 78.3±2.6 75.5±6.5 74.7±11 72.9±2.4

CDAN 84.9±1.8 72.4±4.2 74.0±3.0 72.1±3.4 70.9±2.7 63.2±14 80.2±3.5 81.0±3.7 78.0±4.9 75.6±13 77.8±1.1 75.8±6.2 79.7±2.8 76.4±3.0

DALN 83.5±1.4 72.6±2.6 76.6±4.5 71.7±7.0 74.1±6.1 51.7±3.4 72.0±5.3 74.2±2.5 69.5±5.1 72.0±5.3 58.9±11 71.8±5.5 71.7±12 70.6±9.0

MCC 86.5±1.3 83.5±2.4 76.4±6.3 79.3±6.3 82.9±2.3 56.2±12 81.6±1.2 81.7±1.2 73.8±14 82.0±1.6 82.8±3.2 73.1±5.5 70.2±12 72.1±13

BNM 84.0±1.1 75.4±5.9 74.5±4.3 78.4±3.6 74.8±2.5 52.1±3.0 72.4±6.7 71.2±14 64.5±16 72.3±6.5 74.4±8.5 70.4±9.8 68.8±9.8 62.9±13

ATDOC 81.5±2.0 51.5±14 54.3±12 61.9±17 66.5±9.7 57.2±7.7 57.2±12 63.0±17 54.9±11 57.2±12 68.5±6.2 60.8±10 51.5±9.8 57.2±11

MCD 84.0±1.2 78.0±4.5 76.9±3.7 69.5±9.3 73.6±1.6 68.9±11 76.2±0.48 69.5±22 74.6±2.6 75.3±1.8 75.5±5.7 78.6±5.0 73.5±1.4 74.9±2.8

CoUDA Avg. Across-Algo TargetOnly 89.5±0.56 84.3±2.3 89.5±0.56 92.7±1.1 84.8±3.0 73.2±4.3 86.2±1.5 76.4±6.3 76.9±6.0 72.6±12 85.1±3.5 75.8±3.3 71.0±17 51.0±0.68 86.8±1.8 78.8±5.9 87.1±1.6 79.0±6.2 85.0±3.3 73.8±14 86.6±1.6 78.8±5.9 82.1±7.8 74.2±8.7 85.1±1.4 70.5±9.7 87.9±1.7 48.5±6.0 85.1±2.7 70.5±12 -

Table 16. Complete UDA results on RSNA→Child CXR (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 15. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 81.9±2.6 73.4±5.6 74.0±5.2 73.0±5.9 68.8±12 71.4±4.9 71.8±4.5 71.4±8.0 71.4±4.6 71.5±7.7 56.8±3.4 73.4±6.4 76.3±14 72.1±9.5

MMD 81.2±2.2 72.3±1.7 74.1±6.0 75.8±11 51.1±9.9 74.7±4.3 79.5±5.2 71.7±1.2 74.7±4.3 76.1±5.2 62.6±13 72.2±4.2 78.9±11 70.7±6.0

DANN 82.1±2.4 74.4±2.8 73.7±1.1 74.2±5.3 56.7±6.9 78.5±2.8 79.5±8.3 74.8±1.6 77.6±3.5 77.8±8.2 77.8±3.5 72.9±2.5 78.0±14 74.4±2.5

CDAN 83.7±2.0 71.0±5.4 75.6±3.5 73.6±3.7 63.6±17 80.8±4.9 80.6±4.7 78.3±6.8 80.8±17 78.3±6.7 77.8±1.5 70.4±3.5 80.3±3.1 74.9±3.5

DALN 83.8±1.9 72.4±3.7 77.8±5.4 73.0±9.4 50.0±3.9 75.2±5.8 75.2±3.0 69.5±6.1 75.2±5.8 73.2±6.5 53.6±14 72.0±7.2 75.9±16 71.0±11

MCC 86.6±1.7 84.4±2.9 75.1±7.8 79.9±8.0 51.2±14 81.7±1.5 81.8±1.5 80.4±17 82.0±2.1 71.9±7.2 84.2±3.9 83.1±3.1 72.7±15 76.0±15

BNM 83.6±1.5 75.0±8.0 73.6±5.8 76.9±4.4 51.1±3.6 70.5±8.6 72.8±17 71.7±20 70.5±8.3 72.2±13 76.9±9.6 74.9±3.1 71.0±12 67.5±15

CoUDA 89.4±0.68 84.9±4.0 85.6±1.7 77.9±7.0 79.7±18 85.9±2.0 86.3±1.6 85.8±4.5 85.9±1.8 85.0±1.7 83.9±10 87.0±4.0 88.0±2.0 85.6±3.5

ATDOC 80.8±2.5 52.3±15 55.0±15 64.6±20 55.6±7.7 62.6±15 64.6±23 53.9±14 62.6±15 63.0±13 65.4±7.6 68.4±13 50.6±14 62.6±14

MCD Avg. Across-Algo TargetOnly 83.5±1.5 83.9±2.7 89.4±0.68 93.0±1.2 77.5±6.0 72.4±6.0 76.7±4.6 75.1±7.8 72.1±12 74.4±16 72.1±13 51.1±0.92 76.1±0.51 81.2±7.2 80.0±26 81.7±7.3 75.8±3.0 80.4±17 75.8±2.4 81.2±7.2 79.1±6.2 70.1±13 75.8±7.6 78.2±9.8 73.0±1.8 75.4±4.0 73.1±1.6 50.3±7.8 75.8±3.4 72.3±15 -

Table 17. Complete UDA results on Child CXR→RSNA (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 75.1±0.36 72.4±2.8 73.5±1.5 71.3±2.0 71.7±2.8 72.4±1.1 72.5±1.0 72.3±0.96 72.6±1.2 72.5±1.0 71.4±1.2 73.0±1.4 72.6±1.1 70.9±2.2

MMD 74.6±0.99 69.3±4.7 71.7±3.8 66.6±5.8 68.4±4.4 66.9±1.4 72.1±2.9 74.0±2.0 67.5±2.1 72.1±2.9 69.2±1.8 73.9±0.93 68.0±2.3 69.8±1.4

DANN 77.2±1.1 68.8±2.7 70.5±3.0 68.8±3.4 68.8±2.7 68.3±1.8 73.2±1.0 76.3±1.1 68.8±1.8 73.2±1.0 69.5±1.7 74.7±5.6 70.3±2.6 70.7±1.3

CDAN 76.2±0.32 71.5±1.2 70.2±1.2 64.6±7.0 71.5±1.2 67.2±4.6 74.5±0.94 75.8±0.36 69.1±3.4 74.5±0.94 70.9±2.5 76.2±0.32 71.7±3.3 72.6±2.8

DALN 78.1±0.45 74.6±2.1 76.1±1.4 75.9±1.3 74.9±2.1 76.2±1.6 75.7±1.2 76.0±0.51 74.3±4.3 75.7±1.2 75.3±0.84 74.7±1.2 76.7±0.72 76.8±0.90

MCC 77.3±0.66 73.5±1.8 70.1±7.5 71.3±7.4 73.5±1.8 74.2±1.6 74.2±2.0 75.5±1.1 74.8±2.1 74.2±2.0 73.8±1.4 73.1±2.1 73.7±0.94 73.6±1.3

BNM 77.2±0.47 71.8±3.1 69.3±4.5 68.7±9.4 71.8±3.1 72.8±3.4 74.1±0.98 73.9±0.92 74.4±1.4 74.0±0.98 72.9±1.5 71.6±4.3 73.2±1.1 72.4±2.9

ATDOC 76.3±0.89 61.9±17 59.2±12 53.5±18 59.4±19 68.0±7.0 65.7±16 71.6±2.9 72.5±2.8 65.7±16 70.4±3.2 63.4±13 61.6±16 56.0±20

MCD 77.0±0.39 67.5±2.9 70.9±3.5 72.9±4.4 67.5±2.9 75.5±0.70 74.2±1.7 74.0±1.9 75.7±0.99 73.8±1.1 70.0±3.7 71.2±2.6 73.3±1.1 74.9±0.98

CoUDA Avg. Across-Algo TargetOnly 73.2±0.44 76.4±1.5 78.3±0.26 82.4±0.46 71.5±0.84 70.9±3.1 71.2±1.3 59.2±12 71.4±0.17 71.3±0.32 72.4±0.56 70.6±3.6 71.2±0.32 74.2±1.6 71.2±0.32 74.2±2.0 71.7±0.49 74.6±1.4 71.2±0.32 72.0±1.3 71.2±0.32 74.2±2.0 71.3±0.36 72.9±2.3 72.4±0.73 74.7±1.2 71.2±0.28 73.3±1.1 71.3±0.29 74.9±0.98 -

Table 18. Complete UDA results on Child CXR→RSNA (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 17. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal

SourceOnly 75.2±0.46 73.5±3.4 73.5±2.0 70.5±2.4 72.5±1.3 72.6±1.3 71.7±1.1 73.1±1.4 72.6±1.3 73.2±1.9 71.3±1.4 70.5±3.4 72.9±1.3

MMD 74.7±1.1 72.2±5.1 73.5±4.6 68.3±7.1 66.2±1.7 71.1±3.2 74.7±2.4 66.5±2.6 71.1±3.2 73.7±1.2 69.2±2.4 67.9±5.1 68.1±3.3

DANN 77.4±1.2 70.1±3.0 71.7±3.9 67.8±4.2 69.2±2.0 73.2±1.3 76.7±1.3 69.2±2.0 73.2±1.3 77.4±6.7 70.3±2.1 70.1±3.0 71.6±3.1

CDAN 76.2±0.40 70.6±1.3 70.4±1.6 68.7±8.2 65.6±5.5 74.8±1.2 75.6±0.36 69.8±4.4 74.8±1.2 76.2±0.40 71.5±3.1 70.6±1.3 72.3±4.1

DALN 78.2±0.58 73.7±2.5 76.3±1.9 76.4±1.4 76.8±1.8 74.9±1.3 75.9±0.61 76.1±5.2 74.9±1.3 75.0±1.5 75.2±1.1 75.9±2.2 76.7±0.88

MCC 77.4±0.87 73.5±2.5 72.2±8.5 75.7±8.5 74.1±2.1 74.0±2.5 76.0±1.3 74.2±2.5 74.0±2.5 74.0±2.5 73.5±1.8 73.5±2.5 74.0±1.1

BNM 77.0±0.62 73.3±3.8 71.0±5.7 73.1±12 73.2±4.2 74.2±1.3 74.1±1.1 75.2±1.6 74.1±1.3 69.6±4.6 72.1±1.8 73.3±3.8 72.7±1.3

CoUDA 72.9±0.47 71.9±1.0 71.6±1.6 71.5±0.19 71.2±0.42 71.2±0.42 72.0±0.57 71.2±0.42 71.2±0.42 72.2±0.95 71.2±0.43 72.1±0.65 71.2±0.35

ATDOC 76.6±1.0 71.4±20 63.1±15 63.1±21 69.2±8.5 73.2±18 71.8±4.0 73.2±3.4 73.2±18 68.9±17 71.8±3.5 71.4±21 68.0±20

MCD Avg. Across-Algo TargetOnly 77.2±0.46 76.4±1.6 78.3±0.37 82.4±0.64 67.4±3.5 72.2±3.9 71.4±4.3 63.1±15 74.1±5.5 71.5±0.39 75.5±0.82 74.1±2.1 74.1±2.3 74.0±2.5 73.4±2.4 74.9±1.9 76.1±1.2 71.7±1.7 74.1±1.5 74.0±2.5 72.0±3.5 75.0±1.5 70.6±4.4 73.0±2.4 67.4±3.5 72.4±4.0 73.4±1.3 73.4±1.3 -

Table 19. Complete UDA results on LDD→CRD (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 79.0±0.82 67.6±1.8 69.5±4.2 74.5±5.1 68.2±2.0 62.4±7.0 74.0±4.3 76.7±2.9 67.4±0.93 72.8±3.5 69.5±7.8 68.5±4.8 69.3±8.8 69.9±4.1

MMD 82.1±0.52 79.7±0.96 79.5±0.43 73.5±9.4 79.9±0.72 73.8±13 81.0±1.3 79.6±1.4 79.8±0.76 80.7±0.93 76.8±2.7 72.6±4.4 80.7±0.95 80.1±1.3

DANN 82.4±0.54 78.7±2.2 77.7±3.4 79.8±0.45 78.2±1.3 68.6±17 81.5±0.67 81.5±0.76 79.9±1.6 81.5±0.67 80.8±1.2 74.6±4.1 80.9±0.85 81.7±0.38

CDAN 82.6±0.42 69.2±5.3 71.8±3.6 78.4±1.5 69.8±5.2 68.7±11 81.4±0.95 79.5±2.4 74.4±1.3 81.4±0.95 72.5±6.2 71.4±2.3 74.6±5.2 79.6±1.0

DALN 83.0±0.52 79.9±2.4 79.2±2.1 79.8±3.8 80.1±2.5 75.2±14 82.2±0.88 82.3±1.0 81.3±1.6 82.2±0.88 79.3±2.6 76.9±1.3 80.7±1.6 81.8±1.4

MCC 82.4±0.90 74.1±2.5 74.7±2.2 72.9±8.8 74.1±2.6 70.5±12 81.5±1.4 79.2±2.6 75.7±5.2 81.5±1.4 78.5±3.9 68.5±4.6 75.9±6.5 81.5±1.4

BNM 83.2±0.50 81.0±1.8 79.7±3.1 76.2±4.7 81.4±1.1 62.7±17 80.6±2.6 80.8±1.4 81.7±0.49 80.6±2.6 79.2±1.9 72.2±3.3 78.5±1.6 81.9±0.88

ATDOC 83.0±0.50 78.8±3.9 75.4±6.5 57.6±12 78.8±3.9 73.5±14 80.6±1.5 78.8±6.2 79.9±3.2 80.6±1.4 80.7±1.2 72.8±4.5 71.2±22 64.4±21

MCD 83.0±0.63 79.5±0.96 73.8±5.4 79.8±3.2 79.2±1.3 80.8±1.8 81.9±0.90 81.9±1.4 81.1±1.9 81.8±1.1 76.6±4.2 72.3±3.4 79.0±4.2 81.7±1.1

CoUDA Avg. Across-Algo TargetOnly 81.7±0.80 82.6±0.47 83.6±0.13 95.7±0.34 75.3±1.4 74.0±6.8 76.1±1.1 74.7±2.2 79.3±1.6 74.8±10 75.5±1.3 73.9±6.8 75.2±0.08 54.3±9.7 78.8±0.85 81.9±0.59 79.2±2.1 81.1±1.6 75.2±0.08 75.7±5.2 78.3±0.93 81.9±0.58 77.2±1.6 78.6±3.2 77.9±3.0 75.8±2.6 76.6±0.63 79.0±4.2 77.0±0.75 81.7±1.1 -

Table 20. Complete UDA results on LDD→CRD (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 19. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data. Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 79.1±1.1 68.8±2.1 70.6±4.7 73.9±5.9 64.8±8.8 73.8±5.6 76.9±3.7 67.8±1.1 73.8±4.8 68.5±6.1 73.3±9.5 68.8±2.5 68.2±11 69.0±5.2

MMD 82.2±0.62 80.1±1.1 79.7±0.56 78.5±11 79.6±15 81.2±1.5 79.6±1.9 79.6±0.98 81.1±1.1 74.7±4.6 76.5±3.2 80.1±0.91 80.7±1.3 79.6±1.5

DANN 82.3±0.66 77.8±2.7 77.8±4.5 79.8±0.61 80.4±16 81.4±0.88 81.5±1.1 80.4±2.0 81.4±0.88 76.4±5.0 80.5±1.4 77.8±1.5 81.4±0.91 81.6±0.50

CDAN 82.5±0.58 72.2±5.8 72.5±4.8 79.0±1.9 72.3±13 81.5±1.2 80.5±2.7 74.7±1.7 81.5±1.2 70.9±3.1 74.0±7.9 72.2±6.4 74.3±6.5 79.8±1.3

DALN 83.3±0.55 80.8±3.1 80.0±2.6 81.3±4.5 80.2±16 82.2±1.0 81.8±1.1 80.7±1.7 82.2±1.0 76.6±1.8 80.2±3.4 80.8±3.1 80.9±2.0 82.2±1.7

MCC 82.5±1.2 74.7±2.9 76.0±2.7 75.9±11 74.8±15 81.4±1.9 80.0±2.9 77.8±6.4 81.4±1.9 69.5±5.7 80.5±4.6 74.7±2.9 77.8±7.5 81.4±1.9

BNM 83.4±0.64 81.9±2.2 81.1±3.8 77.6±6.3 50.0±16 81.7±3.2 81.0±1.7 81.6±0.63 81.7±3.2 71.5±4.0 78.9±2.3 81.9±1.3 79.1±2.1 81.8±1.0

CoUDA 82.1±0.87 75.0±1.9 75.4±1.2 79.8±2.0 75.2±0.11 78.5±1.0 79.5±2.6 75.2±0.11 77.9±1.1 78.1±3.9 77.3±2.1 75.9±1.6 77.0±0.73 77.1±0.93

ATDOC 82.8±0.56 80.6±4.7 77.7±8.2 63.3±12 80.4±16 80.8±1.9 81.3±7.4 81.0±3.9 80.8±1.9 72.8±5.5 80.3±1.4 80.9±4.7 81.0±25 77.0±25

MCD Avg. Across-Algo TargetOnly 82.8±0.73 82.7±0.46 83.6±0.15 96.0±0.36 79.5±1.2 77.4±7.8 74.7±7.3 76.0±2.7 80.8±3.8 80.0±12 81.2±2.2 50.0±11 81.7±1.1 81.8±0.71 82.4±1.7 81.7±1.7 82.0±2.2 77.8±6.4 81.7±1.5 81.8±0.71 70.8±4.2 76.3±3.6 76.7±5.7 78.3±4.3 79.5±1.5 76.5±7.8 80.6±5.1 80.6±5.1 82.3±1.3 81.8±1.4 -

Table 21. Complete UDA results on CRD→LDD (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 77.2±3.0 68.2±1.6 67.7±7.3 68.2±7.2 68.2±1.6 65.4±8.9 67.1±2.5 64.1±1.6 68.0±2.1 67.4±1.9 68.2±3.9 60.6±2.9 67.7±1.8 67.9±1.8

MMD 78.6±1.6 44.9±1.3 46.1±3.6 58.1±9.9 51.5±13 44.8±1.6 50.5±11 63.1±11 44.8±1.6 50.4±11 50.7±11 61.5±11 47.7±9.4 60.9±15

DANN 77.9±5.8 64.7±11 61.7±15 67.1±11 67.7±13 59.2±14 73.1±9.8 73.1±9.2 59.8±15 73.1±9.8 66.2±16 71.2±4.6 71.7±11 67.8±12

CDAN 81.0±1.5 66.2±14 70.0±10 66.6±11 63.5±15 73.9±13 69.9±10 63.5±10 70.0±12 69.3±11 66.2±8.4 67.8±8.8 64.6±11 68.2±14

DALN 77.7±1.9 58.4±12 56.5±6.4 62.0±11 56.4±12 58.8±11 66.4±8.3 67.0±8.1 62.6±11 66.4±8.3 62.4±11 53.3±8.4 61.8±13 52.1±5.5

MCC 78.5±3.6 57.1±9.7 59.2±11 60.5±11 57.1±9.7 68.1±7.9 55.0±11 53.3±2.6 63.0±14 55.0±11 52.7±9.3 51.0±2.0 63.0±13 61.2±17

BNM 80.0±3.6 60.0±8.2 58.3±11 58.0±11 59.3±9.2 52.2±9.6 54.4±11 55.4±2.3 55.1±12 54.4±11 58.9±6.6 57.9±9.2 54.2±15 57.1±8.3

ATDOC 74.8±3.4 49.8±7.5 62.0±17 48.3±15 74.1±4.3 47.6±3.0 51.5±10 51.2±14 48.6±4.6 45.2±16 47.1±14 62.8±8.2 37.7±6.5 51.8±10

CoUDA Avg. Across-Algo TargetOnly 63.9±1.4 76.5±4.8 82.7±1.1 98.7±0.17 56.6±0.80 54.8±3.8 56.3±3.1 63.7±6.1 54.1±4.2 49.6±16 63.2±2.4 64.6±14 56.2±1.3 58.4±7.7 56.2±1.3 55.0±11 55.3±1.3 68.1±12 56.2±1.3 59.3±15 56.2±1.3 55.0±11 55.1±2.3 72.5±9.3 52.8±2.3 51.8±7.2 56.0±2.2 67.2±2.4 55.6±1.7 56.7±5.6 -

MCD 76.3±5.0 56.0±8.4 57.8±8.0 60.1±5.7 56.0±8.4 61.5±9.9 61.6±9.6 55.8±7.9 49.4±3.7 61.6±9.6 64.2±1.8 64.9±8.3 67.2±2.4 54.5±6.2

Table 22. Complete UDA results on CRD→LDD (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 21. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal

SourceOnly 76.7±4.1 68.1±2.0 65.4±9.5 68.2±9.5 70.0±11 66.9±3.2 64.9±1.9 68.1±2.3 66.9±2.4 60.0±3.4 68.6±4.4 68.1±2.0 67.4±2.1

MMD 77.7±1.9 45.0±1.7 45.7±4.5 51.8±10 44.6±2.1 45.4±13 69.8±11 44.6±2.1 45.4±13 59.8±14 46.0±14 47.0±15 43.4±11

DANN 80.1±7.0 64.6±12 52.9±15 71.5±12 50.2±14 77.0±11 76.5±11 50.2±15 77.0±11 72.2±5.6 75.6±17 75.1±13 75.9±13

CDAN 81.4±1.9 67.0±17 70.9±13 68.3±13 80.0±15 75.7±11 58.4±12 73.8±13 75.7±11 69.2±11 63.6±11 67.0±17 61.8±13

DALN 78.2±2.3 59.0±12 59.5±7.9 58.6±12 57.6±15 68.5±11 68.5±11 62.4±14 68.5±11 52.6±10 61.5±13 49.0±12 68.3±13

MCC 76.5±3.8 60.4±13 62.4±14 62.4±14 65.8±9.5 53.0±14 54.7±2.8 62.8±19 53.0±14 51.0±2.8 58.8±9.5 60.4±13 65.4±18

BNM 80.0±4.6 62.9±8.9 64.7±12 64.7±12 50.1±13 50.4±14 55.0±3.1 51.9±13 50.4±14 53.0±11 56.1±7.5 62.9±11 50.1±20

CoUDA 64.0±1.6 56.7±1.0 55.7±4.0 54.6±5.9 56.4±1.6 56.4±1.6 55.2±1.8 56.4±1.6 56.4±1.6 53.1±3.0 56.2±2.7 63.7±2.9 56.3±2.9

ATDOC 75.5±4.0 44.9±8.2 70.8±18 51.8±20 47.6±3.1 55.4±12 51.0±18 47.6±5.5 55.4±18 65.3±10 45.5±19 75.5±5.2 36.8±8.0

MCD Avg. Across-Algo TargetOnly 76.1±6.6 76.6±5.1 82.7±1.4 98.8±0.20 58.5±9.3 56.3±4.7 58.0±11 64.7±8.3 62.9±6.6 51.8±20 66.6±10 62.8±7.9 64.5±11 53.0±14 53.7±10 76.3±12 47.6±4.4 62.6±19 64.5±11 53.0±14 64.7±11 52.6±10 64.2±2.1 77.3±11 58.5±9.3 69.8±17 66.3±2.6 66.3±2.6 -

Table 23. Complete UDA results on OCT→SLO (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 61.9±1.3 59.5±1.1 58.4±2.3 55.3±3.4 61.9±1.3 58.0±3.3 59.2±1.2 58.9±1.9 59.1±1.6 59.2±1.2 55.5±6.7 58.9±1.8 56.4±6.5 59.0±1.8

MMD 66.7±0.52 63.3±1.6 63.9±1.4 63.8±1.5 66.6±0.64 63.2±1.5 63.7±1.9 63.8±2.0 63.3±1.4 63.4±1.6 64.8±1.1 62.7±1.0 62.8±1.3 64.0±1.4

DANN 67.0±0.39 66.3±0.59 65.9±1.5 60.8±4.7 66.4±0.07 66.1±0.53 65.8±0.23 65.8±1.1 66.1±0.53 66.2±0.43 65.3±0.98 62.5±0.94 64.8±0.91 65.1±0.75

CDAN 67.2±0.53 64.0±0.65 64.0±1.1 62.4±3.2 66.8±1.0 63.6±1.6 66.4±0.58 65.9±0.74 65.0±0.97 66.4±0.58 63.7±1.0 61.6±2.9 65.6±0.70 65.2±0.94

DALN 66.1±0.79 63.8±1.2 64.6±0.67 55.4±5.0 66.1±0.79 64.0±0.60 64.0±0.60 64.5±0.21 64.0±0.60 64.0±0.60 64.8±0.38 59.4±0.87 64.0±0.25 64.4±0.40

MCC 64.2±1.4 57.6±3.9 49.7±5.6 61.2±2.7 63.8±1.5 50.4±3.9 54.8±7.9 59.7±3.0 54.0±7.4 52.2±5.3 54.5±7.1 60.8±3.8 58.7±4.5 56.8±4.3

BNM 65.0±0.66 55.8±5.0 49.1±6.5 55.9±5.8 64.7±0.79 50.1±0.71 56.4±6.4 54.5±5.1 54.8±8.8 56.4±6.4 60.4±2.8 57.0±10 51.3±2.9 48.9±2.7

ATDOC 60.4±2.5 38.1±4.1 47.6±3.2 51.1±1.6 60.4±2.5 45.4±4.6 44.6±3.9 41.5±4.7 45.4±4.6 44.6±3.9 41.6±8.0 55.3±8.5 44.8±6.6 45.9±2.8

MCD Avg. Across-Algo TargetOnly 64.2±2.4 65.1±2.1 67.4±0.15 75.0±0.58 54.2±3.5 57.6±13 55.2±2.9 63.9±1.4 51.4±3.8 51.6±3.7 63.1±3.3 67.3±0.22 48.8±4.8 50.2±3.9 49.5±4.2 49.7±4.0 53.5±4.3 57.3±7.0 48.8±4.8 54.0±7.4 49.5±4.2 49.7±4.0 51.1±7.9 50.5±6.8 60.3±8.0 59.4±0.87 51.5±3.7 51.5±3.7 53.8±3.6 53.8±3.6 -

Table 24. Complete UDA results on OCT→SLO (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 23. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 61.5±1.3 59.1±1.0 58.1±2.3 55.4±3.4 59.7±3.0 59.4±1.2 59.8±1.8 59.7±1.5 59.4±1.2 59.8±1.6 58.7±6.1 61.5±1.3 60.0±5.7 59.6±1.7

MMD 66.5±0.49 62.9±1.6 64.4±1.3 63.0±1.3 63.9±1.4 64.3±1.8 64.8±1.8 63.9±1.2 64.2±1.5 63.0±0.99 65.0±1.1 66.5±0.64 63.1±1.3 64.6±1.3

DANN 66.9±0.39 66.5±0.54 66.4±1.4 60.7±4.7 65.9±0.49 65.9±0.23 66.2±1.0 65.9±0.49 66.1±0.41 62.7±0.91 65.1±0.97 66.5±0.07 64.9±0.91 64.8±0.71

CDAN 67.4±0.50 63.6±0.56 64.4±1.1 64.2±2.8 64.1±1.5 66.1±0.52 66.0±0.73 65.0±0.97 66.1±0.52 63.0±2.6 63.4±0.97 67.1±0.95 65.7±0.69 65.5±0.90

DALN 66.5±0.73 63.7±1.2 64.6±0.67 53.7±4.8 64.0±0.60 64.0±0.60 64.4±0.20 64.0±0.60 64.0±0.60 59.1±0.82 64.7±0.37 66.5±0.73 64.0±0.25 64.4±0.40

MCC 64.1±1.4 55.5±3.4 50.4±5.5 61.1±2.7 50.6±3.9 56.2±7.8 61.3±2.6 54.2±7.4 54.1±5.0 62.4±3.6 58.2±6.3 63.4±1.4 60.5±4.2 55.9±4.2

BNM 65.3±0.63 58.4±4.5 50.6±6.4 57.2±5.7 50.0±0.70 58.6±6.1 54.8±5.1 50.0±7.7 58.6±6.1 62.9±9.2 58.8±2.5 64.3±0.73 50.4±2.8 50.0±2.5

ATDOC 61.3±2.4 37.2±4.0 47.7±3.2 50.3±1.4 42.9±4.1 43.5±3.8 41.5±4.7 42.9±4.1 43.5±3.8 59.0±7.8 37.2±7.1 61.3±2.4 47.9±6.0 45.1±2.7

MCD Avg. Across-Algo TargetOnly 63.8±2.4 65.2±2.0 67.4±0.15 75.7±0.57 55.4±3.4 63.6±12 54.6±2.8 64.4±1.3 49.5±3.4 50.1±3.4 46.7±4.4 50.0±3.9 48.8±4.2 48.8±3.9 54.4±4.2 61.3±6.1 46.7±4.4 54.2±7.4 48.8±4.2 48.8±3.9 62.5±7.7 59.1±0.82 47.9±7.4 47.9±6.4 62.1±3.2 67.3±0.21 51.4±3.7 51.4±3.7 53.7±3.6 53.7±3.6 -

Table 25. Complete UDA results on SLO→OCT (target accuracy, %; mean with std). Each cell reports the mean target accuracy (std) of the checkpoint a validator selects for an algorithm. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) ClassAMI SND MixVal TransScore

SourceOnly 60.7±1.2 57.9±1.4 56.6±0.78 58.4±1.7 57.0±3.0 57.1±1.3 56.9±1.6 57.9±1.9 56.9±1.5 56.9±1.5 57.8±2.1 59.0±2.4 58.1±1.9 57.8±1.8

MMD 63.0±1.0 60.7±0.83 56.2±0.90 56.8±3.4 61.6±2.3 49.6±4.8 49.7±4.8 50.1±4.4 49.6±4.8 49.7±4.8 59.8±1.5 58.5±3.2 49.8±4.4 60.9±2.2

DANN 64.6±1.5 61.1±1.6 60.5±2.5 61.4±2.3 62.4±0.69 60.5±1.7 60.1±2.3 61.4±2.5 60.5±1.7 60.5±1.7 59.5±2.5 60.6±2.3 59.7±2.1 60.2±1.8

CDAN 65.4±0.34 63.3±1.4 60.8±0.25 60.3±1.8 62.2±1.6 61.9±1.4 61.4±2.6 62.6±2.1 61.0±1.2 61.4±2.6 60.4±1.1 60.1±2.3 62.3±2.2 62.3±0.26

DALN 63.6±1.1 59.6±1.3 58.7±0.39 56.1±1.9 60.3±1.7 61.4±1.1 61.6±1.1 61.7±1.4 61.4±1.1 61.6±1.1 61.8±1.6 57.0±1.4 60.0±1.4 59.7±0.90

MCC 62.3±1.4 53.5±4.6 54.1±3.6 55.9±3.4 52.6±6.2 45.6±3.2 45.3±4.9 55.0±5.4 45.5±3.1 45.3±4.9 52.6±7.5 57.6±4.4 46.2±4.6 47.2±3.5

BNM 61.3±3.3 49.7±4.2 48.6±1.2 51.8±4.9 49.4±4.5 50.3±0.90 48.3±1.6 49.3±1.9 49.7±1.6 48.5±1.9 47.0±1.4 54.2±6.7 50.7±2.7 50.2±3.8

ATDOC 58.4±5.2 45.1±0.56 51.2±3.5 46.2±1.5 43.7±2.5 44.9±4.0 45.2±3.9 47.0±4.6 44.9±4.0 45.2±3.9 43.6±3.8 54.4±5.9 44.8±3.5 48.0±6.2

MCD Avg. Across-Algo TargetOnly 62.0±1.6 62.6±2.0 65.6±0.62 74.4±0.38 54.6±6.7 55.2±6.7 55.7±4.7 52.0±5.4 57.5±3.0 54.2±7.8 49.7±3.8 60.1±4.9 51.1±6.0 45.9±3.6 52.1±5.8 45.3±4.9 51.8±5.3 49.3±1.9 50.8±5.6 45.5±3.1 52.2±5.9 45.3±4.9 52.7±6.0 46.0±5.9 58.7±2.1 57.0±1.4 53.6±5.6 52.2±6.5 57.3±3.2 57.3±3.2 -

Table 26. Complete UDA results on SLO→OCT (target accuracy, %; median with 95% CI). Each cell reports the median target accuracy (95% confidence interval) of the checkpoint a validator selects for an algorithm, computed over the same runs as the mean±std results in Table 25. The per-algorithm columns report selection within a single algorithm, Avg. is their mean, and Across-Algo pools the checkpoints of all algorithms and selects across them. Oracle selects using target labels and is the actual best model, SourceOnly is trained on source data only, and TargetOnly is trained on labeled target data.

Oracle Source-Risk IWCV DEV Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI DEV-N MixVal TransScore

SourceOnly 61.2±1.1 58.1±1.4 56.2±0.73 58.3±1.7 56.4±1.2 56.2±1.5 58.6±1.8 56.2±1.4 56.2±1.4 59.3±2.4 58.7±2.0 58.1±2.8 58.6±1.8 58.6±1.6

MMD 62.4±0.89 61.1±0.76 56.6±0.81 56.9±3.4 48.4±4.7 48.5±4.7 48.5±4.2 48.4±4.7 48.5±4.7 58.9±3.2 60.4±1.4 61.1±2.2 47.9±4.1 59.6±2.0

DANN 64.0±1.4 61.6±1.5 60.6±2.5 60.1±2.0 60.6±1.7 60.6±2.3 60.5±2.4 60.6±1.7 60.6±1.7 60.0±2.3 58.3±2.2 62.4±0.68 59.7±2.1 60.6±1.8

CDAN 65.5±0.33 62.7±1.3 60.9±0.23 60.7±1.8 62.4±1.4 60.4±2.4 62.4±2.1 60.3±1.0 60.4±2.4 60.5±2.3 59.8±0.94 62.9±1.5 61.7±2.2 62.4±0.25

DALN 63.3±1.1 59.3±1.3 58.8±0.37 56.2±1.9 60.9±1.1 61.5±1.1 60.9±1.3 60.9±1.1 61.5±1.1 56.8±1.4 61.0±1.4 61.1±1.5 60.7±1.3 59.2±0.81

MCC 63.0±1.2 53.1±4.6 53.1±3.5 56.8±3.3 44.2±3.0 44.5±4.8 53.5±5.2 44.2±2.9 44.5±4.8 58.5±4.3 52.2±7.5 53.7±6.2 48.2±4.3 48.3±3.3

BNM 62.8±3.0 48.8±4.1 48.2±1.1 49.9±4.6 50.0±0.85 47.5±1.4 48.5±1.8 49.6±1.6 47.5±1.7 56.5±6.5 46.9±1.3 47.7±4.3 50.9±2.7 48.4±3.5

ATDOC 60.2±4.9 45.2±0.54 50.3±3.4 45.3±1.3 46.6±3.7 47.1±3.6 44.5±4.0 46.6±3.7 47.1±3.6 56.0±5.7 42.5±3.6 44.5±2.5 45.3±3.5 46.9±6.2

MCD Avg. Across-Algo TargetOnly 61.2±1.4 62.8±1.6 65.5±0.62 75.8±0.30 53.2±6.6 54.2±6.7 57.4±4.5 50.3±5.2 59.2±2.6 58.2±6.9 49.8±5.9 44.2±3.3 54.5±5.4 44.5±4.8 53.2±5.2 48.5±1.8 49.8±5.6 44.2±2.9 54.5±5.5 44.5±4.8 59.8±1.9 56.8±1.4 50.4±5.6 45.2±5.9 50.1±3.7 62.4±4.4 52.4±5.5 48.8±5.8 56.6±3.2 56.6±3.2 -

Table 27. Within-algorithm Spearman’s ρ on ADNI-1→ADNI-2. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.80 0.49 0.52 0.77 0.45 0.78 0.82 0.70 0.75 0.75 0.29 0.22 0.30

DANN 0.76 0.29 0.47 0.74 0.61 0.87 0.79 0.83 0.86 0.58 0.70 0.33 0.43

CDAN 0.74 0.31 0.45 0.72 0.60 0.87 0.81 0.82 0.86 0.57 0.70 0.37 0.52

DALN 0.74 -0.28 0.02 0.71 0.57 0.87 0.83 0.81 0.85 0.65 0.73 0.33 0.54

MCC 0.74 0.29 0.23 0.73 0.55 0.83 0.78 0.76 0.82 0.50 0.69 0.35 0.75

BNM 0.70 0.04 0.20 0.69 0.51 0.79 0.74 0.73 0.79 0.45 0.67 0.35 0.70

ATDOC 0.53 0.05 0.18 0.53 0.33 0.61 0.62 0.55 0.60 0.38 0.53 0.25 0.47

MCD 0.79 0.48 0.53 0.78 0.72 0.81 0.82 0.78 0.80 0.78 0.47 0.31 0.53

AD2A 0.80 0.38 0.53 0.79 0.67 0.89 0.87 0.86 0.88 0.66 0.71 0.29 0.50

Table 28. Within-algorithm Spearman’s ρ on ADNI-1→ADNI-3. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.52 0.16 0.24 0.51 0.35 0.46 0.40 0.45 0.46 0.34 0.02 0.13 0.22

DANN 0.52 0.14 0.28 0.52 0.53 0.61 0.35 0.66 0.63 0.14 0.56 0.20 0.39

CDAN 0.61 0.22 0.33 0.61 0.44 0.58 0.37 0.64 0.61 0.24 0.58 0.27 0.43

DALN 0.40 -0.12 0.09 0.39 0.22 0.39 0.21 0.41 0.40 0.18 0.37 0.13 0.18

MCC 0.47 0.15 0.19 0.48 0.20 0.39 0.29 0.40 0.39 0.24 0.46 0.20 0.35

BNM 0.40 0.11 0.21 0.40 0.24 0.18 0.03 0.30 0.20 -0.01 0.41 0.19 0.15

ATDOC 0.48 0.13 0.07 0.28 0.05 0.43 0.20 0.42 0.42 0.24 0.44 0.26 0.14

MCD 0.72 0.50 0.47 0.70 0.55 0.66 0.64 0.64 0.64 0.62 0.41 0.27 0.33

AD2A 0.75 0.36 0.53 0.75 0.56 0.73 0.64 0.70 0.72 0.52 0.61 0.29 0.21

Table 29. Within-algorithm Spearman’s ρ on ADNI-2→ADNI-1. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.73 0.43 0.17 0.71 0.43 0.80 0.75 0.73 0.78 0.53 0.16 0.24 0.52

DANN 0.58 -0.08 -0.02 0.57 0.39 0.72 0.72 0.59 0.70 0.35 0.53 0.25 0.34

CDAN 0.53 -0.24 -0.03 0.54 0.50 0.74 0.69 0.64 0.73 0.22 0.55 0.25 0.33

DALN 0.59 -0.37 -0.03 0.57 0.49 0.77 0.67 0.69 0.76 0.32 0.56 0.36 0.55

MCC 0.58 -0.28 -0.01 0.56 0.46 0.76 0.68 0.63 0.74 0.25 0.60 0.29 0.74

BNM 0.56 -0.12 -0.07 0.55 0.44 0.77 0.73 0.65 0.76 0.37 0.67 0.26 0.64

ATDOC 0.68 0.03 0.07 0.68 0.38 0.04 0.72 0.76 0.40 0.38 0.62 0.55 0.57

MCD 0.76 0.45 0.51 0.76 0.67 0.54 0.82 0.77 0.66 0.71 0.58 0.30 0.56

AD2A 0.79 0.05 -0.02 0.77 0.57 0.84 0.84 0.77 0.82 0.62 0.72 0.35 0.42

Table 30. Within-algorithm Spearman’s ρ on ADNI-2→ADNI-3. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.72 0.38 0.49 0.64 0.30 0.76 0.63 0.70 0.75 0.53 -0.01 0.37 0.42

DANN 0.68 0.30 0.46 0.61 0.20 0.71 0.51 0.64 0.72 0.39 0.66 0.29 0.36

CDAN 0.66 0.23 0.41 0.57 0.15 0.68 0.55 0.60 0.68 0.45 0.57 0.27 0.36

DALN 0.69 -0.23 0.21 0.58 0.17 0.68 0.56 0.62 0.69 0.49 0.56 0.32 0.33

MCC 0.67 0.20 0.39 0.58 0.19 0.64 0.48 0.60 0.65 0.35 0.65 0.30 0.59

BNM 0.57 0.01 0.33 0.49 0.19 0.51 0.33 0.57 0.54 0.24 0.60 0.23 0.46

ATDOC 0.57 0.24 0.25 0.52 0.19 0.36 0.19 0.41 -0.04 0.17 0.55 0.24 0.24

MCD 0.74 0.54 0.58 0.71 0.63 0.63 0.61 0.68 0.55 0.52 0.23 0.28 0.53

AD2A 0.82 0.60 0.68 0.78 0.49 0.80 0.58 0.73 0.78 0.44 0.55 0.31 0.41

Table 31. Within-algorithm Spearman’s ρ on ADNI-1+2→AIBL. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.71 0.52 0.32 0.71 0.55 0.65 0.42 0.65 0.66 0.34 -0.25 0.31 0.51

DANN 0.56 0.15 0.12 0.56 0.46 0.40 -0.04 0.54 0.47 -0.14 0.37 0.27 0.36

CDAN 0.65 0.18 0.05 0.62 0.46 0.58 0.24 0.62 0.61 0.09 0.38 0.21 0.54

DALN 0.56 -0.28 0.03 0.56 0.41 0.46 0.06 0.54 0.50 0.01 0.33 0.13 0.36

MCC 0.62 0.25 0.29 0.62 0.50 0.45 0.04 0.58 0.50 -0.11 0.46 0.16 0.49

BNM 0.54 0.08 0.20 0.56 0.41 0.31 -0.12 0.44 0.35 -0.20 0.50 0.13 0.32

ATDOC 0.58 0.22 0.31 0.57 0.43 0.54 -0.09 0.48 -0.13 -0.18 0.39 0.18 0.33

MCD 0.66 0.41 0.29 0.67 0.72 0.75 0.41 0.72 0.51 0.24 0.29 0.24 0.63

AD2A 0.77 0.53 0.30 0.76 0.69 0.71 0.35 0.76 0.73 0.16 0.60 0.27 0.58

Table 32. Within-algorithm Spearman’s ρ on RSNA→Child CXR. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.11 -0.05 -0.02 -0.20 -0.16 0.29 0.75 0.10 0.24 0.76 -0.18 0.43 0.13

DANN 0.20 0.14 0.02 -0.17 -0.15 0.32 0.75 0.11 0.27 0.76 0.64 0.37 0.06

CDAN -0.08 -0.06 -0.08 -0.30 -0.39 0.01 0.63 -0.22 -0.04 0.68 0.44 0.46 -0.17

DALN 0.21 -0.02 -0.02 0.23 -0.19 0.09 0.18 -0.08 0.08 0.24 -0.28 0.36 0.09

MCC 0.49 0.41 0.23 0.39 0.33 0.65 0.58 0.45 0.65 0.39 0.57 0.30 0.63

BNM 0.22 0.17 0.07 0.22 0.07 0.58 0.59 0.23 0.57 0.50 0.44 0.12 0.57

ATDOC -0.10 0.03 0.10 -0.08 -0.14 0.01 0.07 -0.14 0.01 0.16 0.04 -0.10 -0.02

MCD 0.47 0.30 -0.04 0.17 0.26 0.52 0.70 0.33 0.49 0.62 0.07 0.37 0.31

CoUDA 0.57 0.42 0.21 0.46 0.34 0.56 0.65 0.47 0.55 0.59 -0.25 0.15 0.54

Table 33. Within-algorithm Spearman’s ρ on Child CXR→RSNA. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.19 0.20 0.12 0.19 -0.12 0.81 0.96 -0.01 0.76 0.90 0.35 0.07 0.22

DANN 0.11 0.16 0.09 0.10 -0.29 0.79 0.97 -0.15 0.72 0.92 -0.08 0.24 -0.06

CDAN -0.08 0.12 0.03 -0.07 -0.25 0.87 0.99 0.04 0.84 0.96 0.16 0.44 -0.11

DALN 0.22 0.14 0.17 0.22 0.44 0.47 0.43 0.42 0.47 -0.27 0.21 0.39 0.19

MCC 0.12 0.23 0.22 0.12 0.07 0.58 0.61 0.15 0.57 0.58 0.05 -0.19 0.40

BNM 0.17 0.09 0.08 0.18 0.32 0.44 0.38 0.34 0.44 0.29 0.12 -0.16 0.39

ATDOC -0.08 -0.02 -0.02 -0.08 -0.01 0.09 0.13 -0.01 0.09 0.15 0.12 0.11 0.06

MCD 0.02 0.14 0.15 0.03 0.60 0.73 0.69 0.61 0.73 0.46 0.45 0.18 0.65

CoUDA -0.11 0.04 -0.05 0.27 -0.43 -0.36 -0.16 -0.42 -0.37 0.58 0.06 -0.14 -0.37

Table 34. Within-algorithm Spearman’s ρ on LDD→CRD. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.57 0.43 0.30 0.55 0.53 0.78 0.65 0.63 0.77 0.16 0.30 0.66 0.50

DANN 0.58 0.22 0.21 0.55 0.57 0.85 0.75 0.66 0.84 0.34 0.15 0.58 0.58

CDAN 0.06 -0.19 -0.07 0.04 -0.06 0.85 0.87 0.08 0.81 0.70 0.23 0.43 0.20

DALN 0.68 -0.03 0.09 0.66 0.68 0.87 0.81 0.78 0.86 0.09 0.36 0.68 0.65

MCC 0.20 -0.18 -0.01 0.18 0.07 0.89 0.87 0.20 0.87 0.74 0.26 0.23 0.73

BNM 0.74 -0.25 -0.14 0.72 0.67 0.84 0.76 0.77 0.84 0.34 0.68 -0.08 0.80

ATDOC 0.30 -0.02 -0.03 0.28 0.26 0.39 0.39 0.34 0.39 0.11 0.45 0.24 0.27

MCD 0.63 -0.16 0.10 0.57 0.69 0.93 0.83 0.76 0.93 0.44 0.48 0.42 0.65

CoUDA -0.08 -0.16 -0.04 -0.10 -0.07 0.40 0.84 0.05 0.35 0.67 0.38 0.42 0.01

Table 35. Within-algorithm Spearman’s ρ on CRD→LDD. Each cell is the mean ρ over five folds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD -0.01 0.28 0.25 0.01 -0.21 -0.02 0.39 -0.14 -0.05 0.50 -0.10 -0.12 -0.12

DANN 0.44 0.13 0.15 0.45 0.34 0.49 0.58 0.39 0.48 0.21 0.44 0.32 0.29

CDAN 0.42 0.19 0.30 0.43 0.28 0.47 0.43 0.34 0.45 0.15 0.34 0.29 0.27

DALN 0.11 0.09 0.13 0.12 -0.02 0.15 0.38 0.05 0.14 0.37 0.19 0.06 0.04

MCC 0.46 0.10 0.14 0.47 0.46 0.03 -0.13 0.47 0.06 -0.25 0.32 0.38 0.14

BNM 0.15 0.13 0.13 0.16 0.06 0.31 0.35 0.10 0.29 0.32 0.17 -0.02 0.28

ATDOC -0.04 0.15 -0.15 0.96 -0.07 0.01 0.05 -0.05 -0.02 0.12 -0.18 -0.18 0.02

MCD -0.18 0.17 0.15 -0.18 -0.23 -0.34 -0.42 -0.24 -0.33 -0.14 -0.15 0.01 -0.28

CoUDA 0.31 -0.02 0.02 0.88 0.27 0.24 0.09 0.26 0.24 -0.34 -0.10 0.18 0.26

Table 36. Within-algorithm Spearman’s ρ on OCT→SLO. Each cell is the mean ρ over three random seeds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD -0.08 -0.19 0.07 0.82 -0.15 -0.11 0.15 -0.14 -0.11 0.26 0.12 -0.13 -0.16

DANN 0.40 0.22 0.04 0.84 0.44 0.48 0.46 0.45 0.48 -0.12 0.44 0.48 0.29

CDAN 0.27 0.16 0.02 0.74 0.22 0.60 0.70 0.26 0.56 0.14 0.29 0.26 0.15

DALN 0.59 0.75 -0.15 0.98 0.76 0.78 0.78 0.77 0.78 0.13 0.26 0.72 0.56

MCC -0.15 0.11 -0.04 0.91 -0.11 0.45 0.52 -0.02 0.44 0.51 0.18 0.27 0.24

BNM -0.12 0.16 -0.12 0.91 -0.16 -0.04 0.05 -0.14 -0.05 0.22 0.07 -0.07 -0.16

ATDOC -0.29 -0.17 -0.11 0.91 -0.34 -0.37 -0.26 -0.35 -0.37 -0.02 -0.06 -0.28 -0.37

MCD 0.15 0.07 -0.01 0.90 0.15 0.32 0.37 0.18 0.32 0.34 -0.10 0.22 0.34

Table 37. Within-algorithm Spearman’s ρ on SLO→OCT. Each cell is the mean ρ over three random seeds between the validation score and target accuracy across checkpoints within a single algorithm. Positive values indicate ranking in the intended direction.

Source-Risk IWCV DEV DEV-N Entropy InfoMax Corr-C MCC (V) BNM (V) SND ClassAMI MixVal TransScore

MMD 0.16 0.01 -0.11 0.12 0.04 0.09 0.07 0.08 0.08 0.07 -0.36 0.08 0.09

DANN 0.36 -0.06 0.19 0.33 0.35 0.36 0.31 0.36 0.36 -0.02 0.41 0.35 0.31

CDAN 0.49 -0.03 0.10 0.37 0.51 0.59 0.41 0.53 0.59 -0.07 0.47 0.50 0.46

DALN 0.53 -0.40 -0.02 0.54 0.65 0.68 0.69 0.67 0.68 0.12 0.60 0.61 0.05

MCC -0.35 0.15 0.01 -0.34 -0.39 -0.18 -0.06 -0.40 -0.19 0.08 -0.23 -0.19 -0.32

BNM -0.09 0.02 0.07 -0.07 -0.18 -0.07 -0.05 -0.15 -0.07 0.06 -0.18 -0.13 -0.07

ATDOC -0.26 0.08 0.05 -0.26 -0.44 -0.43 -0.33 -0.44 -0.43 0.01 -0.26 -0.19 -0.38

MCD 0.19 0.04 -0.04 0.18 0.13 0.18 0.26 0.14 0.18 0.18 -0.03 0.34 0.24

Related documents

• RESAMPL-UDA: Leveraging foundation models for unsupervised domain adaptation in biomedical images#922978
• Domain Adaptation for Breast Cancer Computational Pathology: Evaluating Feature-Level and Pixel-Level Approaches Under Leave-One-Domain-Out Framework#215387
• CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Survey#1049631
• Semi-Supervised Domain Adaptation with Latent Diffusion for Pathology Image Classification#986815
• Unsupervised domain adaptation for multitask image analysis in realistic context with extreme label shift; Application to the CTAO first large sized telescope#630386