arXiv CSopen access

ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts

ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts Jiawen Li1∗ , Tian Guan1∗ , Huijuan Shi3∗ , Xitong Ling1 , Mingxi Fu1 , Anjia Han3+ , Chao He2+ , Yonghong He1,4+

arXiv:2607.09526v1 [cs.CV] 10 Jul 2026

1

Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China 2 Department of Engineering Science, University of Oxford, Oxford, UK 3 Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China 4 Medical Optical Technology R&D Center, Research Institute of Tsinghua, Pearl River Delta, Guangzhou, China ∗ Contributed equally +

Corresponding Authors: Anjia Han ([email protected]), Chao He ([email protected]) Yonghong He ([email protected])

Abstract Foundation models are reshaping computational pathology, yet their capabilities remain shaped by pretraining objectives, data sources, and spatial scales, fragmenting complementary expertise across separate backbones. Here we present ALICE, a unified foundation model trained through multi-stage agglomerative distillation that sequentially distills eight vision-only, vision–language, and slide-level teacher models into dedicated modules of a single backbone. ALICE is pretrained on 24,985,184 tile-level pathology images and 155,604 high-resolution images, and evaluated across 21 task scenarios, 96 downstream tasks, and 48 data sources, spanning region-of-interest tissue analysis, vision–language multimodal evaluation, and whole-slide clinical assessment. In all three evaluation settings, ALICE achieved the best average rank among task-matched pathology foundation models. These results demonstrate that agglomerative distillation can consolidate complementary capabilities from specialized models into a unified backbone for broad computational pathology applications. The model is available at https://github.com/WonderLandxD/ALICE.

1

Introduction Histopathological assessment of tissue remains the cornerstone of clinical oncology, forming the basis for diagnostic, prognostic, and treatment decisions across many cancer types 1 . Digitizing slides into whole-slide images (WSIs) has created unprecedented opportunities to develop computational tools that enhance pathology analysis at scale 2,3 . Prior deep learning methods have shown potential for specific tasks such as tumor detection, molecular biomarker prediction, and survival analysis 4–7 . However, most of these models are trained from scratch on task-specific, typically small labeled cohorts, limiting their performance and generalization capabilities. Largescale foundation models pretrained on massive histopathology datasets through self-supervised or multimodal learning have fundamentally changed this paradigm, providing transferable vision representations adaptable to a wide range of downstream clinical tasks with substantially reduced annotation requirements 8–11 . Consequently, a growing number of pathology foundation models (PFMs) have been developed, demonstrating substantial potential across diverse clinical applications. Despite their common goal of learning general-purpose representations, existing PFMs are developed under different pretraining paradigms, leading to distinct but fragmented capabilities 12,13 . For instance, vision-only models, such as UNI 8 and Virchow 14 , rely on self-supervised learning to capture dense morphological representations, but they lack explicit alignment with pathological concepts and language-level semantics. Vision-language models, such as CONCH 9 and MUSK 15 , rely on multimodal contrastive or generative learning to align visual features with textual semantics, but they underperform on fine-grained visual discrimination tasks 16 . Slide-level models, such as GigaPath 11 and TITAN 17 , rely on whole-slide context or weak clinical signals to capture global tissue information, but their practical transferability remains constrained, as they can be difficult to fine-tune or insufficiently expressive for broad downstream use. Therefore, no existing model consistently covers the full spectrum of morphological, semantic, and slide-level capabilities required across pathology tasks. A natural way to address this fragmentation is to use a unified architecture to integrate these capabilities. In the general computer vision community, agglomerative models 18,19 have shown that experts trained with distinct objectives can be distilled into a unified backbone, treating pretrained models as complementary knowledge sources rather than isolated alternatives. In computational pathology, GPFM 20 has provided a first step by showing that unified knowledge distillation can improve the generalization of PFMs across diverse clinical tasks. However, pathology expertise spans local morphology, language-aligned diagnostic concepts, and whole-slide clinical context. Existing distillation frameworks primarily strengthen general visual representations but do not explicitly organize these heterogeneous sources of expertise by modality, scale, and level of clinical abstraction. Thus, to build a truly general-purpose PFM, a more structured aggregation strategy is required to absorb complementary expert knowledge step by step into a single backbone. Here, we introduce Agglomerative Learning via Integrated Computational pathology Embedding (ALICE), a general-purpose histopathology foundation model that provides a unified representation for diverse computational pathology tasks spanning region-of-interest (ROI) tissue analysis, vision-language alignment, and whole-slide clinical analysis. ALICE is trained through a 2

Pathology image (low res.)

b.

Image token grid

Vision-only PFMs

MUSK

H-Opt-1

Pathology image (high res.)

Image token grid

Slide-level PFMs KEEP

CONCH

TITAN

CARE

Alignment training

Alignment training

Alignment training

ALICE (vision-only stage)

ALICE (multimodal stage)

ALICE (slide-level stage)

Unused

Training part Frozen part

Multimodal transformer

Vision-only transformer

Patch embedding

Patch embedding

Training part

Slide-level transformer

Tasks with vision-only PFMs (using low resolution image features)

Slide-level transformer

e.

0.709

0.782

0.759

Tasks with multimodal PFMs (using low resolution image & text features)

0.793

ROI image-to-text retrieval

0.566

0.717

0.656 0.824

ROI image-to-image retrieval 0.834

0.718 0.840

0.599 0.607

ROI semantic segmentation (PMT)

UNI-2

Virchow-2

GPFM

ALICE

H-Opt-1

0.688

0.689

0.716 0.755

WSI few-shot classification (10-shot PathPT)

0.693 0.760

0.648 0.631

0.697

0.842

ROI instance segmentation (PMT)

f.

Tasks with slide-level PFMs (using high resolution WSI features) WSI diagnosis classification (KNN)

0.623

WSI multimodal analysis (MICA)

0.732

ROI few-shot 0.696 classification (8-shot ProtoNet)

Slide-level transformer Training part

ROI classification (zero-shot) ROI classification (KNN)

0.742

Multimodal transformer

Vision-only transformer

Unused

ROI classification (linear probe) WSI classification (ABMIL)

Unused

Frozen part

Multimodal transformer

Vision-only transformer

Unused

d.

c.

Image token grid

Multimodal PFMs

Virchow-2

UNI-2

Pathology image (low res.)

Patch embedding

a.

ROI text-to-image retrieval

0.652

MUSK

KEEP

CONCH

ALICE

0.449 0.392

0.713

0.568

0.716

WSI few-shot classification (8-shot, linear probe)

WSI diagnosis classification (linear probe)

0.670

0.761 0.720

ROI Visual Question Answering

WSI zero-shot classification (MI-zero)

0.675

WSI fine-tuning classification

0.750

0.550 0.568

0.752

WSI survival Prediction (Coxph)

WSI slide-to-slide retrieval TITAN

WSI biomarker prediction

0.623

CARE

ALICE

Figure 1: Overview of ALICE training and evaluation. a-c, Multi-stage agglomerative distillation framework for ALICE. a, In the vision-only stage, ALICE learns morphology-oriented representations from pathology tile images by aligning with three vision-only PFMs, UNI-2, Virchow-2, and H-Opt-1. The patch embedding module and vision-only transformer are trained, whereas the multimodal and slide-level transformers are inactive. b, In the multimodal stage, the pre-trained visual encoder is frozen, and ALICE learns language-aligned representations from three multimodal PFMs, MUSK, KEEP, and CONCH. c, In the slide-level stage, ALICE extends its representation to whole-slide analysis by aligning high-resolution pathology image features with two slide-level PFMs, TITAN and CARE. The slide-level transformer is trained while the visual backbone remains frozen. d-f, Benchmark evaluation of ALICE against task-matched PFMs across seven vision-only, seven vision-language, and seven slide-level task scenarios. Radar plots show average performance for each task scenario.

3

multi-stage agglomerative distillation strategy using 24,985,184 low-resolution and 155,604 highresolution histopathology images, with eight pathology foundation models trained under different data sources, learning objectives, and spatial scales serving as teachers (Figure 1a-c). Specifically, ALICE first learns morphology-oriented visual representations from three vision-only models, UNI-2 8 , Virchow-2 21 , and H-Opt-1 22 . It then incorporates language-aligned representations from three vision-language models, CONCH 9 , MUSK 15 , and KEEP 23 , enabling tissue patterns to be associated with diagnostic concepts expressed in natural language. Finally, ALICE learns whole-slide contextual representations from two slide-level models, TITAN 17 and CARE 24 , extending its applicability from local tissue analysis to direct WSI-level analysis. To evaluate ALICE, we establish a comprehensive benchmark organized around three complementary evaluation settings: vision-only, vision-language multimodal, and slide-level analysis. The benchmark spans 21 task scenarios, comprising 96 downstream tasks across 48 data sources (Figure 1d–f). Across all three evaluation settings, ALICE achieved the best average rank among task-matched PFMs, exceeding the second-best model by 1.79, 6.39, and 3.04 percentage points. Compared with the average performance of all remaining models, ALICE further improved performance by 3.10, 7.41, and 4.00 percentage points, respectively. These results demonstrate that structured agglomerative distillation can consolidate morphology-oriented, language-aligned, and whole-slide contextual expertise into a single framework, establishing ALICE as a broadly applicable foundation model for digital pathology.

Results ALICE outperforms vision-only PFMs in visual feature transfer We first evaluated ALICE in a vision-only feature transfer setting, a common deployment scenario in computational pathology in which ROIs or low-resolution tiles from WSIs are encoded into embeddings for downstream diagnostic, retrieval, and segmentation tasks 8 . We used each PFM as a frozen feature extractor and assessed the resulting embeddings either directly or with lightweight task-specific adapters. ALICE was compared with UNI-2, Virchow-2, H-Opt-1, and GPFM across seven representative evaluation scenarios, including ROI classification, image-to-image retrieval, ROI semantic and instance segmentation, few-shot classification, and weakly supervised WSI diagnostic classification using multiple instance learning (MIL). At the ROI level, ALICE showed the strongest overall transferability across diverse evaluation protocols (Figure 2a). ALICE achieved the best average ranks under both linear probing and KNN classification settings, with average ranks of 1.25 and 1.58, respectively. ALICE also ranked first in image-to-image retrieval, with an average rank of 1.33 based on MVAcc@5. Although ALICE performed slightly below UNI-2 in ROI instance segmentation under the PMT setting (0.599 versus 0.607, AP@50), it still achieved the best overall rank across the seven segmentation tasks. Detailed per-dataset ROI classification, retrieval and segmentation results are provided in Extended Data Figure 2 and Extended Data Table 32-35. The advantage of ALICE was also particularly pronounced in data-efficient and non-parametric transfer settings. In few-shot classification, ALICE achieved an overall average rank of 1.08 across 12 ROI tasks, outperforming the second-best model by 1.50 rank positions, and consistently surpassed GPFM across all shot numbers in both 4

ROI classification (KNN, 12 tasks)

ROI classification (linear probe, 12 tasks)

1.33

1.58

1.25 ALICE

ALICE

b.

5 4 3 2 1 Average rank by BAcc

5 4 3 2 1 Average rank by AP@50 & DICE

Query image (label: tumor)

1.0 0.8

BAcc

0.4

GPFM

ALICE

0.0 1

Top-3 retrieved images 0.477

0.477

0.463

0.529

0.523

0.521

ALICE

0.6 0.4

2

4

8

16

32

64

128 256

0.2

GPFM

Source: CRC-100K 1

2

4

8

16

32

64

Top-3 retrieved images

Mucus

Gastric mucosa

Debris

Areas of ulceration

Muscularis propria

Adventitial tissue

Source: BRACS

Vital tumor tissue Oesophageal mucosa

UMAP 1

UMAP 1

Adipose

UMAP 2

0.576

0.502

0.572

ALICE

UMAP 2

e.

UNI-2

Virchow-2

P<0.0001

P=0.45

P=0.17

P=0.0009

P=0.04

GPFM

ALICE

P<0.0001

P=0.0005

P=0.001

GPFM

Subspecialty diagnosis P=0.01

P=0.10 P=0.12

H-Opt-1

Subtyping & grading

Screening & detection P=0.05

0.505

Submucosa

Regression areas

Submucosal glands

Colorectal adenocarcinoma epithelium

Smooth muscle

0.581

Query image (label: malignant tumors)

Lamina muscularis mucosae

Lamina propria mucosae

Background

Cancer-associated stroma

0.563 Oesophagogastric tissue types (ESCA-Tolkach, 11 classes)

Normal colon mucosa

Lymphocytes

128 256

Shot number

Colorectal tissue types (CRC-100K, 9 classes)

d.

Source: GCHTID

ALICE

0.0

Shot number

0.8

5 4 3 2 1 Average rank by BAcc

GPFM

0.2

1.0

2.58 UNI-2

c.

Colorectal polyps classification (Unitopatho, 6 classes)

0.6

BAcc

Virchow-2 1.86

UNI-2

5 4 3 2 1 Average ranking by MVAcc@5

Breast carcinoma subtyping (BRACS, 7 classes)

4.17

Virchow-2

2.92 UNI-2

0.8

H-Opt-1 3.86

Virchow-2 2.83

UNI-2

5 4 3 2 1 Average rank by BAcc

3.33

H-Opt-1

3.00

Virchow-2

UNI-2

GPFM 4.57

H-Opt-1 3.17

3.08

3.83

3.00 GPFM

3.67

H-Opt-1

Virchow-2

ALICE

3.83 GPFM

3.42

2.33

1.08

ALICE

4.00 GPFM

3.92 H-Opt-1

1.71

ALICE

4.42 GPFM

ROI few-shot classification (8-shot ProtoNet, 12 tasks)

ROI segmentation (PMT, 7 tasks)

ROI retrieval (similarity search, 12 tasks)

Source: BreakHis

a.

P=0.003

P=0.88

P=0.0003 P=0.007

BAcc

0.6

P=0.23

P=0.0002

P=0.23

0.4

0.2

m ph

Ly

Ly

Pr

os ta

te (P ca AN nc D er s no A, c m de 2 ree ph cl (C me no as nin am ta de se g s) el sta m yo sis et as n+ s c ta , 2 re tic e cl (C b u as nin am rd se g s) el en C yo cl ol n+ as or ec , 4 s if ic ta Bo c ln la atio ne ss (H e o m es n un pla et ) C sia as R ta C s si , 4 cre s pr cl en a s in i (S mar se g YS y s) B M s ite , 1 pre 0 d cl ict C as io er se n (S L u vi ca s) YS ng le F L le pi +, s i o th el 4 nt ia c la riag N ( T l le s ss on is es e -s su ion ) m e c al N la lc et ss (C el , 4 if ic PT l lu c a l as tio AC ng se n -N ca s) SC nc O LC er s va ri , 2 ub t (U an cl yp a s in BC ca se g -O rcin s) C om R EA a en al Br N , 5 sub ce ea t l y st c l (D c la pin ca ss H arc M rc es g C ino in ) -R m om C a a C co , 5 sub Br ar c l ty p ea se as in st se g (B gra ca s) R ine rc AC d in om S, su a bt 3 fin cl yp as in ese g (B gra Pr s) R ine os AC d t a Lu te S, su ng ca 7 bty ca c nc la pin nc e ss er (P r G es g su AN lea ) bt D so yp A, n in g 6 g cl rad an as in se g ( S d in C N s) YS va S FL sio fin +, n egr 7 gra a cl as din ( E in e se g BR d s) AI tum C N N o S S, r d tu i 3 m 0 ag or n cl as os (E -fam se is BR il s) AI y c N la s S, si C f N 9 S cl icat as io (E tum se n BR or Pe s) di AI W at N HO ric S, Pe ra 4 gra di cl re at as din ric t se g (K um ra s) id or re R cl tu a re ass m or , 4 if i c fin cl as atio (K e-g se n id rai s) R ar ned e, 13 sub cl typ a s in se g s)

0.0

Figure 2: Vision-only feature transfer performance of ALICE. Caption on next page.

5

(Previous page.) Figure 2: Vision-only feature transfer performance of ALICE. a. Average ranks of ALICE and four vision-only pathology foundation models across ROI-level transfer tasks, including linear-probe classification, KNN classification, image-to-image retrieval, segmentation using Plain Mask Transformer (PMT), and 8-shot few-shot classification with ProtoNet. b. ProtoNet few-shot ROI classification performance of ALICE and GPFM across different shot numbers on breast carcinoma subtyping (BRACS) and colorectal polyp classification (Unitopatho). c. Representative cross-source image-to-image retrieval examples comparing ALICE and GPFM. Query images and the top three retrieved images are shown with similarity scores; green checkmarks and red crosses indicate diagnostically matched and mismatched retrievals, respectively. d. UMAP visualization of ALICE embeddings for colorectal tissue typing in CRC-100K and oesophagogastric tissue typing in ESCA-Tolkach. e. Weakly supervised WSI classification using frozen vision-only features and an ABMIL classifier across 19 clinical tasks grouped into screening and detection, subtyping and grading, and CNS subspecialty diagnosis.

breast carcinoma subtyping and colorectal polyp classification (Figure 2b, detailed results can be found in Extended Data Figure 3 and Extended Data Table 36-47). In cross-source image-toimage retrieval, ALICE more reliably retrieved morphologically and diagnostically matched tissue regions than GPFM (Figure 2c). UMAP visualization further showed clear class-wise separation of ALICE features in colorectal and oesophagogastric tissue typing tasks (Figure 2d). To investigate whether the low-resolution image features extracted by ALICE can support a standard weakly supervised slide-level diagnostic workflow, we further used ABMIL 25 as a feature aggregator for WSI-derived tile images. We evaluated this setting across 19 clinical WSI classification tasks spanning early cancer detection, tumor typing and grading, and subspecialty diagnosis of the central nervous system. ALICE achieved best performance among the evaluated PFMs on 17 tasks and significantly outperformed the second-best model on 12 tasks (Figure 2e and Extended Data Table 48). These results indicate that the advantages of ALICE are not limited to ROI-level representation learning, but can also be translated into improved weakly supervised WSI classification when combined with the standard MIL aggregator. ALICE outperforms multimodal PFMs with text and vision features Pathology interpretation is inherently multimodal, requiring the integration of visual morphology with diagnostic terminology, textual descriptions, and clinical information 26 . By aligning low-resolution histopathology image features with textual representations and transferring them to downstream tasks, multimodal PFMs can support a broader range of diagnostic workflows than vision-only models, including zero-shot classification, cross-modal retrieval, visual question answering (VQA), and text-guided WSI analysis. We compared ALICE with MUSK, KEEP, and CONCH, and benchmarked it across seven representative multimodal task scenarios: ROI zeroshot classification, ROI image-to-text and text-to-image retrieval, ROI VQA, WSI zero-shot classification, WSI few-shot classification, and multimodal WSI clinical prediction. In the ROI setting, ALICE achieved the best overall performance (Figure 3a), ranking first across 12 zero-shot classification tasks with an average rank of 1.50 based on balanced accuracy. 6

a.

ROI cross-modal retrieval (similarity search, 3 data source)

ROI zero-shot classification (similarity search, 12 tasks)

1.17

1.50

3.17

2.00

1.83

3.17

4 3 2 1 Average rank by Recall@50

c.

0.9

0.6

0.8

0.5

0.7

BAcc

BAcc

1.0

0.4 0.3 0.2

MUSK

KEEP

CONCH

ALICE

5

10

15

Low-power view of an IDC-P in prostate biopsy showing several large, irregular, branching cribriform glands with dense cribriform architecture and focal intraluminal comedonecrosis. Top-3 retrieved images

0.619

Ground truth caption

0.6 0.5

MUSK

0.4

0.1 1

Query text

Pediatric rare tumor classification (KidRare, 4 classes)

0.7

4 3 2 1 Average rank by BAcc

4 3 2 1 Average rank by BAcc

4 3 2 1 Average rank by Acc

b. Brain tumor subtyping (EBRAINS, 30 classes)

3.83 MUSK

MUSK

MUSK

0.8

KEEP

2.30

MUSK

2.50

KEEP

KEEP

2.75

4 3 2 1 Average rank by BAcc

CONCH

2.97

KEEP

MUSK

2.50

CONCH

CONCH

3.83

2.75 KEEP

ALICE

3.67

2.87

CONCH

1.17

ALICE

ALICE

3.00 CONCH

1.17

1.87

ALICE

ALICE

WSI few-shot classification (10-shot PathPT, 6 tasks)

WSI zero-shot classification (MI-zero, 6 tasks)

ROI VQA understanding (similarity search, 10 data source)

Intraductal carcinoma of the prostate is characterized by glands that are much more likely to have comedo necrosis.

KEEP

CONCH

ALICE

0.3

20

1

5

10

15

20

Shot number

Shot number

Ground truth caption

0.615

Central zone prostate glands are large complex acini with cribriform and intralum inal ridges with dense stroma . Central zone glands may be mistaken for HGPIN. Unlike HGPIN, central zone glands lack significant nuclear atypia, including nuclear enlargement and prominent nucleoli.

d.

20 μm

0.612

Ground truth caption

2 mm

2 mm

20 μm

Nephroblastoma WSI (KidRare)

e.

Heatmap and top-2 highest-scoring patches

MUSK Breast cancer

CONCH Ovarian cancer

P=0.03

P=0.005

P=0.63

f.

ALICE

P=0.72

0.8

P=0.01

P=0.05 P=0.39

P=0.39

0.6

BAcc

P=0.05

0.4

0.4

0.2

0.0

0.0

Ax

illa ry

ly

m ph (B nod C N em B, e 2 tas cl as tas Va se is sc s) ul ar ( H in v an as co io ck n d , 2 et cl ect a s io se n Tu s) (P m TR o r C or - H ig G in SO cl M el C ass an , 2 if om cl icat a s io a re se n la s) ps e pr (V e is dic io M tion el , 2 w/ cl o p as rio se r s)

0.2

ry illa Ax

P=0.003

1.0 P=0.34

ALICE (WSI-only) ALICE (Multimodal)

P=0.02

P=0.003

ALICE (text-only)

Skin melanoma

ly m ph (B nod C N em Ax B, e illa 2 tas ry cl as tas ly m se is ph s) (B nod C N em B, e 3 tas Ke cl as tas ra se is tin s) (H izi an ng Ly m co SC ph ck C ov ,2 g as cl rad cu a s in la se g r s) ( H in v an as c o io n ck , 2 det Va cl ect sc a s io ul se n ar s) (H inv an as c o io Tr ck n d ea , 2 et t cl ect (P me a s io TR nt se n C res s) -H p G on SO se C pr ,2 e T cl dict (P um as io TR or se n C or s) M - H ig el an G in SO cl om a C ss a , 2 if re la cl icat ps as io e se n p s) (V re d is io ic ti M on e M w l, el / 2 an cl o p as rio om se r a s) (V re is lap io M se el pr ,2 e cl dict as io se n s)

BAcc

0.6

KEEP

Head and neck cancer

1.0

0.8

The Gleason pattern 4 cribriform carcinoma spectrum includes small but irregular cribriforming glands.

Figure 3: Multimodal feature transfer performance of ALICE. Caption on next page.

7

(Previous page.) Figure 3: Multimodal feature transfer performance of ALICE. a. Average ranks of ALICE and three multimodal pathology foundation models across multimodal evaluation settings, including ROI zero-shot classification, ROI cross-modal retrieval, ROI visual question answering understanding, WSI zero-shot classification using MI-Zero, and WSI few-shot classification using 10-shot PathPT. b. WSI PathPT few-shot classification performance of ALICE, MUSK, KEEP, and CONCH across different shot numbers on brain tumor subtyping (EBRAINS) and pediatric rare tumor classification (KidRare). c. Representative text-to-image retrieval example using a prostate pathology text description. The top three retrieved images are shown with similarity scores and corresponding ground-truth captions. d. Representative WSI-level prediction example for nephroblastoma from KidRare, showing the original WSI, attention heatmap, and the top two highest-scoring patches. e. Multimodal WSI clinical analysis using MICA adaptors across nine tasks spanning breast cancer, head and neck cancer, ovarian cancer, and skin melanoma. f. Modality ablation of ALICE on representative clinical WSI tasks, comparing text-only, WSI-only, and multimodal variants.

It also achieved the best average rank across three data sources in image-to-text and text-to-image retrieval, with an average rank of 1.17 based on recall@50. Furthermore, ALICE achieved the best average rank in ROI VQA, outperforming competing models in 6 of 10 tasks and reaching an overall average rank of 1.87. These results demonstrate that ALICE learns coordinated visual and textual representations that generalize to language-guided pathological interpretation. More detailed ROI vision-language transfer results are shown in Extended Data Figure 4 and Extended Data Table 49-51. The strengths of ALICE were also evident in slide-level multimodal transfer. In six WSI zeroshot classification tasks using MI-Zero 27 , ALICE achieved the best average rank of 1.17 based on balanced accuracy (See detailed results in Extended Data Table 52). In the 10-shot PathPT 28 setting, ALICE also outperformed the baseline models, achieving an average rank of 1.17 across six few-shot classification tasks. For example, in brain tumor subtyping and pediatric rare tumor classification, ALICE achieved competitive or better average performance across sample sizes (Figure 3b), demonstrating the effectiveness of its multimodal representations in data-limited slidelevel diagnostic settings. More WSI few-shot classification results using PathPT can be found in Extended Data Figure 5 and Extended Data Table 53-58. Qualitative analysis further confirmed the multimodal alignment capability learned by ALICE. In text-to-image retrieval, the histological regions retrieved by ALICE were morphologically consistent with the query descriptions and corresponding ground truth labels, including prostate lesions with cribriform structures and comedo necrosis-related features (Figure 3c). In WSI-level prediction, by mapping patch-level target-class scores back onto the original WSI, the evidence heatmap for nephroblastoma highlighted diagnostically significant tumor regions, and the highestscoring regions showed cellular and structural patterns consistent with the slide-level prediction (Figure 3d). These examples demonstrate that ALICE can link textual pathology concepts with local visual evidence. More representative ROI text-to-image, image-to-text, and WSI heatmaps are shown in Extended Data Figure 6, 7, and 8. 8

Clinical WSI analysis often extends beyond morphology-based slide classification, as many clinically relevant endpoints require joint interpretation of histological evidence and disease-specific clinical context. Tasks such as vascular invasion detection 29 and treatment response prediction 30 may involve diagnostic clues that cannot be fully captured by image morphology alone. Therefore, we further evaluated ALICE using the MICA 31 adapter on nine multimodal clinical WSI analysis tasks covering breast cancer, head and neck cancer, ovarian cancer, and cutaneous melanoma (Figure 3e and Extended Data Table 59). ALICE achieved strong performance across these tasks, including axillary lymph node metastasis classification, keratinizing squamous cell carcinoma grading, lymphovascular and vascular invasion detection, ovarian cancer treatment response prediction, tumor origin classification, and melanoma recurrence prediction. To examine the contribution of each modality, we compared text-only, WSI-only, and multimodal versions of ALICE on representative tasks (Figure 3f). The multimodal version achieved the highest performance across all evaluation settings, particularly in vascular invasion detection and tumor origin classification. These results demonstrate that ALICE can use clinical text representations as supplementary information to WSI morphology, providing more robust multimodal slide-level inference for complex clinical diagnostic tasks. ALICE outperforms slide-level PFMs with high-resolution WSI features Unlike ROI-level or tile-level feature transfer, slide-level pathology foundation models aim to encode the global morphology, spatial organization, and tissue heterogeneity of whole-slide images into compact slide representations 11,17,32,33 . This setting is particularly important for clinical tasks in which diagnostic evidence is distributed across large tissue areas or depends on the overall architectural context of the slide, such as biomarker prediction and survival analysis. We compared ALICE with two representative slide-level PFMs, TITAN and CARE, across seven representative WSI task scenarios, including WSI diagnostic classification with KNN and linear probing, WSI few-shot classification, slide-to-slide retrieval, biomarker prediction, survival analysis, and taskspecific fine-tuning. ALICE achieved the strongest overall performance across slide-level transfer tasks (Figure 4a). In WSI diagnostic classification, ALICE achieved the best average rank across 19 tasks under both KNN and linear probing settings, with average ranks of 1.53 and 1.47 based on balanced accuracy, respectively. ALICE also achieved the best average rank in slide-to-slide retrieval across 21 tasks (1.62, MVAcc@3), WSI biomarker prediction across seven tasks (1.71, AUC), and WSI survival analysis across six tasks (1.67, C-index). These results indicate that WSI features extracted by ALICE generalize across both morphology-driven and clinically oriented slide-level prediction tasks. Detailed slide-level diagnostic, biomarker and survival results are provided in Extended Data Figure 9 and Extended Data Table 60-61, 63-64. The advantage of ALICE was further supported by data-efficient and fixed-feature analyses. In few-shot pan-cancer classification, ALICE consistently achieved strong balanced accuracy across different shot numbers on both CMB and CPTAC, suggesting that its slide-level representations remain effective when only limited labeled slides are available (Figure 4b). UMAP visualization on CPTAC showed clear separation among nine cancer types, including breast cancer, clear cell renal cell carcinoma, colon adenocarcinoma, glioblastoma, head and neck squamous cell carci9

noma, lung adenocarcinoma, lung squamous cell carcinoma, ovarian cancer, and pancreatic ductal adenocarcinoma (Figure 4c). In slide-to-slide retrieval, ALICE identified slides with identical or closely related diagnostic labels from three data sources, further supporting the ability of its high-resolution WSI features to capture global diagnostic similarity (Figure 4d). More representative slide-to-slide retrieval examples are shown in Extended Data Figure 10. Detailed few-shot slide-level classification can be found in Extended Data Table 65. Beyond frozen feature transfer, clinically specific WSI tasks often require adapting slide-level features to fine-grained morphological patterns and molecularly associated visual signals 17 . We therefore assessed whether high-resolution WSI features extracted by ALICE could be effectively specialized through task-specific fine-tuning (Figure 4e and Extended Data Table 66-67). Specifically, we jointly fine-tuned all slide-level branch parameters together with the downstream task-specific adapter. The evaluation included morphological diagnosis tasks, covering breast carcinoma subtyping, lymph node metastasis classification, and pediatric rare tumor classification, as well as biomarker prediction tasks, including breast cancer ER and PR status prediction and PBRM1, BAP1, and SETD2 mutation prediction in renal cell carcinoma. Fine-tuned ALICE achieved the best performance across the evaluated diagnostic and biomarker prediction settings, with notable gains in fine-grained tumor subtyping, lymph node metastasis burden classification, rare tumor subtyping, and renal cell carcinoma mutation prediction. These findings indicate that ALICE learns high-resolution WSI representations with strong adaptation capacity, enabling taskspecific fine-tuning to extract clinically relevant slide-level signals more effectively than existing slide-level PFMs. Discussion In this paper we introduced ALICE, a unifed PFM that combines the complementary strengths of existing models into a single backbone. Although recent PFMs have substantially expanded the capabilities of computational pathology, most remain specialized by design: some excel at capturing local tissue morphology 8,14,22 , others at aligning histology with diagnostic language 9,15,23 , and others at modeling whole-slide context 11,17,24,32 . This specialization creates a fragmented landscape in which different clinical tasks often require different pretrained models. ALICE addresses this challenge through multi-stage agglomerative distillation, progressively integrating morphology-oriented, language-aligned, and slide-level expertise into a unified representation framework. Across vision-only, multimodal, and slide-level evaluations, ALICE showed broad transferability across ROI-level tissue analysis, language-guided pathology tasks, and highresolution WSI representation learning. These findings suggest that the diverse capabilities of current PFMs can be consolidated into a single broadly applicable model, providing a more unified foundation for computational pathology across local, semantic, and whole-slide clinical contexts. Several observations from our benchmark are noteworthy. First, ALICE did not simply reproduce the average behavior of its teacher models. Instead, it retained strong local visual discrimination from vision-only PFMs, acquired language-guided interpretation from vision-language PFMs, and remained effective for high-resolution WSI representation learning. This suggests that the apparent boundaries between different families of PFMs are not fixed, but can be softened through structured knowledge integration. Second, the benefits of ALICE were observed across distinct 10

a.

WSI diagnostic classification (KNN, 19 tasks)

W S I re t ri e v a l (similarity search, 21 tasks)

WSI diagnostic classification (linear probe, 19 tasks)

1 .6 2

1 .4 7

1 .5 3

2 .2 4

1 .9 5

TITAN

TITAN

3 2 1 Average rank by MVAcc@3

3 2 1 Average rank by BAcc

3 2 1 Average rank by C-index

3 2 1 Average rank by AUC

b.

c. Pan-cancer classification (CMB, 9 classes)

Different cancer types (CPTAC, 9 classes)

Pan-cancer classification (CPTAC, 9 classes)

1 .0

1 .0

0 .8

0 .8

0 .6

0 .6

BAcc

0 .4 0 .2 CARE

0 .4

ALICE

0 .0

TITAN

CARE

ALICE

0 .0

1

2

4

8

16

32

1

2

4

8

16

32

Head and neck squamous cell carcinoma (HNSC) Lung adenocarcinoma (LUAD)

Pancreatic ductal adenocarcinoma (PDA)

Lung squamous cell carcinoma (LSCC)

UMAP 2

Shot number

Shot number

d.

Colon adenocarcinoma (COAD)

Clear cell renal cell carcinoma (CCRCC) Glioblastoma (GBM)

0 .2

TITAN

Ovarian cancer (OV)

Breast cancer (BRCA)

UMAP 1

BAcc

2 .1 7

2 .2 9

TITAN

TITAN

3 2 1 Average rank by BAcc

CARE

CARE

1 .9 5

2.11

2 .1 7

2 .0 0

CARE

CARE

TITAN

ALICE

ALICE

2 .5 8

2 .3 7 CARE

1 .6 7

1 .7 1

ALICE

ALICE

ALICE

WSI survival analysis (c o x p h , 6 t a s k s )

WSI biomarker prediction (linear probe, 7 tasks)

Top-5 retrieved slide

Query slide (label: LCA)

Label: LCA 0.684

5 mm

2 mm

Source: CMB

Label: LUAD

Label: LCA 0.656

2 mm

2 mm

Source: CMB Source: CMB

Label: LCA

Label: kidney metastasis

0.630

0.658

0.625

1 mm

High

2 mm

Source: CMB

Source: CPTAC

Source: SYSBM

Cosine Similarity

Low

e. CARE (linear probe)

TITAN (linear probe)

ALICE (linear probe)

CARE (fine-tuning)

Morphological diagnosis

1.0

P<0.0001

0.8

P=0.008

P=0.0005

P=0.03

P=0.02

P=0.04

P=0.002 P=0.02

P=0.23

P=0.0001

ALICE (fine-tuning)

Biomarker prediction P=0.03

P=0.01

1.0

TITAN (fine-tuning)

P=0.0001

P=0.02

P<0.0001 P=0.0003

P=0.002 P=0.002

0.8

P=0.17

P=0.24

P=0.0004

0.6

A UC

P=0.78

0.4

0.0

0.0

Br

Br

ea st ca n

ce r

ER (B sta C tus N B, pr ea 2 edi st cl ca a s ct i o R nc se n en e s) rP al ce R ll (B sta ca C tus rc N in B, pr om 2 edi a cl PB as ctio (M R se n R M s) U en 1 T al -H m ce ET uta ll - R tio ca C np rc C in , 2 r ed om cl ict a as i o B R (M A se n en P s) U al T- 1 m ce H u ET t a ll ca - R tio rc C np in C om , 2 r ed a cl ict SE as i o (M T se n D s) U T- 2 m H u ET ta - R tio C np C , 2 red cl ict as i o se n s)

0.2

co ar C A BR

0.4

0.2

se ( B gr a R ine AC d S, su 3 bty Ly BR cl m as pi n C ph A se g no fin s) de em (B gra et i R as AC ned ta S, su si s 7 bt y co cl Ly as pi n ar s m se g (C eph s) A M gr a no E L i ne de YO d c m et N l as as +, s ta 2 ific si cl at s as i o fin se n ( C es) A M gr a EL ine YO d c N l as Pe +, s di 4 ific at cl at r ic as i o se n ra s) re Pe t ( K um di at i d or r ic R c ar las ra e, s re 4 ific tu cl at m as i o or se n fin s) ( K e- g i d r ai R n ar ed e, 13 sub cl typ as i n se g s)

BAcc

0.6

Figure 4: Slide-level feature transfer performance of ALICE. Caption on next page.

11

(Previous page.) Figure 4: Slide-level feature transfer performance of ALICE. a. Average ranks of ALICE and two slide-level pathology foundation models across WSI-level transfer tasks, including diagnostic classification with KNN and linear probing, slide-to-slide retrieval, biomarker prediction, and survival analysis. b. Linear probe few-shot pan-cancer classification performance of ALICE, TITAN, and CARE across different shot numbers on CMB and CPTAC. c. UMAP visualization of ALICE slide-level embeddings for nine cancer types from CPTAC. d. Representative slide-to-slide retrieval example using ALICE slide-level embeddings. A lung cancer adenocarcinoma (LCA) query slide from CMB is used to retrieve the top five most similar WSIs across CPTAC, CMB, and SYSBM data sources. e. Linear probe and fine-tuning performance of slidelevel PFMs on morphological diagnosis and biomarker prediction tasks.

transfer regimes, including frozen feature evaluation, nonparametric retrieval, few-shot learning, multimodal inference, and task-specific fine-tuning. This indicates that agglomerative distillation can produce representations that are not only broadly reusable but also adaptable when downstream supervision is available. Third, the slide-level results support the value of integrating local morphology and whole-slide context for clinically complex endpoints, including fine-grained diagnosis, biomarker prediction, and survival analysis. Together, these findings suggest that future PFM development may benefit from being organized along the continuum of clinical reasoning, from local tissue morphology to semantic interpretation and whole-slide decision-making. Despite these encouraging results, several limitations point to future directions. First, ALICE was evaluated primarily in retrospective settings. Prospective, multi-institutional studies will be needed to assess its robustness under real-world variation in staining protocols, scanner platforms, tissue processing, and patient populations. Second, the current pretraining data from TCGA and HISTAI 34 provide an effective basis for multi-stage agglomerative distillation, but their scale and source distribution can be further expanded. Future work could incorporate larger, more diverse institutional, international, and disease-specific cohorts to improve tissue coverage, domain robustness, and generalization to rare diagnostic entities 35 . Third, although the current multi-stage architecture enables structured integration of vision-only, multimodal, and slide-level expertise, it may not yet be optimal for computational efficiency and streamlined deployment. A more unified and efficient model framework could preserve the benefits of staged knowledge aggregation while reducing structural fragmentation, improving inference speed, and facilitating scalable downstream adaptation. Finally, additional clinically informative modalities, such as immunohistochemistry 36 and genomic profiles, could be integrated to extend ALICE toward a broader computational pathology system. In conclusion, ALICE demonstrates that complementary capabilities from existing pathology foundation models can be integrated into a unified and broadly transferable backbone. By consolidating morphology-oriented, language-aligned, and slide-level expertise, ALICE provides a flexible foundation for computational pathology tasks spanning local tissue analysis, multimodal interpretation, and whole-slide clinical inference. We envision ALICE and its future iterations serving as a general-purpose platform for developing more efficient, scalable, and clinically adaptable pathology AI systems. 12

Methods Pretraining datasets and image preprocessing ALICE pretraining consisted of three stages, including vision-only distillation, multimodal distillation, and slide-level distillation. The first two stages were performed using low-resolution histopathology image patches, and the final stage used high-resolution regions derived from wholeslide images (WSIs). For low-resolution image pretraining, we used TCGA-12K 37 , a large-scale patch dataset containing 24,985,184 image patches of 224 × 224 pixels extracted from approximately 12K WSIs from The Cancer Genome Atlas (TCGA), covering 32 cancer types. The patches were randomly sampled across multiple magnification levels, and tissue-containing patches were retained using HSV-threshold-based tissue filtering. For high-resolution image pretraining, we used TCGA-UT-8K 17 and additional high-resolution image regions cropped from WSIs in HISTAI 34 . TCGA-UT-8K contains 25,495 tumor-centered ROI images of 8192 × 8192 pixels extracted from 9,662 H&E-stained FFPE diagnostic WSIs in TCGA, covering 32 pan-cancer subtyping classes. Each ROI was center-cropped from pathologistannotated tumor contours to include both dense tumor areas and surrounding tissue context. For the HISTAI-derived high-resolution data, tissue-containing regions were tiled at 20× magnification into non-overlapping image regions of 8192 × 8192 pixels. After tissue filtering and quality control, we obtained 130,109 high-resolution regions from 35,363 slides. These regions were distributed across 5 HISTAI subsets, including breast, colorectal, mixed, skin, and thorax cohorts. The resulting TCGA-UT-8K and HISTAI high-resolution feature sets were used for slide-level distillation. Teacher foundation models We selected eight publicly available expert pathology foundation models as teacher models for ALICE pretraining. These models were organized into three groups corresponding to the three stages of agglomerative distillation: vision-only PFMs, vision-language PFMs, and slide-level PFMs. This grouping was designed to cover complementary representation capabilities, including dense morphological encoding, language-aligned diagnostic semantics, and whole-slide clinical context. For the vision-only distillation stage, we used UNI-2 8 , Virchow-2 21 , and H-Opt-1 22 as teacher models. This group was selected to provide strong morphology-oriented visual supervision from large-scale self-supervised histopathology pretraining. UNI-2 is a vision-only PFM based on a ViT-H/14 architecture with 681 million parameters, pretrained with DINOv2 38 on more than 200 million H&E- and IHC-stained histology tiles sampled from over 350,000 slides from Mass General Brigham. Virchow-2 is a vision-only PFM based on a ViT-H/14 architecture with 632 million parameters, pretrained with a pathology-adapted DINOv2 strategy on 3.1 million WSIs from diverse institutions. H-Opt-1 is a vision-only PFM based on a ViT-G/14 architecture with 1.1 billion parameters, pretrained with self-supervised learning on billions of histology images sampled from more than one million H&E-stained slides across over 50 organs. 13

For the multimodal distillation stage, we used CONCH 9 , MUSK 15 , and KEEP 23 as teacher models. This group was selected to introduce language-aligned diagnostic semantics into ALICE, enabling histological morphology to be associated with natural-language descriptions, diagnostic concepts, and pathology-specific knowledge. CONCH is a vision-language PFM based on a ViT-B/16 image encoder paired with a text encoder, jointly pretrained on 1.17 million histopathology image-caption pairs using contrastive captioning. MUSK is a vision-language PFM based on a unified multimodal transformer, pretrained with unified masked modeling on 50 million pathology images and one billion pathology-related text tokens from unpaired sources. KEEP is a knowledge-enhanced vision-language PFM based on a ViT-L/16 image encoder, pretrained with disease-knowledge-guided contrastive learning using a disease knowledge graph containing 11,454 disease entities to organize pathology image-text pairs into semantically structured groups. For the slide-level distillation stage, we used TITAN 17 and CARE 24 as teacher models. This group was selected to provide high-resolution WSI-level supervision and to extend ALICE from local image representation to whole-slide contextual modeling. TITAN is a slide-level PFM that operates on CONCH 1.5 patch features and aggregates them into slide-level representations through a transformer backbone with ALiBi-based self-attention 39 . It was pretrained on 335,645 WSIs through visual self-supervised learning and vision-language alignment using pathology reports and synthetic ROI captions. CARE is a slide-level PFM that also operates on CONCH 1.5 patch features but adopts adaptive region modeling to partition WSIs into morphologically coherent regions. It was pretrained on 34,277 WSIs through visual self-supervised learning and cross-modal alignment with RNA and protein expression profiles. ALICE architecture ALICE is organized into three transformer-based modules that are sequentially activated during pretraining: a vision-only transformer, a multimodal transformer, and a slide-level transformer (Extended Data Figure 1). The vision-only transformer encodes low-resolution histopathology patches and provides the shared image representation backbone. The multimodal transformer is attached after the vision-only transformer to adapt visual tokens for vision-language alignment. The slide-level transformer aggregates high-resolution patch features and their spatial coordinates into WSI-level representations. This modular design allows ALICE to progressively incorporate morphology-oriented, language-aligned, and slide-level knowledge while keeping each stage structurally distinct. The vision-only transformer is implemented as a ViT-H/14 encoder. Input images are first converted into patch tokens through a patch embedding layer, followed by conditional positional embedding 40 and learnable prefix tokens 41 . The prefix tokens comprise three summary tokens for image-level representation learning and additional register tokens to stabilize token-level encoding. The vision-only transformer processes the resulting token sequence and outputs both summarylevel representations and spatial patch-token representations, enabling feature distillation at both global and local levels. The multimodal transformer is implemented as a lightweight self-attention adaptor placed on top of the pretrained vision-only transformer. During multimodal distillation, the vision-only trans14

former is frozen, and only this multimodal transformer is optimized. The module contains two transformer layers and operates on the visual tokens produced by the frozen image encoder. Its role is to transform the morphology-oriented visual representations learned in the first stage into a representation space suitable for alignment with vision-language teacher models. The slide-level transformer is not used during this stage. The slide-level transformer is trained in the final distillation stage using offline patch features extracted from the pretrained image encoder. Specifically, each high-resolution ROI was first divided into several image patches, which were then encoded by the vision-only transformer pretrained in the first stage to obtain fixed patch-level features. In this stage, the vision-only transformer is effectively frozen, the multimodal transformer is unused, and only the slide-level transformer is updated. For each high-resolution region or WSI, the slide-level transformer takes the extracted patch features and their spatial coordinates as input. Patch features are projected from 3,840 dimensions into a 1,024-dimensional token space, and normalized patch coordinates are encoded through a coordinate projection module to provide spatial information. The slide-level transformer uses two learnable summary tokens as global aggregation tokens. These tokens are prepended to the patch-token sequence and are updated together with the patch tokens by two ALiBi 39 transformer blocks. Each block uses multi-head self-attention with spatial bias computed from pairwise distances between patch coordinates, allowing the model to account for the relative organization of tissue regions within the slide. The final slide-level representation is obtained by flattening the two summary tokens into a 2,048-dimensional embedding. Through this architecture, ALICE combines local patch morphology, multimodal semantic adaptation, and spatially aware whole-slide representation learning within a staged transformer framework. Multi-stage agglomerative distillation pretraining Existing PFMs encode complementary forms of expertise, but these capabilities are distributed across separate architectures, objectives, and spatial scales. Inspired by RADIO 18 , we pretrained ALICE through a three-stage agglomerative distillation process that sequentially transfers knowledge from vision-only, vision-language, and slide-level teachers into a unified student representation. In the first stage, ALICE learned morphology-oriented visual representations from three frozen vision-only PFMs: H-Opt-1, Virchow-2, and UNI-2. The student model contained three learnable summary tokens, each assigned to a single teacher, allowing the student to absorb teacher-specific global representations without forcing all teachers into a shared target space. During training, for an image i, each teacher-specific summary token was projected to the corresponding teacher feature dimension and aligned with the frozen teacher CLS embedding using a balanced angular distillation loss: Ltcls =

Θ(Pt (zit ), cti )2 , Disp(Θt )

Θ(x, y) = arccos(cos(x, y)),

  Disp(Θt ) = E Θ(cti , µt )2 ,

(1)

where zit is the student summary token assigned to teacher t, cti is the teacher CLS embedding, Pt is a teacher-specific projection head, and µt is the expected unit direction of teacher CLS embeddings over the training set. This normalization balances teachers with different feature-space 15

dispersions and prevents teachers with larger angular variance from dominating the global distillation objective. To further transfer local morphology, we used shift-equivariant patch distillation, in which the student and each teacher independently received grid-aligned crops sampled from the same augmented image, and patch tokens were matched only at overlapping spatial positions. The patch-level loss is defined as: Ltpatch =

1 X |Ωt |

2

t , Qt (pi,u ) − qi,u 2

(2)

(i,u)∈Ωt

t where pi,u denotes the student patch tokens and qi,u denotes PHI-S-normalized teacher patch tokens at spatial position u, Qt is a teacher-specific patch projection head, and Ωt denotes the set of overlapping spatial positions between the student and teacher crops. This loss transfers local morphological information while preserving spatial correspondence under independently sampled crops. Finally, to stabilize patch-level learning, an exponential moving-average (EMA) copy of the student was maintained as an additional self-distillation target.

In the second stage, ALICE incorporated language-aligned knowledge from three vision-language PFMs: CONCH, MUSK, and KEEP. To preserve the morphology-oriented representations learned in the first stage, the vision-only transformer was initialized from the first-stage checkpoint and kept frozen. The multimodal transformer was then trained on top of the frozen backbone tokens. For each summary token, ALICE combined it with the average patch representation and aligned it with the image embedding using Ltcls . For patch token supervision, each crop was generated at the native input resolution, then spatially resampled onto the student grid before shift-equivariant alignment. In the third stage, ALICE was extended from local tissue representation learning to slide-level representation learning by distilling high-resolution image knowledge from TITAN and CARE. For each 8192 × 8192 high-resolution region, we divided the image into a 16 × 16 grid of nonoverlapping crops, generating 256 tiles per region. Tile-level features were extracted using the first-stage ALICE vision-only transformer and used as fixed inputs to the slide-level transformer. The corresponding teacher slide embeddings were obtained from TITAN and CARE using their required tile-level CONCH 1.5 feature inputs. Each ALICE tile feature was projected into a 1,024dimensional hidden space through a projection and augmented with coordinate embeddings derived from normalized tile coordinates. Two learnable summary tokens were prepended to the tile sequence and jointly processed with all tile tokens by the slide-level transformer. The first summary token was aligned with the TITAN slide embedding, and the second summary token was aligned with the CARE slide embedding using Lclst . Since TITAN additionally produces tile-level output tokens together with its slide embedding, we applied Lpatcht between the student tile tokens and the corresponding TITAN tile tokens at matched spatial positions, providing fine-grained contextual supervision within the slide representation. CARE was used only for slide-level supervision. Through this stage, ALICE learned to aggregate fixed local tissue features into spatially aware slide-level representations.

16

Pretraining settings For all three pretraining stages, we used distributed training on eight 80GB NVIDIA A100 GPUs. For vision-only and multimodal stages, each input image first underwent a shared augmentation pipeline consisting of random resized cropping, horizontal flipping, and color jittering, followed by teacher-specific cropping and normalization for summary-level and patch-level distillation. For the slide-level stage, since no raw image pixels were used, we applied embedding-level feature-brightness perturbation as feature-space augmentation. We used gradient accumulation to keep the total effective batch size fixed at 1,024 across stages. To further enhance the robustness of our ALICE, we adopted DAMP 42 , which applies multiplicative noise to the model weight across three stages. Training was monitored every 1,000 optimization steps, and early stopping was used if the monitored loss did not improve for 5 consecutive intervals. Complete stage-specific hyperparameters are reported in Extended Data Table 1, 2, and 3. Downstream benchmark We evaluated ALICE on a broad downstream benchmark spanning ROI analysis, vision-language multimodal analysis and whole-slide image analysis. The benchmark included 96 downstream tasks from 48 data sources. AGGC 43 was constructed from the AGGC2022 prostate histopathology dataset for ROI-level automated Gleason grading. We conducted experiments on the biopsy subset, which included 37 training WSIs and 16 test WSIs. Using the provided label masks, we extracted non-overlapping 224×224 image patches and retained only patches for which more than 50% of the patch area overlapped with one target annotation mask. The resulting dataset consisted of 42,157 patches classified into five categories: Gleason pattern 3 (10,486), Gleason pattern 4 (5,615), Gleason pattern 5 (106), normal glands (12,776), and stroma (13,174). We used the officially defined AGGC2022 train:test split, yielding 29,366 training patches and 12,791 test patches (69.66%:30.34%). We used AGGC for downstream ROI classification, ROI retrieval, ROI few-shot classification, and ROI zero-shot classification tasks. BRACS 44 was constructed from the BRACS H&E breast histopathology dataset for ROI- and WSI-level breast cancer subtyping. For ROI-level experiments, we used 4,539 annotated ROI images under two label granularities. The 3-class setting consisted of benign tumors (1,837), atypical tumors (1,263), and malignant tumors (1,439), and the 7-class setting consisted of normal (484), pathological benign (836), usual ductal hyperplasia (517), flat epithelial atypia (756), atypical ductal hyperplasia (507), ductal carcinoma in situ (790), and invasive carcinoma (649). We used the officially defined ROI train:test split for both settings, yielding 3,969 training images and 570 test images (87.44%:12.56%). For WSI-level experiments, BRACS consisted of 546 WSIs. The coarse WSI task used benign tumors (264), atypical tumors (89), and malignant tumors (193), and the fine-grained WSI task used normal (43), pathological benign (147), usual ductal hyperplasia (74), flat epithelial atypia (41), atypical ductal hyperplasia (48), ductal carcinoma in situ (61), and invasive carcinoma (132). We used the officially defined WSI train:val:test split for both WSI tasks, yielding 394 training, 65 validation, and 87 test WSIs (72.16%:11.90%:15.93%). We used BRACS for downstream ROI classification, ROI retrieval, ROI few-shot classification, ROI 17

zero-shot classification, WSI classification, and WSI retrieval tasks. BreakHis 45 is a dataset of H&E-stained breast tumor histopathological images for ROI-level tasks. We used the 2-class setting, which included 7,909 images from 82 cases across four magnification levels (40×, 100×, 200×, and 400×). The dataset consisted of two categories: benign tumors (2,480) and malignant tumors (5,429). The benign group included adenosis, fibroadenoma, phyllodes tumor, and tubular adenoma, whereas the malignant group included ductal carcinoma, lobular carcinoma, mucinous carcinoma, and papillary carcinoma. We split the dataset at the case level with stratification by histological subtype, yielding 5,344 training images and 2,565 test images (67.57%:32.43%). We used BreakHis for downstream ROI classification, ROI retrieval, ROI fewshot classification, and ROI zero-shot classification tasks. CCRCC 46 is constructed from a clear-cell renal cell carcinoma tissue classification dataset for ROI-level renal tissue recognition. The dataset consisted of 52,713 ROI images classified into six categories: blood (996), cancer (13,057), empty or background regions (16,026), normal tissue (8,652), other tissue (8,522), and stroma (5,460). We split the dataset at the slide level with case stratification, yielding 36,900 training images and 15,813 test images (70.00%:30.00%). We used CCRCC for downstream ROI classification, ROI retrieval, ROI few-shot classification, and ROI zero-shot classification tasks. Chaoyang 47 is a colorectal biopsy histopathology dataset for ROI-level gastrointestinal tissue classification. The dataset consisted of 6,160 ROI images classified into four categories: normal (1,816), serrated (1,163), adenocarcinomas (2,244), and adenomas (937). We used the officially defined train:test split, yielding 4,021 training images and 2,139 test images (65.28%:34.72%). We used Chaoyang for downstream ROI classification, ROI retrieval, ROI few-shot classification, and ROI zero-shot classification tasks. CRC-100K 48 is constructed from the NCT-CRC-HE-100K and CRC-VAL-HE-7K H&E colorectal histopathology patch datasets for tissue classification. The dataset consisted of 87,174 image patches classified into nine tissue categories: adipose tissue (ADI, 9,663), background (BACK, 9,299), debris (DEB, 9,548), lymphocytes (LYM, 9,879), mucus (MUC, 8,151), smooth muscle (MUS, 11,420), normal colon mucosa (NORM, 7,751), cancer-associated stroma (STR, 8,777), and colorectal adenocarcinoma epithelium (TUM, 12,686). We used NCT-CRC-HE-100K as the training set and CRC-VAL-HE-7K as the held-out test set, yielding 79,994 training patches and 7,180 test patches (91.76%:8.24%). We used CRC-100K for downstream ROI classification, ROI retrieval, ROI few-shot classification, ROI zero-shot classification, and PathMMU PathCLS VQA tasks. CRC-MSI 49 is constructed from H&E colorectal cancer histopathology patches with microsatellite instability labels for biomarker-associated classification. The dataset consisted of 51,918 image patches from 428 slides and 423 patients, classified into two categories: microsatellite instability-high (MSIH, 15,002) and non-microsatellite instability-high (nonMSIH, 36,916). We used the officially defined train:test split, yielding 19,557 training patches and 32,361 test patches (37.67%:62.33%).We used CRC-MSI for downstream ROI classification, ROI retrieval, ROI fewshot classification, and ROI zero-shot classification tasks.

18

ESCA-Tolkach 50 is constructed from an H&E esophageal cancer histopathology tissue classification dataset for esophageal tissue recognition. The dataset consisted of 334,533 image patches classified into eleven categories using the original directory labels: adventitia (68,118), lamina propria (1,901), muscularis mucosae (2,542), muscularis propria (73,532), regressive tumor (56,490), gastric mucosa (40,928), esophageal mucosa (17,276), submucosa (19,150), submucosal glands (1,406), tumor (52,642), and ulcer (548). We used a cohort-level train:test split, with the UKK and WNS cohorts for training and the CHA cohort for testing, yielding 156,346 training patches and 178,187 test patches (46.74%:53.26%). We used ESCA-Tolkach for downstream ROI classification, ROI retrieval, ROI few-shot classification, and ROI zero-shot classification tasks. GCHTID 51 is a gastric cancer histopathology tissue image dataset for gastric tissue classification. The dataset consisted of 26,744 image patches classified into eight balanced tissue categories: adipose tissue (3,343), debris (3,343), lymphocytes (,343), mucus (3,343), muscle (3,343), normal tissue (3,343), stroma (3,343), and tumor (3,343). We used the benchmark train:test split, yielding 17,408 training patches and 9,336 test patches (65.09%:34.91%). We used GCHTID for downstream ROI classification, ROI retrieval, ROI few-shot classification, and ROI zero-shot classification tasks. OTA 52 is the H&E histopathology image dataset for ROI-level osteosarcoma tissue classification. The dataset consisted of 1,091 image patches classified into three categories: non-tumor tissue (536), non-viable tumor (263), and viable tumor (292). We split the dataset into train:test sets, yielding 763 training patches and 328 test patches (69.94%:30.06%). We used OTA for downstream ROI classification, ROI retrieval, ROI few-shot classification, and ROI zero-shot classification tasks. Unitopatho 53 is a colorectal polyp histopathology dataset for ROI-level polyp classification and adenoma dysplasia grading. The dataset consisted of 8,669 H&E image patches extracted from colorectal polyp WSIs. The dataset contained six categories: normal tissue (950), hyperplastic polyp (545), tubular adenoma with high-grade dysplasia (454), tubular adenoma with low-grade dysplasia (3,618), tubulo-villous adenoma with high-grade dysplasia (916), and tubulo-villous adenoma with low-grade dysplasia (2,186). We used the officially defined train:test split, yielding 6,065 training patches and 2,604 test patches (69.96%:30.04%). We used Unitopatho for downstream ROI classification, ROI retrieval, ROI few-shot classification, and ROI zero-shot classification tasks. CoNSeP 54 is a colorectal adenocarcinoma H&E nuclear segmentation dataset. The dataset contains 41 ROI images, and a total of 24,332 kernel instances are labeled. The annotations covered seven nuclear categories: other (932), inflammatory (5,579), healthy epithelial (1,620), dysplastic or malignant epithelial (7,131), fibroblast (7,554), muscle (1,396), and endothelial (120). We used the officially defined train:test split and further divided the training set into train and validation sets, yielding 23 training images, 4 validation images, and 14 test images (56.10%:9.76%:34.15%). We used CoNSeP for the downstream ROI instance segmentation task. CRAG 55 is a colorectal adenocarcinoma H&E gland segmentation dataset. The dataset consisted of 213 ROI images with binary gland masks, from which gland instances were derived using con-

19

nected components. The dataset contained 3,047 annotated instances. We used the officially defined train:val:test split, yielding 158 training images, 15 validation images, and 40 test images (74.18%:7.04%:18.78%). We used CRAG for the downstream ROI instance segmentation task. GlaS 56 is a colorectal H&E gland segmentation dataset containing benign and malignant histology images with instance-level annotations. The dataset consisted of 165 ROI images and 1,530 annotated gland instances. We used the officially defined train, testA, and testB cohort as train:val:test, yielding 85 training images, 60 validation images, and 20 test images (51.52%:36.36%:12.12%). We used GlaS for the downstream ROI instance segmentation task. CoCaHis 57 is an ROI-level H&E segmentation dataset for colorectal cancer histopathology. The dataset consisted of 82 ROI images, each with a size of+- 1388×1037 pixels. The pixel-level annotations covered two categories: background and tumor region, with one mask for each category in every image. We used the officially defined train:val:test split, yielding 65 training images, 6 validation images, and 11 test images (79.27%:7.32%:13.41%). We used CoCaHis for the downstream ROI semantic segmentation task. COSAS24 is an ROI-level H&E segmentation dataset for colorectal adenocarcinoma histopathology. The dataset consisted of 360 ROI images with pixel-level segmentation masks. The annotations covered 350 background regions and 301 adenocarcinoma regions. We used the officially defined train:val:test split, yielding 289 training images, 33 validation images, and 38 test images (80.28%:9.17%:10.56%). We used COSAS24 for the downstream ROI semantic segmentation task. EBHI 58 is an endoscopic biopsy H&E histopathology dataset for ROI-level segmentation tasks. The dataset consisted of 2,174 image patches, each with a size of 224 × 224 pixels. The pixel-level annotations covered seven categories: background (2,139 regions), adenocarcinoma (775), highgrade intraepithelial neoplasia (177), low-grade intraepithelial neoplasia (615), normal tissue (76), polyp (473), and serrated adenoma (58). We used the officially defined train:val:test split, yielding 1,719 training images, 223 validation images, and 232 test images (79.07%:10.26%:10.67%). We used EBHI for the downstream ROI semantic segmentation task. Janowczyk 59 is an ROI-level histopathology segmentation dataset consisting of 141 ROI images, each with a size of 2000 × 2000 pixels. The pixel-level annotations covered two categories: background and foreground, with one mask for each category in every image. We used the benchmark train:val:test split, yielding 106 training images, 20 validation images, and 15 test images (75.18%:14.18%:10.64%). We used Janowczyk for the downstream ROI semantic segmentation task. BookSet 60 is an English pathology image-text dataset derived from the ARCH histopathology captioning resource, which collects pathology textbook and article figures with dense diagnostic and morphological captions. The original matched set contained 4,267 image-caption pairs. We filtered non-H&E-stained images and retained 3,335 image-caption pairs. We used BookSet for the downstream ROI image-to-text and text-to-image retrieval task. ChineseBook is a self-constructed pathology image-text dataset collected from 8 Chinese clinical diagnostic atlas books. To support the vision-language benchmark, we extracted histopathology 20

figures and their accompanying Chinese captions using PaddleOCR and translated the captions into English using Qwen2.5-14B, creating 1,437 image-English text pairs for evaluation. We used ChineseBook for the downstream ROI cross-modal retrieval task. EnglishBook is a self-constructed pathology image-text dataset collected from 31 English pathology textbooks. We extracted histopathology figures and their accompanying English captions using PaddleOCR, creating 2,724 image-English text pairs for evaluation. We used ChineseBook for the downstream ROI cross-modal retrieval task. Atlas is a PathMMU 61 VQA source derived from pathology atlases, textbooks, and guidelines. We used the official PathMMU val, test-tiny, and test splits, and evaluated 77, 189, and 745 pairs. We used Atlas for the downstream ROI VQA task. EduContent is a PathMMU 61 VQA source derived from educational pathology teaching content, including histopathology video material. We used the official PathMMu val, test-tiny, and test benchmarks, obtaining 143, 239, and 1626 evaluation pairs, respectively. We used EduContent for the downstream ROI VQA task. LC25000 62 is a PathMMU 61 PathCLS VQA source reformulated from the lung and colon histopathology image classification dataset. The questions cover 5 diagnostic categories: benign colonic tissue, colon adenocarcinoma tissue, benign lung tissue, lung adenocarcinoma tissue, and lung squamous cell carcinoma tissue. We used the official PathMMU val, test-tiny, and test benchmarks, obtaining 10, 20, and 170 evaluation pairs, respectively. We used LC25000 for the downstream ROI VQA task. Osteo 63 is a PathMMU 61 PathCLS VQA source derived from osteosarcoma histopathology images. The questions cover 3 tissue categories: non-tumor tissue, necrotic tumor, and viable tumor. We used the official PathMMU val, test-tiny, and test benchmarks, obtaining 6, 12, and 102 evaluation pairs, respectively. We used Osteo for the downstream ROI VQA task. SICAPv2 64 is a PathMMU 61 PathCLS VQA source derived from prostate H&E histopathology images for Gleason-pattern recognition. The questions cover 4 categories: non-cancerous tissue, Gleason grade 3, Gleason grade 4, and Gleason grade 5. We used the official PathMMU val, test-tiny, and test benchmarks, obtaining 8, 16, and 136 evaluation pairs, respectively. We used SICAPv2 for the downstream ROI VQA task. SkinCancer 65 is a PathMMU 61 PathCLS VQA source derived from skin histopathology images for tissue and tumor subtype recognition. The questions use 16 candidate categories, including non-tumor skin structures and common skin tumor subtypes. We used the official PathMMU val, test-tiny, and test benchmarks, obtaining 18, 35, and 306 evaluation pairs after label filtering, respectively. We used SkinCancer for the downstream ROI VQA task. WSSSLUAD 66 is a PathMMU 61 PathCLS VQA source derived from lung adenocarcinoma histopathology images. The questions cover 3 categories: tumor epithelial tissue, tumor-associated stroma tissue, and normal tissue. We used the official PathMMU val, test-tiny, and test benchmarks, obtaining 6, 12, and 102 evaluation pairs, respectively. We used WSSSLUAD for the downstream ROI VQA task. 21

PubMed is a PathMMU 61 VQA source constructed from pathology image-text pairs in openaccess PubMed Central scientific documents. We used the official PathMMU val, test-tiny, and test benchmarks, obtaining 224, 274, and 2,712 evaluation pairs, respectively. We used PubMed for the downstream ROI VQA task. SocialPath is a PathMMU 61 VQA source constructed from pathology image-text pairs shared by pathology experts on social media. We used the official PathMMU val, test-tiny, and test benchmarks, obtaining 135, 210, and 1,466 evaluation pairs after image and label filtering, respectively. We used SocialPath for the downstream ROI VQA task. BCNB 67 is a breast cancer WSI cohort consisting of 1,058 biopsy slides. We used BCNB for WSI biomarker prediction, WSI-text multimodal analysis and WSI fine-tuning tasks. For biomarker prediction, we used case-level label-stratified splitting to conduct experiments for ER status prediction (831 positive and 227 negative, train:val:test = 677:169:212), HER2 status prediction (277 positive and 781 negative, train:val:test = 677:169:212), and PR status prediction (790 positive and 268 negative, train:val:test = 677:169:212). For WSI-text multimodal analysis, we conducted experiments for the 2-class axillary lymph node metastasis (655 N0 and 403 N+; train:val:test = 677:169:212) and the 3-class axillary lymph node metastasis (655 N0, 210 N+(1-2), and 193 N+(¿2); train:val:test = 677:169:212). We used 10 clinical variables as textual information, including age, tumor size, tumor type, ER status, PR status, HER2 status, HER2 expression score, histological grade, Ki-67 index, and molecular subtype. CAMELYON+ 68 is a breast cancer sentinel lymph node metastasis WSI cohort consisting of 1,349 slides. We used CAMELYON+ for WSI classification, WSI retrieval, WSI zero-shot, WSI fewshot classification, and WSI fine-tuning tasks. For classification, we used case-level label-stratified splitting to conduct experiments for the 2-class lymph node metastasis coarse-grained classification task (870 negative and 479 positive; train:val:test = 869:220:260) and the 4-class metastatic finegrained classification task (870 negative, 54 isolated tumor cells, 174 micro metastases, and 251 macro metastases; train:val:test = 865:220:264). For WSI zero-shot classification, we used the 2-class setting. CMB is a pan-cancer primary-site WSI cohort consisting of 947 slides. We used CMB for slidelevel PFM few-shot classification and WSI retrieval tasks. For classification, we used case-level label-stratified splitting to conduct experiments for the 9-class pan-cancer classification task (28 AML, 119 BRCA, 186 CRC, 32 GEC, 294 LCA, 119 MEL, 11 MML, 69 OV, and 89 PCA; train:val:test = 602:149:196). CPTAC is a pan-cancer WSI cohort covering multiple tumor types. We used CPTAC for WSI classification, WSI retrieval, WSI zero-shot classification, WSI few-shot classification, and WSI survival prediction tasks. For classification, we conducted experiments for a 9-class pan-cancer primary-site task (2,059 WSIs: BRCA 112, CCRCC 245, COAD 107, GBM 243, HNSC 259, LUAD 326, LSCC 304, OV 221, and PDA 242; train:val:test = 1313:340:406) and a 2-class nonsmall cell lung cancer subtyping task (326 LUAD and 304 LSCC; train:val:test = 402:108:120). For zero-shot, we used the 2-class setting. For survival prediction, we used 4 CPTAC cancer types: CCRCC (218 WSIs, 48 events), HNSC (243 WSIs, 73 events), LUAD (313 WSIs, 54 events), and

22

PDA (227 WSIs, 167 events). DHMC-RCC 69 is a renal cell carcinoma WSI cohort consisting of 563 slides. We used DHMCRCC for WSI classification and WSI retrieval tasks. For classification, we used case-level labelstratified splitting to conduct experiments for the 5-class renal cell carcinoma subtyping task (29 benign, 23 chromophobe renal cell carcinoma, 344 clear-cell renal cell carcinoma, 66 oncocytoma, and 101 papillary renal cell carcinoma; train:val:test = 383:23:157). EBRAINS 70 is a CNS tumor WSI cohort consisting of 2,310, including diagnostic, tumor family, WHO grading, and molecular annotations. We used EBRAINS for WSI classification, WSI retrieval, WSI zero-shot classification, WSI few-shot classification, and WSI biomarker prediction tasks. For classification, we case-level label-stratified splitting to conduct experiments for the 30-class CNS tumor diagnosis task (83 adamantinomatous craniopharyngioma, 47 anaplastic astrocytoma IDH-mutant, 47 anaplastic astrocytoma IDH-wildtype, 50 anaplastic ependymoma, 46 anaplastic meningioma, 91 anaplastic oligodendroglioma IDH-mutant and 1p/19q codeleted, 31 angiomatous meningioma, 82 atypical meningioma, 70 diffuse astrocytoma IDH-mutant, 59 diffuse large B-cell lymphoma of the CNS, 46 ependymoma, 57 fibrous meningioma, 88 ganglioglioma, 34 glioblastoma IDH-mutant, 469 glioblastoma IDH-wildtype, 59 gliosarcoma, 88 haemangioblastoma, 30 haemangioma, 34 haemangiopericytoma, 32 Langerhans cell histiocytosis, 38 lipoma, 32 medulloblastoma non-WNT/non-SHH, 104 meningothelial meningioma, 47 metastatic tumours, 85 oligodendroglioma IDH-mutant and 1p/19q codeleted, 172 pilocytic astrocytoma, 99 pituitary adenoma, 81 schwannoma, 41 secretory meningioma, and 68 transitional meningioma; train:val:test = 1386:353:571), the 9-class CNS tumor-family classification task (1,074 diffuse glioma, 91 hematolymphoid/histiocytic tumors, 429 meningioma, 38 mesenchymal/lipomatous tumors, 47 metastatic tumors, 81 nerve sheath tumors, 216 other neuroepithelial tumors, 182 sellarregion tumors, and 152 vascular/mesenchymal tumors; train:val:test = 1395:344:571), and the 4class WHO grade prediction task (815 grade I, 295 grade II, 299 grade III, and 594 grade IV; train:val:test = 1206:304:493). For biomarker prediction, we used glioma IDH status prediction (327 IDH-mutant and 516 IDH-wildtype; train:val:test = 509:124:210). For WSI retrieval, we used the diagnosis, tumor family, and WHO grade settings. For WSI zero-shot classification, we used the 9-class setting, and for WSI few-shot classification, we used the 30-class CNS diagnosis setting. Hancock 29 is a head-and-neck squamous cell carcinoma WSI cohort with clinicopathological annotations, consisting of 693 slides. We used Hancock for WSI-text multimodal analysis and WSI survival prediction tasks. For multimodal analysis, we used case-level label-stratified splitting to conduct experiments for keratinizing squamous cell carcinoma grading (185 G2 and 197 G3; train:val:test = 244:60:78), lymphovascular invasion prediction (555 negative and 121 positive; train:val:test = 432:110:134), and vascular invasion prediction (632 negative and 44 positive; train:val:test = 433:109:134). For keratinizing squamous cell carcinoma grading, we used age, sex, smoking status, primary tumor site, and HPV/p16 status as textual clinical variables. For the other two task settings, we additionally used histological type and grade as two supplementary variables. For survival prediction, we used 203 WSIs with post-treatment survival records, including 52 events. 23

HunCRC 71 is a colorectal neoplasia WSI cohort consisting of 199 WSIs. We used HunCRC for WSI classification and WSI retrieval tasks. For classification, we used case-level label-stratification to conduct experiments for 4-class colorectal neoplasia screening (38 colorectal cancer, 66 adenoma, 14 negative, and 81 non-neoplastic lesions; train:val:test = 125:32:42). KidRare 23 is a pediatric rare tumor WSI cohort consisting of 1,283 slides. We used KidRare for WSI classification, WSI retrieval, WSI zero-shot classification, WSI few-shot classification, and WSI fine-tuning tasks. For classification, we used case-level label-stratified splitting to conduct experiments for the 4-class pediatric rare tumor classification task (442 hepatoblastoma, 238 medulloblastoma, 467 nephroblastoma, and 136 neuroblastoma; train:val:test = 821:206:256) and the 13-class pediatric rare tumor fine-grained subtyping task (197 classic medulloblastoma, 30 desmoplastic nodular medulloblastoma, 33 differentiating neuroblastoma, 10 epithelial macrotrabecular pattern of hepatoblastoma, 177 epithelial mixed fetal and embryonal hepatoblastoma, 33 ganglioneuroblastoma intermixed, 11 large cell/anaplastic medulloblastoma, 76 mixed epithelial and mesenchymal hepatoblastoma, 51 normal, 70 poorly differentiated neuroblastoma, 179 pure fetal hepatoblastoma with low mitotic activity, 59 tumor, and 357 unknown; train:val:test = 821:206:256). For WSI zero-shot classification, we used the coarse-grained settings. PANDA 72 is a prostate cancer WSI cohort consisting of 9,555 WSIs. We used PANDA for WSI classification and WSI retrieval. For classification, we used case-level label-stratified splitting to conduct experiments for the 2-class prostate cancer screening (2,603 non-tumor and 6,952 tumor; train:val:test = 6115:1530:1910) and the 6-class Gleason grading (ISUP 0 to 5 with 2,603, 2,399, 1,209, 1,118, 1,124, and 1,102, respectively; train:val:test = 6121:1530:1904). PTRC-HGSOC 30 is a high-grade serous ovarian carcinoma WSI cohort consisting of 348 slides with treatment response and tumor-type annotations. We used PTRC-HGSOC for predicting treatment response (202 sensitive and 146 refractory; train:val:test = 222:56:70) and tumor origin classification (174 primary and 174 metastatic; train:val:test = 226:54:68) in WSI-text multimodal analysis tasks. For tumor origin classification, we used age, race, ethnicity, histological grade, neoadjuvant treatment, tumor stage, tumor substage, and other cancer diagnosis as textual information. For treatment response, we additionally used tumor type, tumor location, and tumor location group as three supplementary variables. SYSBM 7 is an internal bone metastasis primary-site WSI cohort consisting of 835 slides. We used SYSBM for WSI classification, WSI retrieval, WSI zero-shot classification, and WSI few-shot classification. For classification, we used case-level label-stratified splitting to conduct experiments for 10-class bone metastasis primary site prediction (106 breast, 35 gastric, 66 kidney, 89 liver, 234 lung, 33 neuroendocrine, 17 no cancer, 54 prostate, 102 squamous carcinoma, and 99 thyroid; train:val:test = 529:135:171). SYSFL+ 73 is an internal frozen lung tissue WSI cohort, consisting of 1,466 slides. We used SYSFL+ for WSI classification, WSI retrieval, and WSI few-shot classification. For classification, we used case-level label-stratified splitting to conduct experiments for 7-class lung cancer subtyping and invasion grading (101 atypical adenomatous hyperplasia, 307 adenocarcinoma in situ, 250 minimally invasive adenocarcinoma, 296 invasive lung adenocarcinoma, 205 lung inflammation,

24

116 lung squamous cell carcinoma, and 191 no carcinoma; train:val:test = 936:232:298) and 4class lung lesion triage (658 favorable prognosis lesions, 296 invasive lung adenocarcinomas, 396 non-neoplastic lesions, and 116 squamous cell carcinomas; train:val:test = 936:232:298). TissueNet 74 is a cervical epithelial lesion WSI cohort consisting of 983 slides. We used TissueNet for WSI classification and WSI retrieval. For classification, we used case-level label-stratified splitting to conduct experiments for 4-class cervical epithelial lesion screening (261 normal or subnormal tissue, 278 low-grade squamous intraepithelial lesion, 225 high-grade squamous intraepithelial lesion, and 219 invasive squamous carcinoma; train:val:test = 633:159:191). UBC-OCEAN 75 is an ovarian carcinoma WSI cohort consisting of 538 slides. We used UBCOCEAN for WSI classification, WSI zero-shot classification, and WSI retrieval. For classification, we used case-level label-stratified splitting to conduct experiments for ovarian carcinoma subtyping (222 high-grade serous ovarian carcinoma, 47 low-grade serous ovarian carcinoma, 99 ovarian clear cell carcinoma, 124 ovarian endometrioid carcinoma, 46 ovarian mucinous carcinoma; train:val:test = 343:85:110). Visiomel is a melanoma WSI cohort consisting of 1,203 slides. We used Visiomel for predicting relapse with (1,006 negative and 197 positive; train:val:test = 766:192:245) and without (424 negative and 122 positive; train:val:test = 350:87:109) prior melanoma in WSI-text multimodal analysis tasks. We used clinical variables as textual information, including age at initial diagnosis, sex, melanoma body site, histological type, Breslow thickness category, and ulceration. MUT-HET-RCC is a renal cell carcinoma WSI cohort consisting of 1,291 slides. We used MUT-HET-RCC for WSI biomarker prediction tasks. We used case-level label-stratified splitting to conduct experiments for BAP1 mutation (162 mutant and 1,129 wildtype; train:val:test = 825:207:259), PBRM1 mutation (669 mutant and 622 wildtype; train:val:test = 825:207:259), and SETD2 mutation prediction (348 mutant and 943 wildtype; train:val:test = 825:207:259). SURGEN 76 is a colorectal cancer WSI cohort consisting of 1,020 slides. We use SR386 subset (140 slides) to conduct experiments of survival prediction since this subset contains survival records. Evaluation settings We evaluated ALICE in three downstream scenarios: vision-only tasks, vision–language tasks, and slide-level tasks. All encoders were kept frozen unless explicitly stated, and the same train, validation, and test splits were used for all compared models within each task. All the downstream tasks were conducted on single 48GB NVIDIA L20 GPU. For ROI vision-only tasks, we compared ALICE with UNI-2, Virchow-2, H-Opt-1, and GPFM. For linear probing, ROI images were encoded with frozen PFMs and L2-normalized before training a single-layer classifier with AdamW (learning rate 10−4 , weight decay 10−4 , batch size 256, 100 epochs). The training split was divided into five stratified folds. Four folds were used for classifier fitting and one fold for validation, with early stopping after 10 epochs without improvement. We selected the checkpoint based on validation balanced accuracy and reported test balanced accuracy as the mean and standard deviation across the five folds. For KNN classification, the training split 25

was used as the reference set and the test split as queries. We evaluated cosine and Euclidean distances with K ∈ {5, 10, 20} and used balanced accuracy as the primary metric. For ROI imageto-image retrieval, the training split was used as the gallery and the test split as queries. Retrieval used cosine similarity and was evaluated at K ∈ {1, 3, 5, 10} using Recall@K, and majority-vote accuracy (MVAcc@5). For ROI few-shot classification, we used all-way ProtoNet 77 episodes with the training split as support and the test split as query. We evaluated n-shot (n ∈ {1, 2, 4, 8, 16, 32, 64, 128, 256}), sampled 20 query examples per class, repeated each valid setting for 500 episodes, and reported balanced accuracy as mean ± s.d. Settings were skipped when any class lacked enough support or query samples. For ROI semantic and instance segmentation, we trained a PMT decoder on top of frozen PFMs for 50 epochs using AdamW (learning rate 10−4 , weight decay 0.05, batch size 8), 224-pixel crops with a stride of 112, and early stopping with patience 10. Each segmentation experiment was repeated with five random seeds (42-46). Semantic segmentation checkpoints were selected by validation mIoU and evaluated by test DICE score. Instance segmentation checkpoints were selected by validation AP@[50:95] and evaluated by test AP@50. For ROI vision-language tasks, we compared ALICE with MUSK, KEEP, and CONCH. For zero-shot ROI classification, each class was represented by five label descriptions and ten pathology prompt templates, yielding 50 prompts per class. We evaluated two schemes: a prompt-level scheme, in which each prompt slot independently produced predictions, and a prompt-ensemble scheme, in which class probabilities were averaged across the 50 prompts. For ALICE, probabilities were additionally averaged across its KEEP, CONCH, and MUSK-aligned heads. We reported balanced accuracy for the ensemble score and mean ± s.d. across prompt-level classifiers. For ROI image-text retrieval, normalized image and text embeddings were evaluated bidirectionally on paired image-caption datasets using Recall@K with K ∈ {1, 5, 10, 20, 25, 50}. For ROI VQA, multiple-choice PathMMU 61 questions were evaluated by converting each answer option into pathology prompts and selecting the option with the highest image-text similarity. Accuracy was used as the metric. The generic ROI zero-shot templates and dataset-specific class prompts are listed in Extended Data Table 4-18. For WSI vision-language tasks, we evaluated MI-Zero 27 , PathPT 28 , and MICA-style multimodal analysis 31 . In MI-Zero zero-shot WSI classification, each class was represented by three label texts and four WSI prompt templates, yielding 12 prompts per class. Slide-level scores were computed by Top-K MIL pooling of patch-text cosine similarities with K ∈ {1, 5, 10, 50}, followed by softmax averaging across prompts and, for ALICE, across teacher-aligned heads. We reported balanced accuracy on the test split. For PathPT few-shot WSI classification, we sampled n ∈ {1, 5, 10, 15, 20} labeled slides per class, optimized learnable 32 context tokens and the lightweight adaptor for 20 epochs with Adam (learning rate 10−4 , batch size 1), and selected checkpoints by validation balanced accuracy. Each setting was repeated over five seeds, and test balanced accuracy was reported as mean ± s.d. For MICA, clinical variables were converted into text segments and encoded by the corresponding text encoder, then input with WSI patch features through co-attention in a 256-dimensional hidden space. MICA models were trained for 20 epochs with Adam (learning rate 10−4 , dropout 0.25, batch size 1, gradient accumulation over 8 steps) and 26

class-weighted cross-entropy loss. Results were averaged over 5 seeds, using validation balanced accuracy for checkpoint selection and test balanced accuracy for reporting. The MI-Zero WSI prompt templates and class prompts are provided in Extended Data Table 19-25. Task-specific clinical text segment templates used for MICA are listed in Extended Data Table 26-31. For slide-level WSI tasks, we compared ALICE with TITAN and CARE. Patch features were extracted with frozen patch encoders and aggregated by pretrained slide encoders. TITAN and CARE used the CONCH 1.5 patch encoder, and ALICE used its own vision-only module. For slide-level KNN diagnosis, we used the training split as the reference set, the test split as queries. We reported the balanced accuracy (mean ± s.d) of K ∈ {5, 10, 20} on both cosine and Euclidean distances. For slide-level linear probing, a single-layer classifier was trained on frozen slide embeddings with AdamW (learning rate 3 × 10−5 , weight decay 10−4 , batch size 16, 500 epochs). The training split was stratified into five folds, checkpoints were selected on the validation fold using the prespecified validation metric, and test performance was averaged across folds. In the fine-tuning-aligned protocol, the official train and validation splits were used, five seeds were run, and test balanced accuracy was reported as mean ± s.d. Biomarker prediction used the same linear-probe protocol and was evaluated by AUC. Slide-to-slide retrieval used cosine similarity, with the training split as the gallery and the test split as queries, and was evaluated with MVAcc@3. WSI few-shot classification used the same linear-probe protocol on n-shot subsets with n ∈ {1, 2, 4, 8, 16, 32, 64}. For slide-level fine-tuning, the slide encoder and classifier were optimized jointly for 50 epochs with Adam (learning rate 3 × 10−5 , weight decay 10−4 , batch size 1), early stopping with a patience of 5, and validation balanced accuracy for checkpoint selection. Experiments were repeated over five seeds. For survival prediction, slide embeddings were evaluated in a cross-validation setting using Cox proportional hazards models, and performance was summarized by the C-index.

Data availability The pretraining dataset TCGA-12K used for the vision-only and multimodal stages can be accessed at https://huggingface.co/datasets/medarc/TCGA-12K-parquet. The pretraining dataset TCGA-UT-8K used for the slide-level stage can be accessed at https: //huggingface.co/datasets/MahmoodLab/TCGA-UniformTumor-8K. The pretraining dataset HISTAI used for the slide-level stage can be accessed at https://huggin gface.co/collections/histai. For downstream benchmarks, all publicly available datasets can be accessed through their original data portals or publications, including AGGC2022 (https://aggc22.grand-chall enge.org/), BRACS (https://www.bracs.icar.cnr.it/), BreakHis (https: //web.inf.ufpr.br/vri/databases/breast-cancer-histopathological -database-breakhis/), CCRCC (https://zenodo.org/records/7898308), Chaoyang (https://bupt-ai-cz.github.io/HSA-NRL/), CRC-100K (https: //zenodo.org/records/1214456), CRC-MSI (https://zenodo.org/records /2530835), ESCA-Tolkach (https://zenodo.org/records/7548828), GCHTID (https://www.kaggle.com/datasets/orvile/gastric-cancer-histopa 27

thology-tissue-image-dataset/data), OTA (https://www.cancerimag ingarchive.net/collection/osteosarcoma-tumor-assessment/), Unitopatho (https://ieee-dataport.org/open-access/unitopatho), CoNSeP (https://www.kaggle.com/datasets/rftexas/tiled-consep-224x224px), CRAG (https://warwick.ac.uk/fac/cross_fac/tia/data/mildnet/), GlaS (https://www.kaggle.com/datasets/sani84/glasmiccai2015-gla nd-segmentation), CoCaHis (https://cocahis.irb.hr/), COSAS24 (https: //cosas.grand-challenge.org/), EBHI (https://www.kaggle.com/datas ets/alibabaei78/ebhi-seg), Janowczyk (https://andrewjanowczyk.com/ use-case-1-nuclei-segmentation/), BookSet (https://warwick.ac.uk/fa c/cross_fac/tia/data/arch/), PathMMU (https://huggingface.co/datas ets/jamessyx/PathMMU), BCNB (https://bupt-ai-cz.github.io/BCNB/), CAMELYON+ (https://www.scidb.cn/en/detail?dataSetId=cc1f911b75c a4610bd02ac33a51898a9), CMB (https://www.cancerimagingarchive.net /research/cmb/), CPTAC (https://www.cancerimagingarchive.net/colle ction), DHMC-RCC (https://bmirds.github.io/KidneyCancer/), EBRAINS (https://doi.org/10.25493/WQ48-ZGX), HunCRC (https://www.cancerim agingarchive.net/collection/hungarian-colorectal-screening/), KidRare (https://huggingface.co/datasets/Firehdx233/KidRare), PANDA (https://panda.grand-challenge.org/data/), PTRC-HGSOC (https://www. cancerimagingarchive.net/collection/ptrc-hgsoc/), TissueNet (https:// www.drivendata.org/competitions/67/competition-cervical-biopsy/), UBC-OCEAN (https://www.kaggle.com/competitions/UBC-OCEAN/), Visiomel (https://www.drivendata.org/competitions/148/visiomel-melanoma/), MUT-HET-RCC (https://doi.org/10.25452/figshare.plus.c.5983795) and SURGEN (https://www.ebi.ac.uk/biostudies/bioimages/studies/S-BIA D1285).

Code availability The code and model weight for ALICE can be accessed at https://github.com/Won derLandxD/ALICE.

Author contributions J.L., T.G., H.S., A.H., C.H., and Y.H. conceived and designed the study. J.L., T.G., and H.S. developed the ALICE framework. J.L. and T.G. implemented the model training, inference, and evaluation pipelines. J.L., H.S., X.L., and M.F. curated and processed the pretraining and downstream evaluation datasets. J.L., T.G., X.L., and M.F. performed the benchmark experiments and statistical analyses. H.S. and A.H. provided pathology expertise and contributed to clinical interpretation. J.L. and X.L. prepared the figures and tables. J.L., T.G., and H.S. wrote the initial manuscript draft. A.H., C.H., and Y.H. supervised the study and revised the manuscript. Y.H. acquired funding. All authors reviewed and approved the final manuscript. 28

Competing interests The authors declare no competing interests.

Acknowledgements This work was supported by the National Natural Science Foundation of China (NSFC) (82430062), the Beijing Municipal Health Commission Research Ward Excellence Clinical Research Program (BRWEP2024WO32240114), the Shenzhen Engineering Research Centre (XMHT20230115004), and the National High Level Hospital Clinical Research Funding (BJ-2024-155).

29

References 1. Song, A. H. et al. Artificial intelligence for digital and computational pathology. Nature Reviews Bioengineering 1, 930–949 (2023). 2. Zheng, Y. et al. Lazyslide: accessible and interoperable whole-slide image analysis. Nature Methods 1–4 (2026). 3. Lu, M. Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nature Biomedical Engineering 5, 555–570 (2021). 4. Jiang, R. et al. A transformer-based weakly supervised computational pathology method for clinical-grade diagnosis and molecular marker discovery of gliomas. Nature Machine Intelligence 6, 876–891 (2024). 5. Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nature Medicine 25, 1301–1309 (2019). 6. Tsai, P.-C. et al. Histopathology images predict multi-omics aberrations and prognoses in colorectal cancer patients. Nature Communications 14, 2102 (2023). 7. Zhu, L. et al. An accurate prediction of the origin for bone metastatic cancer using deep learning on digital pathological images. EBioMedicine 87 (2023). 8. Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nature Medicine 30, 850–862 (2024). 9. Lu, M. Y. et al. A visual-language foundation model for computational pathology. Nature Medicine 30, 863–874 (2024). 10. Yan, F. et al. Pathorchestra: A comprehensive foundation model for computational pathology with over 100 diverse clinical-grade tasks. npj Digital Medicine 8, 695 (2025). 11. Xu, H. et al. A whole-slide foundation model for digital pathology from real-world data. Nature 630, 181–188 (2024). 12. Neidlinger, P. et al. Benchmarking foundation models as feature extractors for weakly supervised computational pathology. Nature Biomedical Engineering 1–11 (2025). 13. Tizhoosh, H. R. Rethinking foundation models in pathology. Nature Biomedical Engineering 1–6 (2026). 14. Vorontsov, E. et al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nature Medicine 30, 2924–2935 (2024). 15. Xiang, J. et al. A vision–language foundation model for precision oncology. Nature 638, 769–778 (2025).

30

16. Campanella, G. et al. A clinical benchmark of public self-supervised pathology foundation models. Nature Communications 16, 3640 (2025). 17. Ding, T. et al. A multimodal whole-slide foundation model for pathology. Nature Medicine 1–13 (2025). 18. Heinrich, G. et al. Radiov2.5: Improved baselines for agglomerative vision foundation models. In Proceedings of the Computer Vision and Pattern Recognition Conference, 22487–22497 (2025). 19. Zhu, C. et al. Efficient universal perception encoder. arXiv preprint arXiv:2603.22387 (2026). 20. Ma, J. et al. A generalizable pathology foundation model using a unified knowledge distillation pretraining framework. Nature Biomedical Engineering 10, 545–564 (2026). 21. Zimmermann, E. et al. Virchow2: Scaling self-supervised mixed magnification models in pathology. arXiv preprint arXiv:2408.00738 (2024). 22. Scalbert, M. et al. Abstract lb174: H-optimus-1: A foundation model for computational histopathology. Cancer Research 86, LB174–LB174 (2026). 23. Zhou, X. et al. Knowledge-enhanced pretraining for vision-language pathology foundation model on cancer diagnosis. Cancer Cell 44, 777–791 (2026). 24. Zhang, D. et al. Care: A molecular-guided foundation model with adaptive region modeling for whole slide image analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21078–21088 (2026). 25. Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In International conference on machine learning, 2127–2136 (PMLR, 2018). 26. Acosta, J. N., Falcone, G. J., Rajpurkar, P. & Topol, E. J. Multimodal biomedical ai. Nature medicine 28, 1773–1784 (2022). 27. Lu, M. Y. et al. Visual language pretrained multiple instance zero-shot transfer for histopathology images. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 19764–19775 (2023). 28. He, D. et al. Boosting pathology foundation models via few-shot prompt-tuning for rare cancer subtyping. Nature Communications (2026). 29. Dörrich, M. et al. A multimodal dataset for precision oncology in head and neck cancer. Nature Communications 16, 7163 (2025). 30. Chowdhury, S. et al. Proteogenomic analysis of chemo-refractory high-grade serous ovarian cancer. Cell 186, 3476–3498 (2023).

31

31. Li, Z. et al. Ai-enabled virtual spatial proteomics from histopathology for interpretable biomarker discovery in lung cancer. Nature Medicine 1–14 (2026). 32. Wang, X. et al. A pathology foundation model for cancer diagnosis and prognosis prediction. Nature 634, 970–978 (2024). 33. Xu, Y. et al. A multimodal knowledge-enhanced whole-slide pathology foundation model. Nature Communications (2025). 34. Nechaev, D., Pchelnikov, A. & Ivanova, E. Histai: an open-source, large-scale whole slide image dataset for computational pathology. arXiv preprint arXiv:2505.12120 (2025). 35. Vaidya, A. et al. Demographic bias in misdiagnosis by computational pathology models. Nature Medicine 30, 1174–1190 (2024). 36. Li, J. et al. Stainnet: Scaling self-supervised foundation models on immunohistochemistry and special stains for computational pathology. In Medical Imaging with Deep Learning, 544–569 (PMLR, 2026). 37. Karasikov, M. et al. Training state-of-the-art pathology foundation models with orders of magnitude less data. In International Conference on Medical Image Computing and ComputerAssisted Intervention, 573–583 (Springer, 2025). 38. Oquab, M. et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 (2023). 39. Press, O., Smith, N. A. & Lewis, M. Train short, test long: Attention with linear biases enables input length extrapolation. arXiv preprint arXiv:2108.12409 (2021). 40. Chu, X., Tian, Z., Zhang, B., Wang, X. & Shen, C. Conditional positional encodings for vision transformers. arXiv preprint arXiv:2102.10882 (2021). 41. Darcet, T., Oquab, M., Mairal, J. & Bojanowski, P. Vision transformers need registers. In International conference on learning representations, vol. 2024, 2632–2652 (2024). 42. Trinh, T., Heinonen, M., Acerbi, L. & Kaski, S. Improving robustness to corruptions with multiplicative weight perturbations. Advances in Neural Information Processing Systems 37, 35492–35516 (2024). 43. Huo, X. et al. A comprehensive ai model development framework for consistent gleason grading. Communications Medicine 4, 84 (2024). 44. Brancati, N. et al. Bracs: A dataset for breast carcinoma subtyping in h&e histology images. Database 2022, baac093 (2022). 45. Spanhol, F. A., Oliveira, L. S., Petitjean, C. & Heutte, L. A dataset for breast cancer histopathological image classification. IEEE Transactions on Biomedical Engineering 63, 1455–1462 (2015). 32

46. Brummer, O., Pölönen, P., Mustjoki, S. & Brück, O. Computational textural mapping harmonises sampling variation and reveals multidimensional histopathological fingerprints. British Journal of Cancer 129, 683–695 (2023). 47. Zhu, C., Chen, W., Peng, T., Wang, Y. & Jin, M. Hard sample aware noise robust learning for histopathology image classification. IEEE Transactions on Medical Imaging 41, 881–894 (2021). 48. Kather, J. N., Halama, N. & Marx, A. 100,000 histological images of human colorectal cancer and healthy tissue (2018). URL https://doi.org/10.5281/zenodo.1214456. 49. Kather, J. N. et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nature medicine 25, 1054–1056 (2019). 50. Tolkach, Y. et al. Artificial intelligence for tumour tissue detection and histological regression grading in oesophageal adenocarcinomas: a retrospective algorithm development and validation study. The Lancet Digital Health 5, e265–e275 (2023). 51. Lou, S. et al. A large histological images dataset of gastric cancer with tumour microenvironment annotation for ai. Scientific Data 12, 138 (2025). 52. Leavey, P. et al. Osteosarcoma data from ut southwestern/ut dallas for viable and necrotic tumor assessment [data set]. Cancer Imaging Arch 14 (2019). 53. Barbano, C. A. et al. Unitopatho, a labeled histopathological dataset for colorectal polyps classification and adenoma dysplasia grading. In 2021 IEEE International Conference on Image Processing (ICIP), 76–80 (IEEE, 2021). 54. Graham, S. et al. Hover-net: Simultaneous segmentation and classification of nuclei in multitissue histology images. Medical Image Analysis 58, 101563 (2019). 55. Graham, S. et al. Mild-net: Minimal information loss dilated network for gland instance segmentation in colon histology images. Medical Image Analysis 52, 199–211 (2019). 56. Sirinukunwattana, K. et al. Gland segmentation in colon histology images: The glas challenge contest. Medical Image Analysis 35, 489–502 (2017). 57. Sitnik, D. et al. A dataset and a methodology for intraoperative computer-aided diagnosis of a metastatic colon cancer in a liver. Biomedical Signal Processing and Control 66, 102402 (2021). 58. Hu, W. et al. Ebhi: A new enteroscope biopsy histopathological h&e image dataset for image classification evaluation. Physica Medica 107, 102534 (2023). 59. Janowczyk, A. & Madabhushi, A. Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases. Journal of Pathology Informatics 7, 29 (2016).

33

60. Gamper, J. & Rajpoot, N. Multiple instance captioning: Learning representations from histopathology textbooks and articles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2021). 61. Sun, Y. et al. Pathmmu: A massive multimodal expert-level benchmark for understanding and reasoning in pathology. In European Conference on Computer Vision, 56–73 (Springer, 2024). 62. Borkowski, A. A. et al. Lung and colon cancer histopathological image dataset (lc25000). arXiv preprint arXiv:1912.12142 (2019). 63. Arunachalam, H. B. et al. Viable and necrotic tumor assessment from whole slide images of osteosarcoma using machine-learning and deep-learning models. PloS One 14, e0210706 (2019). 64. Silva-Rodrı́guez, J., Colomer, A., Sales, M. A., Molina, R. & Naranjo, V. Going deeper through the gleason scoring scale: An automatic end-to-end system for histology prostate grading and cribriform pattern detection. Computer Methods and Programs in Biomedicine 195, 105637 (2020). 65. Kriegsmann, K. et al. Deep learning for the detection of anatomical tissue structures and neoplasms of the skin on scanned histopathological tissue sections. Frontiers in Oncology 12, 1022967 (2022). 66. Han, C. et al. Wsss4luad: Grand challenge on weakly-supervised tissue semantic segmentation for lung adenocarcinoma. arXiv preprint arXiv:2204.06455 (2022). 67. Xu, F. et al. Predicting axillary lymph node metastasis in early breast cancer using deep learning on primary tumor biopsy slides. Frontiers in Oncology 11, 759007 (2021). 68. Ling, X. et al. Comprehensive benchmark dataset for pathological lymph node metastasis in breast cancer sections. Scientific Data 12, 1381 (2025). 69. Zhu, M. et al. Development and evaluation of a deep neural network for histologic classification of renal cell carcinoma on biopsy and surgical resection slides. Scientific Reports 11, 7080 (2021). 70. Roetzer-Pejrimovsky, T. et al. The digital brain tumour atlas, an open histopathology resource. Scientific Data 9, 55 (2022). 71. Pataki, B. Á. et al. Huncrc: annotated pathological slides to enhance deep learning applications in colorectal cancer screening. Scientific Data 9, 370 (2022). 72. Bulten, W. et al. Artificial intelligence for diagnosis and gleason grading of prostate cancer: the panda challenge. Nature medicine 28, 154–163 (2022). 73. Li, J. et al. Diagnostic text-guided representation learning in hierarchical classification for pathological whole slide image. Medical Image Analysis 103894 (2025). 34

74. Fick, R. H. et al. A partial label-based machine learning approach for cervical whole-slide image classification: the winning tissuenet solution. In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2127–2131 (IEEE, 2021). 75. Asadi-Aghbolaghi, M. et al. Machine learning-driven histotype diagnosis of ovarian carcinoma: insights from the ocean ai challenge. medRxiv 2024–04 (2024). 76. Myles, C., Um, I. H., Marshall, C., Harris-Birtill, D. & Harrison, D. J. Surgen: 1020 h&e-stained whole-slide images with survival and genetic markers. GigaScience 14, giaf086 (2025). 77. Snell, J., Swersky, K. & Zemel, R. Prototypical networks for few-shot learning. Advances in Neural Information Processing Systems 30 (2017).

35

a. Vision-only transformer

b. Multimodal transformer Tokens output from the vision-only stage

Image

…

…

Patch embedding layer

Patch tokens

…

Register tokens

Transformer layer

Summary tokens

…

Transformer layer × 2 𝑙𝑎𝑦𝑒𝑟𝑠

…

Transformer layer

… Teacher-align tokens

Transformer layer

c. Slide-level transformer Tokens output from the vision-only stage

…

Transformer layer

………

Transformer layer

Transformer layer

Transformer layer × 32 𝑙𝑎𝑦𝑒𝑟𝑠

…

Summary tokens

× 2 𝑙𝑎𝑦𝑒𝑟𝑠

…

… Teacher-align tokens

Teacher-align tokens

Extended Data Figure 1: Transformer architectures used in ALICE. a. The vision-only transformer converts pathology images into patch tokens and combines them with register and summary tokens before processing them through N transformer layers. Output summary tokens serve as teacher-alignment tokens. b. The multimodal transformer takes tokens from the vision-only stage and applies a two-layer transformer to generate teacher-alignment tokens for multimodal representation learning. c. The slide-level transformer combines vision-only tokens with slidelevel summary tokens and applies a two-layer transformer to produce teacher-alignment tokens for slide-level representation learning.

36

a.

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

All tasks

1.0

0.8

0.70

0.65

BAcc

0.4

0.2

0.55

0.0

0.50

(5

se as

s)

) es

BR

S

(3

s as

BR

AC

S

(7

cl

s) se as

e Br

s) se

) es

H ak

is

(2

cl

s as

C

C

R

C C

(6

as cl

C

a oy ha

ng

(4

cl

se as

s)

C

R

-1 C

K 00

(9

cl

se as

s)

C

R

C

SI -M

(2

a cl

ss

ES

b.

UNI-2

Virchow-2

H-Opt-1

GPFM

) es

AC

lk To

h ac

(1

1

cl

se as

s)

G

C

H

D TI

(8

a cl

) es

ss

O

TA

(3

s as cl

) es

U

pa to ni

o th

(6

cl

se as

s)

ALICE

All tasks

1.0

0.80

0.8

0.78

0.6

0.76

BAcc

BAcc

AC

cl

0.4

0.2

AG

0.70

(5

cl

se as

s)

BR

S

(3

s)

c

BR

AC

S

(7

c

e ss la

s)

Br

kH ea

is

(2

cl

se as

s)

C

C

C R

C

(6

cl

se as

C

s)

h

n ya ao

g

(4

c

e ss la

C

s)

R

-1 C

K 00

(9

cl

se as

s)

C

R

SI -M C

(2

a cl

UNI-2

Virchow-2

H-Opt-1

GPFM

) es ss

ASC

lk To

h ac

(1

1

cl

se as

s)

G

C

TI H

D

(8

cl

se as

s)

O

TA

(3

cl

se as

U

s)

ni

p to

ho at

(6

cl

s) se as

ALICE

All tasks

1.0

0.85

0.8

0.83

MVAcc@5

MVAcc@5

AC

e ss la

E

c.

0.6

0.4

0.2

AG

C G

0.81

0.79

0.77

0.0

0.75

(5

cl

se as

s)

BR

AC

S

(3

cl

se as

s)

BR

AC

S

(7

se as cl

s)

Br

kH ea

is

(2

cl

se as

s)

C

C

C R

C

(6

cl

se as

C

s)

h

n ya ao

g

(4

c

) es ss la

C

R

C

0K 10

(9

cl

se as

s)

C

R

SI -M C

(2

cl

s) se as

AC ES

h ac lk To

(1

1

cl

s) se as

G

C

TI H

D

(8

cl

se as

s)

O

TA

(3

cl

se as

U

s)

ni

p to

ho at

(6

cl

s) se as

e. UNI-2

Virchow-2

H-Opt-1

GPFM

All tasks

ALICE

1.0

0.8

0.84

0.8

0.6

0.83

0.2

0.0

CoCaHis

COSAS24

EBHI

0.82

AP@50

0.4

0.8395 0.8352 0.8329 0.8385 0.8417

0.85

DICE

1.0

UNI-2

Virchow-2

H-Opt-1

GPFM

0.4

0.2

0.80

0.0

0.57

0.55

0.53

0.51

CoNSeP

Janowczyk

0.61

0.59

0.6

0.81

All tasks

ALICE

AP@50

d.

DICE

0.74

0.72

0.0

C G

0.7786 0.7815 0.7685 0.7690 0.7930

C G

cl

0.8228 0.8242 0.8144 0.8179 0.8340

AG

0.60

0.6070 0.5825 0.5675 0.5894 0.5990

BAcc

0.6

0.6989 0.7089 0.6623 0.6144 0.7422

0.75

CRAG

GlaS

Extended Data Figure 2: Detailed ROI vision-only transfer results. a-e. ROI-level evaluation of ALICE against UNI-2, Virchow-2, H-Opt-1 and GPFM using frozen image features. a. Linear-probe classification. b. KNN classification. c. Image-to-image retrieval. d. Semantic segmentation. e. Instance segmentation. 37

AGGC (5 classes) Shot = 1, 2, 4, 8, 16, 32 ,64

BRACS (7 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256

BRACS (3 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256

1.0

1.0

0.8

0.8

0.6

0.6

0.8

0.4

BAcc

BAcc

BAcc

0.6

0.4

0.4

0.2 0.2

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.2

0.0

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

UNI-2

1

2

4

8

16

32

64

1

2

4

BreakHis (2 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256

8

16

32

64

128

1

256

0.6

0.4

0.4

0.2

0.2

H-Opt-1

GPFM

ALICE

4

8

16

32

64

128

UNI-2

32

ALICE 64

128

256

Virchow-2

H-Opt-1

GPFM

ALICE

0.2

UNI-2

Virchow-2

H-Opt-1 ALICE

GPFM 0.0

1

256

2

4

8

16

32

64

128

1

256

2

4

8

16

32

64

128

256

ESCA-Tolkach (11 classes) Shot = 1, 2, 4, 8, 16, 32

CRC-MSI (2 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256

CRC-100K (9 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256 1.0

16

0.4

0.0

0.0

8

BAcc

0.8

0.6

BAcc

0.8

0.6

BAcc

0.8

2

4

Chaoyang (4 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256 1.0

1

2

CCRCC (6 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256 1.0

Virchow-2

H-Opt-1

GPFM

1.0

UNI-2

Virchow-2

0.0

0.0

1.0

1.0

0.8

0.8

0.6

0.6

BAcc

BAcc

BAcc

0.8

0.4

0.4 0.6

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.2

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.2

1

2

4

8

16

32

64

128

1

256

UNI-2

2

4

8

16

32

64

128

H-Opt-1 ALICE

GPFM 1

256

2

4

8

16

32

Unitopatho (6 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256

OTA (3 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128

GCHTID (8 classes) Shot = 1, 2, 4, 8, 16, 32, 64, 128, 256

0.8

1.0

0.8

Virchow-2

0.0

0.0

0.4

0.7

0.7

0.8

0.6

0.6

0.5

0.4

BAcc

0.6

BAcc

BAcc

0.5

0.4

0.3

0.4 0.3 0.2

0.2

UNI-2

0.1

Virchow-2

H-Opt-1

GPFM

ALICE

0.2

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

1

2

4

8

16

32

64

128

256

UNI-2

0.1

Virchow-2

H-Opt-1 ALICE

GPFM 0.0

0.0

0.0

1

2

4

8

16

32

64

128

1

2

4

8

16

32

64

128

256

Extended Data Figure 3: Detailed ROI few-shot classification performance. Few-shot ROI classification of ALICE and four vision-only PFMs across 12 datasets using all-way ProtoNet episodes.

38

b. CONCH

C G

(5

cl

BR

AC

S

(3

cl

s) se

as

BR

AC

S

(7

cl

s) se

as

e Br

H ak

is

(2

cl C

s) se as

C

C R

C

(6 C

cl

s) se as

ha

a oy

ng

(4

C

cl

R

s) se

as

-1 C

K 00

a cl

(9 C

R

) es ss

SI -M C

(2

AC

a cl

) es ss

h ac lk To

(1

1

a cl

G

C

) es ss

TI H

D

(8

a cl

) es ss O

TA

(3

U

cl

s) se as

pa to ni

o th

(6

cl

a

(5

a cl

e ss

) es

s)

(3

cl

s as

(7

cl

s as

MUSK

KEEP

CONCH

) es

(2

cl

se as

s)

(6

ALICE

MUSK

KEEP

CONCH

0.6

0.6

0.4

BAcc

0.6

BAcc

0.8

0.4

0.0

recall@25

recall@50

recall@1

recall@5

KEEP

CONCH

1.0

ALICE

recall@10

recall@20

recall@25

0.2947

0.5572 0.5664 0.5403 0.6234

0.6943 0.7239 0.7625

0.4850

0.8690

recall@1

recall@50

MUSK

KEEP

CONCH

0.6

0.6

0.4

0.2

0.0

0.0

recall@5

recall@10

recall@20

recall@25

BAcc

0.6

BAcc

0.8

0.2

ALICE

recall@5

recall@10

recall@20

recall@25

recall@50

MUSK

KEEP

CONCH

ALICE

0.4

0.2

0.0

recall@1

recall@50

CONCH

EnglishBook (text-to-image) 1.0

ALICE

0.8

0.4

KEEP

0.4

0.8

recall@1

MUSK

ChineseBook (text-to-image)

BookSet (text-to-image) MUSK

cl

0.0

0.0

recall@20

(4

0.2

0.2

0.2

recall@10

s)

EnglishBook (image-to-text) 1.0

ALICE

0.8

recall@5

se as

e. 1.0

0.8

recall@1

cl

ChineseBook (image-to-text)

BookSet (image-to-text)

1.0

0.7403

s) s) s) s) s) s) s) ks se se se se se se se as lt as as as as as as as cl cl cl Al cl cl cl 8 3 2 9 6 1 ( ( ( ( ( (1 g o S S K D C C is SI h TA th C G TI an 00 kH AC AC O -M R ac H pa -1 oy C C lk AG C ea to C BR BR R C ha G To Br R ni C C C U A C ES

) es ss

d.

c. 1.0

0.4823 0.5644 0.5287 0.5685

0.7824 0.7790 0.7773 0.6595

0.0

s) se as

ES

BAcc

0.4795 0.4158 0.4288 0.5211

0.2

0.0

BAcc

0.7054

0.4

0.2

AG

0.5657

0.6

0.4682 0.4298 0.3514

0.4

0.5204 0.4729 0.6014

BAcc

BAcc

0.8

0.6

0.5993 0.5753 0.5383 0.6866

0.8

ALICE

0.4494 0.4215

KEEP

0.5445

MUSK 1.0

0.5620 0.6438 0.6494 0.6274

ALICE

0.4373 0.3714 0.4504 0.4538

CONCH

0.9103

KEEP

0.7663

MUSK

1.0

0.5582 0.5081 0.4995 0.5207

a.

recall@5

recall@10

recall@20

recall@25

recall@1

recall@50

recall@5

recall@10

recall@20

recall@25

recall@50

f. Extended Data Figure 4: Detailed ROI vision-language transfer results. a. Prompt-level Non-small cell lung cancer subtyping Ovain tumor fine-grained subtyping zero-shot ROI classification of5 ALICE, MUSK, KEEP and CONCH across2 classes) 12 ROI datasets. b. (CPTAC-NSCLC, (UBC-OCEAN, classes) Prompt-ensemble zero-shot ROI classification across the 1.0 1.0 same 12 datasets. c-e. Image-to-text and text-to-image retrieval on BookSet, ChineseBook and EnglishBook, respectively. 0.8 0.6

BAcc

BAcc

0.9

0.4 0.2

MUSK

KEEP

CONCH

ALICE

0.7

0.0

MUSK

KEEP

CONCH

ALICE

0.6 1

5

10

15

20

1

5

10

15

20

Shot number

Shot number

Lymph node metastasis screening (Camelyon+, 2 classes)

Lung cancer subtyping and invasion grading (SYSFL, 7 classes)

1.0

0.5

0.8

0.4

0.6

0.3

BAcc

BAcc

0.8

0.4 0.2

MUSK

KEEP

CONCH

ALICE

0.2 0.1

MUSK

KEEP

CONCH

ALICE

0.0

0.0 1

5

10 Shot number

15

1

20

39

5

10 Shot number

15

20

0.4

0.6

0.4

0.2

0.2

0.0

0.0

recall@1

recall@5

recall@10

recall@20

recall@25

BAcc

BAcc

BAcc

0.6

0.4

0.2

0.0

recall@1

recall@50

0.6

recall@5

recall@10

recall@20

recall@25

recall@1

recall@50

recall@5

recall@10

recall@20

recall@25

recall@50

f. Non-small cell lung cancer subtyping (CPTAC-NSCLC, 2 classes)

Ovain tumor fine-grained subtyping (UBC-OCEAN, 5 classes) 1.0

1.0 0.9

0.6

BAcc

BAcc

0.8

0.4 0.2

MUSK

KEEP

CONCH

ALICE

0.7

0.0

MUSK

KEEP

CONCH

ALICE

0.6 1

5

10

15

20

1

5

10

15

20

Shot number

Shot number

Lymph node metastasis screening (Camelyon+, 2 classes)

Lung cancer subtyping and invasion grading (SYSFL, 7 classes)

1.0

0.5

0.8

0.4

0.6

0.3

BAcc

BAcc

0.8

0.4 0.2

MUSK

KEEP

CONCH

ALICE

0.2 0.1

MUSK

KEEP

CONCH

ALICE

0.0

0.0 1

5

10

15

1

20

5

10

15

20

Shot number

Shot number

Extended Data Figure 5: Detailed WSI few-shot vision-language classification results. PathPT-based few-shot WSI classification of ALICE, MUSK, KEEP and CONCH across four clinical tasks using 1, 5, 10, 15, and 20 labelled slides per class.

40

Query text

Query text

At intermediate magnification, the cells in chordoma grow in cords, strands, and clusters amid a loose myxoid stroma. Top-3 retrieved images 0.564

Query text

Perineural invasion: histological feature specific for prostate cancer. It is defined as tight circumferential or near circumferential encircling of a nerve fiber by cancer glands.

Flat carcinoma in situ. The urothelium is partially denuded, but the cells that remain show enlarged, round, hyperchromatic nuclei appearing to pop off the surface.

Top-3 retrieved images

Ground truth caption Chordoma. This tumor occurs very rarely outside the vertebral column and skull either as metastatic disease or as extra-axial chordoma. The appearance is similar to that of skeletal chordoma. Cords of epithelioid cells are dispersed in myxoid stroma

0.646

Top-3 retrieved images Ground truth caption

Ground truth caption A cribriform lesion that may be graded as Gleason pattern 3. Such pattern 3 cribriform glands are small and have round and regular contour with regular lumina. They should be differentiated from cribriform PIN.

0.645

Normal urothelium. The urothelial cells form a layer five to seven cells thick, with large umbrella cells sitting on top (arrow). The urothelial nuclei are generally polarized and oriented perpendicular to the surface, with the exception of the umbrella cells. The nuclei are two to three times the size of a lymphocyte (arrowhead).

0.619

Ground truth caption

0.556

Perineural invasion. A nerve (N) is identified by the undulating axons and nerve sheath nuclei. Malignant glands are seen nearly surrounding the nerve (arrow).

Ground truth caption Low-power view of a lobular growth pattern in a chordoma.

0.615

Ground truth caption Von Brunns nest and cystitis glandularis. The normal urothelium has invaginated down into the lamina propria, forming a rounded von Brunns nest (arrow). The center of the nest has acquired a lumen and columnar cell metaplasia (asterisk).

Ground truth caption

0.553 0.629

Prostate adenocarcinoma with atrophic features. At higher magnification, the cancer glands are not completely atrophic and still retain a small amount of cytoplasm. Also note that the cancer glands display unequivocal malignant histologic features.

Ground truth caption Spindling of chondrocytes is a feature of chondrosarcoma.

Query text

Query text

0.622

Top-3 retrieved images

0.620

PASM-Masson stain at 40x magnification shows nodular diabetic glomerulosclerosis.

Top-3 retrieved images

Ground truth caption Later in the secretory phase, the cytoplasmic vacuoles are gone, and the epithelium is more cuboidal in shape, with small round nuclei (arrow). The stroma is edematous, and early decidualization (accumulation of pink cytoplasm) is beginning around the spiral arteries (arrowhead).

0.611

0.608

Ground truth caption

0.613

Ground truth caption IgG4-related lymphadenopathy, HE staining: low magnification shows in the central area blurring of the follicular germinal centermantle zone boundary, with expansion of the mantle zone, consistent with progressive transformation of the germinal center.

Top-3 retrieved images Ground truth caption

0.569

PASM-Masson stain at 40x magnification shows diffuse mesangial proliferative diabetic nephropathy with moderate mesangial proliferation and widened mesangial areas with increased matrix. A "ballooning drop" is observed at the 8 o'clock position of the Bowman's capsule, along with crescentic plasma protein deposits on the inner wall of the capsule.

0.561

Ground truth caption Lymph node paraffin section shows PTGC has follicles at different stages of transformation.

Early secretory endometrium. Prominent subnuclear vacuoles are seen.

0.605

Ground truth caption The basement membrane shows a double-track appearance at 40x magnification. 0.561

Ground truth caption

Ground truth caption

Morphological features of reactive follicular hyperplasia. Germinal center, mantle zone, and marginal zone structures are clear.

Round to tubular glands exhibit nuclear pseudostratification and mitotic activity.

Query image

Ground truth caption The squamous epithelium appears flattened with decreased to absent intracytoplasmic glycogen. Query text

Lymphoid follicular reactive hyperplasia, HE staining

The endometrial stroma shows prominent edema.

0.612

Top-3 retrieved texts

Ground truth caption PASM stain at 40x magnification shows endothelial cell proliferation within blood vessels.

Top-3 retrieved texts

Extended Data Figure 6: Representative ROI text-to-image cross-modal retrieval examples. Text-to-image retrieval examples using ALICE on paired pathology image-caption data. Each query text is used to retrieve the top three ROI images, with cosine similarity scores and the corresponding ground-truth captions shown. Red text highlights key pathological terms shared between the query and retrieved results. Ground truth caption Melanophages in an intradermal nevus. The nevus cells (arrowhead) have focal small specks of pigment, but the macrophages digesting the excess melanin (arrow) stand out.

Query image

Query image

Clark level IV: melanoma extends into the reticular dermis. Coarse reticular dermal collagen bundles surround melanoma cells.

Leiomyoma composed of fascicles of bland spindled cells with blunt-ended nuclei and cytoplasmic vacuolation.

melanoma has metastasized to the site of basal cell carcinoma (collision phenomenon).

Primary ovarian leiomyoma. Spindled smooth muscle cells contain bland, boxcar-shaped, elongated nuclei.

Acral melanoma in situ with prominent lentiginous pattern: melanocytes are arranged as solitary units along the base of elongated rete ridges.

2

Top-3 retrieved texts

Query image

Lymph node of patient with drug hypersensitivity syndrome, panoramic view. Germinal center hyperplasia, interfollicular area markedly expanded. Lymphoid infiltration involves capsule and perinodal fat.

Ground truth caption Giant follicular hyperplasia, HE staining: Geographic lymphoid follicular reactive hyperplasia. Note the presence of multiple reactive secondary lymphoid follicles of varying sizes and serpentine borders. All follicles have their own

Ground truth caption Fascicular pattern, leiomyoma.

2

2

Top-3 retrieved texts

Tumor contains dilated hemorrhagic spaces, similar to spaces in aneurysmal bone cyst, HE×40.

41 HE×20 tumor cells show very significant atypia, multinucleated tumor giant cells are visible.

Lymphoid follicular reactive hyperplasia, HE staining.

Lymph node changes of nodular variant LRCHL. HE staining low power view shows multinodular structure (solid blue arrow), similar to NLPHL. However, no moth-eaten structure is seen within the lymph node.

Submucosal leiomyoma. Intersecting bundles of benign-appearing smooth muscle cells.

Ground truth caption Multinucleated giant cells, hemorrhage, and fibrous hyperplasia

2

Osteoclast-like giant cells in the tumor.

shows a double-track appearance at 40x magnification. 0.613

0.605

Morphological features of reactive follicular hyperplasia. Germinal center, mantle zone, and marginal zone structures are clear.

Round to tubular glands exhibit nuclear pseudostratification and mitotic activity.

Query image

Ground truth caption Melanophages in an intradermal nevus. The nevus cells (arrowhead) have focal small specks of pigment, but the macrophages digesting the excess melanin (arrow) stand out.

Query image

0.561

Ground truth caption

Ground truth caption

Top-3 retrieved texts

Query image

Giant follicular hyperplasia, HE staining: Geographic lymphoid follicular reactive hyperplasia. Note the presence of multiple reactive secondary lymphoid follicles of varying sizes and serpentine borders. All follicles have their own mantle zones at the edges

Top-3 retrieved texts

Clark level IV: melanoma extends into the reticular dermis. Coarse reticular dermal collagen bundles surround melanoma cells.

Leiomyoma composed of fascicles of bland spindled cells with blunt-ended nuclei and cytoplasmic vacuolation.

melanoma has metastasized to the site of basal cell carcinoma (collision phenomenon).

Primary ovarian leiomyoma. Spindled smooth muscle cells contain bland, boxcar-shaped, elongated nuclei.

Acral melanoma in situ with prominent lentiginous pattern: melanocytes are arranged as solitary units along the base of elongated rete ridges.

2

Ground truth caption Fascicular pattern, leiomyoma.

Top-3 retrieved texts

Query image

Lymph node of patient with drug hypersensitivity syndrome, panoramic view. Germinal center hyperplasia, interfollicular area markedly expanded. Lymphoid infiltration involves capsule and perinodal fat.

Ground truth caption

Ground truth caption PASM stain at 40x magnification shows endothelial cell proliferation within blood vessels.

Submucosal leiomyoma. Intersecting bundles of benign-appearing smooth muscle cells.

2

Top-3 retrieved texts

Tumor contains dilated hemorrhagic spaces, similar to spaces in aneurysmal bone cyst, HE×40.

HE×20 tumor cells show very significant atypia, multinucleated tumor giant cells are visible.

Lymphoid follicular reactive hyperplasia, HE staining.

Lymph node changes of nodular variant LRCHL. HE staining low power view shows multinodular structure (solid blue arrow), similar to NLPHL. However, no moth-eaten structure is seen within the lymph node.

Ground truth caption

2

Multinucleated giant cells, hemorrhage, and fibrous hyperplasia in tumor tissue, HE×100.

2

Osteoclast-like giant cells in the tumor.

Extended Data Figure 7: Representative ROI image-to-text cross-modal retrieval examples. Image-to-text retrieval examples using ALICE on paired pathology image-caption data. Each query image is used to retrieve the top three text descriptions, with the ground-truth caption shown for reference. Red text highlights key pathological terms shared between the query and retrieved results.

42

Heatmap and top-2 highest-scoring patches

F202404153024 (Nephroblastoma)

Heatmap and top-2 highest-scoring patches

F202405020005 (Hepatoblastoma)

Heatmap and top-2 highest-scoring patches

NULL-20241226-200144 (Hepatoblastoma)

Heatmap and top-2 highest-scoring patches

NULL-20241219-042625 (Medulloblastoma)

Heatmap and top-2 highest-scoring patches

NULL-20241218-222637 (Medulloblastoma)

Heatmap and top-2 highest-scoring patches

NULL-20241218-124142 (Neuroblastoma)

Heatmap and top-2 highest-scoring patches

NULL-20241218-131445 (Neuroblastoma)

Heatmap and top-2 highest-scoring patches

NULL-20241227-150124 (Nephroblastoma)

Extended Data Figure 8: Representative WSI heatmaps and high-scoring regions for pediatric rare tumor classification. Representative ALICE predictions on KidRare whole-slide images covering nephroblastoma, hepatoblastoma, medulloblastoma and neuroblastoma. For each case, the original H&E-stained WSI is shown together with the corresponding prediction heatmap and the top two highest-scoring patches.

43

H ea d d

ea st ca n

m

ou s

ce l

ca r ci n

C-index

0.6

0.45

0.40

0.2

0.0

1.0

d.

0.4

0.0

44 TITAN

0.7

CARE

0.8

0.7

0.5

0.2

0.3 0.35

0.30 0.25

All tasks

0.4

0.54

0.52

0.4492

0.8

0.6750

0.6

0.56

0.6703

0.6597

0.64

0.5679

0.0

0.3924

0.2

0.66

0.3310

BAcc

BAc c 0.6

0.5500

no d

0.8

0.5483

0.4

BAcc

m

Pr

os ta te (P ca AN nc D er s ph A, c e 2 re e no (C me cl de a s n in am ta se g m el sta et s) s y as on is ta +, sc tic re 2 (C b u cl en as in a m rd se g C el en ol s) yo cl or n+ as ec , 4 sifi ta c ln c la atio (H eo s se n u n p la s) C si a Bo R C sc ne , 4 re m cl en e a (S ta ss ing YS sta es ) BM sis , 1 sc C 0 ree er c la n in vi ss (S L u ca es g YS ng le ) pi FL l e th +, sio el ia 4 nt N r cl on ( T l le s as iag -s is se e m su i o n s) al eN cl l et ass (C cel , 4 ifi P T l lu cl cat AC ng a s io c -N a se n SC nc O s) va LC er s ri , 2 ub (U a n cl typ B C ca as in - O r ci n se g R C om en s) EA a al N ce , 5 sub ( D ll c cl typ H arc a i s M BR se ng C in o s) -R m C A C a co C , 5 sub ar se cl typ a s in (B g ra se g R in e s) BR AC d C S A , 3 sub fin cl typ ea s in g Pr (B ra se g os R in e s) Lu ta AC d te ng S, su ca ca 7 bty nc nc c er la p in er s ( G s su P A le es g bt N as ) yp D A, on in gr g 6 an cl ad a s in C ( S d in N se g S YS va s) fin FL sio e+, n gr 7 g ra a cl (E ine as din BR d se g C AI tum N s) S N S, or d tu m 3 0 ia or gn (E -fam clas os B R il se is s) AI y c N las C S, s N 9 ific S c la atio (E tum Pe s BR or se n di s at AI W Pe r ic N HO ) S, di ra at 4 g ra re ric cl t as din ra (K u m se g re id or s) tu R m ar clas or e, s fin 4 ific cl e as atio (K -gr se n id ai R s) ar ned e, s 13 ub cl typ a s in se g s) Ly

Ly m ph

0.4

C-index

ALICE

Lu

0.6230

0.5683

0.6

sq ua

el l

0.5

al c

0.4

ck

0.6

ne

0.8

0.5612

CARE

lr en

TITAN

ar ce l

c. CARE

le

TITAN

C

BAc c

b.

CARE

AUC

AUC

Pr

os ta te (P ca AN nc Ly no m D er s ph A, de c 2 re e no (C me cl de a s n in am ta se g m el sta et s) s y as on is ta +, sc tic re 2 (C b u cl en as in a m rd se g C el en ol s) yo cl or n+ as ec , 4 sifi ta c ln c la atio (H eo s se n u n p la s) C si a Bo R C sc ne , 4 re m cl en e a (S ta ss ing YS sta es ) BM sis , 1 sc C 0 ree er c la n in vi ss (S L u ca es g YS ng le ) pi FL l e th +, sio el ia 4 nt N r cl on ( T l le s as iag -s is se e m su ion s) al eN cl l et ass (C cel , 4 ifi P T l lu cl cat AC ng a s io c -N a se n SC nc O s) va LC er s ri , 2 ub (U a n cl typ BC ca as in -O rcin se g R C om en s) EA a al N ce , 5 sub ( D ll c cl typ H arc a i s M BR se ng C in o s) -R m C A C a co C , 5 sub ar se cl typ a s in (B g ra se g R in e s) BR AC d C S A , 3 sub fin cl typ ea s in g Pr (B ra se g os R in e s) Lu ta AC d te ng S, su ca ca 7 bty nc nc c er la p in er s ( G s su P A le es g bt N as ) yp D A, on in gr g 6 an cl ad a s in C ( S d in N se g S YS va s) fin FL sio e+, n gr 7 g ra a cl (E ine as din BR d se g C AI tum N s) S N S, or d tu m 3 0 ia or gn (E -fam clas os B R il se is s) AI y c N las C S, s N 9 ific S c la atio (E tum Pe s BR or se n di s at AI W Pe r ic N HO ) S, di ra at 4 g ra re ric cl t as din ra (K u m se g re id or s) tu R m ar clas or e, s fin 4 ific cl e as atio (K -gr se n id ai R s) ar ned e, s 13 ub cl typ a s in se g s) Ly m ph

TITAN

om (C a s P u lc ar TA rviv C ci Pa -C al p no ng nc C re m ad R re a C di c en s at ic oc (CP urv C, tion du ar TA iva OS c ct ) in C lp al H o H re m ea ad N di a d S c en s an oc (C urv C, tion d PT i v O a ne S) rc A al in ck om C-L pre ca U d a nc su AD icti er o Pr , O n tre (CP rviv im S) TA al at ar p m y en C-P red co ts lo D ict re A, io ur ct O n (H viv al S) A N al ca nc C pre O di er (S su CK ctio , ur rv n G iva OS en ) l SR pre 38 dic 6, t i o O n S)

an

Br ce ea rE st R ca nc ( sta er BC tu H NB s p Br ER , r ea 2 2 edi st ca (B sta clas ctio nc C tu se n N sp s R e r P B, en r ) al R 2 edi c c G ce lio (B stat las tio ll R m CN us se n ca en a s r p ci al ID B, re ) n ( ce om E H 2 di ll a BR sta clas ctio ca R rc (M BA AIN tus se n en in U P1 S p s) om Tal , 2 re m ce a HE ut c dic l ll ca ( PBR T-R atio ass tion n e rc M in UT M1 CC pr s) om m , 2 ed a HE uta cla ict io S T (M ET -RC tion sse n U D2 C pr s) T, e H mu 2 c dic ET ta la ti -R tio ss on C n p es C , 2 red ) cl ict as i o se n s)

Br

a. 1.0

ALICE

0.70

All tasks

0.68

0.62 0.60

1.0

ALICE

0.50

All tasks

0.20

0.9

ALICE

0.60

All tasks

0.58

0.3

0.2

0.50

Extended Data Figure 9: Detailed slide-level WSI transfer results. a. Slide-level diagnostic classification using KNN on frozen WSI embeddings from ALICE, TITAN and CARE across 19 tasks. b. Slide-level diagnostic classification using linear probing on frozen WSI embeddings across the same 19 tasks. c. Slide-level biomarker prediction using linear probing across seven tasks. d, Slide-level survival prediction across six tasks using Cox proportional hazards models.

Query slide (CCRCC, CPTAC)

Query slide (Kidney, SYSBM)

Query slide (AML, CMB)

High

Query slide (PCA, CMB)

0.911

0.996

0.949 0.942

Kidney, SYSBM

PCA, CMB AML, CMB

0.925

0.904

Kidney, SYSBM

0.911

CCRCC, CPTAC 0.986

PCA, CMB

0.872

Cosine Similarity

AML, CMB 0.900

CCRCC, CPTAC

0.970

AML, CMB

CCRCC, CPTAC

Kidney, SYSBM 0.862

Kidney, SYSBM

PCA, CMB

0.968

0.896

AML, CMB PCA, CMB 0.868

0.855

CCRCC, CPTAC 0.957

Low

PCA, CMB Kidney, SYSBM

AML, CMB

CCRCC, CPTAC

Extended Data Figure 10: Representative slide-to-slide retrieval examples. Slide-to-slide retrieval using ALICE slide-level embeddings. Four query WSIs from SYSBM, CMB and CPTAC are shown with their top five retrieved slides ranked by cosine similarity. Similarity scores, diagnostic labels and data sources are shown for each retrieved slide.

45

Hyperparameter

Value

Transformer layers Transformer heads Transformer embedding dimension Transformer patch size Summary tokens Register tokens

32 16 1280 14 3 7

Patch crop size Patch crop number Patch crop grid Random resized crop scale Random resized crop aspect ratio Horizontal flip probability Vertical flip probability Colour jitter EMA decay (cosine schedule) DAMP

98 × 98 8 7×7 (0.5, 1.0) (0.75, 1.33) 0.5 0.5 0.4 (0.996, 1.0) 0.05

Training precision AdamW β Weight decay Linear warmup steps Training steps Peak learning rate (cosine decay) Minimum learning rate (cosine decay) Gradient clipping

fp16 (0.9, 0.999) 0.05 104 6 × 105 3 × 10−4 1 × 10−5 1.0

Extended Data Table 1: ALICE vision-only pretraining hyperparameters. 8 × NVIDIA A100 80GB GPUs were used for training. The total effective batch size is 1024 with 64 per GPU and 2 gradient accumulations.

46

Hyperparameter

Value

Transformer layers Transformer heads Transformer embedding dimension Patch crop size Patch crop number Patch crop grid Random resized crop scale Random resized crop aspect ratio Horizontal flip probability Vertical flip probability Colour jitter KEEP input size CONCH input size MUSK input size EMA decay (cosine schedule) DAMP

2 16 1280 98 × 98 4 7×7 (0.5, 1.0) (0.75, 1.33) 0.5 0.5 0.4 224 × 224 448 × 448 384 × 384 (0.996, 1.0) 0.02

Training precision AdamW β Weight decay Linear warmup steps Training steps Peak learning rate Minimum learning rate Gradient clipping

bf16 (0.9, 0.999) 0.02 104 6 × 105 3 × 10−4 1 × 10−5 1.0

Extended Data Table 2: ALICE multimodal pretraining hyperparameters. 8 × NVIDIA A100 80GB GPUs were used for training. The total effective batch size is 1024 with 32 per GPU and 4 gradient accumulations.

47

Hyperparameter

Value

Transformer layers 2 Transformer heads 8 Transformer input dimension 3840 Transformer hidden dimension 1024 Attention heads 8 Dropout 0.25 Summary tokens 2 Feature brightness strength TITAN patch loss weight EMA decay (cosine schedule) DAMP

0.05 0.5 (0.996, 1.0) 0.02

Training precision AdamW β Weight decay Linear warmup steps Training steps Peak learning rate Minimum learning rate Gradient clipping

bf16 (0.9, 0.999) 0.05 5 × 103 2 × 105 3 × 10−4 1 × 10−5 1.0

Extended Data Table 3: ALICE slide-level pretraining hyperparameters. 8 × NVIDIA A100 80GB GPUs were used for training. The total effective batch size is 1024 with 128 per GPU.

48

ID Prompt template 1 2 3 4 5 6 7 8 9 10

an H&E stained pathology image patch of {label text}. a histopathology image showing {label text}. a microscopic pathology image of {label text}. a tissue patch containing {label text}. a pathology slide region with {label text}. a pathology ROI consistent with {label text}. a diagnostic histology image of {label text}. a histologic tissue patch of {label text}. a pathology image whose main finding is {label text}. a microscopic image of {label text}.

Extended Data Table 4: Generic prompt templates for ROI zero-shot classification. Class

Class names

Gleason pattern 3

prostate tissue with Gleason pattern 3 prostatic adenocarcinoma of Gleason pattern 3 Gleason grade 3 prostate cancer well formed glands of Gleason pattern 3 prostate tumor tissue showing Gleason pattern 3

Gleason pattern 4

prostate tissue with Gleason pattern 4 prostatic adenocarcinoma of Gleason pattern 4 Gleason grade 4 prostate cancer fused or cribriform glands of Gleason pattern 4 prostate tumor tissue showing Gleason pattern 4

Gleason pattern 5

prostate tissue with Gleason pattern 5 prostatic adenocarcinoma of Gleason pattern 5 Gleason grade 5 prostate cancer poorly differentiated Gleason pattern 5 tumor prostate tumor tissue showing Gleason pattern 5

Normal prostate tissue

normal prostate tissue benign prostate tissue non neoplastic prostate glands histologically normal prostate normal prostatic glandular tissue

Prostatic stroma

prostatic stroma stromal tissue of the prostate fibromuscular prostate stroma non epithelial stromal prostate tissue prostate stromal region

Extended Data Table 5: Class prompts for AGGC ROI zero-shot classification. 49

Class

Class names

Benign breast tumors

a benign breast lesion benign breast tumor tissue non malignant breast lesion benign breast pathology pathologically benign breast tissue

Atypical breast tumors

an atypical breast lesion atypical breast tumor tissue borderline atypical breast pathology breast tissue with atypia an atypical proliferative breast lesion

Malignant breast tumors

a malignant breast tumor malignant breast carcinoma tissue invasive or in situ malignant breast lesion breast cancer tissue pathologically malignant breast tissue

Extended Data Table 6: Class prompts for BRACS (3 classes) ROI zero-shot classification.

50

Class

Class names

Normal breast tissue

normal breast tissue non neoplastic breast tissue histologically normal breast parenchyma normal mammary tissue benign normal breast epithelium

Pathologically breast lesion

benign

a pathologically benign breast lesion benign proliferative breast lesion pathological benign breast tissue benign breast abnormality non malignant pathological breast lesion

Usual ductal hyperplasia

usual ductal hyperplasia of the breast breast tissue with usual ductal hyperplasia usual ductal hyperplasia lesion UDH breast lesion benign ductal hyperplasia in breast tissue

Flat epithelial atypia

flat epithelial atypia of the breast breast tissue with flat epithelial atypia FEA breast lesion flat epithelial atypia lesion atypical flat epithelial breast lesion

Atypical ductal hyperplasia

atypical ductal hyperplasia of the breast breast tissue with atypical ductal hyperplasia ADH breast lesion atypical proliferative ductal breast lesion atypical ductal hyperplasia lesion

Ductal carcinoma in situ

ductal carcinoma in situ of the breast breast tissue with ductal carcinoma in situ DCIS breast lesion in situ ductal breast carcinoma non invasive ductal breast carcinoma

Invasive breast carcinoma

invasive breast carcinoma breast tissue with invasive carcinoma infiltrating breast carcinoma IC breast lesion malignant invasive breast tumor

Extended Data Table 7: Class prompts for BRACS (7 classes) ROI zero-shot classification.

51

Class

Class names

Benign breast tumor

a benign breast tumor benign breast lesion non malignant breast pathology benign mammary tumor tissue histologically benign breast tissue

Malignant breast tumor

a malignant breast tumor breast carcinoma tissue malignant breast pathology cancerous breast lesion histologically malignant breast tissue

Extended Data Table 8: Class prompts for BreakHis ROI zero-shot classification.

52

Class

Class names

Blood-rich renal tissue

blood rich renal tissue hemorrhagic tissue region blood filled histology patch red blood cell rich tissue vascular blood region

Clear cell renal cell carcinoma

clear cell renal cell carcinoma ccRCC tumor tissue renal carcinoma tissue kidney cancer tissue clear cell kidney tumor

Background or empty region

background or empty region empty histology patch blank slide background tissue free region non tissue background patch

Normal kidney tissue

normal kidney tissue non neoplastic renal tissue histologically normal renal parenchyma benign kidney tissue normal renal tissue

Other renal tissue component

other renal tissue component miscellaneous kidney tissue kidney tissue of other type other non tumor renal component renal tissue not in the main categories

Renal stroma

renal stroma stromal kidney tissue fibrous tumor associated stroma stromal tissue region connective stromal tissue

Extended Data Table 9: Class prompts for CCRCC ROI zero-shot classification.

53

Class

Class names

Normal colorectal tissue

normal colorectal tissue histologically normal colon mucosa normal colorectal mucosa non neoplastic colorectal tissue benign normal colorectal epithelium

Serrated colorectal lesion

a serrated colorectal lesion serrated colorectal polyp colorectal tissue with serrated architecture serrated lesion of the colon serrated colorectal pathology

Colorectal adenocarcinoma

colorectal adenocarcinoma colorectal cancer tissue adenocarcinoma of the colon or rectum malignant colorectal glandular tumor tumor epithelium of colorectal adenocarcinoma

Colorectal adenoma

colorectal adenoma adenomatous colorectal polyp colorectal tissue with adenoma benign colorectal adenomatous lesion adenoma of the colon or rectum

Extended Data Table 10: Class prompts for Chaoyang ROI zero-shot classification.

54

Class

Class names

Adipose tissue

adipose tissue fat tissue in colorectal histology adipocytes colorectal adipose tissue fat rich tissue region

Background

histology image background background region non tissue background patch slide background empty histologic background

Debris

debris necrotic debris cellular debris in colorectal tissue degenerated tissue debris tumor associated debris

Lymphocytes

lymphocytes lymphocyte rich tissue immune cell rich region lymphoid infiltrate dense lymphocytic infiltrate

Mucus

mucus mucin rich tissue mucus producing region mucinous material colorectal mucus region

Smooth muscle

smooth muscle muscular tissue smooth muscle region muscle layer tissue colorectal smooth muscle

Extended Data Table 11: Class prompts for CRC-100K ROI zero-shot classification. Continued.

55

Class

Class names

Normal colon mucosa

normal colon mucosa histologically normal colorectal mucosa non neoplastic colon mucosa normal colorectal epithelium benign colon mucosal tissue

Cancer-associated stroma

cancer associated stroma stromal tissue around tumor fibrovascular stroma tumor stroma stromal connective tissue

Colorectal adenocarcinoma epithelium

colorectal adenocarcinoma epithelium tumor epithelium colorectal cancer epithelium adenocarcinoma tissue malignant colorectal glandular epithelium

Extended Data Table 12: Class prompts for CRC-100K ROI zero-shot classification. Continued.

Class

Class names

MSI-high colorectal cancer

colorectal cancer with microsatellite instability high status MSI high colorectal adenocarcinoma microsatellite instability high colorectal tumor MMR deficient colorectal cancer MSI H colorectal carcinoma

Non-MSI-high cancer

colorectal cancer without microsatellite instability high status microsatellite stable or low instability colorectal cancer non MSI high colorectal adenocarcinoma MSS or MSI low colorectal tumor non MSIH colorectal carcinoma

colorectal

Extended Data Table 13: Class prompts for CRC-MSI ROI zero-shot classification.

56

Class

Class names

Esophageal adventitia

esophageal adventitia adventitial connective tissue outer adventitial layer of the esophagus adventitia around the esophagus esophageal outer fibrous tissue

Lamina propria

lamina propria lamina propria of the esophagus mucosal lamina propria loose connective tissue of the lamina propria esophageal lamina propria tissue

Muscularis mucosae

muscularis mucosae esophageal muscularis mucosae thin smooth muscle of the mucosa muscularis mucosa layer mucosal muscle layer

Muscularis propria

muscularis propria esophageal muscularis propria thick muscle wall of the esophagus muscularis propria layer smooth muscle of the esophageal wall

Regressive tumor tissue

regressive tumor tissue tumor regression bed regressed esophageal adenocarcinoma tissue post treatment regressive tumor area regression associated tumor tissue

Gastric mucosa

gastric mucosa stomach mucosa gastric glandular mucosa mucosa of the stomach gastric epithelial mucosa

Extended Data Table 14: Class prompts for ESCA-Tolkach ROI zero-shot classification. Continued.

57

Class

Class names

Squamous esophageal mucosa

squamous esophageal mucosa normal squamous epithelium of the esophagus esophageal squamous mucosa squamous epithelial esophageal tissue non glandular esophageal mucosa

Submucosa

submucosa esophageal submucosa submucosal connective tissue submucosal layer submucosal tissue of the esophagus

Submucosal glands

submucosal glands esophageal submucosal glands glandular tissue in the submucosa submucosal gland region mucous glands of the submucosa

Esophageal adenocarcinoma

esophageal adenocarcinoma tumor tissue of esophageal adenocarcinoma malignant esophageal gland forming tumor esophageal tumor epithelium adenocarcinoma of the esophagus

Ulcer

ulcer ulcerated tissue ulcer bed ulcer associated tissue damage esophageal ulceration

Extended Data Table 15: Class prompts for ESCA-Tolkach ROI zero-shot classification. Continued.

58

Class

Class names

Adipose tissue

adipose tissue fat tissue in gastric histology adipocytes gastric adipose tissue fat rich tissue region

Debris

debris necrotic debris cellular debris in gastric tissue degenerated tissue debris tumor associated debris

Lymphocytes

lymphocytes lymphocyte rich tissue immune cell rich region lymphoid infiltrate dense lymphocytic infiltrate

Mucus

mucus mucin rich tissue mucus producing region mucinous material gastric mucus region

Smooth muscle

smooth muscle muscular tissue smooth muscle region muscle layer tissue gastric smooth muscle

Normal gastric mucosa

normal gastric mucosa histologically normal stomach mucosa non neoplastic gastric mucosa normal gastric epithelium benign stomach mucosal tissue

Cancer-associated stroma

cancer associated stroma stromal tissue around gastric tumor fibrovascular stroma tumor stroma stromal connective tissue

Gastric tumor epithelium

gastric tumor epithelium gastric cancer tissue tumor epithelium in stomach histology malignant gastric epithelial tumor gastric adenocarcinoma epithelium

Extended Data Table 16: Class prompts for GCHTID ROI zero-shot classification. 59

Class

Class names

Non-tumor tissue

non tumor osteosarcoma bed tissue background bone or soft tissue without tumor non neoplastic tissue in osteosarcoma histology osteosarcoma slide region without tumor benign non tumor tissue region

Non-viable osteosarcoma

non viable osteosarcoma tumor necrotic osteosarcoma tissue treated non viable tumor tissue dead osteosarcoma tumor cells non viable malignant osteoid tumor

Viable osteosarcoma

viable osteosarcoma tumor living osteosarcoma tissue viable malignant osteoid tumor active osteosarcoma tumor cells residual viable osteosarcoma

Extended Data Table 17: Class prompts for OTA ROI zero-shot classification.

60

Class

Class names

Hyperplastic polyp

hyperplastic polyp colorectal hyperplastic polyp hyperplastic polyp tissue benign hyperplastic colorectal lesion hyperplastic mucosal polyp

Normal colorectal tissue

normal colorectal tissue histologically normal colon mucosa normal colorectal mucosa non neoplastic colorectal tissue benign normal colorectal epithelium

Tubular adenoma with highgrade dysplasia

tubular adenoma with high grade dysplasia high grade tubular adenoma colorectal tubular adenoma high grade tubular adenoma showing severe dysplasia high grade dysplastic tubular adenoma

Tubular adenoma with lowgrade dysplasia

tubular adenoma with low grade dysplasia low grade tubular adenoma colorectal tubular adenoma low grade tubular adenoma showing mild dysplasia low grade dysplastic tubular adenoma

Tubulovillous adenoma with high-grade dysplasia

tubulovillous adenoma with high grade dysplasia high grade tubulovillous adenoma colorectal tubulovillous adenoma high grade tubulovillous adenoma showing severe dysplasia high grade dysplastic tubulovillous adenoma

Tubulovillous adenoma with low-grade dysplasia

tubulovillous adenoma with low grade dysplasia low grade tubulovillous adenoma colorectal tubulovillous adenoma low grade tubulovillous adenoma showing mild dysplasia low grade dysplastic tubulovillous adenoma

Extended Data Table 18: Class prompts for Unitopatho ROI zero-shot classification.

61

Template ID

Prompt template

1 2 3 4

{label text}. a histopathological image showing {label text}. a histopathological image of {label text}. an H&E stained image of {label text}.

Extended Data Table 19: Generic prompt templates for MI-Zero WSI zero-shot classification.

Class

Class names

Negative lymph node

negative for tumor in lymph node benign lymph node tissue lymph node without metastatic carcinoma

Tumor-positive lymph node

lymph node with metastatic carcinoma tumor-positive lymph node lymph node involved by carcinoma

Extended Data Table 20: Class prompts for MI-Zero lymph node metastasis screening on CAMELYON+.

Class

Class names

Lung adenocarcinoma

lung adenocarcinoma adenocarcinoma of the lung LUAD lung adenocarcinoma

Lung squamous cell carcinoma

lung squamous cell carcinoma squamous cell carcinoma of the lung LSCC lung squamous cell carcinoma

Extended Data Table 21: Class prompts for MI-Zero non-small cell lung cancer subtyping on CPTAC-NSCLC.

62

Class

Class names

Diffuse glioma

diffuse glioma diffuse glioma in CNS histopathology diffuse glioma brain tumor

Hematolymphoid or histiocytic tumor

hematolymphoid or histiocytic tumor hematolymphoid or histiocytic tumor in CNS histopathology hematolymphoid or histiocytic brain tumor

Meningioma

meningioma meningioma in CNS histopathology meningioma brain tumor

Mesenchymal or lipomatous tumor

mesenchymal or lipomatous tumor mesenchymal or lipomatous tumor in CNS histopathology mesenchymal or lipomatous brain tumor

Metastatic tumor

metastatic tumor metastatic tumor in CNS histopathology metastatic brain tumor

Nerve sheath tumor

nerve sheath tumor nerve sheath tumor in CNS histopathology nerve sheath brain tumor

Other neuroepithelial tumor

other neuroepithelial tumor other neuroepithelial tumor in CNS histopathology other neuroepithelial brain tumor

Sellar-region tumor

sellar-region tumor sellar-region tumor in CNS histopathology sellar-region brain tumor

Vascular or mesenchymal tumor

vascular or mesenchymal tumor vascular or mesenchymal tumor in CNS histopathology vascular or mesenchymal brain tumor

Extended Data Table 22: Class prompts for MI-Zero CNS tumor-family classification on EBRAINS.

63

Class

Class names

Hepatoblastoma

hepatoblastoma pediatric liver tumor hepatoblastoma embryonal liver tumor hepatoblastoma

Medulloblastoma

medulloblastoma cerebellar medulloblastoma embryonal cerebellar tumor medulloblastoma

Nephroblastoma

nephroblastoma Wilms tumor of the kidney pediatric kidney tumor nephroblastoma

Neuroblastoma

neuroblastoma adrenal neuroblastoma peripheral neuroblastic tumor neuroblastoma

Extended Data Table 23: Class prompts for MI-Zero pediatric rare tumor classification on KidRare.

64

Class

Class names

Breast carcinoma

breast carcinoma breast cancer malignant breast tumor

Gastric carcinoma

gastric carcinoma gastric cancer stomach cancer

Renal cell carcinoma

renal cell carcinoma kidney cancer renal tumor

Hepatocellular carcinoma

hepatocellular carcinoma liver cancer hepatic malignancy

Lung carcinoma

lung carcinoma lung cancer pulmonary malignancy

Neuroendocrine tumor

neuroendocrine tumor neuroendocrine carcinoma neuroendocrine malignancy

Normal bone marrow

normal bone marrow benign bone marrow tissue non-neoplastic bone marrow

Prostate adenocarcinoma

prostate adenocarcinoma prostate cancer prostatic carcinoma

Squamous cell carcinoma

squamous cell carcinoma squamous carcinoma malignant squamous tumor

Thyroid carcinoma

thyroid carcinoma thyroid cancer thyroid malignancy

Extended Data Table 24: Class prompts for MI-Zero bone metastasis primary site prediction on SYSBM.

65

Class

Class names

Ovarian clear cell carcinoma

ovarian clear cell carcinoma clear cell carcinoma of the ovary ovarian clear cell adenocarcinoma

Ovarian endometrioid carcinoma

ovarian endometrioid carcinoma endometrioid carcinoma of the ovary ovarian endometrioid adenocarcinoma

High-grade serous ovarian carcinoma

high-grade serous ovarian carcinoma high-grade serous carcinoma of the ovary ovarian high-grade serous carcinoma

Low-grade serous ovarian carcinoma

low-grade serous ovarian carcinoma low-grade serous carcinoma of the ovary ovarian low-grade serous carcinoma

Ovarian mucinous carcinoma

ovarian mucinous carcinoma mucinous carcinoma of the ovary ovarian mucinous adenocarcinoma

Extended Data Table 25: Class prompts for MI-Zero ovarian carcinoma subtyping on UBCOCEAN.

Clinical variable

Text segment template

Age Tumour size Tumour type ER status PR status HER2 status HER2 expression Histological grade Ki-67 index Molecular subtype

Patient age is [x] years. Tumour size is [x] cm. Tumour type is [x]. ER status is [x]. PR status is [x]. HER2 status is [x]. HER2 expression score is [x]. Histological grade is [x]. Ki-67 index is [x]. Molecular subtype is [x].

Extended Data Table 26: Clinical text segment templates for the breast cancer axillary lymph node metastasis (BCNB, 2 classes and 3 classes). The placeholder [x] represents the specific value of each clinical variable.

66

Clinical variable

Text segment template

Age at initial diagnosis Sex Smoking status Primary tumour site HPV association by p16

Patient age at initial diagnosis is [x] years. Patient sex is [x]. Smoking status is [x]. Primary tumour site is [x]. HPV association by p16 is [x].

Extended Data Table 27: Clinical text segment templates for the head and neck keratinizing SCC grading (Hancock, 2 classes). The placeholder [x] represents the specific value of each clinical variable.

Clinical variable

Text segment template

Age at initial diagnosis Patient age at initial diagnosis is [x] years. Sex Patient sex is [x]. Smoking status Smoking status is [x]. Primary tumour site Primary tumour site is [x]. Histologic type Histologic type is [x]. Histologic grade Histologic grade is [x]. HPV association by p16 HPV association by p16 is [x].

Extended Data Table 28: Clinical text segment templates for the head and neck lymphovascular invasion detection (Hancock, 2 classes) and vascular invasion detection (Hancock, 2 classes). The placeholder [x] represents the specific value of each clinical variable.

67

Clinical variable

Text segment template

Age Ethnicity Race Neo-adjuvant treatment status Other cancer diagnosis Tumour type Tumour location Tumour location group Histological grade Tumour stage Tumour substage

Patient age is [x] years. Patient ethnicity is [x]. Patient race is [x]. Neo-adjuvant treatment status is [x]. Other cancer diagnosis is [x]. Tumour type is [x]. Tumour location is [x]. Tumour location group is [x]. Histological grade is [x]. Tumour stage is [x]. Tumour substage is [x].

Extended Data Table 29: Clinical text segment templates for the ovarian cancer treatment response prediction (PTRC-HGSOC, 2 classes). The placeholder [x] represents the specific value of each clinical variable.

Clinical variable

Text segment template

Age Patient age is [x] years. Ethnicity Patient ethnicity is [x]. Race Patient race is [x]. Neo-adjuvant treatment status Neo-adjuvant treatment status is [x]. Other cancer diagnosis Other cancer diagnosis is [x]. Histological grade Histological grade is [x]. Tumour stage Tumour stage is [x]. Tumour substage Tumour substage is [x].

Extended Data Table 30: Clinical text segment templates for the ovarian cancer tumor type classification (PTRC-HGSOC, 2 classes). The placeholder [x] represents the specific value of each clinical variable.

68

Clinical variable

Text segment template

Age at initial diagnosis interval Patient age at initial diagnosis is [x] years. Sex Patient sex is [x]. Breslow tumor thickness category Breslow tumor thickness category is [x]. Epidermal ulceration Epidermal ulceration is [x]. Histological melanoma type Histological melanoma type is [x]. Primary melanoma body site Primary melanoma body site is [x].

Extended Data Table 31: Clinical text segment templates for the melanoma relapse prediction without prior melanoma (Visiomel, 2 classes) and overall relapse prediction (Visiomel, 2 classes). The placeholder [x] represents the specific value of each clinical variable.

Dataset

UNI-2

Virchow-2

H-Opt-1

GPFM

AGGC (5 classes) 0.6565±0.0012 0.6573±0.0008 0.6549±0.0016 0.6615±0.0008 BRACS (3 classes) 0.7290±0.0504 0.7384±0.0067 0.7084±0.0087 0.6611±0.0202 BRACS (7 classes) 0.5625±0.0055 0.5553±0.0044 0.4285±0.0058 0.4666±0.0078 BreakHis (2 classes) 0.8367±0.0352 0.7854±0.0550 0.7412±0.0701 0.5353±0.0488 CCRCC (6 classes) 0.8407±0.0014 0.8857±0.0009 0.8750±0.0009 0.8616±0.0011 Chaoyang (4 classes) 0.6498±0.0301 0.7047±0.0039 0.6818±0.0234 0.5292±0.0967 CRC-100K (9 classes) 0.9641±0.0005 0.9650±0.0005 0.9531±0.0004 0.9495±0.0001 CRC-MSI (2 classes) 0.7751±0.0083 0.7529±0.0030 0.7636±0.0006 0.7293±0.0005 ESCA-Tolkach (11 classes) 0.8467±0.0011 0.8632±0.0014 0.8371±0.0015 0.8124±0.0007 GCHTID (8 classes) 0.6128±0.0017 0.6342±0.0003 0.6158±0.0020 0.6329±0.0022 OTA (3 classes) 0.6884±0.0562 0.7503±0.0274 0.4964±0.1523 0.3519±0.0243 Unitopatho (6 classes) 0.2243±0.0715 0.2142±0.0117 0.1919±0.0178 0.1821±0.0147 Overall

0.6989

0.7089

0.6623

0.6144

ALICE 0.6651±0.0015 0.7544±0.0064 0.6007±0.0035 0.8850±0.0026 0.9065±0.0009 0.6856±0.0538 0.9611±0.0008 0.7825±0.0007 0.8648±0.0012 0.6586±0.0008 0.7782±0.0617 0.3632±0.1806 0.7422

Extended Data Table 32: ROI-level classification performance across twelve benchmarks under linear probing. Balanced accuracy was evaluated on the held-out test set for each of five train–validation splits and is reported as the mean ± standard deviation. Bold indicates the bestperforming model, and underline indicates the second-best-performing model for each dataset.

69

Dataset

UNI-2

Virchow-2

H-Opt-1

GPFM

AGGC (5 classes) 0.6142±0.0038 0.6056±0.0075 0.6079±0.0032 0.6047±0.0069 BRACS (3 classes) 0.7633±0.0098 0.7493±0.0116 0.7423±0.0013 0.7533±0.0114 BRACS (7 classes) 0.5888±0.0165 0.5888±0.0061 0.5453±0.0046 0.6025±0.0088 BreakHis (2 classes) 0.9078±0.0003 0.8896±0.0060 0.8955±0.0024 0.8787±0.0014 CCRCC (6 classes) 0.8333±0.0018 0.8708±0.0030 0.8470±0.0028 0.8189±0.0036 Chaoyang (4 classes) 0.7667±0.0084 0.7873±0.0044 0.7874±0.0093 0.7642±0.0148 CRC-100K (9 classes) 0.9552±0.0008 0.9550±0.0004 0.9339±0.0003 0.9481±0.0012 CRC-MSI (2 classes) 0.7139±0.0040 0.6963±0.0043 0.7189±0.0043 0.6749±0.0051 ESCA-Tolkach (11 classes) 0.8463±0.0007 0.8647±0.0015 0.8268±0.0017 0.8118±0.0016 GCHTID (8 classes) 0.6085±0.0126 0.6278±0.0108 0.6148±0.0129 0.6340±0.0111 OTA (3 classes) 0.9470±0.0036 0.9378±0.0094 0.9531±0.0082 0.9422±0.0077 Unitopatho (6 classes) 0.7981±0.0329 0.8052±0.0398 0.7493±0.0486 0.7942±0.0453 Overall

0.7786

0.7815

0.7685

0.7690

ALICE 0.6159±0.0028 0.7633±0.0104 0.6068±0.0087 0.9136±0.0029 0.8771±0.0005 0.7924±0.0090 0.9528±0.0013 0.7144±0.0045 0.8611±0.0007 0.6333±0.0112 0.9466±0.0023 0.8381±0.0450 0.7930

Extended Data Table 33: ROI-level classification performance across twelve benchmarks under KNN evaluation. Balanced accuracy is reported as the mean ± standard deviation across six KNN settings, comprising cosine and Euclidean distance metrics with k = 5, 10 and 20. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each dataset.

Dataset

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

AGGC (5 classes) BRACS (3 classes) BRACS (7 classes) BreakHis (2 classes) CCRCC (6 classes) Chaoyang (4 classes) CRC-100K (9 classes) CRC-MSI (2 classes) ESCA-Tolkach (11 classes) GCHTID (8 classes) OTA (3 classes) Unitopatho (6 classes)

0.8052 0.7579 0.5807 0.9099 0.9081 0.8382 0.9652 0.7616 0.9402 0.5844 0.9543 0.8675

0.8017 0.7439 0.5667 0.9060 0.9177 0.8471 0.9596 0.7621 0.9513 0.6081 0.9512 0.8744

0.8068 0.7561 0.5404 0.8932 0.9116 0.8537 0.9564 0.7423 0.9500 0.5890 0.9421 0.8314

0.7945 0.7561 0.5930 0.8873 0.8979 0.8420 0.9628 0.7307 0.9320 0.6108 0.9421 0.8656

0.8122 0.7596 0.6018 0.9092 0.9227 0.8560 0.9631 0.7717 0.9513 0.6073 0.9543 0.8990

Overall

0.8228

0.8242

0.8144

0.8179

0.8340

Extended Data Table 34: ROI-level retrieval performance across twelve benchmarks. Performance is reported as MVAcc@5. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each dataset.

70

Metric

CoNSeP CRAG GlaS

AP@50 AP@50 AP@50

0.2959±0.0060 0.2470±0.0079 0.2130±0.0020 0.2604±0.0076 0.2580±0.0042 0.8447±0.0113 0.8388±0.0152 0.8254±0.0172 0.8349±0.0124 0.8533±0.0108 0.6803±0.0077 0.6617±0.0197 0.6640±0.0121 0.6728±0.0089 0.6857±0.0116

CoCaHis COSAS24 EBHI Janowczyk

DICE DICE DICE DICE

0.9464±0.0016 0.9438±0.0022 0.9399±0.0017 0.9444±0.0012 0.9463±0.0013 0.8988±0.0019 0.8899±0.0047 0.8900±0.0032 0.8959±0.0010 0.9004±0.0015 0.8348±0.0125 0.8367±0.0146 0.8275±0.0284 0.8313±0.0244 0.8316±0.0147 0.6782±0.0156 0.6704±0.0092 0.6742±0.0199 0.6824±0.0054 0.6886±0.0060

Overall Overall

UNI-2

Instance (AP@50) Semantic (DICE)

Virchow-2

0.6070 0.8395

0.5825 0.8352

H-Opt-1

0.5675 0.8329

GPFM

ALICE (Ours)

Dataset

0.5894 0.8385

0.5990 0.8417

Extended Data Table 35: ROI-level segmentation performance across seven benchmarks using Plain Mask Transformer (PMT). Instance segmentation is evaluated using AP@50, and semantic segmentation is evaluated using DICE. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the secondbest-performing model for each task.

Shot

UNI-2

Virchow-2

1-shot 0.4425±0.0949 0.3825±0.0798 2-shot 0.5090±0.0811 0.4232±0.0746 4-shot 0.5907±0.0673 0.4924±0.0674 8-shot 0.6396±0.0606 0.5378±0.0636 16-shot 0.6754±0.0493 0.5681±0.0573 32-shot 0.6901±0.0431 0.5852±0.0497 64-shot 0.7004±0.0410 0.5931±0.0420

H-Opt-1 0.4573±0.0930 0.5276±0.0792 0.5950±0.0602 0.6323±0.0528 0.6556±0.0451 0.6627±0.0442 0.6662±0.0420

GPFM

ALICE

0.4518±0.0894 0.4719±0.0967 0.5174±0.0786 0.5438±0.0803 0.5882±0.0620 0.6196±0.0637 0.6272±0.0556 0.6566±0.0560 0.6549±0.0504 0.6846±0.0458 0.6683±0.0468 0.6929±0.0444 0.6784±0.0431 0.7021±0.0395

Extended Data Table 36: AGGC (5 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

71

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.5463±0.1066 0.6176±0.0922 0.6775±0.0714 0.7148±0.0619 0.7334±0.0596 0.7409±0.0545 0.7401±0.0548 0.7471±0.0518 0.7425±0.0548

0.4890±0.1009 0.5508±0.0921 0.6080±0.0820 0.6578±0.0756 0.6929±0.0616 0.7149±0.0545 0.7210±0.0555 0.7292±0.0526 0.7279±0.0532

0.4672±0.0984 0.5076±0.0861 0.5541±0.0749 0.5861±0.0677 0.6068±0.0643 0.6102±0.0616 0.6089±0.0577 0.6167±0.0538 0.6077±0.0584

0.4770±0.0941 0.5195±0.0864 0.5629±0.0740 0.5889±0.0692 0.6084±0.0630 0.6084±0.0596 0.6064±0.0549 0.6132±0.0598 0.6023±0.0599

0.5409±0.1103 0.6173±0.0907 0.6751±0.0700 0.7156±0.0630 0.7366±0.0545 0.7457±0.0523 0.7463±0.0528 0.7488±0.0540 0.7411±0.0542

Extended Data Table 37: BRACS (3 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.3900±0.0625 0.4468±0.0529 0.4961±0.0425 0.5369±0.0403 0.5608±0.0360 0.5794±0.0330 0.5881±0.0309 0.5954±0.0317 0.5991±0.0311

0.3575±0.0616 0.4061±0.0551 0.4578±0.0460 0.4966±0.0421 0.5219±0.0398 0.5394±0.0367 0.5475±0.0331 0.5517±0.0304 0.5545±0.0314

0.3012±0.0609 0.3467±0.0574 0.3969±0.0468 0.4375±0.0393 0.4586±0.0388 0.4760±0.0405 0.4787±0.0356 0.4805±0.0348 0.4857±0.0341

0.3239±0.0593 0.3700±0.0583 0.4223±0.0486 0.4618±0.0436 0.4836±0.0419 0.4992±0.0388 0.5025±0.0363 0.5076±0.0348 0.5083±0.0331

0.3845±0.0651 0.4489±0.0562 0.5056±0.0465 0.5458±0.0378 0.5688±0.0351 0.5859±0.0354 0.5914±0.0316 0.5950±0.0309 0.5932±0.0318

Extended Data Table 38: BRACS (7 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

72

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.6719±0.1243 0.7356±0.1076 0.7833±0.0930 0.8266±0.0747 0.8448±0.0647 0.8618±0.0599 0.8692±0.0546 0.8699±0.0539 0.8696±0.0555

0.5893±0.1346 0.6255±0.1396 0.6737±0.1303 0.7241±0.1155 0.7645±0.1043 0.8019±0.0808 0.8232±0.0725 0.8370±0.0590 0.8423±0.0585

0.5875±0.1074 0.6336±0.1047 0.6759±0.0934 0.7309±0.0943 0.7780±0.0808 0.8017±0.0705 0.8283±0.0664 0.8429±0.0570 0.8530±0.0584

0.5778±0.0986 0.6206±0.0983 0.6602±0.0965 0.7016±0.0912 0.7406±0.0784 0.7588±0.0703 0.7783±0.0644 0.7768±0.0624 0.7827±0.0633

0.6705±0.1244 0.7347±0.1033 0.7860±0.0880 0.8345±0.0702 0.8544±0.0606 0.8675±0.0546 0.8749±0.0534 0.8778±0.0512 0.8765±0.0531

Extended Data Table 39: BreakHis (2 classes ROI-level few-shot classification) performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

0.5069±0.0896 0.5935±0.0715 0.6654±0.0577 0.6995±0.0519 0.7134±0.0480 0.7158±0.0430 0.7195±0.0415 0.7135±0.0378 0.7153±0.0325

0.5321±0.0866 0.6079±0.0656 0.6750±0.0538 0.7155±0.0478 0.7459±0.0469 0.7666±0.0436 0.7782±0.0386 0.7850±0.0353 0.7931±0.0321

0.6183±0.0858 0.6893±0.0641 0.7416±0.0491 0.7623±0.0457 0.7744±0.0440 0.7780±0.0427 0.7815±0.0378 0.7816±0.0371 0.7845±0.0331

GPFM

ALICE

0.5301±0.0847 0.6324±0.0862 0.6128±0.0752 0.7078±0.0667 0.6875±0.0616 0.7674±0.0506 0.7341±0.0552 0.7951±0.0452 0.7576±0.0526 0.8132±0.0423 0.7626±0.0478 0.8189±0.0380 0.7670±0.0425 0.8241±0.0326 0.7622±0.0367 0.8248±0.0326 0.7641±0.0340 0.8289±0.0315

Extended Data Table 40: CCRCC (6 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

73

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

0.5244±0.0953 0.5763±0.0791 0.6098±0.0693 0.6472±0.0585 0.6726±0.0547 0.6875±0.0480 0.6958±0.0480 0.7030±0.0457 0.7118±0.0441

0.5019±0.0896 0.5570±0.0750 0.5992±0.0649 0.6324±0.0589 0.6661±0.0546 0.6830±0.0494 0.6961±0.0474 0.7054±0.0484 0.7114±0.0463

0.5233±0.0937 0.5865±0.0768 0.6245±0.0642 0.6654±0.0562 0.6928±0.0508 0.7053±0.0452 0.7128±0.0471 0.7159±0.0471 0.7189±0.0454

GPFM

ALICE

0.4828±0.0933 0.5326±0.0944 0.5423±0.0829 0.5827±0.0771 0.5909±0.0734 0.6241±0.0655 0.6280±0.0603 0.6594±0.0601 0.6566±0.0540 0.6851±0.0537 0.6754±0.0488 0.6995±0.0489 0.6780±0.0463 0.7106±0.0462 0.6885±0.0483 0.7141±0.0468 0.6911±0.0465 0.7196±0.0446

Extended Data Table 41: Chaoyang (4 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.8032±0.0643 0.8710±0.0457 0.9142±0.0289 0.9348±0.0209 0.9458±0.0171 0.9522±0.0157 0.9543±0.0149 0.9551±0.0133 0.9560±0.0137

0.7571±0.0820 0.8323±0.0548 0.8846±0.0394 0.9087±0.0245 0.9176±0.0218 0.9232±0.0189 0.9254±0.0181 0.9256±0.0174 0.9264±0.0180

0.7672±0.0567 0.8281±0.0415 0.8696±0.0345 0.8911±0.0277 0.9032±0.0226 0.9118±0.0202 0.9145±0.0204 0.9163±0.0175 0.9179±0.0177

0.8000±0.0576 0.8691±0.0386 0.9089±0.0261 0.9268±0.0211 0.9339±0.0182 0.9376±0.0173 0.9392±0.0171 0.9409±0.0152 0.9410±0.0157

0.8682±0.0504 0.9167±0.0310 0.9397±0.0206 0.9502±0.0167 0.9546±0.0143 0.9579±0.0146 0.9581±0.0139 0.9588±0.0127 0.9591±0.0137

Extended Data Table 42: CRC-100K (9 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

74

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.5466±0.0902 0.5746±0.0838 0.6017±0.0893 0.6467±0.0833 0.6735±0.0807 0.6884±0.0788 0.7043±0.0736 0.7060±0.0726 0.7055±0.0706

0.5188±0.0725 0.5231±0.0818 0.5445±0.0798 0.5678±0.0865 0.5906±0.0835 0.6161±0.0871 0.6383±0.0816 0.6488±0.0797 0.6534±0.0747

0.5399±0.0859 0.5698±0.0841 0.5940±0.0882 0.6315±0.0882 0.6545±0.0824 0.6773±0.0797 0.6882±0.0710 0.6879±0.0741 0.6881±0.0732

0.5326±0.0840 0.5466±0.0827 0.5665±0.0877 0.6010±0.0852 0.6303±0.0824 0.6435±0.0803 0.6697±0.0787 0.6680±0.0774 0.6728±0.0714

0.5490±0.0866 0.5764±0.0850 0.6071±0.0895 0.6483±0.0826 0.6773±0.0797 0.7013±0.0780 0.7152±0.0709 0.7176±0.0727 0.7203±0.0682

Extended Data Table 43: CRC-MSI (2 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot. Shot

UNI-2

1-shot 0.6939±0.0494 2-shot 0.7625±0.0423 4-shot 0.8103±0.0317 8-shot 0.8432±0.0260 16-shot 0.8633±0.0227 32-shot 0.8729±0.0211

Virchow-2

H-Opt-1

GPFM

0.5947±0.0568 0.6998±0.0528 0.6582±0.0513 0.6797±0.0497 0.7685±0.0400 0.7302±0.0427 0.7495±0.0373 0.8167±0.0300 0.7825±0.0343 0.7918±0.0311 0.8470±0.0236 0.8148±0.0286 0.8164±0.0264 0.8628±0.0218 0.8331±0.0261 0.8229±0.0250 0.8730±0.0205 0.8428±0.0257

ALICE 0.7394±0.0468 0.7971±0.0385 0.8363±0.0288 0.8586±0.0240 0.8711±0.0216 0.8772±0.0207

Extended Data Table 44: ESCA (11 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot. Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

GPFM

0.2763±0.0511 0.2536±0.0514 0.2881±0.0541 0.2947±0.0546 0.3364±0.0478 0.3029±0.0517 0.3469±0.0491 0.3541±0.0487 0.4028±0.0490 0.3581±0.0507 0.4084±0.0466 0.4200±0.0473 0.4647±0.0467 0.4085±0.0448 0.4692±0.0419 0.4839±0.0434 0.5153±0.0413 0.4526±0.0414 0.5182±0.0395 0.5356±0.0398 0.5411±0.0371 0.4774±0.0408 0.5480±0.0367 0.5665±0.0370 0.5537±0.0380 0.4960±0.0401 0.5600±0.0378 0.5819±0.0383 0.5602±0.0381 0.5060±0.0394 0.5694±0.0376 0.5903±0.0371 0.5657±0.0384 0.5181±0.0393 0.5749±0.0401 0.5948±0.0392

ALICE 0.3124±0.0585 0.3805±0.0526 0.4422±0.0506 0.5030±0.0441 0.5485±0.0405 0.5699±0.0363 0.5785±0.0390 0.5825±0.0381 0.5878±0.0387

Extended Data Table 45: GCHTID (8 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot. 75

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot

UNI-2

Virchow-2

H-Opt-1

GPFM

ALICE

0.6323±0.1186 0.7345±0.0992 0.8118±0.0704 0.8630±0.0560 0.8825±0.0430 0.8958±0.0411 0.8982±0.0344 0.8965±0.0379

0.5490±0.1023 0.6360±0.1045 0.6964±0.1048 0.7414±0.0994 0.7652±0.0913 0.7726±0.0823 0.7666±0.0702 0.7621±0.0537

0.6212±0.1172 0.7192±0.0963 0.7977±0.0693 0.8399±0.0587 0.8609±0.0475 0.8755±0.0454 0.8767±0.0409 0.8785±0.0381

0.6147±0.1103 0.7133±0.0921 0.7980±0.0725 0.8472±0.0592 0.8766±0.0457 0.8911±0.0399 0.8939±0.0376 0.8959±0.0363

0.6466±0.1188 0.7459±0.0989 0.8244±0.0722 0.8725±0.0534 0.8986±0.0412 0.9103±0.0382 0.9150±0.0332 0.9179±0.0328

Extended Data Table 46: OTA (3 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

Shot 1-shot 2-shot 4-shot 8-shot 16-shot 32-shot 64-shot 128-shot 256-shot

UNI-2

Virchow-2

H-Opt-1

0.3449±0.0673 0.4039±0.0702 0.4766±0.0621 0.5377±0.0544 0.5816±0.0478 0.6114±0.0436 0.6212±0.0409 0.6308±0.0392 0.6314±0.0404

0.3684±0.0683 0.4262±0.0627 0.4794±0.0538 0.5135±0.0473 0.5346±0.0480 0.5432±0.0429 0.5489±0.0421 0.5544±0.0417 0.5565±0.0384

0.3097±0.0642 0.3627±0.0638 0.4238±0.0565 0.4755±0.0515 0.5126±0.0452 0.5336±0.0432 0.5475±0.0385 0.5534±0.0429 0.5574±0.0402

GPFM

ALICE

0.2991±0.0584 0.3887±0.0722 0.3446±0.0614 0.4561±0.0690 0.4016±0.0554 0.5286±0.0564 0.4482±0.0508 0.5793±0.0485 0.4901±0.0456 0.6117±0.0418 0.5156±0.0443 0.6308±0.0414 0.5275±0.0420 0.6378±0.0391 0.5342±0.0416 0.6389±0.0380 0.5350±0.0423 0.6393±0.0384

Extended Data Table 47: Unitopatho (6 classes) ROI-level few-shot classification performance under all-way ProtoNet evaluation. Results are reported as the mean ± standard deviation across 500 episodes. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

76

Dataset Prostate cancer screening (PANDA, 2 classes) Lymph node metastasis screening (CAMELYON+, 2 classes) Lymph node metastatic burden classification (CAMELYON+, 4 classes) Colorectal neoplasia screening (HunCRC, 4 classes) Bone metastasis primary site prediction (SYSBM, 10 classes) Lung lesion triage (SYSFL+, 4 classes) Cervical epithelial lesion classification (TissueNet, 4 classes) Non-small cell lung cancer subtyping (CPTAC-NSCLC, 2 classes) Ovarian carcinoma subtyping (UBC-OCEAN, 5 classes) Renal cell carcinoma subtyping (DHMC-RCC, 5 classes) Breast carcinoma coarse-grained subtyping (BRACS, 3 classes) Breast carcinoma fine-grained subtyping (BRACS, 7 classes) Prostate cancer Gleason grading (PANDA, 6 classes) Lung cancer subtyping and invasion grading (SYSFL+, 7 classes) CNS fine-grained tumor diagnosis (EBRAINS, 30 classes) CNS tumor-family classification (EBRAINS, 9 classes) CNS tumor WHO grading (EBRAINS, 4 classes) Pediatric rare tumor classification (KidRare, 4 classes) Pediatric rare tumor fine-grained subtyping (KidRare, 13 classes) Overall

UNI-2 0.9756±0.0037

Virchow-2

H-Opt-1

0.9820±0.0017 0.9786±0.0028

GPFM

ALICE

0.9825±0.0018 0.9835±0.0016

0.9385±0.0093 0.9458±0.0044 0.9438±0.0077

0.9446±0.0063

0.9488±0.0066

0.6332±0.0062

0.6448±0.0119 0.6136±0.0263

0.6351±0.0399

0.7046±0.0101

0.3701±0.0422

0.4909±0.0198

0.4686±0.0233 0.4269±0.0280

0.4622±0.0732

0.7007±0.0352

0.7624±0.0169 0.7409±0.0320

0.9464±0.0127

0.9397±0.0194

0.7518±0.0097

0.7542±0.0139 0.7529±0.0324

0.8979±0.0197

0.8361±0.0349

0.7403±0.0733

0.7586±0.0092 0.7141±0.0271

0.6508±0.0380

0.8014±0.0335

0.9632±0.0203 0.9308±0.0126

0.9715±0.0202

0.7035±0.0217

0.7738±0.0230

0.8917±0.0204 0.8311±0.0180

0.8856±0.0304

0.6806±0.0359

0.7822±0.0039

0.7043±0.0589

0.6392±0.0310

0.7269±0.0304

0.4166±0.0236

0.4339±0.0247 0.4091±0.0560

0.3989±0.0316

0.4900±0.0300

0.7048±0.0034

0.7255±0.0109 0.7106±0.0048

0.6985±0.0094

0.7427±0.0100

0.9278±0.0122

0.9283±0.0091 0.9105±0.0179

0.8520±0.0270

0.9448±0.0117

0.4573±0.0161

0.4586±0.0203 0.4578±0.0204

0.4329±0.0268

0.4693±0.0102

0.9056±0.0145

0.9116±0.0096

0.9116±0.0095 0.8172±0.0113

0.9413±0.0086

0.6942±0.0088

0.6917±0.0074 0.7018±0.0080 0.6250±0.0079

0.7164±0.0072

0.7124±0.0054

0.7492±0.0195 0.7498±0.0090 0.6906±0.0151

0.7506±0.0097

0.5019±0.0256

0.5263±0.0657 0.5121±0.0156

0.7082±0.0262 0.6926±0.0888

0.6726±0.0202 0.6806±0.0047 0.6677±0.0157 0.7187

0.7322

0.7264

0.5281±0.0560 0.6144±0.0406 0.6445±0.0196

0.7075±0.0068

0.6901

0.7588

Extended Data Table 48: WSI-level classification performance across 19 benchmarks using vision-only PFM features with ABMIL. Balanced accuracy is reported as the mean ± standard deviation. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each dataset.

77

Dataset

KEEP

CONCH

ALICE

0.4298±0.0771 0.5204±0.0521 0.4158±0.0582 0.7824±0.0471 0.5753±0.0528 0.5644±0.0311 0.8690±0.0869 0.5081±0.0234 0.6438±0.0598 0.3714±0.0320 0.6943±0.1369 0.4215±0.0399

0.3514±0.0597 0.4729±0.0676 0.4288±0.0739 0.7790±0.0690 0.5383±0.0772 0.5287±0.0771 0.7663±0.0719 0.4995±0.0290 0.6494±0.0687 0.4504±0.0397 0.7239±0.1498 0.2947±0.0585

0.5657±0.0635 0.6014±0.0649 0.5211±0.0471 0.7773±0.0657 0.6866±0.0838 0.5685±0.0568 0.9103±0.0445 0.5207±0.0273 0.6274±0.0497 0.4538±0.0214 0.7625±0.1448 0.4850±0.0544

0.5664

0.5403

0.6234

MUSK

AGGC (5 classes) 0.4682±0.0564 BRACS (3 classes) 0.7054±0.0454 BRACS (7 classes) 0.4795±0.0296 BreakHis (2 classes) 0.6595±0.0626 CCRCC (6 classes) 0.5993±0.0458 Chaoyang (4 classes) 0.4823±0.0426 CRC-100K (9 classes) 0.7403±0.0913 CRC-MSI (2 classes) 0.5582±0.0185 ESCA-Tolkach (11 classes) 0.5620±0.0294 GCHTID (8 classes) 0.4373±0.0325 OTA (3 classes) 0.5445±0.1118 Unitopatho (6 classes) 0.4494±0.0273 Overall

0.5572

Extended Data Table 49: ROI-level zero-shot classification performance across twelve benchmarks. Balanced accuracy is reported with the per-prompt standard deviation. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each dataset. Source

Recall@K

MUSK t2i i2t

KEEP t2i

i2t

CONCH t2i i2t

ALICE t2i i2t

BookSet

K=1 K=5 K=10 K=20 K=25 K=50

0.1805 0.3970 0.5112 0.6318 0.6717 0.7778

0.1784 0.4024 0.5244 0.6411 0.6720 0.7754

0.0921 0.2771 0.4036 0.5271 0.5712 0.7043

0.0759 0.2135 0.3031 0.4039 0.4462 0.5718

0.1583 0.3550 0.4522 0.5610 0.5985 0.7106

0.1598 0.3298 0.4420 0.5475 0.5832 0.7019

0.1448 0.3793 0.5031 0.6366 0.6765 0.7949

0.0966 0.2870 0.4030 0.5460 0.5877 0.7172

ChineseBook

K=1 K=5 K=10 K=20 K=25 K=50

0.1044 0.2937 0.3946 0.5129 0.5539 0.6681

0.1106 0.2923 0.4015 0.5101 0.5623 0.6994

0.0703 0.2248 0.3194 0.4461 0.4906 0.6214

0.0710 0.2025 0.2867 0.3981 0.4412 0.5685

0.1294 0.2999 0.3987 0.5150 0.5518 0.6541

0.1141 0.2916 0.3820 0.4913 0.5421 0.6500

0.1545 0.3646 0.4809 0.6013 0.6514 0.7697

0.1301 0.3111 0.4245 0.5421 0.5915 0.7321

EnglishBook

K=1 K=5 K=10 K=20 K=25 K=50

0.0602 0.2269 0.3491 0.4692 0.5059 0.6336

0.0532 0.2126 0.3308 0.4299 0.4692 0.5918

0.0896 0.2368 0.3385 0.4537 0.4886 0.5969

0.0694 0.1894 0.2658 0.3594 0.3902 0.4883

0.0848 0.0642 0.2383 0.1898 0.3286 0.2680 0.4317 0.3638 0.4611 0.4012 0.5789 0.5015

0.1322 0.0977 0.3473 0.2702 0.4490 0.3656 0.5584 0.4692 0.5988 0.5037 0.7144 0.6138

Extended Data Table 50: ROI cross-modal retrieval performance across three benchmarks. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each dataset, K value and retrieval direction. 78

Source

Test

Test-tiny

Val

MUSK

KEEP

CONCH

ALICE MUSK

KEEP

CONCH

ALICE MUSK

KEEP

CONCH

ALICE

Atlas EduContent CRC100K LC25000 Osteo SICAPv2 SkinCancer WSSSLUAD PubMed SocialPath

0.5919 0.5387 0.6961 0.9588 0.6373 0.5956 0.7516 0.5686 0.4963 0.4386

0.5221 0.4471 0.7778 0.8941 0.5588 0.3235 0.6765 0.7647 0.4270 0.3956

0.5732 0.4287 0.6209 0.9588 0.5980 0.5588 0.6373 0.5294 0.4554 0.3990

0.5987 0.4533 0.7582 0.9941 0.6863 0.5735 0.8464 0.6863 0.4572 0.4161

0.6720 0.5900 0.6471 1.0000 0.3333 0.7500 0.6857 0.5833 0.4891 0.4095

0.5820 0.5272 0.7353 1.0000 0.5833 0.2500 0.6571 0.7500 0.4307 0.4238

0.6720 0.5439 0.7647 1.0000 0.5000 0.5625 0.5714 0.5000 0.5292 0.4333

0.6614 0.5356 0.8235 1.0000 0.8333 0.5625 0.8857 0.5000 0.4672 0.4571

0.5584 0.5455 0.7222 0.9000 0.6667 0.7500 0.8889 0.5000 0.5089 0.4519

0.5584 0.4196 0.7222 1.0000 0.6667 0.2500 0.7222 0.8333 0.4420 0.3778

0.6104 0.3706 0.6667 0.9000 0.8333 0.6250 0.6111 0.5000 0.4509 0.4370

0.6364 0.3916 0.8333 0.9000 0.8333 0.6250 0.8333 0.5000 0.3839 0.4370

Overall

0.6274

0.5787

0.5760

0.6470 0.6160

0.5939

0.6077

0.6726 0.6492

0.5992

0.6005

0.6374

Extended Data Table 51: PathMMU VQA accuracy across ten data sources. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each source and cohort.

79

Task

TopK pooling MUSK

KEEP

CONCH ALICE

Lymph node metastasis screening (CAMELYON+, 2 classes)

K=1 K=5 K=10 K=50

0.6549 0.6415 0.6304 0.5804

0.6471 0.6627 0.6497 0.6438

0.5444 0.5278 0.5222 0.5222

0.8180 0.7827 0.7634 0.6971

CNS tumor-family classification (EBRAINS, 9 classes)

K=1 K=5 K=10 K=50

0.4415 0.4464 0.4549 0.4920

0.4388 0.5074 0.5531 0.5957

0.3272 0.3476 0.3597 0.3889

0.5471 0.5744 0.5809 0.5979

Pediatric rare tumor classification (KidRare, 4 classes)

K=1 K=5 K=10 K=50

0.6461 0.6555 0.6671 0.6849

0.6628 0.7121 0.7147 0.7303

0.5591 0.5704 0.5576 0.6019

0.7094 0.7240 0.7147 0.7149

Non-small cell lung cancer subtyping (CPTAC-NSCLC, 2 classes)

K=1 K=5 K=10 K=50

0.8502 0.8701 0.8784 0.8772

0.9111 0.9413 0.9349 0.9401

0.8163 0.8328 0.8329 0.8379

0.9169 0.9220 0.9205 0.9208

K=1 Bone metastasis primary site prediction K=5 (SYSBM, 10 classes) K=10 K=50

0.2212 0.2296 0.2389 0.2622

0.3966 0.4104 0.4159 0.4155

0.4167 0.4279 0.4251 0.4366

0.4638 0.4881 0.4866 0.4937

K=1 K=5 K=10 K=50

0.5862 0.6411 0.6411 0.6882

0.6340 0.6533 0.6622 0.6967

0.5284 0.5689 0.5813 0.6178

0.6780 0.6896 0.6976 0.7131

Ovarian carcinoma subtyping (UBC-OCEAN, 5 classes)

Extended Data Table 52: Zero-shot WSI-level classification performance across six benchmarks using MI-Zero TopK pooling. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each task and TopK setting.

Shot

MUSK

KEEP

CONCH

ALICE

1-shot 0.4341±0.0181 0.4940±0.0239 0.4577±0.0209 0.5237±0.0221 5-shot 0.5266±0.0128 0.6024±0.0163 0.5741±0.0109 0.6079±0.0135 10-shot 0.5658±0.0193 0.6296±0.0089 0.6135±0.0158 0.6525±0.0147 15-shot 0.5917±0.0171 0.6693±0.0183 0.6512±0.0122 0.6940±0.0149 20-shot 0.6153±0.0201 0.6769±0.0145 0.6507±0.0099 0.7081±0.0200

Extended Data Table 53: EBRAINS (30 classes) WSI-level few-shot classification performance using PathPT. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each shot.

80

Shot

MUSK

KEEP

CONCH

ALICE

1-shot 5-shot 10-shot 15-shot 20-shot

0.5661±0.0776 0.7166±0.0320 0.7596±0.0318 0.7862±0.0305 0.8118±0.0137

0.5077±0.0729 0.7966±0.0293 0.8258±0.0370 0.8671±0.0099 0.8690±0.0304

0.5841±0.0637 0.7231±0.0174 0.7877±0.0480 0.8177±0.0148 0.8277±0.0317

0.6712±0.0210 0.8012±0.0290 0.8699±0.0198 0.8832±0.0163 0.8905±0.0059

Extended Data Table 54: KidRare (4 classes) WSI-level few-shot classification performance using PathPT. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each shot.

Shot

MUSK

KEEP

CONCH

ALICE

1-shot 0.2661±0.0197 0.3283±0.0391 0.2954±0.0209 0.3347±0.0258 5-shot 0.3131±0.0351 0.3567±0.0208 0.3593±0.0314 0.3731±0.0232 10-shot 0.3855±0.0123 0.3980±0.0163 0.3987±0.0216 0.4057±0.0160 15-shot 0.3904±0.0304 0.4185±0.0194 0.4175±0.0388 0.4229±0.0306 20-shot 0.3984±0.0290 0.4265±0.0205 0.4053±0.0365 0.4292±0.0380

Extended Data Table 55: SYSFL+ (7 classes) WSI-level few-shot classification performance using PathPT. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each shot.

Shot

MUSK

KEEP

CONCH

ALICE

1-shot 5-shot 10-shot 15-shot 20-shot

0.5568±0.0536 0.7089±0.0403 0.7383±0.0402 0.7781±0.0506 0.8413±0.0230

0.4902±0.0599 0.7974±0.0216 0.7928±0.0384 0.8199±0.0272 0.8425±0.0373

0.5143±0.0900 0.7263±0.0705 0.8054±0.0290 0.8285±0.0200 0.8363±0.0189

0.5914±0.0720 0.8156±0.0342 0.8339±0.0377 0.8273±0.0177 0.8524±0.0176

Extended Data Table 56: UBC-OCEAN (5 classes) WSI-level few-shot classification performance using PathPT. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

81

Shot

MUSK

KEEP

CONCH

ALICE

1-shot 5-shot 10-shot 15-shot 20-shot

0.5133±0.0179 0.5995±0.0864 0.7376±0.1038 0.8465±0.0637 0.8171±0.0681

0.5941±0.0492 0.6476±0.0430 0.6948±0.0394 0.7941±0.0695 0.8433±0.0492

0.5051±0.0212 0.6201±0.1020 0.7765±0.0935 0.8620±0.0518 0.8875±0.0428

0.5089±0.0182 0.6667±0.0993 0.8274±0.0484 0.8770±0.0531 0.9101±0.0300

Extended Data Table 57: Camelyon+ (2 classes) WSI-level few-shot classification performance using PathPT. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each shot.

Shot

MUSK

KEEP

CONCH

ALICE

1-shot 0.8348±0.0286 0.7679±0.1063 0.9161±0.0201 0.9056±0.0139 5-shot 0.8727±0.0354 0.8994±0.0357 0.9090±0.0145 0.9181±0.0063 10-shot 0.9076±0.0203 0.9435±0.0088 0.9136±0.0284 0.9395±0.0131 15-shot 0.9139±0.0212 0.9455±0.0242 0.9395±0.0103 0.9476±0.0079 20-shot 0.9213±0.0316 0.9557±0.0200 0.9541±0.0202 0.9557±0.0119

Extended Data Table 58: CPTAC-NSCLC (2 classes) WSI-level few-shot classification performance using PathPT. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each shot.

82

MUSK

KEEP

CONCH

ALICE

Task

Modality

Breast cancer axillary lymph node metastasis (BCNB, 2 classes)

WSI-only 0.7040±0.0134 0.6354±0.0191 0.6610±0.0329 0.6658±0.0303 Text-only 0.5490±0.0286 0.6071±0.0409 0.5839±0.0475 0.5667±0.0353 Multimodal 0.6003±0.0856 0.6624±0.0116 0.6636±0.0330 0.6693±0.0293

Breast cancer axillary lymph node metastasis (BCNB, 3 classes)

WSI-only Text-only Multimodal

Head and neck keratinizing SCC grading (Hancock, 2 classes)

WSI-only 0.6347±0.0163 0.6607±0.0404 0.6583±0.0368 0.6385±0.0142 Text-only 0.5200±0.0240 0.5303±0.0258 0.5509±0.0111 0.5567±0.0273 Multimodal 0.6105±0.0680 0.6446±0.0129 0.6483±0.0211 0.6978±0.0326

Head and neck lymphovascular invasion detection (Hancock, 2 classes)

WSI-only Text-only Multimodal

Head and neck vascular invasion detection (Hancock, 2 classes)

WSI-only 0.6131±0.0288 0.5819±0.1122 0.6512±0.0905 0.6577±0.0923 Text-only 0.5661±0.0474 0.5466±0.0266 0.6327±0.0534 0.6405±0.0153 Multimodal 0.5381±0.0700 0.7171±0.1264 0.7141±0.0586 0.9032±0.0243

Ovarian cancer treatment response prediction (PTRC-HGSOC, 2 classes)

WSI-only Text-only Multimodal

0.5108±0.0453 0.5775±0.0240 0.5308±0.0295 0.6058±0.0278 0.5750±0.0736 0.6566±0.0037 0.6767±0.0354 0.6583±0.0000 0.4800±0.0471 0.5817±0.0309 0.5625±0.0466 0.6617±0.0173

Ovarian cancer tumor type classification (PTRC-HGSOC, 2 classes)

WSI-only Text-only Multimodal

0.8056±0.0099 0.8660±0.0219 0.5000±0.0000 0.5111±0.0249 0.8254±0.0334 0.8212±0.0379

0.5010±0.0318 0.4401±0.0245 0.4297±0.0265 0.4534±0.0186 0.3613±0.0302 0.3749±0.0342 0.4283±0.0315 0.3675±0.0290 0.3934±0.0458 0.3920±0.0209 0.4022±0.0289 0.4530±0.0132

0.6185±0.0358 0.6760±0.0381 0.6586±0.0201 0.6410±0.0196 0.6705±0.0045 0.6522±0.0310 0.6681±0.0069 0.6725±0.0171 0.5970±0.0512 0.6441±0.0145 0.6360±0.0482 0.7261±0.0332

0.8163±0.0225 0.8320±0.0119 0.5167±0.0373 0.5167±0.0373 0.8358±0.0388 0.8674±0.0294

WSI-only 0.6856±0.0183 0.6981±0.0354 0.6396±0.0131 0.6743±0.0226 Melanoma relapse prediction without prior melanoma Text-only 0.7327±0.0172 0.7426±0.0000 0.7383±0.0199 0.7150±0.0179 (Visiomel, 2 classes) Multimodal 0.7139±0.0234 0.6908±0.0422 0.6684±0.0418 0.7215±0.0318 Melanoma overall relapse prediction (Visiomel, 2 classes)

WSI-only Text-only Multimodal

0.7005±0.0262 0.7208±0.0187 0.7174±0.0092 0.7322±0.0175 0.7717±0.0100 0.7792±0.0179 0.7641±0.0254 0.7741±0.0076 0.7423±0.0162 0.7468±0.0240 0.7465±0.0217 0.7539±0.0174

Extended Data Table 59: Modality comparison on MICA multimodal WSI-text tasks. Results are reported as the mean ± standard deviation across five random seeds. Bold indicates the bestperforming model, and underline indicates the second-best-performing model for each task and modality.

83

Task Breast carcinoma coarse-grained subtyping (BRACS, 3 classes) Breast carcinoma fine-grained subtyping (BRACS, 7 classes) Lymph node metastasis screening (CAMELYON+, 2 classes) Lymph node metastatic burden classification (CAMELYON+, 4 classes) Non-small cell lung cancer subtyping (CPTAC-NSCLC, 2 classes) Renal cell carcinoma subtyping (DHMC-RCC, 5 classes) CNS fine-grained tumor diagnosis (EBRAINS, 30 classes) CNS tumor-family classification (EBRAINS, 9 classes) CNS tumor WHO grading (EBRAINS, 4 classes) Colorectal neoplasia screening (HunCRC, 4 classes) Pediatric rare tumor classification (KidRare, 4 classes) Pediatric rare tumor fine-grained subtyping (KidRare, 13 classes) Prostate cancer Gleason grading (PANDA, 6 classes) Prostate cancer screening (PANDA, 2 classes) Bone metastasis primary site prediction (SYSBM, 10 classes) Lung lesion triage (SYSFL+, 4 classes) Lung cancer subtyping and invasion grading (SYSFL+, 7 classes) Cervical epithelial lesion classification (TissueNet, 4 classes) Ovarian carcinoma subtyping (UBC-OCEAN, 5 classes)

TITAN

CARE

ALICE

0.6798±0.0343 0.6164±0.0237 0.6082±0.0141 0.3206±0.0228

0.3297±0.0274 0.3455±0.0259

0.7234±0.0229

0.7339±0.0172 0.7595±0.0020

0.4114±0.0146

0.4382±0.0313 0.4745±0.0179

0.9605±0.0080 0.9330±0.0067 0.9774±0.0040 0.7449±0.0070 0.7790±0.0023 0.7288±0.0150 0.6525±0.0133

0.5943±0.0185 0.6489±0.0069

0.8677±0.0152 0.8173±0.0181

0.8609±0.0176

0.7298±0.0078 0.6940±0.0041

0.7155±0.0092

0.6320±0.0198 0.6375±0.0138 0.4830±0.0151 0.9032±0.0043

0.8629±0.0095 0.9030±0.0123

0.5876±0.0354 0.5242±0.0552 0.4657±0.0467 0.5717±0.0214

0.6185±0.0316 0.6318±0.0263

0.8786±0.0093

0.8972±0.0077 0.9340±0.0060

0.6725±0.0416

0.5593±0.0631 0.6332±0.0511

0.6080±0.0183 0.6176±0.0144 0.6080±0.0027 0.4211±0.0123 0.4238±0.0094 0.4200±0.0113 0.6068±0.0293

0.6230±0.0222 0.6772±0.0113

0.8536±0.0232 0.8344±0.0297 0.8612±0.0171

Extended Data Table 60: Slide-level clinical diagnosis KNN evaluation. Balanced accuracy is reported as the mean ± standard deviation across five random seeds. Bold indicates the bestperforming model, and underline indicates the second-best-performing model for each task and modality.

84

Task Breast carcinoma coarse-grained subtyping (BRACS, 3 classes) Breast carcinoma fine-grained subtyping (BRACS, 7 classes) Lymph node metastasis screening (CAMELYON+, 2 classes) Lymph node metastatic burden classification (CAMELYON+, 4 classes) Non-small cell lung cancer subtyping (CPTAC-NSCLC, 2 classes) Renal cell carcinoma subtyping (DHMC-RCC, 5 classes) CNS fine-grained tumor diagnosis (EBRAINS, 30 classes) CNS tumor-family classification (EBRAINS, 9 classes) CNS tumor WHO grading (EBRAINS, 4 classes) Colorectal neoplasia screening (HunCRC, 4 classes) Pediatric rare tumor classification (KidRare, 4 classes) Pediatric rare tumor fine-grained subtyping (KidRare, 13 classes) Prostate cancer Gleason grading (PANDA, 6 classes) Prostate cancer screening (PANDA, 2 classes) Bone metastasis primary site prediction (SYSBM, 10 classes) Lung lesion triage (SYSFL+, 4 classes) Lung cancer subtyping and invasion grading (SYSFL+, 7 classes) Cervical epithelial lesion classification (TissueNet, 4 classes) Ovarian carcinoma subtyping (UBC-OCEAN, 5 classes)

TITAN

CARE

ALICE

0.3800±0.0554 0.3278±0.0442 0.4458±0.0462 0.2064±0.0464

0.2568±0.0303 0.2333±0.0533

0.5354±0.0510 0.5674±0.0652 0.7399±0.0904 0.2602±0.0172

0.2753±0.0426 0.3065±0.0621

0.9528±0.0205 0.8991±0.0340 0.9915±0.0060 0.2657±0.0549 0.3558±0.1012 0.2290±0.0584 0.1276±0.0332 0.0704±0.0238 0.2901±0.2013 0.3727±0.2099 0.1560±0.0410 0.4451±0.2614 0.4728±0.0034 0.3848±0.0736 0.6019±0.0677 0.3388±0.0907

0.2542±0.0505 0.2475±0.0481

0.6210±0.0678 0.4318±0.0155 0.6268±0.2081 0.2178±0.0025

0.1305±0.0305 0.2109±0.0056

0.4655±0.0076 0.3535±0.1181 0.5593±0.0156 0.5909±0.0980

0.5932±0.1210 0.8998±0.0187

0.2322±0.0582 0.1500±0.0457 0.2414±0.0491 0.3133±0.0644

0.2707±0.0300 0.2830±0.0280

0.2925±0.0885 0.1707±0.0491

0.2430±0.1048

0.4696±0.0126 0.3983±0.0799 0.6167±0.0637 0.3396±0.0949

0.2432±0.0558 0.3225±0.0863

Extended Data Table 61: Slide-level clinical diagnosis linear probe evaluation. Balanced accuracy is reported as the mean ± sample standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each row.

85

Dataset Breast carcinoma coarse-grained subtyping (BRACS, 3 classes) Breast carcinoma fine-grained subtyping (BRACS, 7 classes) Lymph node metastasis screening (CAMELYON+, 2 classes) Lymph node metastatic burden classification (CAMELYON+, 4 classes) Pan-cancer primary site prediction (CMB, 9 classes) Non-small cell lung cancer subtyping (CPTAC-NSCLC, 2 classes) Pan-cancer primary site prediction (CPTAC, 9 classes) Renal cell carcinoma subtyping (DHMC-RCC, 5 classes) CNS fine-grained tumor diagnosis (EBRAINS, 30 classes) CNS tumor-family classification (EBRAINS, 9 classes) CNS tumor WHO grading (EBRAINS, 4 classes) Colorectal neoplasia screening (HunCRC, 4 classes) Pediatric rare tumor classification (KidRare, 4 classes) Pediatric rare tumor fine-grained subtyping (KidRare, 13 classes) Prostate cancer Gleason grading (PANDA, 6 classes) Prostate cancer screening (PANDA, 2 classes) Bone metastasis primary site prediction (SYSBM, 10 classes) Lung lesion triage (SYSFL+, 4 classes) Lung cancer subtyping and invasion grading (SYSFL+, 7 classes) Cervical epithelial lesion classification (TissueNet, 4 classes) Ovarian carcinoma subtyping (UBC-OCEAN, 5 classes) Overall

TITAN

CARE

ALICE

0.6782

0.6782

0.7126

0.3793

0.4138

0.4368

0.8000

0.7885

0.8192

0.7348

0.7348

0.7879

0.8112

0.7551

0.7959

0.9417

0.9250

0.9750

0.9581

0.9532

0.9458

0.8344

0.8535

0.8025

0.6883

0.6497

0.7075

0.9194

0.9037

0.9317

0.7911

0.7748

0.7992

0.8095

0.8333

0.6429

0.9141

0.8750

0.8984

0.7148

0.6641

0.6562

0.6024

0.6413

0.6623

0.8859

0.9010

0.9288

0.7544

0.7310

0.7310

0.6510

0.6577

0.6779

0.4128

0.4329

0.4329

0.5864

0.5288

0.5969

0.8727

0.8636

0.8455

0.7495

0.7409

0.7518

Extended Data Table 62: Slide-level retrieval performance across 21 benchmarks using WSI features. Performance is reported as MVAcc@3. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each dataset.

86

Task

TITAN

ER status prediction (BCNB, 2 classes) HER2 status prediction (BCNB, 2 classes) PR status prediction (BCNB, 2 classes) Glioma IDH status prediction (EBRAINS, 2 classes) BAP1 mutation prediction (MUT-HET-RCC, 2 classes) PBRM1 mutation prediction (MUT-HET-RCC, 2 classes) SETD2 mutation prediction (MUT-HET-RCC, 2 classes)

ALICE

CARE

0.5728±0.1189 0.6608±0.0657

0.5019±0.0856

0.5118±0.0548 0.4682±0.0509

0.5667±0.0735

0.5726±0.0820 0.6095±0.0405

0.5329±0.0792

0.6515±0.1593 0.5633±0.0782

0.9259±0.0658

0.4621±0.1252

0.5171±0.1226 0.5925±0.1256

0.6392±0.0464 0.6070±0.0669

0.7001±0.0191

0.5519±0.0696

0.5412±0.0588

0.5182±0.0682

Extended Data Table 63: Slide-level biomarker prediction linear probe evaluation. AUC is reported as the mean ± standard deviation across five random seeds. Bold indicates the bestperforming model, and underline indicates the second-best-performing model for each task.

Task CPTAC-CCRCC (OS) CPTAC-HNSC (OS) CPTAC-LUAD (OS) CPTAC-PDAC (OS) Hancock (OS) SURGEN (OS)

TITAN

CARE

ALICE

0.5953±0.1875 0.5549±0.1531 0.5987±0.1215 0.5019±0.0543 0.5022±0.0814 0.5368±0.0731

0.5835±0.1015 0.6498±0.1064 0.5367±0.0884 0.5308±0.0728 0.4558±0.0563 0.5435±0.0384

0.5989±0.2200 0.6304±0.0776 0.5707±0.0918 0.4949±0.1000 0.5586±0.0744 0.5537±0.0919

Extended Data Table 64: Slide-level survival prediction evaluation. C-index is reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each task.

87

Task

Shot

Pan-cancer classification (CPTAC, 9 classes)

K=1 K=2 K=4 K=8 K=16 K=32

TITAN

CARE

ALICE

0.7266±0.0754 0.5469±0.0303 0.7255±0.0706 0.8049±0.0468 0.5401±0.0622 0.7993±0.0283 0.8718±0.0205 0.7522±0.0245 0.8696±0.0218 0.8818±0.0292 0.8011±0.0232 0.8960±0.0087 0.8993±0.0047 0.8461±0.0139 0.9153±0.0136 0.9037±0.0073 0.8521±0.0140 0.9288±0.0058

K=1 0.3474±0.0884 0.2683±0.0786 0.3083±0.1255 K=2 0.4435±0.0935 0.4217±0.0319 0.4337±0.0614 0.5322±0.0440 0.4877±0.0342 0.5310±0.0779 Bone metastasis primary site prediction K=4 (SYSBM, 10 classes) K=8 0.6176±0.0495 0.5273±0.0213 0.5766±0.0225 K=16 0.5992±0.0282 0.5163±0.0309 0.6087±0.0575 K=32 0.4983±0.0186 0.4525±0.0191 0.4852±0.0318

Pan-cancer classification (CMB, 9 classes)

K=1 K=2 K=4 K=8 K=16 K=32

0.4647±0.0656 0.5040±0.0492 0.6207±0.0582 0.6388±0.0359 0.6786±0.0330 0.5653±0.0419

0.4220±0.0348 0.4011±0.0812 0.4726±0.0485 0.5384±0.0360 0.5468±0.0612 0.4964±0.0232

0.4718±0.0861 0.5139±0.0640 0.6676±0.0813 0.6757±0.0270 0.6995±0.0168 0.6613±0.0257

Extended Data Table 65: Comparison of few-shot slide-level classification performance using WSI features under linear probing. Results are reported as the mean ± standard deviation. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each task and shot.

88

TITAN

CARE

ALICE

Task

Setting

Breast carcinoma coarse-grained subtyping (BRACS, 3 classes)

Linear probe 0.3800±0.0554 0.3278±0.0442 0.4458±0.0462 Fine-tuning 0.6704±0.0362 0.6913±0.0473 0.7258±0.0340

Breast carcinoma fine-grained subtyping (BRACS, 7 classes)

Linear probe 0.2064±0.0464 0.2568±0.0303 0.2333±0.0533 Fine-tuning 0.3727±0.0198 0.4397±0.0340 0.4447±0.0200

Lymph node metastasis coarse-grained classification Linear probe 0.5354±0.0510 0.5675±0.0652 0.7399±0.0904 (CAMELYON+, 2 classes) Fine-tuning 0.7508±0.0202 0.9013±0.0125 0.9229±0.0085 Lymph node metastasis fine-grained classification (CAMELYON+, 4 classes)

Linear probe 0.2602±0.0172 0.2753±0.0426 0.3065±0.0621 Fine-tuning 0.4454±0.0112 0.5738±0.0246 0.6155±0.0220

Pediatric rare tumor classification (KidRare, 4 classes)

Linear probe 0.6210±0.0678 0.4318±0.0155 0.6268±0.2081 Fine-tuning 0.9013±0.0072 0.8977±0.0247 0.9298±0.0197

Pediatric rare tumor fine-grained subtyping (KidRare, 13 classes)

Linear probe 0.2178±0.0025 0.1305±0.0305 0.2109±0.0056 Fine-tuning 0.5374±0.0436 0.4955±0.0630 0.5774±0.0544

Extended Data Table 66: Slide-level clinical diagnosis comparison between linear probing and fine-tuning. Balanced accuracy is reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-bestperforming model for each setting.

TITAN

CARE

ALICE

Task

Setting

ER status prediction (BCNB, 2 classes)

Linear probe 0.5728±0.1329 0.6608±0.0734 Fine-tuning 0.8557±0.0172 0.8491±0.0200

0.5019±0.0957 0.8900±0.0260

PR status prediction (BCNB, 2 classes)

Linear probe 0.5726±0.0917 0.6095±0.0453 Fine-tuning 0.8032±0.0076 0.7813±0.0219

0.5329±0.0885 0.8175±0.0085

PBRM1 mutation prediction Linear probe 0.6392±0.0518 0.6070±0.0748 (MUT-HET-RCC, 2 classes) Fine-tuning 0.7391±0.0202 0.7369±0.0220

0.7001±0.0213 0.8225±0.0081

BAP1 mutation prediction (MUT-HET-RCC, 2 classes)

Linear probe 0.4621±0.1399 0.5171±0.1370 0.5925±0.1404 Fine-tuning 0.8254±0.0297 0.8413±0.0103 0.8997±0.0213

SETD2 mutation prediction (MUT-HET-RCC, 2 classes)

Linear probe 0.5182±0.0763 0.5519±0.0778 0.5412±0.0657 Fine-tuning 0.6663±0.0163 0.7080±0.0213 0.7305±0.0256

Extended Data Table 67: Slide-level biomarker prediction comparison between linear probing and fine-tuning. AUC is reported as the mean ± standard deviation across five random seeds. Bold indicates the best-performing model, and underline indicates the second-best-performing model for each setting.

89