Disentangled Shared Representations Improve Morpho-Transcriptomic Integration

Disentangled Shared Representations Improve Morpho-Transcriptomic Integration Julian Ostermaier1,2∗ , Swann Ruyter2 , Reuben Dorent2 , and Daniel Racoceanu2 ESPCI Paris, PSL University, Paris, France julian.ostermaier@gmx.net 2 Sorbonne Université, CNRS, Inserm, AP-HP, Inria, Paris Brain Institute – ICM, Paris, France swann.ruyter@icm-institute.org, reuben.dorent@inria.fr, daniel.racoceanu@sorbonne-universite.fr ∗ Corresponding author.

arXiv:2608.14355v1 [cs.AI] 14 Aug 2026

1

Abstract. Spatial transcriptomics (ST) enables the simultaneous profiling of gene expression and tissue morphology, creating an opportunity to learn multimodal representations capturing shared morpho-transcriptomic structure. However, standard multimodal models often compress modalities into a common latent space without explicitly separating shared and modality-specific sources of variation, which may limit downstream utility. We investigate whether explicit disentanglement of shared and private latent components improves multimodal representation learning for paired Hematoxylin & Eosin (H&E) and ST data. We compare VAEbased and contrastive approaches, each in standard and disentangled variants, across two cancer cohorts under matched experimental conditions. Representations are evaluated using cross-modal reconstruction, downstream probing and cross-modal probe transfer. The experiments suggest two main trends. First, contrastive objectives yield higher downstream probing performance than VAE-based models. Second, disentangled variants improve the selected reconstruction and probing metrics, although the gains depend on the model family, task, direction, and disentanglement strength. Overall, our results suggest that explicitly factorizing shared and modality-specific information can improve multimodal representation learning for spatial transcriptomics and provides a useful evaluation framework for future foundation models. Keywords: Representation Learning · Multimodal Data · Spatial Transcriptomics · Variational Autoencoders · Contrastive Learning.

1

Introduction

Spatial transcriptomics (ST) enables spatially resolved molecular profiling while preserving tissue architecture, providing paired measurements of histology and gene expression from the same tissue section [12]. Beyond direct prediction of gene expression from H&E [23,17], these paired data offer an opportunity to learn multimodal representations of tissue organization.

2

J. Ostermaier et al.

In histopathology, foundation models have demonstrated that compact image representations capture transferable tissue features across diverse tasks and cohorts [8,3]. A similar objective is appealing for ST, where joint representations of H&E morphology and gene expression can encode the morpho-transcriptomic organization of a tissue section in a low-dimensional latent space. Recent advances in representation learning, including in world models, suggest that learning compact latent representations of complex data can provide a more efficient alternative to directly modeling high-dimensional observations [10]. In the context of ST, this motivates learning latent representations that capture morphotranscriptomic information, rather than directly predicting virtual gene expression from H&E. Such representations may provide compact transcriptomicsenriched representations for downstream analyses and transfer tasks. Accordingly, several multimodal representation learning methods have recently been proposed to learn a shared representation of these modalities using paired histology and ST data. Approaches range from contrastive objectives [22,4] to autoencoder-based and generative models [5,1,21,9]. However, while standard multimodal models encode both modalities into the same space, they do not control which information is shared and which is private (or modalityspecific). This disentanglement is needed in representation learning [9,2], since the true amount of shared signal between H&E and gene expression data is unknown and both modalities contain a large amount of modality-specific information. The idea of disentangling shared from modality-specific information is well established in the broader multimodal learning literature [13,11,19]. Within ST specifically, [9] is the closest prior work: it uses a latent variable model that explicitly decomposes histological and ST data into distinct generative factors including spatial, morphological, and transcriptomic variation. However, it relies on expert annotations for supervision which makes it impractical for large-scale training, where such annotations are costly and often unavailable. More broadly, across this line of work, previously proposed representation learning approaches are evaluated under heterogeneous experimental settings and different evaluation criteria, leaving it unclear if their representations are both well aligned and simultaneously keep information of downstream utility. In this work, we test whether disentanglement of multimodal representations improves joint H&E and ST representation learning. We compare two main lines of models, variational autoencoder models (VAE) and contrastive models, each in standard shared-latent and variants that separate shared from modality-specific components [13,11,19]. Under matched experimental conditions, we evaluate the learned representations across multiple tissue cohorts using cross-modal reconstruction, downstream probing, and cross-modal probe transfer. Our experiments suggest that contrastive objectives yield stronger downstream representations than VAE-based models, while disentanglement generally improves cross-modal prediction and downstream performance.

Disentangled shared representations in ST

2

3

Methods

In this study, we compared five representative multimodal representation learning models, including multimodal variational autoencoders and contrastive learning methods, and their disentangled extensions, across two cancer cohorts, using a unified probing and reconstruction evaluation framework. 2.1

Representation Learning Model Baselines.

Representation methods were selected to represent two common families of multimodal representation learning methods: autoencoder-based objectives that learn shared representations through cross-modal reconstruction, as in GenST [21], and contrastive alignment objectives, as exemplified by omiCLIP [4]. Crucially, to explore the effect of disentanglement, both lines were represented by a standard shared-representation model and a disentangled variant, enabling a systematic comparison of representation strategies within each paradigm (see Table 2). Multimodal Variational Autoencoders (MMVAE) [16]. MMVAE encodes each modality independently into a shared latent space. The mixture-of-experts VAE [16], which approximates the joint posterior distribution using a mixtureof-experts, serves as the non-disentangled MMVAE baseline. We further implemented MMVAE+[13], a MMVAE extension that introduces an explicit disentanglement mechanism between shared and modality-specific information. We denote the modalities by M = {G, HE}, where xm is the observation from modality m. The shared latent code is zs , and zm p denotes the private latent code } . The encoders qsm and qpm approximate of modality m. We write zp = {zm p m∈M the modality-specific posteriors over shared and private latents, respectively. Its objective can be written as    LMMVAE+ =

Y    1 X Rmm  Rmn  E qsm (zs |xm )  log   , (1)   |M| qΦ (zs , zp | x1:M ) q m (zm |xm ) m∈M

p

p

n∈M, n̸=m

n where Rmm = pm (xm , zs , zm p ) is self-reconstruction, Rmn = pθn (xn | zs , z̃p ) n n cross-modal reconstruction and z̃p ∼ r(zp ) is sampled from a modality-specific prior distribution. Additionally, the recent disentanglement variant MMVAE+sg [11] was included. MMVAE+sg redefines the training objective to prevent leakage of modality-specific information into the shared space, using a stop-gradient operator that blocks the gradient flow from same-view reconstruction into the shared latent variable zs .

Contrastive Learning (CLIP) [14]. Motivated by recent foundation models for ST such as omiCLIP [4] and BLEEP [22], contrastive learning methods were included as a second model family. Modality-specific encoders were trained with the symmetric InfoNCE

4

J. Ostermaier et al.

loss [14], which maximizes the cosine similarity between representations of paired multimodal observations while minimizing the similarity to independent observations within each training batch, thereby aligning representations across modalities without requiring data reconstruction: " # HE+ exp(zG⊤ /τ ) exp(zHE⊤ zG+ 1 s zs s s /τ ) + log P , (2) Lcon = − E log P G⊤ HE HE⊤ zG /τ ) 2 k exp(zs zs,k /τ ) k exp(zs s,k where τ is a temperature parameter, superscript + denotes the matched (positive) sample, and the sums run over all samples in the batch. Disentangled SSL (disSSL) [19] extends this contrastive framework by separating shared from modality-specific latent information. The shared representation is learned using the same contrastive objective Lcon with a penalty that
minimizes I ZG , X | X , the amount of modality-specific information reHE G s tained in zs , using the Kullback-Leibler divergence (DKL ) as an upper bound: Lzs =

Lcon |{z}

contrastive loss

HE HE +β DKL (qsG (ZG | XHE )) . s | XG ) || qs (Zs | {z }

(3)

redundancy reduction

The coefficient β controls the strength of this penalty. When β = 0, the objective reduces to standard contrastive alignment. Uni-modal Baselines. To contextualize the performance of our multimodal models, uni-modal baselines were included using Principal Component Analysis (PCA) and uni-modal Variational Autoencoders (VAE), thereby providing linear and nonlinear reference representation baselines within each modality. Implementation Details. All models were trained using a leave-one-slide-out cross-validation scheme to learn a 32-dimensional shared latent representation zs , capturing information shared across gene expression and histology modalities. Architectural choices and training settings were matched across models where applicable. Experiments were implemented in Python with PyTorch and run on NVIDIA V100 GPUs. MMVAE-based models were adapted from MultiVae [15], with preprocessing performed using HEST and scanpy. Probing experiments were repeated five times with different random seeds; training hyperparameters are reported in Supplementary Table 4. 2.2

Datasets and Pre-processing

Cohorts. We used two Visium ST cohorts: 7 colorectal cancer (Colon) samples from HEST [7] and 10 glioblastoma (GBM) samples acquired in this project from Owkin - MOSAIC Window initiative. For Colon, one sample per patient was retained, selected based on artifact burden and spot count, to avoid patient overlap during leave-one-slide-out cross-validation. The GBM cohort complements Colon by introducing stronger inter-patient molecular heterogeneity, a known feature of GBM [20]. After pre-processing this results in a total of 35,002 spots for the

Disentangled shared representations in ST A. Pre-processing Gene expression

B. Representation Learning Normalized expression

Standard CLIP MMVAE

C. Cross-modal Reconstruction

Disentangled Models Disentangled SSL MMVAE +

cosine similarity cos( ℎ𝑖𝑗 , ℎ'𝑖𝑗 )

𝑔 𝑖𝑗

Patch

ℎ'𝑖𝑗 top-k PCC PCC(𝑔 𝑖𝑗𝑘 , 𝑔'𝑖𝑗𝑘)

𝑔 𝑖𝑗 UNI features

5

𝑔 𝑖𝑗

𝑔 𝑖𝑗

ℎ𝑖𝑗

𝑔'𝑖𝑗

D. Downstream Tasks Tissue Niches (scRNA-seq derived)

ℎ𝑖𝑗

ℎ𝑖𝑗

Tissue Types

(pathologist annotations)

ℎ𝑖𝑗

Fig. 1. Multimodal representation learning pipeline. (A) Pre-processing of the paired gene expression and H&E image patch data. (B) Representations are learned using different model families including contrastive and VAE methods as well as their disentangled versions. Representations are evaluated using (C) cross-modal reconstruction and (D) prediction of downstream labels.

GBM and 62,730 spots for the Colon cohort. Spatial Transcriptomics Pre-processing. Gene expression data were filtered by retaining spots with more than 200 expressed genes and fewer than 8% mitochondrial counts, normalized to 10,000 counts per spot, and log-transformed. For each dataset, gene panels were obtained from the union of highly variable genes (HVGs) across samples. HVGs were ranked by their maximum variance across slides, and the top K genes were retained (K ∈ {200, 500, 1000}). Histology Image Pre-processing. Histology image patches were extracted centered on each Visium spot, with a patch size of 224 × 224 pixels at a resolution of 0.5 microns per pixel, fully encompassing the 55 µm Visium spot diameter. Patches were processed with the UNI histopathology foundation model [3] to obtain a 1024-dimensional image embedding for each spot, capturing morphologically relevant features from the image modality.

2.3

Experimental Setup

Each model was trained to map multimodal inputs into a low-dimensional joint representation space. Once trained, the encoder weights were frozen and the learned representations were evaluated using probing models with a fixed architecture (Figure 1) to isolate the quality of the learned representations. Evaluation comprised cross-modal reconstruction and downstream prediction tasks. Probing on each task was repeated five times with different random seeds.

6

J. Ostermaier et al.

Cross-modal Reconstruction Task. Cross-modal reconstruction evaluates if a shared representation obtained from one modality can be used to predict the other modality. Specifically, MLP decoders were trained to reconstruct gene expression from the H&E-conditioned representation zHE (HE→G), and image representations s from the gene-expression-conditioned representation zG s (G→HE). HE→G reconstruction is evaluated using top-50 PCC, whereas G→HE reconstruction is evaluated using cosine similarity. Top-50 PCC is defined as the mean PCC across the 50 best-predicted genes, while cosine similarity is computed between the predicted and ground-truth image embeddings. Results are reported in Table 1. Downstream Probing Task. We evaluated downstream utility on the Colon cohort using linear probes for two complementary label sets from [18]: pathologist tissue-type annotations and tissue niches derived from scRNA-seq-based cell-type composition estimates. Tissue-type labels provide a morphology-oriented task, expected to be most directly captured by H&E-conditioned representations, whereas tissue-niche labels provide a molecularly oriented task, expected to be most directly captured by gene-expression-conditioned representations. Label construction is described in Supplementary Section 4. We report two probing settings. 1) Cross-modality probes evaluate whether representations conditioned on one modality predict labels primarily associated with the other modality: HE→G probes predict tissue niches from H&Econditioned representations, whereas G→HE probes predict tissue-type annotations from gene-expression-conditioned representations. 2) Cross-modality transfer probing assesses if a probe trained on one modality-conditioned representation can be applied to the other without retraining. This allows to assess if the shared latent space only contains shared information and if this information is rich. AUC results are reported for tissue niche classification and tissue-type classification in Figure 2.

3

Results

This section presents the cross-modal and downstream results of our study. Cross-modal Reconstruction Results. Within the VAE family, MMVAE+ and MMVAE+sg achieved higher mean reconstruction performance than MMVAE across both cohorts, panel sizes, and reconstruction directions. After multipletest correction, only a subset of these comparisons reached statistical significance (Table 3), indicating that this pattern should be interpreted as a descriptive trend rather than a uniform effect. Among contrastive models, disSSL showed a direction- and β-dependent pattern. For HE→G reconstruction, the best disSSL configuration exceeded CLIP on GBM at larger panel sizes (K = 500 and K = 1000), but not at K = 200. On Colon, disSSL matched or slightly exceeded CLIP at K = 200 and K = 1000,

Disentangled shared representations in ST

7

Table 1. Cross-modal reconstruction results on Colon and GBM datasets for different gene panel sizes K. Metrics are reported as percentages and presented as mean±standard deviation over five repeated runs with different random seeds. For disSSL, 10−3 , 10−2 , and 10−1 denote the value of β. Bold and underline indicate the best and second-best values per column within each model family. Family

Colon

Model

top-50 PCC (HE→G)

cosine similarity (G→HE)

K 200

500

1000

K 200

500

1000

47.9±1.5 43.7±0.6

51.4±1.0 46.7±0.8

52.7±0.8 47.2±1.0

59.2±1.0 57.0±0.5

60.7±0.5 58.5±0.4

60.7±0.8 58.9±0.5

Unimodal

PCA VAE

VAE

MMVAE 42.3±0.4 48.6±0.2 49.7±0.5 60.0±0.3 59.9±0.3 61.4±0.2 MMVAE+ 47.5±0.6 50.8±0.8 52.5±0.8 61.5±0.4 60.8±0.3 62.5±0.5 MMVAE+sg 47.6±0.5 51.8±0.4 53.2±0.6 62.3±0.3 61.5±0.5 62.7±0.4

CLIP 50.2±0.5 53.4±0.5 52.8±0.6 63.1±0.3 62.0±0.3 62.4±0.2 Contras- disSSL 10−3 50.6±0.8 53.1±0.3 53.3±0.5 61.8±0.5 61.9±0.5 62.5±0.5 −2 tive disSSL 10 48.8±0.4 52.3±0.9 53.9±0.4 61.3±0.7 60.8±0.9 62.2±0.5 disSSL 10−1 45.7±0.3 46.5±0.3 47.4±0.6 60.9±0.9 60.9±0.9 60.8±0.4

GBM

Unimodal

PCA VAE

43.0±0.3 42.2±0.4

46.9±0.8 43.6±0.7

47.4±0.7 43.5±0.7

38.7±0.5 39.8±0.1

40.7±0.4 39.5±0.4

41.6±0.3 40.0±0.3

VAE

MMVAE 46.1±0.4 50.0±0.3 51.7±0.2 41.9±0.3 42.6±0.3 44.6±0.2 MMVAE+ 48.3±0.3 51.0±0.2 52.3±0.5 43.7±0.2 44.4±0.3 45.0±0.1 MMVAE+sg 48.0±0.3 51.4±0.3 51.8±0.3 42.8±0.4 43.9±0.2 44.5±0.5

CLIP 48.6±0.2 50.1±0.2 50.8±0.2 42.9±0.4 44.4±0.6 45.7±0.7 Contras- disSSL 10−3 48.3±0.2 50.5±0.5 51.0±0.3 42.5±0.3 45.0±0.4 44.3±0.2 tive disSSL 10−2 47.9±0.3 51.0±0.5 51.7±0.3 41.6±0.5 44.0±0.7 43.8±0.5 disSSL 10−1 41.8±0.2 44.8±0.5 46.0±0.3 38.5±0.6 38.4±0.8 40.0±0.6

whereas CLIP remained higher at K = 500. For G→HE reconstruction, disSSL did not consistently improve over CLIP. This asymmetry is consistent with stronger disentanglement removing information useful for image-embedding reconstruction, rather than uniformly improving cross-modal prediction. Because most comparisons did not reach significance after correction, these patterns should be interpreted as suggestive. Downstream Probing Results. Contrastive models achieved higher cross-modal probe AUCs than VAEbased models for both tissue niche and tissue-type classification. MMVAE showed the lowest cross-modality probe performance, whereas MMVAE+ and MMVAE+sg improved over MMVAE but generally remained below CLIP and disSSL. This pattern suggests that, under this probing protocol, contrastive alignment yields more task-useful shared representations than VAE-based reconstruction objectives. Within the VAE family, the benefit of disentanglement depended on the probing setting. MMVAE+ and MMVAE+sg improved cross-modality probing across both label sets, supporting a benefit of factorized latent representations for this readout. Transfer probing showed a more task-dependent pattern: MMVAE+sg improved transfer performance for tissue niche classification in several settings,

J. Ostermaier et al. A

Colon Tissue Niches (cell type decon. Belgian cohort) K 500 Transfer probe AUC

Transfer probe AUC

K 200

Cross-modal probe AUC

K 1000 MMVAE

Transfer probe AUC

8

Cross-modal probe AUC

MMVAE+ MMVAE+sg CLIP disSSL 10⁻³ disSSL 10⁻² disSSL 10⁻¹

Cross-modal probe AUC

Colon Expert Tissue type annotations

B

MMVAE

Transfer probe AUC

Transfer probe AUC

Transfer probe AUC

Cross-modal probe AUC

K 1000

K 500

K 200

Cross-modal probe AUC

MMVAE+ MMVAE+sg CLIP disSSL 10⁻³ disSSL 10⁻² disSSL 10⁻¹

Cross-modal probe AUC

Fig. 2. Downstream task performance for tissue niche classification (HE → G) and tissue type classification (G → HE). AUC prediction performance (five repeated runs with random seeds) with gene panel sizes K for (A) tissue niche classification derived from scRNA-seq reference Belgian cohort and (B) pathologist tissue types classification. Stars indicate non-disentangled and squares disentangled versions. The gray arrow connects contrastive models from CLIP (β = 0) to increasing disentanglement strength.

whereas disentangled VAE variants showed lower transfer AUCs for tissue-type annotations. This reversal suggests that the shared latent representation may retain less morphology-associated information when the VAE objective is more strongly factorized. Among contrastive models, disSSL matched or improved over CLIP in several probing settings, but the effect depended on β, panel size, and task. For cross-modality probes, disSSL generally matched or exceeded CLIP up to an intermediate disentanglement strength, with weaker gains at the smallest panel size (K = 200). For transfer probes, disSSL improved over CLIP in several conditions, consistent with improved latent-space compatibility and/or better retention of task-relevant information. However, performance was sensitive to the choice of β: stronger disentanglement did not uniformly improve AUC, and the optimal value varied across tasks and panel sizes. These results suggest that disentanglement can improve selected downstream readouts, but its benefit is not uniform across objectives, tasks, and hyperparameter settings.

Disentangled shared representations in ST

4

9

Conclusion

We evaluated VAE-based and contrastive multimodal representation learning models for paired H&E and ST data, comparing standard shared latent models with disentangled variants under a common evaluation protocol. The results support two main observations. First, disentangled VAE variants improved mean cross-modal reconstruction performance relative to MMVAE, although statistically significant gains were limited to a subset of configurations. Second, contrastive models achieved higher downstream linear-probe AUCs than VAEbased models on the Colon classification tasks. In contrast, the benefit of disSSL over CLIP was more variable and depended on the task, panel size, and disentanglement strength. Together, these findings suggest that disentanglement can improve selected readouts of multimodal representation quality, but does not provide a uniform benefit across objectives or evaluation settings. They also highlight the need to combine cross-modal reconstruction, cross-modality probing, and transfer probing, as each criterion captures a different aspect of learned H&E–ST representations.

Acknowledgments and Disclosure of Funding The research leading to these results has received funding from Agence Nationale de la Recherche as part of the “France 2030” program (reference ANR-23-IACL0008, PRAIRIE-PSAI) and as part of the ”Investissements d’avenir” program (reference ANR-19-P3IA-0001, PRAIRIE 3IA Institute; and reference ANR-10IAIHU-0006). The ARAMIS Lab is affiliated with DIM C-BRAINS, funded by the Conseil Régional d’Ile-de-France. This work was performed using HPC resources from GENCI–IDRIS (Grant 2025-AD011016416). R.D. received a Marie Sklodowska-Curie grant No 101154248 (project: SafeREG). This study makes use of data generated by the MOSAIC consortium (Owkin, Charité – Universitätsmedizin Berlin, Lausanne University Hospital - CHUV, Erlangen Hospital, Gustave Roussy Institute, University of Pittsburgh) and made available through the MOSAIC Window initiative. Readers should note that the MOSAIC consortium bears no responsibility for the further analysis or interpretation of these data beyond what published by the MOSAIC consortium partners. Disclosure of Interests The authors have no competing interests in the paper.

References 1. Bao, F., Deng, Y., Wan, S., Shen, S.Q., Wang, B., Dai, Q., Altschuler, S.J., Wu, L.F.: Integrative spatial analysis of cell morphologies and transcriptional states with muse. Nature Biotechnology 40(8), 1200–1209 (Mar 2022). https://doi. org/10.1038/s41587-022-01251-z

10

J. Ostermaier et al.

2. Chelebian, E., Avenel, C., Wählby, C.: Combining spatial transcriptomics with tissue morphology. Nature Communications 16(1), 4452 (may 2025). https:// doi.org/10.1038/s41467-025-58989-8 3. Chen, R.J., Ding, T., Lu, M.Y., Williamson, D.F., Jaume, G., Chen, B., Zhang, A., Shao, D., Song, A.H., Shaban, M., et al.: Towards a general-purpose foundation model for computational pathology. Nature Medicine (2024) 4. Chen, W., Zhang, P., Tran, T.N., et al.: A visual–omics foundation model to bridge histopathology with spatial transcriptomics. Nature Methods 22(7), 1568–1582 (2025). https://doi.org/10.1038/s41592-025-02707-1 5. Dai, J., Nonchev, K., Koelzer, V., Rätsch, G.: Towards cross-sample alignment for multi-modal representation learning in spatial transcriptomics (03 2026). https: //doi.org/10.64898/2026.03.02.709002 6. Gloor, G.B., Macklaim, J.M., Pawlowsky-Glahn, V., Egozcue, J.J.: Microbiome datasets are compositional: And this is not optional. Front. Microbiol. 8, 2224 (Nov 2017) 7. Jaume, G., Doucet, P., Song, A.H., Lu, M.Y., Almagro-Perez, C., Wagner, S.J., Vaidya, A.J., Chen, R.J., Williamson, D.F.K., Kim, A., Mahmood, F.: Hest-1k: A dataset for spatial transcriptomics and histology image analysis. In: Advances in Neural Information Processing Systems (Dec 2024) 8. Lu, M.Y., Chen, B., Williamson, D.F., Chen, R.J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L.P., Gerber, G., et al.: A visual-language foundation model for computational pathology. Nature Medicine 30, 863–874 (2024) 9. Maan, H., Ji, Z., Sicheri, E., Tan, T.J., Selega, A., Gonzalez, R., Krishnan, R., WANG, B., Campbell, K.R.: Multi-modal disentanglement of spatial transcriptomics and histopathology imaging. In: Learning Meaningful Representations of Life (LMRL) Workshop at ICLR 2025 (2025) 10. Maes, L., Lidec, Q.L., Scieur, D., LeCun, Y., Balestriero, R.: Leworldmodel: Stable end-to-end joint-embedding predictive architecture from pixels (2026), https:// arxiv.org/abs/2603.19312 11. Märtens, K., Yau, C.: Disentangling shared and private latent factors in multimodal variational autoencoders. In: Knowles, D.A., Mostafavi, S. (eds.) Proceedings of the 18th Machine Learning in Computational Biology meeting. Proceedings of Machine Learning Research, vol. 240, pp. 60–75. PMLR (30 Nov–01 Dec 2024) 12. Moses, L., Pachter, L.: Museum of spatial transcriptomics. Nat. Methods 19(5), 534–546 (May 2022) 13. Palumbo, E., Daunhawer, I., Vogt, J.E.: MMVAE+: Enhancing the generative quality of multimodal VAEs without compromises. In: The Eleventh International Conference on Learning Representations (2023) 14. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision (2021) 15. Senellart, A., Chadebec, C., Allassonnière, S.: Multivae: A python package for multimodal variational autoencoders on partial datasets. Journal of Open Source Software 10(110), 7996 (2025). https://doi.org/10.21105/joss.07996 16. Shi, Y., Siddharth, N., Paige, B., Torr, P.H.S.: Variational mixture-of-experts autoencoders for multi-modal deep generative models. In: Advances in Neural Information Processing Systems (NeurIPS 2019). vol. 32. Curran Associates, Inc. (2019) 17. Tran, M., Gindra, R.H., Putze, P., Senbai, K., Palla, G., Kos, T., Falcomatà, C., Wang, C., Guo, R.R., Boxberg, M., Berclaz, L.M., Lindner, L.H., Bergmayr, L.,

Disentangled shared representations in ST

11

Knösel, T., Jurmeister, P., Klauschen, F., Homicsko, K., Gottardo, R., Eckstein, M., Matek, C., Mock, A., Theis, F.J., Saur, D., Peng, T.: Pan-cancer virtual spatial transcriptomics from routine histology with phoenix. bioRxiv (Apr 2026), preprint 18. Valdeolivas, A., Amberg, B., Giroud, N., Richardson, M., Gálvez, E.J.C., Badillo, S., Julien-Laferrière, A., Túrós, D., Voith von Voithenberg, L., Wells, I., Pesti, B., Lo, A.A., Yángüez, E., Das Thakur, M., Bscheider, M., Sultan, M., Kumpesa, N., Jacobsen, B., Bergauer, T., Saez-Rodriguez, J., Rottenberg, S., Schwalie, P.C., Hahn, K.: Profiling the heterogeneity of colorectal cancer consensus molecular subtypes using spatial transcriptomics. NPJ Precis. Oncol. 8(1), 10 (Jan 2024) 19. Wang, C., Gupta, S., Zhang, X., Tonekaboni, S., Jegelka, S., Jaakkola, T., Uhler, C.: An information criterion for controlled disentanglement of multimodal data. In: The Thirteenth International Conference on Learning Representations (2025) 20. Wick, W., Kessler, T.: New glioblastoma heterogeneity atlas — a shared resource. Nature Reviews Neurology 14(8), 453–454 (2018). https://doi.org/10.1038/ s41582-018-0038-3 21. Wood, R., Hu, Y., Rittscher, J., Li, B.: GenST: A generative cross-modal model for predicting spatial transcriptomics from histology images. In: MICCAI Workshop on Computational Pathology with Multimodal Data (COMPAYL) (2025), https: //openreview.net/forum?id=LHVSSaCKL6 22. Xie, R., Pang, K., Chung, S.W., Perciani, C., MacParland, S., WANG, B., Bader, G.: Spatially resolved gene expression prediction from histology images via bimodal contrastive learning. In: Thirty-seventh Conference on Neural Information Processing Systems (2023) 23. Zhu, S., Zhu, Y., Tao, M., Qiu, P.: Diffusion generative modeling for spatially resolved gene expression inference from histology images. In: The Thirteenth International Conference on Learning Representations (2025)

12

J. Ostermaier et al.

Supplementary Material Model Overview

Table 2. Summary of the evaluated representation learning models. Model

Contrastive Generative Shared latent Private latent

MMVAE [16] MMVAE+ [13] MMVAE+sg [11] CLIP [14] disSSL [19]

– – – ✓ ✓

✓ ✓ ✓ – –

– ✓ ✓ – ✓

✓ ✓ ✓ ✓ ✓

Statistical Tables

Table 3. Statistically significant Wilcoxon signed-rank test results after Benjamini– Hochberg FDR correction (p < 0.05) across Colon and GBM dataset (HVG, log1p preprocessing). For each slide performance metrics were averaged across repeated runs, and paired slide-level metric values were used as observations in the Wilcoxon signedrank test. Significance levels refer to the BH-adjusted p-value: ∗ p < 0.05; ∗ ∗ p < 0.01; ∗ ∗ ∗ p < 0.001. Comparison

Metric

p (BH) Sig.

Colon —

—

—

GBM MMVAE vs MMVAE+sg PCC (HE→G) 500 MMVAE vs MMVAE+ Cosine (G→HE) 200 MMVAE vs MMVAE+ Cosine (G→HE) 500 Cosine (G→HE) 1000 CLIP vs disSSL β −3 CLIP vs disSSL β −2 Cosine (G→HE) 200 CLIP vs disSSL β −2 Cosine (G→HE) 1000 CLIP vs disSSL β −1 PCC (HE→G) 200 CLIP vs disSSL β −1 PCC (HE→G) 500 CLIP vs disSSL β −1 Cosine (G→HE) 200 CLIP vs disSSL β −1 Cosine (G→HE) 500 CLIP vs disSSL β −1 Cosine (G→HE) 1000

0.035 0.015 0.019 0.019 0.042 0.015 0.028 0.032 0.015 0.019 0.015

∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗

Disentangled shared representations in ST

Training Configurations

Table 4. Training hyperparameters shared across all model configurations. Hyperparameter Optimizer Learning rate Weight decay Optimizer Training schedule Batch size Number of epochs Architecture Latent dimension Private latent dim. Encoder model layers VAE β-parameter β

VAE-based

Contrastive

1 × 10−4 1 × 10−4 −5 1 × 10 — Adam (AMSGrad) Adam (AMSGrad) 64 50

256 50

32 16 [512, 256, 64]

32 16 [512, 256, 64]

0.1

—

13

14

J. Ostermaier et al.

Sample Overview for HEST Colorectal Cancer Cohort

Table 5. Overview of the retained samples used for model training after filtering of low quality samples and enforcing one sample per patient. HEST ID corresponds to the unique identifier of the sample in the HEST database [7]. Cohort

HEST IDs

Colorectal Cancer ZEN37, ZEN40, ZEN42, ZEN43, ZEN44, ZEN47, ZEN49

Downstream Task Label Generation Colorectal Cancer Cohort Pathologist Tissue Annotations. Raw pathologist annotations were obtained from the original publication [18] for each slide. Due to heterogeneity in naming conventions across slides that did not match the categorization criteria, annotations were mapped to a standardized set of tissue region categories. The mapping resolved typographic and formatting variants (normalizing underscores to spaces, converting to lowercase) and merged biologically equivalent labels. Specifically, connective tissue classes from slides ZEN37 and ZEN43 were remapped onto corresponding stroma categories based on their immune cell (IC) infiltration grade. lamina propria, squamous epithelium, and glandular tissue were collapsed into non neoplastic epithelium. Labels marked as uncertain or appearing in only a single slide were excluded to avoid slide-identity leakage. Each spot was one-hot encoded over the retained categories; unannotated spots were excluded. Cell Type Proportion Cluster Labels. Cell type count estimations from the original publication were downloaded [18], then normalized to per-spot cell type proportions. To address the compositional nature of these data, proportions were CLR-transformed [6] prior to k-means clustering. Clustering was performed once on the pooled, CLR-transformed proportions from all slides jointly, so that a given cluster label corresponds to the same tissue niche across the dataset without requiring post-hoc label harmonization. The optimal k was selected by silhouette score, yielding four tissue niche clusters in both deconvolution estimations (Belgian & Korean cohort scRNA-seq reference).

Disentangled shared representations in ST

15

Additional Results on Downstream Tasks

Colon Tissue Niches (cell type decon. Korean cohort)

MMVAE

Transfer probe AUC

Transfer probe AUC

Transfer probe AUC

Cross-modal probe AUC

K 1000

K 500

K 200

Cross-modal probe AUC

MMVAE+ MMVAE+sg CLIP disSSL 10⁻³ disSSL 10⁻² disSSL 10⁻¹

Cross-modal probe AUC

Fig. 3. Downstream task performance for tissue niche classification (HE → G). AUC prediction performance (mean over five repeated runs with different random seeds) from different gene panel sizes K for tissue niche classification derived from scRNA-seq reference Korean cohort. Stars indicate the non-disentangled model and circles the disentangled versions. The gray arrow connects contrastive models from CLIP (β = 0) to increasing disentanglement strength.

Related documents

• A deep learning representation and spatial Bayesian cell-type deconvolution for spatial transcriptomics.#406424
• Disentangling Representation Evolution in Transformers through Directional Decomposition#1110000
• From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flows#1143681
• NicheDeSig: niche-aware deconvolution and adaptive signature analysis for spatial transcriptomics.#469365
• Group-based Learning of Disentangled Representations with Generalizability for Novel Contents#478167