Multi-Modal Disentanglement of Spatial Transcriptomics and Histopathology Imaging

Hassaan Maan, Zongliang Ji, Elliot Sicheri, Tiak Ju Tan, Alina Selega, Ricardo Gonzalez, Rahul Krishnan, BO WANG, Kieran R. Campbell

Track: Full Paper Track

Keywords: Spatial transcriptomics, Histopathology, Disentanglement, Representation learning, Multi-modal, Cancer research, Genomics, Variational inference, Probabilistic modeling

TL;DR: We propose a multi-modal disentanglement method to learn and control generative factors of spatial transcriptomics and histopathology data.

Abstract: Spatially-resolved expression profiling data has revolutionized biological research with multiple emerging clinical applications. Spatial transcriptomic assays are often jointly measured with histopathology imaging data, which is frequently used for diagnosing and staging various diseases. However, determining the extent to which the spatial transcriptomic and histopathology data represent overlapping or unique sources of variation is challenging, particularly given the myriad of factors influencing both, including expression variation, spatial context, tissue morphology, and batch effects. Here, we view this challenge as multi-modal disentanglement and develop an evaluation framework. We introduce SpatialDIVA, a disentanglement technique for jointly measured spatially resolved transcriptomics and histopathology data. We demonstrate that SpatialDIVA outperforms baseline techniques in disentangling salient factors of variation in curated pathologist-annotated multi-sample colorectal and pancreatic cancer cohorts. Further, SpatialDIVA removes batch effects from multi-modal data, allows for factor covariance analysis, and yields actionable biological insights through a novel conditional multi-modal generation method.

Attendance: Hassaan Maan

Submission Number: 56