Integrating Pathology Foundation Models and Spatial Transcriptomics for Cellular Decomposition from Histology Images

ICML 2025 Workshop FM4LS Submission7 Authors

Keywords: Pathology foundation models, Spatial transcriptomics, Cell-type composition prediction

TL;DR: We present a lightweight multi-modal framework combining pretrained pathology model embeddings and spatial transcriptomics references to efficiently infer cell-type composition from H&E images without costly assays.

Abstract: The rapid development of digital pathology and modern deep learning has facilitated the emergence of pathology foundation models that are expected to solve general pathology problems under various disease conditions in one unified model, with or without fine-tuning. In parallel, spatial transcriptomics has emerged as a transformative technology that enables the profiling of gene expression on hematoxylin and eosin (H&E) stained histology images. Spatial transcriptomics unlocks the unprecedented opportunity to dive into existing histology images at a more granular, cellular level. In this work, we propose a lightweight and training-efficient approach to predict cellular composition directly from H&E-stained histology images by leveraging information-enriched feature embeddings extracted from pre-trained pathology foundation models. By training a lightweight multi-layer perceptron (MLP) regressor on cell-type abundances derived via cell2location, our method efficiently distills knowledge from pathology foundation models and demonstrates the ability to accurately predict cell-type compositions from histology images, without physically performing the costly spatial transcriptomics. Our method demonstrates competitive performance compared to existing methods such as Hist2Cell, while significantly reducing computational complexity.

Submission Number: 7