Genomics-Embedded Histopathology Whole Slide Image Encoding for Data-efficient Survival Prediction

Kun Wu, Zhiguo Jiang, Xinyu Zhu, Jun Shi, Yushan Zheng

Keywords: Survival Prediction, Multi-modal Learning, Whole Slide Images

Abstract: Extensive works have shown that fusing histopathology images with genomics features can significantly improve the performance of survival prediction. However, the current methods still require both image and genomics data during the inference phase. In this work, we proposed the Genomics-Embedded WSI Encoding (GEE) framework where a proxy branch is built to guide the WSI encoder to extract genomics-related features from image modality. It makes the model achieve comparable inference accuracy solely based on image modality input when compared to the SOTA multi-modal-based survival prediction methods.

Submission Number: 162