Deep learning-based integration of histology, radiology, and genomics for improved survival prediction in glioma patients

Luoting Zhuang, Jana Lipková, Richard Chen, Faisal Mahmood

Abstract: Management of aggressive malignancies, such as glioma, is complicated by a lack of predictive biomarkers that could reliably stratify patients based on treatment outcome. The complex mechanisms driving glioma recurrence and treatment resistance cannot be fully understood without the integration of multiscale factors such as cellular morphology, tissue microenvironment, and macroscopic features of the tumor and the host tissue. We present a weakly-supervised, interpretable, multimodal deep learning-based model fusing histology, radiology, and genomics features for glioma survival predictions. The proposed framework demonstrates the feasibility of multimodal integration for improved survival prediction in glioma patients.