SurvivMIL: A Multimodal, Multiple Instance Learning Pipeline for Survival Outcome of Neuroblastoma Patients
Keywords: Multiple Instance Learning, Multimodal Fusion, Digital Pathology, Survival Analysis
Abstract: Integrating Whole Slide Images (WSIs) and patient-specific health records (PHRs) can facilitate survival analysis of high-risk neuroblastoma (NB) cancer patients. However, this integration is challenging due to extreme differences in data dimensionality. Specifically, while PHRs are at the patient level and contain sparse information, WSIs are highly information-dense and processed at high resolution. Adjacent to this challenge, specifically in the context of survival analysis under the Multiple Instance Learning (MIL) framework, there are limitations with approximating the hazard function because of varying size WSIs and implicitly limited batch sizes. To address these challenges, we propose SurvivMIL, a late fusion MIL model that integrates multimodal prognostic data for predicting NB patient outcomes. Our approach fuses predictions from both modalities and incorporates a novel concordance-based loss function via a specifically designed buffer branch, which mitigates the batch size limitation by accumulating survival predictions. Our model is evaluated on an in-house pediatric NB patient dataset, providing insights into the contributions of each modality to predictive performance.
Submission Number: 13
Loading