A machine learning approach for multimodal data fusion for survival prediction in cancer patients

Nikolaos Nikolaou

Keywords: Multimodal Data, Survival Analysis, Precision Oncology, Biological Data, AI for Health, Ensemble Learning

TL;DR: We develop a versatile pipeline to explore multimodal data fusion in the context of cancer patient survival analysis; we use it to demonstrate its advantages over single-modality approaches.

Abstract:

Technological advancements of the past decade have transformed cancer research, improving patient survival predictions through genotyping and multimodal data analysis. However, there is no comprehensive machine learning pipeline for comparing methods to enhance these predictions. To address this, a versatile pipeline using The Cancer Genome Atlas (TCGA) data was developed, incorporating various data modalities such as transcripts, proteins, metabolites, and clinical factors. This approach manages challenges like high dimensionality, small sample sizes, and data heterogeneity. By applying different feature extraction and fusion strategies, notably late fusion ensemble models, the effectiveness of integrating diverse data types was demonstrated. Late fusion models consistently outperformed single-modality approaches in most TCGA cancer types (including lung, breast, and pan-cancer datasets), offering higher accuracy and robustness. This advantage is amplified in larger datasets and adding modalities is found to generally improve performance, with clinical features and gene expression emerging as top predictors. This research highlights the potential of comprehensive multimodal data integration in precision oncology to improve survival predictions for cancer patients. The study provides a reusable pipeline for the research community, suggesting future work on larger cohorts.

Where was the work published:

The pipeline itself and the extensive study on the TCGA dataset focusing on late modality fusion is presented in the uploaded paper:
[1] Nikolaou, N., Salazar, D., RaviPrakash H., Gonçalves, M., Mulla, R., Burlutskiy, N., Markuzon, N. and Jacob E., A machine learning approach for multimodal data fusion for survival prediction in cancer patients. npj Precision Oncology 9, 128 (2025). https://doi.org/10.1038/s41698-025-00917-6

An older version of this work is available as a preprint:
[2] Nikolaou, N., Salazar, D., RaviPrakash H., Gonçalves, M., Mulla, R., Burlutskiy, N., Markuzon, N. and Jacob E., Quantifying the advantage of multimodal data fusion for survival prediction in cancer patients. bioArxiv Preprint, doi: https://doi.org/10.1101/2024.01.08.574756

We have also successfully applied our pipeline in two additional multimodal data fusion contexts; the relevant publications are listed below; In the presentation, we will briefly discuss why different techniques worked in the different contexts:

I. Intermediate fusion of clinical features and CT images for survival prediction and risk classification of metastatic non-small cell lung cancer patients (NSCLC) using data from multiple clinical studies:
[3] Patwardhan, K. A., RaviPrakash, H., Nikolaou, N., Gonzalez-García, I., Salazar, J. D., Metcalfe, P., & Reischl, J. (2024). Towards a survival risk prediction model for metastatic NSCLC patients on durvalumab using whole-lung CT radiomics. Frontiers in Immunology, 15, 1383644.

II. Intermediate fusion of clinical features and several -omics modalities for survival prediction of non-small cell lung cancer patients (NSCLC) on TCGA data using autoencoders as the per-modality dimensionality reduction technique:
[4] Ellen, J. G., Jacob, E., Nikolaou, N., & Markuzon, N. (2023). Autoencoder-based multimodal prediction of non-small cell lung cancer survival. Scientific Reports, 13(1), 15761.

Submission Number: 33