A Multi-modal Deep Learning Framework for Head and Neck Tumor Segmentation and Survival Prediction

Fuyou Mao, Yanfeng Jiang, Naye Ji, Yanbing Jiang, Xinyuan Zheng, Hao Zhang, Yan Tang

Keywords: Head and Neck Cancer, Tumor Segmentation, Survival Prediction, Deep Learning, Mamba, 3D ResNet, Multi-modal Fusion, PET/CT Imaging

Abstract: Accurate segmentation and prognosis of head and neck cancer are crucial for effective treatment planning and personalized medicine. This study addresses two key challenges from the HECKTOR 2025 challenge: automated segmentation of primary gross tumor volume (GTVp) and prediction of Recurrence-Free Survival (RFS).For segmentation (Task 1), we employed the HecMamba architecture, leveraging its powerful HecMamab encoder to capture global context from PET/CT images. For prognosis (Task 2), we developed a multi-modal fusion model that combines a 3D ResNet for deep feature extraction from PET/CT images with a dedicated multi-layer perceptron (MLP) for processing clinical data. An ensemble of these models, trained using a 5-fold cross-validation strategy, was used to predict RFS.Our segmentation model achieved a mean Dice Similarity Coefficient (DSC) of 0.785. The prognosis model achieved a high Concordance Index (C-index) of 0.902 on the test set, demonstrating strong predictive power by effectively integrating imaging and clinical features. This work presents a comprehensive deep learning framework that successfully addresses both segmentation and prognosis prediction for head and neck cancer. The HecMamba proves highly effective for segmentation, while our multi-modal fusion network demonstrates that integrating deep-learned imaging features with clinical data significantly enhances survival prediction accuracy.

Submission Number: 4