Lymph Node Metastasis Classification with Prototype-Guided Multiple Instance Aggregation and Heterogeneous Feature Fusion

Haoshen Li, Tashan Ai, Yirui Wang, Zhanghexuan Ji, Qinji Yu, Le Lu, Bin Dong, Li Zhang, Xianghua Ye, Kuaile Zhao, Dakai Jin

Abstract: Lymph node metastasis diagnosis in computed tomography (CT) scans is an essential yet very challenging task for esophageal cancer staging and treatment planning. Recent advances in deep learning have markedly improved the performance in lymph node (LN) metastasis classification. However, these methods often focus more on the averaged features of all CT slices containing a 3D LN instance, lacking effective fusion of key slice-wise features, which is important in the LN metastasis analysis by physicians. In addition, existing deep learning models are trained using CT scans in an end-to-end fashion, thus lacking the explicit incorporation of clinically relevant meta-imaging features (i.e., morphological and radiomic features). Meta-imaging features play a crucial role in LN assessment and may not be effectively captured by direct end-to-end deep learning models. To address these issues, we formulate the 3D LN metastasis classification as a multiple instance learning (MIL) problem by extracting and fusing slice-level features (instance) into a comprehensive bag representation. Building on this, we propose a two-streamed MIL framework with a prototype-guided aggregation method that effectively captures LN characteristics at both local and global scales. Furthermore, a multi-scale multi-source fusion module is introduced to integrate the heterogeneous meta-imaging features with deep learning features, enhancing the comprehensive representation of LN. Five-fold cross-validation on a cohort of 284 esophageal cancer patients with 809 pathology-confirmed LN instances demonstrate the superiority of our methods compared to the state-of-the-art approaches with +2.66% in AUROC and +4.81% in sensitivity improvements.

External IDs:dblp:conf/miccai/LiAWJYLDZYZJ25