CMIB: A Cross Modal Interaction with Information Bottleneck Framework for Multimodal Survival Analysis

Zhouyou Lai, Lifan Long, Jiaqi Cui, Deng Xiong, Yuanjun Liu, Hao Yin, Yan Wang

Abstract: Recently, multimodal survival analysis that integrates histology images and genomic data has become a hot topic. Existing multimodal survival analysis methods have evolved from direct fusion strategies to cross-modal attention mechanisms to incorporate multimodal features. However, these methods ignore the redundancy and noise in the fusion features. To solve this problem, we introduced a Cross Modal Interaction with Information Bottleneck (CMIB) framework for multimodal survival analysis, which filters out redundancy and noise while exploring the latent complementary information across modalities. Specifically, CMIB uses the Private Feature Extraction Block (PFEB) and Common Feature Extraction Block (CFEB) to extract the private and the common features of different modalities, respectively. Subsequently, it captures the deep interactions between these features through Co-Attention (CA). Additionally, a Multimodal Information Bottleneck (MIB) is employed to yield a robust representation of the fused features. To verify the effectiveness of CMIB, we conducted extensive experiments on three public TCGA datasets. The results show that CMIB outperforms the current state-of-the-art methods.