Multimodal Deep Fusion in Hyperbolic Space for Mild Cognitive Impairment Study

Lu Zhang, Saiyang Na, Tianming Liu, Dajiang Zhu, Junzhou Huang

Abstract: Multimodal fusion of different types of neural image data offers an invaluable opportunity to leverage complementary cross-modal information and has greatly advanced our understanding of mild cognitive impairment (MCI), a precursor to Alzheimer’s disease (AD). Current multi-modal fusion methods assume that both brain’s natural geometry and the related feature embeddings are in Euclidean space. However, recent studies have suggested that non-Euclidean hyperbolic space may provide a more accurate interpretation of brain connectomes than Euclidean space. In light of these findings, we propose a novel graph-based hyperbolic deep model with a learnable topology to integrate the individual structural network with functional information in hyperbolic space for the MCI/NC (normal control) classification task. We comprehensively compared the classification performance of the proposed model with state-of-the-art methods and analyzed the feature representation in hyperbolic space and its Euclidean counterparts. The results demonstrate the superiority of the proposed model in both feature representation and classification performance, highlighting the advantages of using hyperbolic space for multimodal fusion in the study of brain diseases. (Code is available here (https://github.com/nasyxx/MDF-HS).)