Multi-Modality Brain Disease Prediction with Progressive Curriculum Graph Learning

Bo Jiang, Xixi Wan, Beibei Wang, Zhengzheng Tu, XIAO WANG, AIHUA ZHENG

Keywords: Multi-Modality Learning, Multi-Modality Brain Disease Prediction, Context-Graph Representation, Curriculum Learning

TL;DR: We propose a novel unified framework for multi-modality brain disease prediction, which integrates the curriculum learning with dual graph convolution to better serve the feature representation learning within and between modalities.

Abstract:

Recently, graph-based multi-modality learning approaches have been studied to handle multi-modality medical brain data analysis.
Although they have achieved some promising performance, they still suffer from two main issues. First, current works generally fail to capture the inherent relationships of subjects (samples) from both feature and semantic/label perspectives. Second, for brain disease prediction tasks, the number of modalities is usually large (usually more than 4) and existing methods generally employ simple multi-modal fusion techniques that fail to carefully capture the dependencies of different modalities. To address these issues, this paper proposes a novel approach for multi-modality brain disease prediction by developing curriculum multi-modality learning. Our approach stems from observing that multi-modality learning becomes more challenging as the number of modalities increases, while recognizing curriculum learning as providing an explicit mechanism for tackling easy-to-hard learning tasks. This motivates us to propose a new progressive multi-modality learning strategy by leveraging the curriculum learning pipeline. Specifically, we first propose to dynamically learn a context-graph representation by jointly modeling the relationships of subjects from both feature and semantic label cues. Then, we propose a new multi-modality brain data representation by employing progressive curriculum learning. Experiments demonstrate the advantages of the proposed curriculum multi-modality learning strategy. The code of our method will be released upon acceptance.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 18939