Self-supervised multimodal learning for group inferences from MRI data: Discovering disorder-relevant brain regions and multimodal links

Alex Fedorov, Eloy Geenjaar, Lei Wu, Tristan Sylvain, Thomas P. DeRamus, Margaux Luck, Maria B. Misiura, Girish Mittapalle, R. Devon Hjelm, Sergey M. Plis, Vince D. Calhoun

Abstract: Highlights•Self-supervised models reveal group discriminative regions.•Self-supervised models reveal multimodal links between regions.•Multi-scale multimodal relationships improve classification performance.•Costly training of a decoder for reconstruction is not required.•A rigorous evaluation framework enabled by an exhaustive taxonomy.