Augmenting Magnetic Resonance Imaging with Tabular Features for Enhanced and Interpretable Medial Temporal Lobe Atrophy Prediction

Dongsoo Lee, Chong Hyun Suh, Jinyoung Kim, Wooseok Jung, Changhyun Park, Kyu-Hwan Jung, Seo Taek Kong, Woo-Hyun Shim, Hwon Heo, Sang Joon Kim

Abstract: Medial temporal lobe atrophy (MTA) score is a key feature for Alzheimer’s disease (AD) diagnosis. Diagnosis of MTA from images acquired using magnetic resonance imaging (MRI) technology suffers from high inter- and intra-observer discrepancies. The recently-developed Vision Transformer (ViT) can be trained on MRI images to classify MTA scores, but is a “black-box” model whose internal working is unknown. Further, a fully-trained classifier is also susceptible to inconsistent predictions by nature of its labels used for training. Augmenting imaging data with tabular features could potentially rectify this issue, but ViTs are designed to process imaging data as its name suggests. This work aims to develop an accurate and explainable MTA classifier. We introduce a multi-modality training scheme to simultaneously handle tabular and image data. Our proposed method processes multi-modality data consisting of T1-weighted brain MRI and tabular data encompassing brain region volumes, cortical thickness, and radiomics features. Our method outperforms various baselines considered, and its attention map on input images and feature importance scores on tabular data explains its reasoning.