REVEAL: Multimodal Vision–Language Alignment of Retinal Morphometry and Clinical Risks for Incident AD and Dementia Prediction

Seowung Leem, Lin Gu, Chenyu You, Kuang Gong, Ruogu Fang

Keywords: Retinal morphometry, risk factors, Alzheimer’s disease and related dementia, Vision-language alignment, Contrastive learning

TL;DR: REVEAL

Abstract: The retina provides a unique, noninvasive window into Alzheimer’s disease and related dementias (ADRD), capturing early structural changes through morphometric features, while systemic and lifestyle risk factors reflect well-established contributors to ADRD susceptibility long before clinical onset. However, current retinal analysis frameworks typically model imaging and risk-factor information separately, preventing them from capturing the joint multimodal patterns that are critical for early risk prediction. Moreover, existing methods rarely incorporate mechanisms to organize or align patients with similar retinal and clinical characteristics, limiting their ability to learn coherent cross-modal associations. To address these limitations, we introduce REVEAL ($\textbf{RE}$tinal–risk $\textbf{V}$ision–language $\textbf{E}$arly $\textbf{A}$lzheimer's $\textbf{L}$earning) that aligns color fundus photographs with individualized disease-specific risk profiles for incident AD and dementia prediction $-$ on average 8 years before diagnosis (range: 1–11 years). Because real-world risk factors are structured questionnaire data, we first translate them into clinically interpretable narratives compatible with pretrained vision–language models (VLMs). We further propose a group-aware contrastive learning (GACL) strategy that clusters patients with similar retinal morphometry and risk factors as positive pairs, strengthening multimodal alignment. This unified representation-learning framework substantially outperforms state-of-the-art retinal imaging models paired with clinical text encoders, as well as general VLMs, demonstrating the value of jointly modeling retinal biomarkers and clinical risk factors. By providing a generalizable, noninvasive approach for early ADRD risk stratification, REVEAL has the potential to enable earlier interventions and improve preventive care at the population level.

Primary Subject Area: Integration of Imaging and Clinical Data

Secondary Subject Area: Unsupervised Learning and Representation Learning

Registration Requirement: Yes

Visa & Travel: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 139