A GPT-4o-powered framework for identifying cognitive impairment stages in electronic health records

Yu Leng, Yingnan He, Samad Amini, Colin Magdamo, Ioannis Paschalidis, Shibani S. Mukerji, Lidia MVR Moura, M Brandon Westover, Ana-Maria Vranceanu, Christine S Ritchie, Deborah Blacker, John R Dickson, Sudeshna Das

Abstract: Alzheimer’s Disease and Related Dementias (ADRD) pose a major public health challenge, with a critical need for accurate and scalable tools for detecting cognitive impairment (CI). Readily available electronic health records (EHRs) contain valuable cognitive health data, but much of it is embedded in unstructured clinical notes. To address this problem, we developed a GPT-4o-powered framework for CI stage classification, leveraging longitudinal patient history summarization, multi-step reasoning, and confidence-aware decision-making. Evaluated on 165,926 notes from 1002 Medicare patients from Mass General Brigham (MGB), our GPT-4o framework achieved high accuracy in CI stage classification (weighted Cohen’s kappa = 0.95, Spearman correlation = 0.93), and outperformed two other language models (weighted Cohen’s kappa 0.82–0.85). Our framework also achieved high performance on Clinical Dementia Rating (CDR) scoring on an independent dataset of 769 memory clinic patients (weighted Cohen’s kappa = 0.83). Finally, to ensure reliability and safety, we designed an interactive AI agent integrating our GPT-4o-powered framework and clinician oversight. This collaborative approach has the potential to facilitate CI diagnoses in real-world clinical settings.