Reinforcement Learning based Disease Progression Model for Alzheimer’s Disease

Krishnakant V. Saboo, Anirudh Choudhary, Yurui Cao, Gregory Worrell, David T Jones, Ravi Iyer

Keywords: disease progression model, Alzheimer's disease, cognition trajectory prediction, reinforcement learning, differential equations

TL;DR: We model Alzheimer's disease progression by combining differential equation-based models with RL using domain knowledge to (i) predict individualized disease progression 10 years into the future, and (ii) understand processes underlying the disease.

Abstract: We model Alzheimer’s disease (AD) progression by combining differential equations (DEs) and reinforcement learning (RL) with domain knowledge. DEs provide relationships between some, but not all, factors relevant to AD. We assume that the missing relationships must satisfy general criteria about the working of the brain, for e.g., maximizing cognition while minimizing the cost of supporting cognition. This allows us to extract the missing relationships by using RL to optimize an objective (reward) function that captures the above criteria. We use our model consisting of DEs (as a simulator) and the trained RL agent to predict individualized 10-year AD progression using baseline (year 0) features on synthetic and real data. The model was comparable or better at predicting 10-year cognition trajectories than state-of-the-art learning-based models. Our interpretable model demonstrated, and provided insights into, "recovery/compensatory" processes that mitigate the effect of AD, even though those processes were not explicitly encoded in the model. Our framework combines DEs with RL for modelling AD progression and has broad applicability for understanding other neurological disorders.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.