PIE: Simulating Disease Progression via Progressive Image Editing

Kaizhao Liang, Xu Cao, Kuei-Da Liao, Tianren Gao, Wenqian Ye, Zhengyu Chen, Jianguo Cao, Tejas Nama, Jimeng Sun

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Disease Progression, Medical Imaging

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Disease progression trajectories can greatly affect the quality and efficacy of clinical diagnosis, prognosis, and treatment. However, one major challenge is the lack of longitudinal medical imaging monitoring of individual patients over time. To address this issue, we propose Progressive Image Editing (PIE) method that enables controlled manipulation of disease-related image features, facilitating precise and realistic disease progression simulation in imaging space. Specifically, we leverage recent advancements in text-to-image generative models to simulate disease progression accurately and personalize it for each patient. We also theoretically analyze the iterative refining process in our framework as a gradient descent with an exponentially decayed learning rate. To validate our framework, we conduct experiments in three medical imaging domains. Our results demonstrate the superiority of PIE over existing methods such as Stable Diffusion Video and Style-Based Manifold Extrapolation based on CLIP score (Realism) and Disease Classification Confidence (Alignment). Our user study collected feedback from 35 veteran physicians to assess the generated progressions. Remarkably, 76.2% of the feedback agrees with the fidelity of the generated progressions. PIE can allow healthcare providers to model disease imaging trajectories over time, predict future treatment responses, fill in missing imaging data in clinical records, and improve medical education. Anonymous code for replicating our results can be found at https://anonymous.4open.science/r/PIE-3332.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 815