Conditional Diffusion Replay for Continual Learning in Medical Settings

Yewon Byun, Saurabh Garg, Sanket Vaibhav Mehta, Praveer Singh, Jayashree Kalpathy-cramer, Bryan Wilder, Zachary Chase Lipton

Keywords: Distribution Shift, Continual Learning, Generative Models, Medical Imaging

TL;DR: A text-to-image generative replay approach for continual learning in medical settings

Abstract:

Episodic replay methods, which store and replay past data, have proven effective
for handling distribution shifts in continual learning.
However, due to regulatory and privacy concerns for data sharing,
their applicability can be limited, especially in healthcare.
In this work, we advance the state of art,
focusing our inquiry on two novel benchmarks
for domain incremental continual learning:
diabetic retinopathy severity classification
and dermoscopy skin lesion detection.
First, we demonstrate the poor forward and backward transferability
of simple baselines.
Then, to overcome these challenges,
we propose a novel method called conditional diffusion replay.
By leveraging a text-to-image diffusion model for synthetic data generation,
our approach effectively preserves
performance on previously encountered domains
while adapting to new ones.
We observe that compared to standard sequential fine-tuning, our conditional diffusion replay method improves average AUC by up to 7.3 points and 3.3 points for the skin lesions and diabetic retinopathy benchmarks, respectively.

Submission Number: 64