Improving dermatology classifiers across populations using images generated by large diffusion models

Luke William Sagers, James Allen Diao, Matthew Groh, Pranav Rajpurkar, Adewole Adamson, Arjun Kumar Manrai

Keywords: computer vision, dermatology, synthetic data, diffusion models, fairness

TL;DR: We show that targeted generation of synthetic images by DALL·E 2 can be used to improve the performance for dermatological classifiers on a benchmark dataset overall and particularly for underrepresented groups.

Abstract: Dermatological classification algorithms developed without sufficiently diverse training data may generalize poorly across populations. While intentional data collection and annotation offer the best means for improving representation, new computational approaches for generating training data may also aid in mitigating the effects of sampling bias. In this paper, we show that DALL·E 2, a large-scale text-to-image diffusion model, can produce photorealistic images of skin disease across skin types. Using the Fitzpatrick 17k dataset as a benchmark, we demonstrate that augmenting training data with DALL·E 2-generated synthetic images improves classification of skin disease overall and especially for underrepresented groups.