DermX: a Dermatological Diagnosis Explainability Dataset

Raluca Jalaboi, Mauricio Orbes Arteaga, Dan Richter Jørgensen, Ionela Manole, Oana Ionescu-Bozdog, Andrei Chiriac, Ole Winther, Alfiia Galimzianova

Keywords: dataset, explainability, dermatology, diagnosis, segmentation

TL;DR: We introduce DermX, a novel dermatological diagnosis explainability dataset annotated by multiple expert dermatologists, and benchmark GradCAM visualisations from two ConvNet architectures against the dermatologist explanations.

Abstract: In this paper, we introduce DermX: a novel dermatological diagnosis and explanations dataset annotated by eight board-certified dermatologists. To date, public datasets for dermatological applications have been limited to diagnosis and lesion segmentation, while validation of dermatological explainability has been limited to visual inspection. As such, this work is a first release of a dataset providing gold standard explanations for dermatological diagnosis to enable a quantitative evaluation of ConvNet explainability. DermX consists of 525 images sourced from two public datasets, DermNetNZ and SD-260, spanning six of the most prevalent skin conditions. Each image was enriched with diagnoses and diagnosis explanations by three dermatologists. Supporting explanations were collected as 15 non-localisable characteristics, 16 localisable characteristics, and 23 additional terms. Dermatologists manually segmented localisable characteristic and described them with additional terms. We showcase a possible use of our dataset by benchmarking the explainability of two ConvNet architectures, ResNet-50 and EfficientNet-B4,trained on an internal skin lesion dataset and tested on DermX. ConvNet visualisations are obtained through gradient-weighted class-activation map (Grad-CAM), a commonly used model visualisation technique. Our analysis reveals EfficientNet-B4 as the most explainable between the two. Thus, we prove that DermX can be used to objectively benchmark the explainability power of dermatological diagnosis models. The dataset is available at https://github.com/ralucaj/dermx.

URL: https://github.com/ralucaj/dermx