A Multiresolution Convolutional Neural Network with Partial Label Training for Annotating Reflectance Confocal Microscopy Images of Skin.

Alican Bozkurt, Kivanç Köse, Christi Alessi-Fox, Melissa Gill, Jennifer G. Dy, Dana H. Brooks, Milind Rajadhyaksha

Abstract: We describe a new multiresolution “nested encoder-decoder” convolutional network architecture and use it to annotate morphological patterns in reflectance confocal microscopy (RCM) images of human skin for aiding cancer diagnosis. Skin cancers are the most common types of cancers, melanoma being the most deadly among them. RCM is an effective, non-invasive pre-screening tool for skin cancer diagnosis, with the required cellular resolution. However, images are complex, low-contrast, and highly variable, so that it requires months to years of expert-level training for clinicians to be able to make accurate assessments. In this paper we address classifying 4 key clinically important structural/textural patterns in RCM images. The occurrence and morphology of these patterns are used by clinicians for diagnosis of melanomas. The large size of RCM images, the large variance of pattern size, the large scale range over which patterns appear, the class imbalance in collected images, and the lack of fully-labelled images all make this a challenging problem to address, even with automated machine learning tools. We designed a novel nested U-net architecture to cope with these challenges, and a selective loss function to handle partial labeling. Trained and tested on 56 melanoma-suspicious, partially labelled, 12k $$\times $$ 12k pixel images, our network automatically annotated RCM images for these diagnostic patterns with high sensitivity and specificity, providing consistent labels for unlabelled sections of the test images. We believe that providing such annotation in a fast manner will aid clinicians in achieving diagnostic accuracy, and perhaps more important, dramatically facilitate clinical training, thus enabling much more rapid adoption of RCM into widespread clinical use process. In addition our adaptation of U-net architecture provides an intrinsically multiresolution deep network that may be useful in other challenging biomedical image analysis applications.