Self-supervised Learning Through Colorization for Microscopy Images

Vaidehi Pandey, Christoph Brune, Nicola Strisciuglio

Abstract: Training effective models for segmentation or classification of microscopy images is a hard task, complicated by the scarcity of adequately labeled data sets. In this context, self-supervised learning strategies can be deployed to learn suitable image representations from the available large quantity of unlabeled data, e.g. the 500k electron microscopy images that compose the CEM500k data sets. In this work, we investigate a self-supervised strategy for representation learning based on a colorization pre-text task on microscopy images. We integrate the colorization task into the BYOL (Bootstrap your own latent) self-supervised contrastive pre-training strategy. We train the self-supervised architecture on the CEM500k data set of electron microscopy images. As backbone of the BYOL framework, we investigate the use of Resnet50 and a Stand-alone Self-Attention network, and subsequently test them as feature extractors for downstream classification and segmentation tasks. The Self-Attention encoders pre-trained with the colorization-based BYOL method are able to learn effective features for segmentation of microscopy images, achieving higher results than those of encoders, both Resnet- and Self-Attention-based, trained with the original BYOL. This shows the effectiveness of colorization as pre-text for a downstream segmentation task on microscopy images. We release the code at https://github.com/nis-research/selfsup-byol-colorization .