DenoiSeg: Joint Denoising and Segmentation

Tim-Oliver Buchholz, Mangal Prakash, Deborah Schmidt, Alexander Krull, Florian Jug

TL;DR: DenoiSeg is a method to improve instance segmentation when only very little segmentation ground truth is available.

Reviews Visibility: The authors agree that reviews are made publicly visible, if the submission is accepted.

Abstract: Microscopy image analysis often requires the segmentation of objects, but training data for this task is typically scarce and hard to obtain. Here we propose DenoiSeg, a new method that can be trained end-to-end on only a few annotated ground truth segmentations. We achieve this by extending Noise2Void, a self-supervised denoising scheme that can be trained on noisy images alone, to also predict dense 3-class segmentations. The reason for the success of our method is that segmentation can profit from denoising, especially when performed jointly within the same network. The network becomes a denoising expert by seeing all available raw data, while co-learning to segment, even if only a few segmentation labels are available. This hypothesis is additionally fueled by our observation that the best segmentation results on high quality (very low noise) raw data are obtained when moderate amounts of synthetic noise are added. This renders the denoising-task non-trivial and unleashes the desired co-learning effect. We believe that DenoiSeg offers a viable way to circumvent the tremendous hunger for high quality training data and effectively enables learning of dense segmentations when only very limited amounts of segmentation labels are available.

Keywords: segmentation, denoising, microscopy, co-learning

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2005.02987/code)