Combining Deep Learning and Active Contours Opens The Way to Robust, Automated Analysis of 3D Brain Cytoarchitectonics

Konstantin Thierbach, Pierre-Louis Bazin, Walter de Back, Filippos Gavriilidis, Evgeniya Kirilina, Carsten Jaeger, Markus Morawski, Stefan Geyer, Nikolaus Weiskopf, Nico Scherf

Abstract: Deep learning has thoroughly changed the field of image analysis yielding impressive results whenever enough annotated data can be gathered. While partial annotation can be very fast, manual segmentation of 3D biological structures is tedious and error-prone. Additionally, high-level shape concepts such as topology or boundary smoothness are hard if not impossible to encode in Feedforward Neural Networks. Here we present a modular strategy for the accurate segmentation of neural cell bodies from light-sheet microscopy combining mixed-scale convolutional neural networks and topology-preserving geometric deformable models. We show that the network can be trained efficiently from simple cell centroid annotations, and that the final segmentation provides accurate cell detection and smooth segmentations that do not introduce further cell splitting or merging. The cell detection stage works sufficiently robust to even uncover actual errors in the reference annotations.

Keywords: Cell Segmentation, 3D Histology, Active Contours, Convolutional Neural Networks

Author Affiliation: Max Planck Institute for Human Cognitive and Brain Sciences, Spinoza Centre for Neuroimaging, Netherlands Institute for Neuroscience Amsterdam, Technische Universität Dresden, Center for Cognitive Neuroscience Berlin, Paul Flechsig Institute of Brain Research