A General Stitching Solution for Whole-Brain 3D Nuclei Instance Segmentation from Microscopy Images

Ziquan Wei, Guorong Wu

Keywords: Image stitching, 3D microscopy image, Whole-brain nucleus instance segmentation, Graph neural network.

TL;DR: We use contextual graph model to stitch partial nuclei instance by any deep learning model for 3D instance segmentation of nuclei in whole-brain microscopy images

Abstract: High-throughput 3D nuclei instance segmentation (NIS) is critical to understanding the complex structure and function of individual cells and their interactions within the larger tissue environment in the brain. Despite the significant progress in achieving accurate NIS within small image stacks using cutting-edge machine learning techniques, there has been a lack of effort to extend this approach towards whole-brain NIS. To address this challenge, we propose an efficient deep stitching neural network built upon a knowledge graph model characterizing 3D contextual relationships between nuclei. Our deep stitching model is designed to be agnostic, enabling existing limited methods (optimized for image stack only) to overcome the challenges of whole-brain NIS, particularly in addressing the issue of inter- and intra-slice gaps. We have evaluated the NIS accuracy on top of state-of-the-art deep models, such as Cellpose, with $128\times 128\times 64$ image stacks.