Combining deep learning and model-based segmentation for labeled spine CT segmentation

Christian Bürger, Jens von Berg, Astrid Franz, Tobias Klinder, Cristian Lorenz, Matthias Lenga

Abstract: Automatic instance segmentation of individual vertebrae from 3D CT is essential for various applications in orthopedics, neurology, and oncology. In case model-based segmentation (MBS) shall be used to generate a mesh-based representation of the spine, a good initialization of MBS is crucial to avoid wrong vertebra labels due to the similar appearance of adjacent vertebrae. Here, we propose to use deep learning (DL) for MBS initialization and for robustly guiding MBS during segmentation to generate 24 instance segmentations for each and every vertebra. We propose a four-step approach: In step 1, we apply a first single-class U-Net to coarsely segment the spine. In step 2, we sample image patches along the coarse segmentation of step 1 and apply a second multi-class U-net to generate a fine segmentation including individual labeling of some key vertebrae and vertebra body landmarks. In step 3, we detect and label landmark coordinates from the classes estimated in step 2. In step 4, we initialize all MBS vertebrae models using the landmarks from step 3 and adapt the model to the joint vertebrae probability map from step 2. We validated our method on segmentation results from 147 patient images. We computed surface distances between segmentation and ground truth meshes and achieved root mean squared distances of RMSDist = 0.90 mm over all cases and vertebrae.