Memory U-Net: Memorizing Where to Vote for Lesion Instance Segmentation

Hang Zhang, Jinwei Zhang, Gufeng Yang, Pascal Spincemaille, Thanh D. Nguyen, Yi Wang

Keywords: Memory Network, Hough Voting, Instance Segmentation, Multiple Sclerosis, U-Net

TL;DR: A novel memory network for multiple sclerosis lesion instance segmentation.

Abstract:

Confluent lesions usually occur when pathologically distinct lesions grow close to each other and form a large spatially-connected lesion.
These confluent lesions pose a great challenge for subsequent image analysis and disease diagnosis, as individual lesions are difficult to separate and segment.
In this paper, we propose a Memory U-Net that takes advantage of recent fully convolutional neural network U-Net and memory networks, to resolve the issue.
The main idea is that we develop a hybrid model with a U-Net for feature extraction and a memory network as the alternative code book for generalized Hough voting.
To alleviate the GPU memory overhead brought by the large code book, we decompose the large code book into three smaller ones, where each one of them accounts for voting in one specific direction.
Through voxel-wise voting, a density map of lesion locations can be obtained by aggregating votes from all lesion voxels, and this density map is further used to generate final instance segmentation results.
Experiments on a large-scale cross-sectional multiple sclerosis study verify the efficiency and the effectiveness of the proposed method.

Registration: I acknowledge that publication of this at MIDL and in the proceedings requires at least one of the authors to register and present the work during the conference.

Authorship: I confirm that I am the author of this work and that it has not been submitted to another publication before.

Paper Type: both

Primary Subject Area: Segmentation

Secondary Subject Area: Application: Radiology

Source Code Url: https://github.com/tinymilky/Memory-UNet