An Interpretable Multi-scale Deep Network for Structure Localization in Medical Images

Yongfei Liu, Bo Wan, Xuming He

Abstract:

Anatomical structure localization plays an important role in image-based medical diagnosis. Despite recent progress achieved by deep learning methods, most existing neural networks for localization lack an interpretable inference process and thus are difficult to use for critical diagnosis in practice.
In this paper, we propose an interpretable multi-scale convolutional networks for structure localization in medical images. Our network employs a modularized architecture consisting of a local branch to encode structure features and a context branch to capture global cues. The local branch adopts a coarse-to-fine strategy to refine the localization. And within each step, it learns a linear voting scheme based on a set of visual landmarks. The context branch uses a deformable pooling to encode contextual anatomical structures for reducing local ambiguities in localization. Given a prediction, we are able to trace back and determine which features are involved and their importance.
We validate the proposed strategy on a Nuchal Translucency (NT) dataset, and the results demonstrate that our method is capable of generating an interpretable localization process and achieves the state-of-the-art detection performance.

Keywords: Interpretable structure localization, Coarse-to-fine, Nuchal Translucency

TL;DR: We propose an interpretable voting-based deep network for structure localization task, which contains a coarse-to-fine strategy and a context module coping with noisy medical images and fine structure.