Multi-field of View Aggregation and Context Encoding for Single-Stage Nucleus Recognition

Tian Bai, Jiayu Xu, Fuyong Xing

Abstract: Automated nucleus/cell recognition is a very challenging task, especially for differentiating tumor nuclei from non-tumor nuclei in Ki67 immunohistochemistry (IHC) stained images. Convolutional neural networks and their variants have been recently introduced to identify different types of nuclei and have achieved state-of-the-art performance. However, previous nucleus recognition approaches do not explicitly encode contextual information in the images, which can be very helpful for network representation learning. In this paper, we propose a novel multi-field-of-view context encoding method for single-stage nuclei identification in Ki67 IHC stained images. Specifically, we learn a deep structured regression model that takes multi-field of views of images as input and conducts feature aggregation on the fly for representation learning; then, we design a context encoding module to explicitly explore the multi-field-of-view contextual information and enhance the model’s representation power. In order to further improve nucleus recognition, we also introduce a novel deep regression loss that can emphasize specific channels of the prediction map with category-aware channel suppression. The proposed method can be learned in an end-to-end, pixel-to-pixel manner for single-stage nucleus recognition. We evaluate our method on a large-scale pancreatic neuroendocrine tumor image dataset, and the experiments demonstrate the superior performance of our method in nucleus recognition.