Prior Consistent CNN with Multi-Task Learning for Colon Image Classification

Chaoyang Yan, Chengfei Cai, Jiawei Xie, Yao Fu, Hui Shuai, Xiangshan Fan, Jun Xu

Keywords: Multi-Task Learning, Gland Prior Attention, Colorectal Cancer, Convolutional Neural Network

TL;DR: We encoded gland tissue regions as prior attention information for deep learning network, guiding the model’s preference for gland information when inferring.

Abstract: As adenocarcinoma is the most common cancer, the pathology diagnoses for it is of great significance. In the field of digital pathology, although deep learning method has achieved good results, it is theorem agnostic and the accumulated pathology-level knowledge is ignored. Specifically, the degree of gland differentiation is vital for defining the grade of adenocarcinoma. Following this domain knowledge, we encoded gland tissue regions as prior information in a multi-task convolutional neural network (CNN), guiding the network's preference for gland information when inferring. Firstly, we validated the effectiveness of the gland prior information by single task with gland ground truth annotations. Then we constructed a multi-task framework with segmentation and classification branches simultaneously. In this architecture, the segmentation probability map acted as the spatial attention for classification, emphasizing the region of gland and masking the noise of irrelevant parts. Experiments showed that the proposed prior consistent CNN with a multi-task learning method achieved 97.04% accuracy, compared with 93.82% of the single task classification model. Meanwhile, proposed multi-task model outputted gland tissue segmentation results. Most importantly, our model is based on the clinical-pathological diagnostic criteria of adenocarcinoma, which provides more ideas on how to make deep learning methods in the field of digital pathology more interpretable.