Fine-grained visual explanations for the convolutional neural network via class discriminative deconvolution

Nianwen Si, Wenlin Zhang, Dan Qu, Heyu Chang, Dongning Zhao

Abstract: Deep convolution neural networks have been widely studied and applied in many computer vision tasks. However, they are commonly treated as black-boxes and plagued by the inexplicability. In this paper, we propose a novel method to visually interpret the convolutional neural network in the field of image classification. Our method is capable of generating fine-grained and class discriminative heatmap that highlights the important input features contributing to specific predictions. Specifically, through the combination of the modified deconvolution and the pixel-wise Grad-CAM, the fine-grained heatmap and discriminative mask can be fused to achieve fine-grained deconvolution characteristics, and retain the class discriminativeness of the Grad-CAM, enhancing the interpretation effect of the heatmap. Both qualitative and quantitative experiments on ILSVRC 2012 dataset and PASCAL VOC 2012 dataset are conducted. The results indicate that the proposed method achieves a better visual effect with less noise in comparison to the previous methods, especially for visualising small objects in simple contexts. Furthermore, this method can realize a moderately effective performance on weakly supervised instance segmentation tasks, whereas the existing methods only work for weakly supervised object localisation.