IA-CNN: A generalised interpretable convolutional neural network with attention mechanism

Zhisong Zhang, Yaran Chen, Haoran Li, Qichao Zhang

Abstract: In recent years, convolutional neural network (CNN) has been widely used in security, autonomous driving, and healthcare. Even though CNN has achieved a great performance, the results produced by CNN are difficult to explain and sometimes irresponsible. The black-box nature of CNN makes it lack trust. In this paper, we propose an attention based CNN structure, named IA -CNN, which highly improves the interpretability of the CNN models. Each feature map of the last conv-layer only has one response (one key point) of the target object, which is directly connected to the output. We also combine the attention mechanism to weakly supervise the last conv-layer. In this way, our model can clearly show that which features the model extracted are the keys to the output prediction. Meanwhile, our IA-CNN structure can be used in various classical models with higher performance in the fine-grained classification and comparative performance in the ordinary classification task. Note that our IA-CNN structure is an end-to-end model, the last conv-layer of which can extract key points from images automatically and is connected to the output prediction linearly.