A-FMI: Learning Attributions from Deep Networks via Feature Map Importance

An Zhang, Xiang Wang, Chengfang Fang, Jie Shi, Xiangnan He, Tat-seng Chua, Zehua Chen

Keywords: Feature Attribution, Convolutional Neural Networks, Explanation Methods

Abstract: Gradient-based attribution methods can aid in the understanding of convolutional neural networks (CNNs). However, the redundancy of attribution features and the gradient saturation problem, which weaken the ability to identify significant features and cause the explanation focus shift, remain open challenges. In this work, we propose 1) an essential characteristic, Relevant, when selecting attribution features; 2) a new concept, feature map importance (FMI), to refine the contribution of each feature map, which is faithful to the CNN model; and 3) a novel attribution method via FMI, termed A-FMI, to address the gradient saturation problem, which couples the target image with a reference image, and assigns the FMI to the “difference-from-reference” at the granularity of feature map. Through visual inspections and qualitative evaluations of common models on the ImageNet dataset, we show the compelling advantages of A-FMI on faithfulness, insensitivity to the choice of reference, class discriminability, and the superior explanation performance compared with popular attribution methods.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Reviewed Version (pdf): https://openreview.net/references/pdf?id=YEcE_3eFkt