Explaining Convolutional Neural Networks via a Concise and Hierarchical Approach

jiawei pan, Xiangyu Zhu, Haoyuan Zhang, Stan Z. Li, Zhen Lei

Abstract: Explainable artificial intelligence (XAI) aims to bring transparency to black-box neural networks. Many innovative explainable methods provide rich and multifaceted explanations. However, these methods often have complex mechanisms or a high complexity of explanations, making them difficult for people to understand. To address this challenge, we introduce a concept-based explainable method which can improve the readability of deep neural network explanations by obtaining a concise set of high-quality explanations. We reduce the explanation redundancy by weighted hierarchical clustering, thereby obtaining a set of explanations that completely describe the input image and are crucial to the network's decision-making. Compared with existing XAI methods, our approach reduces the complexity while ensuring the integrity and comprehensibility of the explanation. In addition, we show that our approach can be traced back to the neuron-level explanation, which can also provide inspiration for model researchers to interpret the model. We validated the effectiveness of our approach through experiments in bird classification and facial recognition tasks. Specifically, we employed XAI to investigate the mechanisms behind face recognition, identifying critical neurons that correspond to key concepts in the recognition process.