Interactive Learning for Interpretable Visual Recognition via Semantic-Aware Self-Teaching Framework

Hao Jiang, Haowei Li, Junhao Chen, Wentao Wan, Keze Wang

Abstract: Aimed at tracing back the decision-making process of deep neural networks on the fine-grained image recognition task, various interpretable methods have been proposed and obtained promising results. However, the existing methods still have limited interpretability in aligning the abstract semantic concepts with the concrete image regions, due to the lack of human guidance during the model training. Attempting to address this issue and inspired by the machine teaching techniques, we formulate the training process of interpretable methods as an interactive learning manner by concisely simulating the human learning mechanism. Specifically, we propose a semantic-aware self-teaching framework to progressively improve the given neural network through an interactive teacher-student learning protocol. After initialing from the well-trained parameters of the given model, the teacher model focuses on minimally providing informative image regions to train the student model to generate interpretable predictions (i.e., semantic image regions) as good feedback. These feedback can encourage the teacher model to further refine the alignment of semantic concepts and image regions. Besides, our proposed framework is compatible with most of the existing network architectures. Extensive and comprehensive comparisons with the existing state-of-the-art interpretable approaches on the public benchmarks demonstrate that our interactive learning manner showcases an improved interpretability, a higher classification accuracy, and a greater degree of generality.