Interpretable Image Recognition by Screening Class-Specific and Class-Shared Prototypes

Xiaomeng Li, Jiaqi Wang, Liping Jing

Abstract: Convolutional neural networks (CNNs) have shown impressive performance in various domains, but their lack of interpretability remains an important issue. Prototype-based methods have been proposed to address this problem. Prototype-based methods assign a fixed number of prototypes to categories. But prototype networks are limited by the non-learnable relationship between prototypes and categories, which restricts each prototype to only one category. Furthermore, the large number of prototypes used in these methods often leads to poor prototype distribution. To address these limitations, we propose a deep learning approach with an active learning concept inspired by the associative function of the human brain. We introduce the Prototype Screening Matrix (PSM). We optimize PSM by label smoothing to describe the relationship between categories and prototypes, so that it can dynamically filter prototypes and retain prototypes that are more suitable for concept learning. PSM enables similar prototypes to be shared among different classes, which significantly reduces the number of prototypes and leads to a more rational distribution of prototypes. We experimentally validate the effectiveness of our proposed method on the CUB-200-2011 and Stanford Cars datasets and show that it achieves higher accuracy than existing methods. Our method is more interpretable, uses fewer prototypes, and has a simpler structure, advancing the state-of-the-art in interpretable and efficient prototype-based image classification methods. The code is available at https://github.com/Lixiaomemg/PSMnet.