DProtoNet: Decoupling Prototype Activation via Multiple Dynamic Masks

Yitao Peng, Yihang Liu, Longzhen Yang, Lianghua He

Abstract: The interpretability of neural networks has recently received extensive attention. The previous prototype-based explainable networks involved prototype activation in both the reasoning process and interpretation process, which requires specific explainable structures for the prototype. This makes the network less accurate as it gains interpretability. To avoid this problem, we propose a new model: decoupling prototypical network (DProtoNet), which contains three modules. 1) encoder module: we propose unrestricted masks to generate expressive features and prototypes. 2) inference module: we propose a multi-image prototype learning method to update prototypes so that the network can learn generalized prototypes. 3) interpretation module: we propose multiple dynamic masks (MDM) decoder to explain the network, which generates heatmaps using the consistent activation of the original image and mask image at the detection nodes of the network. It decouples the inference module and interpretation module of a prototype-based network by avoiding the use of prototype activation to explain the network's decisions, so that the accuracy and interpretability of the network can be simultaneously improved. We test on multiple public general and medical datasets. The accuracy of our method is improved compared with the previous methods, which can be improved by up to 5%. DProtoNet achieves state-of-the-art interpretability.