Interpretability Benchmark for Evaluating Spatial Misalignment of Prototypical Parts Explanations

Mikolaj Sacha, Bartosz Jura, Dawid Rymarczyk, Lukasz Struski, Jacek Tabor, Bartosz Zielinski

Abstract: Prototypical parts-based networks are becoming increasingly popular due to their faithful self-explanations. However, their similarity maps are calculated in the penultimate network layer. Therefore, the receptive field of the prototype activation region often depends on parts of the image outside this region, which can lead to misleading interpretations. We name this undesired behavior a spatial explanation misalignment and introduce an interpretability benchmark with a set of dedicated metrics for quantifying this phenomenon. In addition, we propose a method for misalignment compensation and apply it to existing state-of-the-art models. We show the expressiveness of our benchmark and the effectiveness of the proposed compensation methodology through extensive empirical studies.