Robust Models Are More Interpretable Because Attributions Look Normal

Zifan Wang, Matt Fredrikson, Anupam Datta

Keywords: Explainability, Decision Boundary, Attribution, Adversarial Robustness

Abstract: Recent work has found that adversarially-robust deep networks used for image classification are more interpretable: their feature attributions tend to be sharper, and are more concentrated on the objects associated with the image's ground-truth class. We show that smooth decision boundaries play an important role in this enhanced interpretability, as the model's input gradients around data points will more closely align with boundaries' normal vectors when they are smooth. Thus, because robust models have smoother boundaries, the results of gradient-based attribution methods will capture more accurate information about nearby decision boundaries. This understanding of robust interpretability leads to our second contribution: \emph{boundary attributions}, which aggregate information about the normal vectors of local decision boundaries to explain a classification outcome. We show that by leveraging the key factors underpinning robust interpretability, boundary attributions produce sharper, more concentrated visual explanations---even on non-robust models.

One-sentence Summary: Boundary Attributions as a means of providing geometrical-based explanation for deep classifiers