Sanity Checks for Saliency Metrics

Richard Tomsett, Supriyo Chakraborty, Alun D. Preece, Dave Braines, Gurram Poorna Prudhvi

Abstract:

Saliency maps are a popular approach to creating post-hoc
explanations of image classifier outputs. These methods pro-
duce estimates of the relevance of each pixel to the classi-
fication output score, which can be displayed as a saliency
map that highlights important pixels. Despite a proliferation
of such methods, little effort has been made to quantify how
good these saliency maps are at capturing the true relevance
of the pixels to the classifier output (i.e. their “fidelity”). We
therefore investigate existing metrics for evaluating the fi-
delity of saliency methods (i.e. saliency metrics). We find that
there is little consistency in the literature in how such metrics
are calculated, and show that such inconsistencies can have a
significant effect on the measured fidelity. Further, we apply
measures of reliability developed in the psychometric testing
literature to assess the consistency of saliency metrics when
applied to individual saliency maps. Our results show that
saliency metrics can be statistically unreliable and inconsis-
tent, indicating that comparative rankings between saliency
methods generated using such metrics can be untrustworthy.