Explaining the Inconsistency of Perturbation-Based Fidelity Metrics

Revoti Prasad Bora, Philipp Terhörst, Raymond Veldhuis, Raghavendra Ramachandra, Kiran Raja

Keywords: XAI, Fidelity Metrics, Saliency Metrics, Deep Learning, Interpretability

Abstract: Saliency maps are one of the most widely used post-hoc approaches for interpreting the behavior of Deep Learning models. Yet, assessing their fidelity is difficult in the absence of ground-truth explanations. To address this, numerous fidelity metrics have been introduced. Previous studies have shown that fidelity metrics can behave inconsistently under different perturbations, and a recent work has attempted to estimate the extent of this inconsistency. However, the underlying reasons behind these observations have not been systematically explained. In this work, we revisit this problem and analyze why such inconsistencies arise. We examine several representative fidelity metrics, apply them across diverse models and datasets, and compare their behavior under multiple perturbation types. To formalize this analysis, we introduce two conformity measures that test the assumptions implicit in existing metrics. Our results show that these assumptions often break down, explaining the observed inconsistencies and calling into question the reliability of current practices. We therefore recommend careful consideration of both metric choice and perturbation design when employing fidelity evaluations in eXplainable Artificial Intelligence (XAI).

Primary Area: interpretability and explainable AI

Submission Number: 7816