Examining Why Perturbation-Based Fidelity Metrics are Inconsistent

Revoti Prasad Bora, Philipp Terhörst, Raymond Veldhuis, Raghavendra Ramachandra, Kiran Raja

Keywords: Fidelity Metric, Perturbation, Inconsistency, XAI, Explainability, Interpretability

Abstract: Saliency maps are commonly employed as a post-hoc method to explain the decision-making processes of Deep Learning models. Despite their widespread use, ensuring the fidelity of saliency maps is challenging due to the absence of ground truth. Therefore, researchers have developed fidelity metrics to evaluate the fidelity of saliency maps. However, prior investigations have uncovered statistical inconsistencies in existing fidelity metrics using multiple perturbation techniques without delving into the underlying causes. Our study aims to explore the origins of these observed inconsistencies. Our analysis examines the correctness of the assumptions made by the existing fidelity metrics using different types of perturbation to perturb the images. Our findings reveal that the assumptions made by existing fidelity metrics do not always hold true. Consequently, the existing fidelity metrics become inconsistent and unreliable. Thus, we recommend a cautious interpretation of fidelity metrics and the choice of perturbation technique when evaluating the fidelity of saliency maps in eXplainable Artificial Intelligence (XAI) applications.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4221