Soundness and Completeness: An Algorithmic Perspective on Evaluation of Feature Attribution

Yawei Li, Yang Zhang, Bernd Bischl, Mina Rezaei

Keywords: explainable AI, explainability, feature attribution

TL;DR: We propose a novel method to evaluate \emph{soundness} and \emph{completeness} of feature attribution methods.

Abstract: Feature attribution is a fundamental approach to explaining neural networks by quantifying the importance of input features for a model's prediction. Although a variety of feature attribution methods have been proposed, there is little consensus on the assessment of attribution methods. In this study, we empirically show the limitations of \emph{order-based} and \emph{model-retraining} metrics. To overcome the limitations and enable evaluation with higher granularity, we propose a novel method to evaluate the \emph{completeness} and \emph{soundness} of feature attribution methods. Our proposed evaluation metrics are mathematically grounded on algorithm theory and require no knowledge of "ground truth" informative features. We validate our proposed metrics by conducting experiments on synthetic and real-world datasets. Lastly, we use the proposed metrics to benchmark a wide range of feature attribution methods. Our evaluation results provide an innovative perspective on comparing feature attribution methods. Code is in the supplementary material.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Social Aspects of Machine Learning (eg, AI safety, fairness, privacy, interpretability, human-AI interaction, ethics)