Sufficient and Necessary Explanations (and What Lies in Between)

ICLR 2025 Conference Submission12912 Authors

Keywords: Explainability, Interpretability, Trustworthiness

TL;DR: This paper explores notions of sufficiency, necessity, and their unification for explaining how general machine learning models make predictions.

Abstract: As complex machine learning models continue to find applications in high-stakes decision making scenarios, it is crucial that we can explain and understand their predictions. Post-hoc explanation methods can provide useful insights by identifying important features in an input ${\bf x}$ with respect to the model output $f({\bf x})$. In this work we formalize and study two precise notions of feature importance for general machine learning models: \emph{sufficiency} and \emph{necessity}. We demonstrate how these two types of explanations, albeit intuitive and simple, can fall short in providing a complete picture of which features a model deems important for its predictions. To this end, we propose a unified notion of importance that circumvents these limitations by exploring a continuum along a necessity-sufficiency axis. Our unified notion, we show, has strong ties to other popular definitions of feature importance, like those based on conditional independence and game-theoretic quantities like Shapley values. Crucially, we demonstrate how studying this spectrum of importance allows us to detect important features that could be missed by either of the previous approaches alone.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 12912