Inherent Inconsistencies of Feature Importance

Published: 27 Oct 2023, Last Modified: 27 Oct 2023NeurIPS XAIA 2023EveryoneRevisionsBibTeX
Abstract: The rapid advancement and widespread adoption of machine learning-driven technologies have underscored the practical and ethical need for creating interpretable artificial intelligence systems. Feature importance, a method that assigns scores to the contribution of individual features on prediction outcomes, seeks to bridge this gap as a tool for enhancing human comprehension of these systems. Feature importance serves as an explanation of predictions in diverse contexts, whether by providing a global interpretation of a phenomenon across the entire dataset or by offering a localized explanation for the outcome of a specific data point. Furthermore, feature importance is being used both for explaining models and for identifying plausible causal relations in the data, independently from the model. However, it is worth noting that these various contexts have traditionally been explored in isolation, with limited theoretical foundations. This paper presents an axiomatic framework designed to establish coherent relationships among the different contexts of feature importance scores. Notably, our work unveils a surprising conclusion: when we combine the proposed properties with those previously outlined in the literature, we demonstrate the existence of an inconsistency. This inconsistency highlights that certain essential properties of feature importance scores cannot coexist harmoniously within a single framework.
Submission Track: Full Paper Track
Application Domain: None of the above / Not applicable
Clarify Domain: Theory of XAI
Survey Question 1: Our work tries to build a consistent framework across different applications of feature importance methods.
Survey Question 2: Methods lacking explainability can be problematic because they obscure the reasons for specific predictions, hindering their adaptations in critical applications like healthcare or finance. Furthermore, without explainability, it's difficult to identify and mitigate biases or errors in the model, making it crucial for ensuring fairness and accountability in AI systems.
Survey Question 3: Our work investigates the theory behind such methods. We explicitly exemplify some of our work on SHAP and MCI.
Submission Number: 26