Inherent Inconsistencies of Feature Importance

Nimrod Harel, Uri Obolski, Ran Gilad-Bachrach

Abstract:

The rapid advancement and widespread adoption of machine learning-driven technologies have underscored the practical and ethical need for creating interpretable artificial intelligence systems. Feature importance, a method that assigns scores to the contribution of individual features on prediction outcomes, seeks to bridge this gap as a tool for enhancing human comprehension of these systems. Feature importance serves as an explanation of predictions in diverse contexts, whether by providing a global interpretation of a phenomenon across the entire dataset or by offering a localized explanation for the outcome of a specific data point. Furthermore, feature importance is being used both for explaining models and for identifying plausible causal relations in the data, independently from the model. However, it is worth noting that these various contexts have traditionally been explored in isolation, with limited theoretical foundations.

This paper presents an axiomatic framework designed to establish coherent relationships among the different contexts of feature importance scores. Notably, our work unveils a surprising conclusion: when we combine the proposed properties with those previously outlined in the literature, we demonstrate the existence of an inconsistency. This inconsistency highlights that certain essential properties of feature importance scores cannot coexist harmoniously within a single framework.

Submission Track: Full Paper Track

Application Domain: None of the above / Not applicable

Clarify Domain: Theory of XAI

Survey Question 1: Our work tries to build a consistent framework across different applications of feature importance methods.

Survey Question 2: Methods lacking explainability can be problematic because they obscure the reasons for specific predictions, hindering their adaptations in critical applications like healthcare or finance. Furthermore, without explainability, it's difficult to identify and mitigate biases or errors in the model, making it crucial for ensuring fairness and accountability in AI systems.

Survey Question 3: Our work investigates the theory behind such methods. We explicitly exemplify some of our work on SHAP and MCI.

Submission Number: 26