Who are playing the games?

Yang An

Keywords: shapley values, model explainability

TL;DR: We show that one cannot get "efficient" Shapley values without correctly identifying the players (features) , and propose a solution to this conundrum

Abstract: The Shapley value has been widely used as the measures of feature importance of a predictive model, by treating a model as a cooperative game $(N, v)$. There have been many discussions on what the correct characteristic function $v$ should be, but almost all literature will take the player set $N$ as the set of features. While in classical cooperative game scenarios, players are obvious and well defined, it is not clear whether we should treat each feature individually as a player in machine learning. In fact, adding or deleting a feature, even a redundant one, will change every feature's Shapley value and its rank among all features in a non-intuitive way. To address this problem, we introduce a new axiom called ``Consistency", which characterizes the ``robustness" of computed Shapley-like values against different player set identifications, and is specific to machine learning setup. We show that while one can achieve Efficiency and Consistency in special cases, such as inessential games and 2-player games, they are contradictory to each other in general. This impossibility theorem signifies a conundrum of applying Shapley values in the feature selection process: The Shapley value is only axiomatically desirable if the players(features) are correctly identified, however, this prerequisite is exactly the purpose of the feature selection task. We then introduce the GroupShapley value to help address this dilemma, and as an additional bonus, GroupShapley valuess have a computational advantage over the classical Shapley values.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Theory (eg, control theory, learning theory, algorithmic game theory)