On Locality of Local Explanation ModelsDownload PDF

21 May 2021, 20:42 (edited 22 Jan 2022)NeurIPS 2021 PosterReaders: Everyone
  • Keywords: Shapley values, model interpretability, reference distribution, neighbourhood sampling
  • TL;DR: We consider the formulation of neighbourhood reference distributions that improve the local interpretability of Shapley values.
  • Abstract: Shapley values provide model agnostic feature attributions for model outcome at a particular instance by simulating feature absence under a global population distribution. The use of a global population can lead to potentially misleading results when local model behaviour is of interest. Hence we consider the formulation of neighbourhood reference distributions that improve the local interpretability of Shapley values. By doing so, we find that the Nadaraya-Watson estimator, a well-studied kernel regressor, can be expressed as a self-normalised importance sampling estimator. Empirically, we observe that Neighbourhood Shapley values identify meaningful sparse feature relevance attributions that provide insight into local model behaviour, complimenting conventional Shapley analysis. They also increase on-manifold explainability and robustness to the construction of adversarial classifiers.
  • Supplementary Material: pdf
  • Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
  • Code: https://medium.com/@lucile.terminassian?p=4c279fb57974
14 Replies

Loading