Debiasing Concept-based Explanations with Causal Analysis

Mohammad Taha Bahadori, David Heckerman

Keywords: Interpretability, Concept-based Explanation

Abstract: Concept-based explanation approach is a popular model interpertability tool because it expresses the reasons for a model's predictions in terms of concepts that are meaningful for the domain experts. In this work, we study the problem of the concepts being correlated with confounding information in the features. We propose a new causal prior graph for modeling the impacts of unobserved variables and a method to remove the impact of confounding information and noise using a two-stage regression technique borrowed from the instrumental variable literature. We also model the completeness of the concepts set and show that our debiasing method works when the concepts are not complete. Our synthetic and real-world experiments demonstrate the success of our method in removing biases and improving the ranking of the concepts in terms of their contribution to the explanation of the predictions.

One-sentence Summary: We use a technique from instrumental variables literature and remove the impact of noise and latent confounding from concept-based explanations.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Data: [CUB-200-2011](https://paperswithcode.com/dataset/cub-200-2011)