Shapley Value-Based Approaches to Explain the Quality of Predictions by Classifiers

Guilherme Dean Pelegrina, Sajid Siraj

Abstract: The use of algorithm-agnostic approaches for explainable machine learning (ML) is an emerging area of research. When explaining the contribution of features toward the predicted outcome, traditionally, the focus remains on explaining the prediction itself, however a little has been done on explaining the quality of prediction of these models, where the quality can be assessed by the algorithm performance when changing the thresholds for classification. In this article, we propose the use of Shapley values to explain the contribution of features toward the overall algorithm performance, measured in terms of receiver operating characteristics (ROC) curve and the area under the ROC curve (AUC). With the help of an illustrative example, we demonstrate the proposed idea of explaining the ROC curve, and visualizing the uncertainties in these curves. For imbalanced datasets, the use of precision-recall curve (PRC) is considered more appropriate, therefore we also demonstrate how to explain the PRCs with the help of Shapley values. The explanation of the model performance can help analysts in a number of ways, for example, in feature selection by identifying the irrelevant features that can be removed to reduce the computational complexity. It can also help in identifying the features having critical contributions toward the overall algorithm performance.