Shapley Values of Structured Additive Regression Models and Application to RKHS Weightings of Functions

Published: 31 Jan 2025, Last Modified: 31 Jan 2025Accepted by TMLREveryoneRevisionsBibTeXCC BY 4.0
Abstract: Shapley values are widely used in machine learning to interpret model predictions. However, they have an important drawback in their computational time, which is exponential in the number of variables in the data. Recent work has yielded algorithms that can efficiently and exactly calculate the Shapley values of specific model families, such as Decision Trees and Generalized Additive Models (GAMs). Unfortunately, these model families are fairly restricted. Consequently, we present STAR-SHAP, an algorithm for efficiently calculating the Shapley values of Structured Additive Regression (STAR) models, a generalization of GAMs which allow any number of variable interactions. While the computational cost of STAR-SHAP scales exponentially in the size of these interactions, it is independent of the total number of variables. This allows the interpretation of more complex and flexible models. As long as the variable interactions are moderately-sized, the computation of the Shapley values will be fast, even on high-dimensional datasets. Since STAR models with more than pairwise interactions (e.g. GA2Ms) are seldom used in practice, we also present a new class of STAR models built on the RKHS Weightings of Functions paradigm. More precisely, we introduce a new RKHS Weighting instantiation, and show how to transform it and other RKHS Weightings into STAR models. We therefore introduce a new family of STAR models, as well as the means to interpret their outputs in a timely manner.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: Revision 4: Camera-ready version. Revision 3: We added a table of acronyms. Revision 2: We revised the abstract, and corrected Table 1 and its caption. Revision 1: The following sections have been added or modified: 1 Introduction: the last three paragraphs have been reworked. 2.2 Shapley values: Major additions to the section. 2.3 RKHS Weightings: Added clarifications to better define RKHS Weightings. 3 Shapley values of STAR models: The final paragraph before 3.1 explains the algorithmic complexity of our algorithm. 3.1 Comparison to other Shapley value algorithms: New section comparing our algorithm to others. 3.2 Applications of STAR-SHAP: New section describing models compatible with our algorithm. 5.1 Generating synthetic datasets: New section explaining how we generated random models and datasets. 5.2 Computation time of the Shapley values: Added multiple algorithms to the experiment. Added a Figure 1b. Adujusted the text in consequence. Added another experiment (Figure 2) which examines the quality of the Shapley values returned with regard to the compution time. 5.4 Time series prediction performance: The final paragraph is new, and places this experiment in the context of future research. 5.5 Shapley values comparison: New experiment (Figures 3 and 4). Comparison of the actual Shapley values of an Explainable Boosting Machine and a STAR RKHS Weighting. 5.6 Discussion: New section summarizing the conclusions we take from the experiments. We improved the language and fixed other mistakes, and added new citations where relevant.
Video: https://youtu.be/hQM7fJ_aBrQ
Code: https://github.com/gadub44/star-rkhs-weightings
Supplementary Material: zip
Assigned Action Editor: ~Dennis_Wei1
Submission Number: 3300
Loading