Shapley Is Not All You Need: Sobol's Total Indices for Feature Selection and Performance Loss Estimation

Zonghan Zhang, Zhiqian Chen

Keywords: Feature selection, Shapley values, Sobol indices, global sensitivity analysis, machine learning

TL;DR: Sobol total indices provide a more computationally efficient, comprehensive, and robust approach to quantifying feature importance for feature selection than the popular Shapley value method.

Abstract:

The selection of pertinent features constitutes a pivotal step in developing interpretable machine learning models, particularly when handling high-dimensional data, where the combinatorial interactions among features must be considered. The Shapley value, a concept originating from cooperative game theory, has gained recognition as a method for quantifying feature importance. However, the Shapley value often fails to precisely reflect the variance reduction that occurs when a feature is removed from the model. As the number of features increases, these challenges are further exacerbated by the high computational complexity of computing the exact Shapley value. Additionally, the common approximation techniques used to calculate the Shapley value are not model-agnostic.
To address these gaps, we propose utilizing Sobol's total indices, a variance-based sensitivity analysis technique, as a more efficient and robust alternative to Shapley values. In this paper, we present both theoretical and empirical studies comparing these two methods. Sobol's total indices provide several key advantages. It captures both main effects and interactions, offering a more accurate importance measure than Shapley values. Its computation scales linearly with the number of features, making it suitable for high-dimensional problems. Additionally, it is derived from the data itself, ensuring complete model-agnosticism.
Experiments on synthetic and real-world datasets demonstrate that feature selection using Sobol's total indices achieves better predictive performance than Shapley-based selection while requiring significantly less computational time. Our findings suggest that Sobol's total indices are a promising alternative to Shapley values, offering greater computational efficiency, comprehensiveness in accounting for interactions, and robustness in estimating variance. This represents a favorable substitute, particularly for high-dimensional feature selection.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 8533