HyperSHAP: Shapley Values and Interactions for Hyperparameter Importance

Marcel Wever, Maximilian Muschalik, Fabian Fumagalli, Marius Lindauer

Keywords: explainability, Shapley, interaction, hyperparameter optimization

TL;DR: We propose HyperSHAP, a framework for quantifying hyperparameter importance based on Shapley values and interactions.

Abstract:

Hyperparameter optimization (HPO) is a crucial step in achieving strong predictive performance, particularly for deep learning with hyperparameters controlling the neural architecture and learning behavior. However, the impact of some hyperparameters on model generalization can vary significantly depending on the dataset and performance measure, making it challenging to generalize their importance. Gaining a better understanding of the importance of hyperparameters is therefore important to deepen our understanding of machine learning and to leverage this knowledge in future downstream HPO tasks, especially if training is expensive and HPO needs to be as efficient as possible.
To address these challenges, we propose a game theoretic framework based on Shapley values and interactions for HPO. These methods offer an additive decomposition of a performance measure across hyperparameters, enabling both local and global explanations of hyperparameter importance and interactions. Our framework, named HyperSHAP, provides insights into ablation studies, tunability of specific hyperparameter configurations, and entire configuration spaces. Through experiments, we demonstrate that focusing on the hyperparameters deemed important by our framework can improve performance during subsequent hyperparameter optimization, while ignoring important hyperparameters or interactions degrades performance. This validates the effectiveness of our approach in enhancing model performance and providing meaningful, interpretable explanations of hyperparameter importance.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10064