Interpretable Surrogate Models: A Clustering Approach for Gaussian Process Posteriors Using Mixed-Integer Quadratic Programming

ICLR 2025 Conference Submission1084 Authors

Keywords: Interpretability, Clustering, Gaussian Process Regression

Abstract:

Gaussian process regression is a flexible Bayesian method for capturing nonlinearity.
Although recent advancements allow us to handle various types of tasks by specifying a covariance function and a likelihood function, the interpretation of its predictions is sometimes challenging due to the large number of parameters.
In this study, we propose a clustering approach to improve the interpretability of Gaussian process posteriors.
Assuming that the parameters corresponding to data points within each cluster are identical, the number of parameters in the posterior distribution is reduced.
The assignment of data points to clusters is formulated as a mixed-integer quadratic programming problem, with the objective function being a weighted squared error from the mean of the posterior distribution approximated by variational inference.
Graph partitioning and decision tree learning can be represented by incorporating linear inequality constraints into this formulation.
Experimental results demonstrated that our approach provided significant advantages in enhancing the interpretability of spatial modeling.
Moreover, our formulation has produced higher-scoring decision trees compared to Classification and Regression Trees algorithm.

Primary Area: interpretability and explainable AI

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1084