Practical and Rigorous Extremal Bounds for Gaussian Process Regression via Chaining
Keywords: Upper and lower bounds, Gaussian Process Regression, Chaining
TL;DR: We introduce a chain-based approach in Gaussian process regression that offers practical and theoretical bounds for common kernels, outperforming existing methods.
Abstract: Gaussian Process Regression (GPR) is a popular nonparametric regression method based on Bayesian principles that, unlike most machine learning techniques, provides uncertainty estimates for its predictions. Recent research has focused on robustness to model misspecification but has neglected improvements to the underlying methods for computing bounds. Inspired by the chaining method, we applied it to the prediction intervals of GPR. This work addresses the limitations of current GPR methods, which rely heavily on scaling posterior standard deviations and assume well-specified models, limiting their adaptability and accuracy. We propose a novel chain-based approach that decomposes the problem into smaller, refined stages, enabling better error control and enhanced robustness, particularly in challenging scenarios. Additionally, we innovate by providing tighter, practical and theoretically sound bounds for commonly used kernels, including RBF and Matérn, improving both their theoretical understanding and practical utility. Numerical experiments validate our theoretical findings, demonstrating that our method outperforms existing approaches on synthetic and real-world datasets.
Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 6727
Loading