Bayesian Optimization over Bounded Domains with Beta Product Kernels

Huy Hoang Nguyen, Han Zhou, Matthew B. Blaschko, Aleksei Tiulpin

Keywords: Bayesian optimization over bounded domains, Gassian process, Non-stationary kernel

TL;DR: We introduce a novel non-stationary boundary-aware kernel, constructed from products of Beta density functions, for Bayesian optimation over bounded domains.

Abstract: Bayesian Optimization with Gaussian Process (GP) and Matérn and Radial Basis Function (RBF) covariance functions is commonly used to optimize black-box functions. The Matérn and the RBF kernels do not make any assumptions about the domain of the function, which may limit their applicability in bounded domains. To address the limitation issue, we introduce a non-stationary Beta Unit Hyper-Cube (BUC) kernel, which is induced by a product of Beta distribution density functions, and allows to model functions on bounded domains. To provide theoretical insights, we provide analyses of information gain and cumulative regret bounds when using the GP Upper Confidence Bound (GP-UCB) algorithm with the BUC kernel. Our experiments show that the BUC kernel consistently outperforms the well-known Matérn and RBF kernels in different problems, including synthetic function optimization and the compression of vision and language models.

Submission Number: 65