Bayesian optimization with derivatives acceleration
Abstract: Bayesian optimization algorithms form an important class of methods to minimize functions that are costly to evaluate, which is a very common situation. These algorithms iteratively infer Gaussian processes from past observations of the function and decide where new observations should be made through the maximization of an acquisition criterion.
Often, the objective function is defined on a compact set such as in a hyper-rectangle of a $d$-dimensional real space,
and the bounds are chosen wide enough so that the optimum is inside the search domain.
In this situation, this work provides a way to integrate in the acquisition criterion the a priori information that
these functions, once modeled as GP trajectories, should be evaluated at their minima, and not at any point as usual acquisition criteria do. We propose an adaptation of the widely used Expected Improvement acquisition criterion that
accounts only for GP trajectories where the first order partial derivatives are zero and the Hessian matrix is positive definite.
The new acquisition criterion keeps an analytical, computationally efficient, expression.
This new acquisition criterion is found to improve Bayesian optimization on a test bed of functions made of Gaussian process trajectories in dimensions 2, 3 and 5. The addition of first and second order derivative information is particularly useful for multimodal functions.
Submission Length: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=SlZWTsnx8z
Changes Since Last Submission: The former version did not follow TMLR's stylefile format. This problem was fixed in the new version of the manuscript.
Assigned Action Editor: ~Jasper_Snoek1
Submission Number: 2540
Loading