Bayesian Optimization in High Dimensional Input Space

Jaemin Son, Samarth Gupta, Gary Tan

Abstract: Many simulations include design parameters that influence the terminal outcome. Often, our aim is to optimize these design parameters through recursive simulation in order to gain the outcome in favor of our interests. However, unlike usual objective functions, simulation cannot be directly expressed in analytic forms. Thus, conventional optimization methods such as gradient descent and convex optimization do not apply straightforwardly. Also, simulation usually incurs high computational costs. Therefore, less number of function evaluations is preferred during the optimization process. We present Bayesian Optimization (BO) to deal with such challenges in simulation-based optimization. BO constructs surrogates of the true function and evaluates the most promising points based on the surrogates. Also, we consider issues in applying BO to problems with high dimensional input space. Dealing with high dimensional input space is especially critical in simulation-based optimization if there exist multiple design parameters. In this paper, we discuss related theory and demonstrate experiments of BO on analytic functions.