Bayesian Optimization on the Cartesian Product of Weighted Graphs to Better Search Discrete Spaces with Irregular Increments
Abstract: Bayesian optimization is a powerful tool for optimizing a black-box function on a compact Euclidean space under a limited evaluation budget. However, in practice, we may want to optimize over discretization of the solution space. For example, in scientific and engineering problems the discretization of the solution space naturally occurs due to measurement precision or standardized parts. In this work, we consider the problem of optimizing a black-box function with a discretized solution space. To address this problem, prior work uses Bayesian optimization on the Cartesian product of graphs. We extend this work to weighted edges which allow us to exploit the problem structure more effectively. Our proposed method outperforms earlier methods in diverse experiments including neural architecture search benchmarks and physics-based simulations with discretized solution spaces. We also investigate the impact of adding multi-hop edges to weighted graphs, which improves performance of our method on the optimization of synthetic benchmark functions.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Based on the Reviewer's comments and suggestions, we have made the following improvements to our paper:
* Added Probabilistic Reparameterization [1] as a baseline
* Changed metrics for synthetic functions to simple regrets
* Moved *the section "Related Work"* to *Section 2*
* Added *the paragraph "Bayesian Optimization with Prior Knowledge"* in *Section 2*
* Enhanced *the paragraph "Analysis on Numerical Results by Weighted Graphs"* in *Section 6*
* Added *the section "Computational Costs for Calculating Eigenvalues and Eigenvectors"*
* Added a description of simple regrets in *Figures 2 and 6*
* Revised minor issues
[1] Samuel Daulton, et al. Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization. Advances in Neural Information Processing Systems, 2022.
Assigned Action Editor: ~Michael_U._Gutmann1
Submission Number: 1206
Loading