Deep Learning for Bayesian Optimization of Scientific Problems with High-Dimensional Structure
Abstract: Bayesian optimization (BO) is a popular paradigm for global optimization of expensive black-box functions, but there are many domains where the function is not completely black-box. The data may have some known structure (e.g. symmetries) and/or the data generation process can yield useful intermediate or auxiliary information in addition to the value of the optimization objective. However, surrogate models traditionally employed in BO, such as Gaussian Processes (GPs), scale poorly with dataset size and do not easily accommodate known structure or auxiliary information. Instead, we propose performing BO on complex, structured problems by using deep learning models with uncertainty, a class of scalable surrogate models that have the representation power and flexibility to handle structured data and exploit auxiliary information. We demonstrate BO on a number of realistic problems in physics and chemistry, including topology optimization of photonic crystal materials using convolutional neural networks, and chemical property optimization of molecules using graph neural networks. On these complex tasks, we show that neural networks often outperform GPs as surrogate models for BO in terms of both sampling efficiency and computational cost.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Added additional comparison experiments with BOHAMIANN (which uses SGHMC to approximate the BNN) and a latent-space method by Tripp et al (which uses a JTVAE to encode the chemical molecule).
Assigned Action Editor: ~Tom_Rainforth1
Submission Number: 4
Loading