Variational Search Distributions

Daniel M. Steinberg, Rafael Oliveira, Cheng Soon Ong, Edwin V. Bonilla

Keywords: black box optimization, Bayesian optimization, variational inference, generative models, level set estimation

TL;DR: A method for learning a generative model of designs of a rare class sequentially

Abstract: We develop variational search distributions (VSD), a method for finding and generating discrete, combinatorial designs of a rare desired class in a batch sequential manner with a fixed experimental budget. We formalize the requirements and desiderata for active generation and formulate a solution via variational inference. In particular, VSD uses off-the-shelf gradient based optimization routines, can learn powerful generative models for designs, and can take advantage of scalable predictive models. We empirically demonstrate that VSD can outperform existing baseline methods on a set of real sequence-design problems in various biological systems.

Submission Number: 30