Enabling efficient experimental design in the context of high-dimensional generative models

Samuel Willis, Alexandru I. Stere, Dragos D Margineantu, Henry T. Oldroyd, John A Fozard, Carl Henrik Ek, Henry Moss, Erik Bodin

Keywords: generative models, latent space, surrogate latent space, latent space optimisation, black-box optimisation, experimental design, latent optimal linear combinations

TL;DR: We introduce surrogate latent spaces and show how they enable effective optimisation for use in experimental design

Abstract:

Many scientific optimisation tasks require finding designs consistent with physical or experimental constraints. Generative models such as diffusion models have proven to be powerful tools to propose designs, but remain challenging to control, often requiring significant model-specific procedures and compute to meet the constraints. We introduce surrogate latent spaces --- Euclidean subspaces defined by examples --- that enable standard optimisation algorithms to search for designs efficiently; including when the objective function is a black-box, non-differentiable, or expensive to evaluate, e.g. be evaluated through simulation or real-world experiments.
We outline the underlying principles governing surrogate spaces, and demonstrate that the approach allows generating protein backbone designs using RFdiffusion with a sequence length that was previously infeasible.

Release To Public: Yes, please release this paper to the public

Submission Number: 34