Training-Free Guidance with Applications to Protein Engineering

Lewis Cornwall, Joshua Meyers, James Day, Lilly S Wollman, Neil Dalchau, Aaron Sim

Keywords: inpainting, diffusion, proteins, langevin, sequence editing, floating inpainting

TL;DR: A novel and principled method for inpainting using unconditional diffusion models

Abstract: Diffusion models facilitate powerful control over the generative process. Here we introduce training-free guidance, a method for sampling from a broad class of conditional distributions that can be considered generalisations of inpainting. The method is grounded in annealed Langevin dynamics which ensures convergence to the exact conditional distribution, unlike popular methods for inpainting which rely on heuristics. We demonstrate training-free guidance using pretrained unconditional models for protein structure, and protein sequence generation and improve upon state-of-the-art approaches. We show the versatility of training-free guidance by addressing a wider range of tasks, including multi-motif scaffolding and amino acid mutagenesis of T cell receptors, with applications to biologics design.

Submission Number: 141