Constrained Synthesis with Projected Diffusion Models

Jacob K Christopher, Stephen Baek, Ferdinando Fioretto

Keywords: Constraint satisfaction, Generative diffusion models, physics-informed models

TL;DR: We propose an alteration of the sampling step in diffusion models to generate outputs that satisfy desired constraints and physical principles

Abstract:

This paper introduces an approach to endow generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional sampling process of generative diffusion models as a constrained optimization problem, steering the generated data distribution to remain within a specified region to ensure adherence to the given constraints.
These capabilities are validated on applications featuring both convex and challenging, non-convex, constraints as well as ordinary differential equations, in domains spanning from synthesizing new materials with precise morphometric properties, generating physics-informed motion, optimizing paths in planning scenarios, and human motion synthesis.

Primary Area: Diffusion based models

Submission Number: 18234