One step further with Monte-Carlo sampler to guide diffusion better

Minsi Ren, Wenhao Deng, Ruiqi Feng, Tailin Wu

Keywords: Conditional Generation; Diffusion Model; Training-free Guidance

Abstract:

Stochastic differential equation (SDE)-based generative models have achieved
substantial progress in conditional generation via training-free differentiable
loss-guided approaches. However, existing methodologies utilizing posterior sam-
pling typically confront a substantial estimation error, which results in inaccurate
gradients for guidance and leading to inconsistent generation results. To mitigate
this issue, we propose that performing an additional backward denoising step and
Monte-Carlo sampling (ABMS) can achieve better guided diffusion, which is a
plug-and-play adjustment strategy. To verify the effectiveness of our method, we
provide theoretical analysis and propose the adoption of a dual-evaluation frame-
work, which further serves to highlight the critical problem of cross-condition
interference prevalent in existing approaches. We conduct experiments across var-
ious task settings and data types, mainly including conditional online handwritten
trajectory generation, image inverse problems (inpainting, super resolution and
gaussian deblurring), and molecular inverse design. Experimental results demon-
strate that our approach consistently improves the quality of generation samples
across all the different scenarios.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 4118