Controlled Denoising For Diffusion Models
Keywords: Generative Models, Computer Vision, Diffusion Models, Guidance
TL;DR: By combining block-wise optimal sampling with an adjustable noise conditioning strategy, C-CoDe offers extra control over reward vs. divergence trade-off outperforming state-of-the-art baselines.
Abstract: Aligning diffusion models for downstream tasks often requires finetuning new models or costly inference-time solutions (e.g., gradient-based guidance) to allow sampling from the reward-tilted posterior. In this work, we explore a simple and low-cost inference-time gradient-free guidance approach, called conditional controlled denoising (C-Code), that circumvents the need for differentiable guidance functions and model finetuning. C-Code is a block-wise sampling method with adjustable conditioning on a reference image applied during intermediate denoising steps, allowing for efficient alignment with downstream rewards. Experiments demonstrate that, despite its simplicity, C-Code offers a balanced trade-off between reward alignment, prompt instruction following, and inference cost, outperforming state-of-the-art baselines. Our code is available at: https://anonymous.4open.science/r/CoDe-Repo.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 11160
Loading