Noise Re-sampling for High Fidelity Image Generation

ICLR 2025 Conference Submission2505 Authors

Keywords: Generative models, Diffusion based models

TL;DR: A Noise Re-sampling plugin for current LDMs, which enhances the generation quality of LDMs by enabling multi-scale generation and preserving high-frequency information

Abstract:

Latent diffusion models (LDMs) have emerged as powerful tools for generating diverse and realistic samples across domains. However, their efficacy in capturing intricate details and small-scale objects remains a challenge.
Our investigation reveals that VAE compression induces errors in the latent space and limits the generation quality. Furthermore, LDMs trained on fixed-resolution images struggle to produce high-resolution outputs without distortions, making simple resolution increases ineffective.
In this paper, we propose a novel **noise re-sampling** strategy that enables multi-scale generation of LDMs, allowing LDMs to "zoom in" and improve generation quality of local regions. By increasing the sampling rates from the noise perspective in the latent space, we effectively bypass the constraints imposed by VAE compression, thus preserving crucial high-frequency information. Our approach, a simple yet effective plugin for current LDMs, enhances the quality of image generation in local regions while maintaining overall structural consistency and providing fine-grained control over the scale of generation in latent diffusion models.
Through extensive experimentation and evaluation, we demonstrate the efficacy of our method in enhancing the generation quality across various LDM architectures. Our approach surpasses existing methods, including stable diffusion (SD) models, SD-based super-resolution methods and high-resolution adaptation methods, in generating high-fidelity samples of complex objects.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2505