Angle Domain Guidance: Latent Diffusion Requires Rotation Rather Than Extrapolation

Cheng Jin, Zhenyu Xiao, Chutao Liu, Yuantao Gu

TL;DR: A diffusion guidance method by enhancing in the angular domain, improving text-image alignment and image quality.

Abstract: Classifier-free guidance (CFG) has emerged as a pivotal advancement in text-to-image latent diffusion models, establishing itself as a cornerstone technique for achieving high-quality image synthesis. However, under high guidance weights, where text-image alignment is significantly enhanced, CFG also leads to pronounced color distortions in the generated images. We identify that these distortions stem from the amplification of sample norms in the latent space. We present a theoretical framework that elucidates the mechanisms of norm amplification and anomalous diffusion phenomena induced by classifier-free guidance. Leveraging our theoretical insights and the latent space structure, we propose an Angle Domain Guidance (ADG) algorithm. ADG constrains magnitude variations while optimizing angular alignment, thereby mitigating color distortions while preserving the enhanced text-image alignment achieved at higher guidance weights. Experimental results demonstrate that ADG significantly outperforms existing methods, generating images that not only maintain superior text alignment but also exhibit improved color fidelity and better alignment with human perceptual preferences.

Lay Summary: Diffusion model is a powerful AI technique for generating images from text descriptions. A key component in this process is Classifier-Free Guidance (CFG), which helps the model better align images with the input text. However, when the guidance weight is set high, the generated images often break down—appearing overly saturated or distorted. We analyzed why this happens and found that CFG tends to push image features too strongly in a linear fashion. To address this, we propose a new method called Angle-Domain Guidance (ADG), which provides more stable control by adjusting the direction of updates rather than their intensity. This leads to more reliable and visually consistent images, improving the usability of AI-generated content in real-world applications.

Link To Code: https://github.com/jinc7461/ADG

Primary Area: Deep Learning->Generative Models and Autoencoders

Keywords: Diffusion Model, Classifier Free Guidance, Text-to-Image Generation

Submission Number: 10986