Beyond Pixel Space: Frequency-Domain Uncertainty for Structure Aware Diffusion Guidance

Tianqi Zhao, Xixi Liu, Yingzhen Li, Zhengrui Xiang, Liangrui Peng

Keywords: Diffusion Models, Frequency-domain Analysis, Uncertainty Estimation

Abstract: Although diffusion models achieve impressive generation quality, they inevitably produce samples with artifacts. Most existing approaches focus on quantifying pixel-level uncertainty to guide sample refinement. However, these methods erroneously assume independence among pixels. In this work, we turn to a frequency-based uncertainty metric that preserves structural information. We empirically show that samples with artifacts exhibit higher uncertainty, and further theoretically prove that the proposed uncertainty can be reinterpreted as an approximation of the optimal posterior covariance. To this end, we develop a structure-aware diffusion sampling guidance framework. For the mean of the reverse process, the gradient of the uncertainty is utilized to penalize specific frequency components; for the posterior covariance, we replace it with the proposed uncertainty. Experiments across U-Net, U-ViT, and SD3 architectures validate our approach.

Submission Number: 31