Instability in Diffusion ODEs: An Explanation for Inaccurate Image Reconstruction

Han Zhang, Jinghong Mao, Shangwen Zhu, Zhantao Yang, Lianghua Huang, Yu Liu, Deli Zhao, Ruili Feng, Fan Cheng

Keywords: Diffusion models, Probability-Flow ODE, Learning theory

Abstract: Diffusion reconstruction plays a critical role in various applications such as image editing, restoration, and style transfer. In theory, the reconstruction should be simple, as it just inverts and regenerates images by numerically solving the Probability-Flow Ordinary Differential Equation (PF-ODE). Yet in practice, noticeable reconstruction errors have been observed, which cannot be well explained by numerical discretization error alone. In this work, we identify an intrinsic property of the PF-ODE generation process, the instability, that can further amplify the reconstruction errors. The root of this instability lies in the sparsity inherent in the generation distribution: the probability mass is concentrated on scattered small regions, while most of the space remains nearly empty. To demonstrate the existence of instability and its amplification on reconstruction error, we conduct experiments on both toy numerical examples and popular open-source diffusion models. Furthermore, based on the characteristics of image data, we theoretically prove that the probability of instability converges to one as the data dimensionality increases. Our findings clarify that instability, besides numerical errors, is a fundamental cause of inaccurate diffusion reconstruction, and offer insights for future improvements.

Primary Area: learning theory

Submission Number: 18932