ST-DDPM: Explore Class Clustering for Conditional Diffusion Probabilistic Models

Zhijie Lin, Zijian Zhang, Zhou Zhao

Keywords: conditional generation, diffusion models, decoupling, interpretability

Abstract: Score-based generative models involve sequentially corrupting the data distribution with noise and then learns to recover the data distribution based on score matching. In this paper, for the diffusion probabilistic models, we first delve into the changes of data distribution during the forward process of the Markov chain and explore the class clustering phenomenon. Inspired by the class clustering phenomenon, we devise a novel conditional diffusion probabilistic model by explicitly modeling the class center in the forward and reverse process, and make an elegant modification to the original formulation, which enables controllable generation and gets interpretability. We also provide another direction for faster sampling and more analysis of our method. To verify the effectiveness of the formulated framework, we conduct extensive experiments on multiple tasks, and achieve competitive results compared with the state-of-the-art methods(conditional image generation on CIFAR-10 with an inception score of 9.58 and FID score of 3.05).

One-sentence Summary: we devise a novel conditional diffusion probabilistic model by explicitly modeling the class center, and make an elegant modification to the original formulation, which enables controllable generation and gets interpretability.