Unsupervised Controllable Generation with Score-based Diffusion Models: Disentangled Latent Code Guidance

Yeongmin Kim, Dongjun Kim, HyeonMin Lee, Il-chul Moon

Keywords: Score-based Diffusion Model, Disentanglement, Controllable generation

TL;DR: We propose Latent Code Guidance Diffusion Model (LCG-DM), enabling unsupervised controllable generation in Score-based diffusion model.

Abstract: From the impressive empirical success of Score-based diffusion models, it is recently spotlighted in generative models. In real-world applications, the controllable generation enriches the impact of diffusion models. This paper aims to solve the challenge by presenting the method of control in an unsupervised manner. We propose the Latent Code Guidance Diffusion Model (LCG-DM), which is the first approach to apply disentanglement on Score-based diffusion models. Disentangled latent code can be considered as a pseudo-label, since it separately expresses semantic information in each dimension. LCG-DM is a Score-based diffusion model that reflects disentangled latent code as the condition. LCG-DM shows the best performance among baselines in terms of both sample quality and disentanglement on dSprites dataset. LCG-DM can manipulate images on CelebA dataset, with comparable FID performance compared to non-disentangling Score-based diffusion models. Furthermore, we provide experimental results of scaling method that reflects more on pseudo-label with MNIST dataset.

Student Paper: Yes