Learning Generative Models with Locally Disentangled Latent Factors

Brady Neal, Alex Lamb, Sherjil Ozair, Devon Hjelm, Aaron Courville, Yoshua Bengio, Ioannis Mitliagkas

Abstract: One of the most successful techniques in generative models has been decomposing a complicated generation task into a series of simpler generation tasks. For example, generating an image at a low resolution and then learning to refine that into a high resolution image often improves results substantially. Here we explore a novel strategy for decomposing generation for complicated objects in which we first generate latent variables which describe a subset of the observed variables, and then map from these latent variables to the observed space. We show that this allows us to achieve decoupled training of complicated generative models and present both theoretical and experimental results supporting the benefit of such an approach.

TL;DR: Decompose the task of learning a generative model into learning disentangled latent factors for subsets of the data and then learning the joint over those latent factors.

Keywords: Generative Models, Hierarchical Models, Latent Variable Models