Spatial Broadcast Decoder: A Simple Architecture for Disentangled Representations in VAEs

Nick Watters, Loic Matthey, Chris P. Burgess, Alexander Lerchner

Keywords: disentangling, VAE, coordconv, representation learning, untangling

TL;DR: We introduce a neural rendering architecture that helps VAEs learn disentangled latent representations.

Abstract: We present a neural rendering architecture that helps variational autoencoders (VAEs) learn disentangled representations. Instead of the deconvolutional network typically used in the decoder of VAEs, we tile (broadcast) the latent vector across space, concatenate fixed X- and Y-“coordinate” channels, and apply a fully convolutional network with 1x1 stride. This provides an architectural prior for dissociating positional from non-positional features in the latent space, yet without providing any explicit supervision to this effect. We show that this architecture, which we term the Spatial Broadcast decoder, improves disentangling, reconstruction accuracy, and generalization to held-out regions in data space. We show the Spatial Broadcast Decoder is complementary to state-of-the-art (SOTA) disentangling techniques and when incorporated improves their performance.