Progressive Disentanglement Using Relevant Factor VAE

Anonymous

Abstract:

Unsupervised learning of meaningful disentanglement remains an open challenge. This problem has roughly two perpendicular objectives: dimensionality reduction of high-dimensional data (such as images) to a low-dimensional latent space, and enforcing a disentanglement structure on the obtained latent space. It has been shown that improved performance in one can potentially hurt the other i.e. increased disentanglement can reduce reconstruction quality. Previous works have developed various reformulations to better decouple these objectives but there is always still a connection which often requires hyperparameter search / tuning to find the right trade-off.
In this work, we propose a systematic approach that automatically adapts the relative weights of both components to obtain the right trade-off.
Based on the Factor VAE approach, we can adaptively increase or decrease the weight of disentanglement objective as a function of the discriminator performance. This makes the unsupervised learning process insensitive to the initial choice of hyperparameters.
Our approach also enables a learning curriculum that initially places focus on the reconstruction and adaptively shifts emphasis to learning disentanglements.