
Unsupervised learning of meaningful disentanglement remains an open challenge. This problem has roughly two perpendicular objectives: dimensionality reduction of high-dimensional data (such as images) to a low-dimensional latent space, and enforcing a disentanglement structure on the obtained latent space. It has been shown that improved performance in one can potentially hurt the other i.e. increased disentanglement can reduce reconstruction quality. Previous works have developed various reformulations to better decouple these objectives but there is always still a connection which often requires hyperparameter search / tuning to find the right trade-off.
In this work, we propose a systematic approach that automatically adapts the relative weights of both components to obtain the right trade-off.
Based on the Factor VAE approach, we can adaptively increase or decrease the weight of disentanglement objective as a function of the discriminator performance. This makes the unsupervised learning process insensitive to the initial choice of hyperparameters.
Our approach also enables a learning curriculum that initially places focus on the reconstruction and adaptively shifts emphasis to learning disentanglements.

Progressive Disentanglement Using Relevant Factor VAE

A disentangled representation of a data set should be capable of recovering the underlying factors that generated it. One question that arises is whether using Euclidean space for latent variable models can produce a disentangled representation when the underlying generating factors have a certain geometrical structure. Take for example the images of a car seen from different angles. The angle has a periodic structure but a 1-dimensional representation would fail to capture this topology. How can we address this problem? The submissions presented for the first stage of the  NeurIPS2019 Disentanglement Challenge consist of a Diffusion Variational Autoencoder ($\Delta$VAE) with a hyperspherical latent space which can for example recover periodic true factors. The training of the $\Delta$VAE is enhanced by incorporating a modified version of the Evidence Lower Bound (ELBO) for tailoring the encoding capacity of the posterior approximate.

Disentanglement with Hyperspherical Latent Spaces using Diffusion Variational Autoencoders

Recent advances in deep learning techniques has shown the usefulness of the deep neural networks in extracting features required to perform the task at hand.
However, these features learnt are in particular helpful only for the initial task. This is due to the fact that the features learnt are very task specific and does not capture the most general and task agnostic features of the input.
In fact the way humans are seen to learn is by disentangling features which task agnostic. This indicates that leaning task agnostic features by disentangling only the most informative features from the input data.
Recently Variational Auto-Encoders (VAEs) have shown to be the de-facto models to capture the latent variables in a generative sense.
As these latent features can be represented as continuous and/or discrete variables, this indicates us to use VAE with a mixture of continuous and discrete variables for the latent space.
We achieve this by performing our experiments using a modified version of joint-vae to learn the disentangled features.


Neural Disentanglement using Mixture Latent Space with Continuous and Discrete Variables

It has long been speculated that deep neural networks function by discovering a hierarchical set of domain-specific core concepts or patterns, which are further combined to recognize even more elaborate concepts for the classification or other machine learning tasks. Meanwhile disentangling the actual core concepts engrained in the word embeddings (like word2vec or BERT) or deep convolutional image recognition neural networks (like PG-GAN) is difficult and some success there has been achieved only recently. In this paper we propose a novel neural network nonlinearity named Differentiable Disentanglement Filter (DDF) which can be transparently inserted into any existing neural network layer to automatically disentangle the core concepts used by that layer. The DDF probe is inspired by the obscure properties of the hyper-dimensional computing theory. The DDF proof-of-concept implementation is shown to disentangle concepts within the neural 3D scene representation - a task vital for visual grounding of natural language narratives.

Differentiable Disentanglement Filter: an Application Agnostic Core Concept Discovery Probe

We present a simple image preprocessing method for training VAEs leading to improved disentanglement compared to directly using the images. In particular, we propose to use regionally aggregated feature maps extracted from CNNs pretrained on ImageNet. Our method achieves the first rank on 3 of 5 metrics on the challenge’s public leaderboard.

Improved Disentanglement through Aggregated Convolutional Feature Maps

Disentangled encoding is an important step towards a better representation learning. However, despite the numerous efforts, there still is no clear winner that captures the independent features of the data in an unsupervised fashion. In this work we empirically evaluate the performance of six unsupervised disentanglement approaches on the mpi3d toy dataset curated and released for the NeurIPS 2019 Disentanglement Challenge. The methods investigated in this work are Beta-VAE, Factor-VAE, DIP-I-VAE, DIP-II-VAE, Info-VAE, and Beta-TCVAE. The capacities of all models were progressively increased throughout the training and the hyper-parameters were kept intact across experiments. The methods were evaluated based on five disentanglement metrics, namely, DCI, Factor-VAE, IRS, MIG, and SAP-Score. Within the limitations of this study, the Beta-TCVAE approach was found to outperform its alternatives with respect to the normalized sum of metrics. However, a qualitative study of the encoded latents reveal that there is not a consistent correlation between the reported metrics and the disentanglement potential of the model.

NeurIPS 2019 Workshop DC S1 Submissions

Progressive Disentanglement Using Relevant Factor VAE

Disentanglement with Hyperspherical Latent Spaces using Diffusion Variational Autoencoders

Neural Disentanglement using Mixture Latent Space with Continuous and Discrete Variables

Differentiable Disentanglement Filter: an Application Agnostic Core Concept Discovery Probe

Improved Disentanglement through Aggregated Convolutional Feature Maps

A Preliminary Study of Disentanglement With Insights on the Inadequacy of Metrics