Mixture Representation Learning with Coupled Autoencoders

Yeganeh Marghi, Rohan Gala, Uygar Sümbül

Keywords: Mixture representation, high-dimensional categorical variable, unsupervised learning, constrained variational framework, neuronal diversity, single-cell RNA sequencing, cell types

Abstract: Latent representations help unravel complex phenomena. While continuous latent variables can be efficiently inferred, fitting mixed discrete-continuous models remains challenging despite recent progress, especially when the discrete factor dimensionality is large. A pressing application for such mixture representations is the analysis of single-cell omic datasets to understand neuronal diversity and its molecular underpinnings. Here, we propose an unsupervised variational framework using multiple interacting networks called cpl-mixVAE that significantly outperforms state-of-the-art in high-dimensional discrete settings. cpl-mixVAE introduces a consensus constraint on discrete factors of variability across the networks, which regularizes the mixture representations at the time of training. We justify the use of this framework with theoretical results and validate it with experiments on benchmark datasets. We demonstrate that our approach discovers interpretable discrete and continuous variables describing neuronal identity in two single-cell RNA sequencing datasets, each profiling over a hundred cortical neuron types.

One-sentence Summary: We propose a novel unsupervised variational framework using multiple interacting autoencoders to discover interpretable discrete and continuous latent variables describing neuronal identity in single-cell omic datasets.