Joint Source Separation and Classification Using Variational Autoencoders

Çaglar Hizli, Ertug Karamatli, Ali Taylan Cemgil, Serap Kirbiz

Abstract: In this paper, we propose a novel multi-task variational auto encoder (VAE) based approach for joint source separation and classification. The network uses a probabilistic encoder for each sources to map the input data to latent space. The latent representation is then used by a probabilistic decoder for the two tasks: source separation and source classification. Throughout a variety of experiments performed on various image and audio datasets, source separation performance of our method is as good as the method that performs source separation under source class supervision. In addition, the proposed method does not require the class labels and can predict the labels.