Problems using deep generative models for probabilistic audio source separation

Maurice Frank, Maximilian Ilse

Keywords: generative models, deep learning, source separation

TL;DR: We find that deep generative models cannot be used for audio source separation using Langevin dynamics because they are either discriminative but too peaked or smooth and non-discriminative.

Abstract: Recent advancements in deep generative modeling make it possible to learn prior distributions from complex data that subsequently can be used for Bayesian inference. However, we find that distributions learned by deep generative models for audio signals do not exhibit the right properties that are necessary for tasks like audio source separation using a probabilistic approach. We observe that the learned prior distributions are either discriminative and extremely peaked or smooth and non-discriminative. We quantify this behavior for two types of deep generative models on two audio datasets.