Deep Audio Prior

Yapeng Tian, Chenliang Xu, Dingzeyu Li

TL;DR: a deep audio network that does not require any external training data

Abstract:

Deep convolutional neural networks are known to specialize in distilling compact and robust prior from a large amount of data. We are interested in applying deep networks in the absence of training dataset. In this paper, we introduce deep audio prior (DAP) which leverages the structure of a network and the temporal information in a single audio file. Specifically, we demonstrate that a randomly-initialized neural network can be used with carefully designed audio prior to tackle challenging audio problems such as universal blind source separation, interactive audio editing, audio texture synthesis, and audio co-separation.

To understand the robustness of the deep audio prior, we construct a benchmark dataset Universal-150 for universal sound source separation with a diverse set of sources. We show superior audio results than previous work on both qualitatively and quantitative evaluations. We also perform thorough ablation study to validate our design choices.

Code: https://iclr-dap.github.io/Deep-Audio-Prior/

Keywords: deep audio prior, blind sound separation, deep learning, audio representation

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1912.10292/code)