BridgeVoC: Insights into Using Schrödinger Bridge for Neural Vocoders

Tong Lei, Andong Li, Rilin Chen, Dong Yu, Meng Yu, Jing Lu, Chengshi Zheng

Track: long paper (up to 8 pages)

Keywords: neural vocoder, Diffusion model, Schrödinger Bridge

TL;DR: BridgeVoC introduces a Schrödinger Bridge-based T-F domain vocoder, shifting from noise-to-data to data-to-data generation by restoring degraded mel-spectrograms, achieving faster inference and superior performance over existing methods.

Abstract: While previous diffusion-based neural vocoders typically follow a noise-to-data generation pipe-line, the linear-degradation prior of the mel-spectrogram is often neglected, resulting in limited generation quality. By revisiting the vocoder task and excavating its connection with the signal restoration task, this paper proposes a novel time-frequency (T-F) domain-based neural vocoder with the Schrödinger Bridge, called \textbf{BridgeVoC}, which is the first to follow the data-to-data generation paradigm. Specifically, the mel-spectrogram can be projected into the target linear-scale domain and regarded as a degraded spectral representation with a deficient rank distribution. Based on this, the Schrödinger Bridge is leveraged to establish a connection between the degraded and target data distributions. During the inference stage, starting from the degraded representation, the target spectrum can be gradually restored rather than generated from a Gaussian noise process. We conduct extensive experiments on the LJSpeech and LibriTTS benchmarks. Quantitative and qualitative results demonstrate that the proposed method enjoys faster inference speed and outperforms existing diffusion-based vocoder baselines, while also achieving competitive or better performance compared to other non-diffusion state-of-the-art methods across multiple evaluation metrics.

Submission Number: 63