HiFi-VC: High Quality ASR-based Voice Conversion

Anton Kashkin, Ivan Karpukhin, Svyatoslav Shishkin

Keywords: Any-to-any voice conversion, speech synthesis

TL;DR: A new voice conversion system with ASR feature extractor and joint decoder-vocoder architecture

Abstract: The goal of voice conversion is to convert input voice to match the target speaker's voice while keeping text and prosody intact. Voice conversion is usually used in entertainment and speaking-aid systems, as well as applied for speech data generation and augmentation. The development of any-to-any voice conversion systems, which are capable of generating voices unseen during model's training, is of particular interest to both researchers and the industry. Despite recent progress, any-to-any conversion quality is still inferior to natural speech. In this work, we propose a new any-to-any voice conversion pipeline, that combines ASR feature extraction, the joint decoder-vocoder model, and GAN training objective. According to multiple subjective and objective evaluations, our method outperforms modern systems in terms of voice quality, similarity and consistency.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/hifi-vc-high-quality-asr-based-voice/code)