StarGAN-VC++: Towards Emotion Preserving Voice Conversion Using Deep EmbeddingsDownload PDF

Published: 15 Jun 2023, Last Modified: 27 Jun 2023SSW12Readers: Everyone
Keywords: voice conversion, emotion preservation, StarGAN
TL;DR: An emotion preserving voice conversion using GAN based model and deep emotion embedding
Abstract: Voice conversion (VC) transforms an utterance to sound like another person without changing the linguistic content. A recently proposed generative adversarial network-based VC method, StarGANv2-VC is very successful in generating natural-sounding conversions. However, the method fails to preserve the emotion of the source speaker in the converted samples. Emotion preservation is necessary for natural human-computer interaction. In this paper, we show that StarGANv2-VC fails to disentangle the speaker and emotion representations, pertinent to preserve emotion. Specifically, there is an emotion leakage from the reference audio used to capture the speaker embeddings while training. To counter the problem, we propose novel emotion-aware losses and an unsupervised method which exploits emotion supervision through latent emotion representations. The objective and subjective evaluations prove the efficacy of the proposed strategy over diverse datasets, emotions, gender, etc.
Supplementary Material: zip
3 Replies