Improving Zero-Shot Voice Style Transfer via Disentangled Representation LearningDownload PDF

28 Sept 2020, 15:50 (edited 17 Mar 2021)ICLR 2021 PosterReaders: Everyone
  • Keywords: Style Transfer, Mutual Information, Zero-shot Learning, Disentanglement
  • Abstract: Voice style transfer, also called voice conversion, seeks to modify one speaker's voice to generate speech as if it came from another (target) speaker. Previous works have made progress on voice conversion with parallel training data and pre-known speakers. However, zero-shot voice style transfer, which learns from non-parallel data and generates voices for previously unseen speakers, remains a challenging problem. In this paper we propose a novel zero-shot voice transfer method via disentangled representation learning. The proposed method first encodes speaker-related style and voice content of each input voice into separate low-dimensional embedding spaces, and then transfers to a new voice by combining the source content embedding and target style embedding through a decoder. With information-theoretic guidance, the style and content embedding spaces are representative and (ideally) independent of each other. On real-world datasets, our method outperforms other baselines and obtains state-of-the-art results in terms of transfer accuracy and voice naturalness.
  • One-sentence Summary: An information-theoretic disentangled representation learning framework for zero-shot voice style transfer.
  • Supplementary Material: zip
  • Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
9 Replies