Improving the quality of neural TTS using long-form content and multi-speaker multi-style modeling

Tuomo Raitio, Javier Latorre, Andrea Davis, Tuuli Morrill, Ladan Golipour

Keywords: Speaking style modeling, multi-speaker modeling, long-form data, neural TTS

TL;DR: We show that we can improve neural TTS quality by using long-form content and multi-speaker multi-style modeling

Abstract: Neural text-to-speech (TTS) can provide quality close to natural speech if an adequate amount of high-quality speech material is available for training. However, acquiring speech data for TTS training is costly and time-consuming, especially if the goal is to generate different speaking styles. In this work, we show that we can transfer speaking style across speakers and improve the quality of synthetic speech by training a multi-speaker multi-style (MSMS) model with long-form recordings, in addition to regular TTS recordings. In particular, we show that 1) multi-speaker modeling improves the overall TTS quality, 2) the proposed MSMS approach outperforms pre-training and fine-tuning approach when utilizing additional multi-speaker data, and 3) long-form speaking style is highly rated regardless of the target text domain.