Local Style Tokens: Fine-Grained Prosodic Representations For TTS Expressive Control

Martin Lenglet, Olivier Perrotin, Gérard Bailly

Keywords: speech synthesis, expressive TTS, style control, prosody modeling

TL;DR: Training of an unsupervised set of local style tokens to provide a fine-grained prosodic control.

Abstract: Neural Text-To-Speech (TTS) models achieve great performances regarding naturalness, but modeling expressivity remains an ongoing challenge. Some success was found through implicit approaches like Global Style Tokens (GST), but these methods model speech style at utterance-level. In this paper, we propose to add an auxiliary module called Local Style Tokens (LST) in the encoder-decoder pipeline to model local variations in prosody. This module can implement various scales of representations; we chose Word-level and Phoneme-level prosodic representations to assess the capabilities of the proposed module to better model sub-utterance style variations. Objective evaluation of the synthetic speech shows that LST modules better capture prosodic variations on 12 common styles compared to a GST baseline. These results were validated by participants during listening tests.