Fine-Grained Quantitative Emotion Editing for Speech Generation

Sho Inoue, Kun Zhou, Shuai Wang, Haizhou Li

Abstract: It remains a significant challenge how to quantitatively control the expressiveness of speech emotion in speech generation. In this work, we propose an approach for quantitative manipulation of the emotion rendering for emotion editing in speech generation. We apply a hierarchical emotion distribution extractor, i.e. Hierarchical ED, that quantifies the intensity of emotions at different levels of granularity. Hierarchical ED is subsequently integrated into the FastSpeech2 framework, guiding the model to learn emotion intensity at phoneme, word, and utterance levels. During synthesis, users can manually edit the emotional intensity of the generated voices. Both objective and subjective evaluations demonstrate the effectiveness of the proposed network in terms of fine-grained quantitative emotion editing.