Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applications

Biel Tura Vecino, Adam Gabrys, Daniel Matwicki, Andrzej Pomirski, Tom Iddon, Marius Cotescu, Jaime Lorenzo-Trueba

Keywords: speech, text-to-speech, end-to-end, on-device, lightweight

TL;DR: We present an E2E TTS model for on-device applications that achieves comparable signal quality to SOTA TTS models while being up to 90% smaller and 10x faster.

Abstract:

Recent works have shown that modelling raw waveform directly from text in an end-to-end (E2E) fashion produces more natural-sounding speech than traditional neural text-to-speech (TTS) systems based on a cascade or two-stage approach. However, current E2E state-of-the-art models are computationally complex and memory-consuming, making them unsuitable for real-time offline on-device applications in low-resource scenarios. To address this issue, we propose a Lightweight E2E-TTS (LE2E) model that generates high-quality speech requiring minimal computational resources.
We evaluate the proposed model on the LJSpeech dataset and show that it achieves state-of-the-art performance while being up to 90% smaller in terms of model parameters and 10x faster in real-time-factor. Furthermore, we demonstrate that the proposed E2E training paradigm achieves better quality compared to an equivalent architecture trained in a two-stage approach.
Our results suggest that LE2E is a promising approach for developing real-time, high quality, low-resource TTS applications for on-device applications.