Generating multilingual voices using speaker space translation based on bilingual speaker data

Soumi Maiti, Erik Marchi, Alistair Conkie

Abstract:

We present progress towards bilingual Text-to-Speech which is
able to transform a monolingual voice to speak a second language
while preserving speaker voice quality. We demonstrate that a bilingual speaker embedding space contains a separate distribution for
each language and that a simple transform in speaker space generated by the speaker embedding can be used to control the degree of
accent of a synthetic voice in a language. The same transform can
be applied even to monolingual speakers.
In our experiments speaker data from an English-Spanish (Mexican) bilingual speaker was used, and the goal was to enable English
speakers to speak Spanish and Spanish speakers to speak English.
We found that the simple transform was sufficient to convert a voice
from one language to the other with a high degree of naturalness. In
one case the transformed voice outperformed a native language voice
in listening tests. Experiments further indicated that the transform
preserved many of the characteristics of the original voice. The degree of accent present can be controlled and naturalness is relatively
consistent across a range of accent values.