Exemplar-based Stylized Gesture Generation from Speech: An Entry to the GENEA Challenge 2022

Saeed Ghorbani, Ylva Ferstl, Marc-André Carbonneau

Keywords: gesture generation, machine learning, style transfer, computer animation

TL;DR: We present a neural network framework for gesture generation from speech audio input with zero-shot style control by example.

Abstract: We present our entry to the GENEA Challenge of 2022 on data-driven co-speech gesture generation. Our system is a neural network that generates gesture animation from an input audio file. The motion style generated by the model is extracted from an exemplar motion clip. Style is embedded in a latent space using a variational framework. This architecture allows for generating in styles unseen during training. Moreover, the probabilistic nature of our variational framework furthermore enables the generation of a variety of outputs given the same input, addressing the stochastic nature of gesture motion. The GENEA challenge evaluation showed that our model produces full-body motion with highly competitive levels of human-likeness.