Exemplar-based Stylized Gesture Generation from Speech: An Entry to the GENEA Challenge 2022Download PDF

Published: 25 Oct 2022, Last Modified: 05 May 2023GENEA Challenge & Workshop 2022 MainproceedingReaders: Everyone
Keywords: gesture generation, machine learning, style transfer, computer animation
TL;DR: We present a neural network framework for gesture generation from speech audio input with zero-shot style control by example.
Abstract: We present our entry to the GENEA Challenge of 2022 on data-driven co-speech gesture generation. Our system is a neural network that generates gesture animation from an input audio file. The motion style generated by the model is extracted from an exemplar motion clip. Style is embedded in a latent space using a variational framework. This architecture allows for generating in styles unseen during training. Moreover, the probabilistic nature of our variational framework furthermore enables the generation of a variety of outputs given the same input, addressing the stochastic nature of gesture motion. The GENEA challenge evaluation showed that our model produces full-body motion with highly competitive levels of human-likeness.
3 Replies