TransGesture: Autoregressive Gesture Generation with RNN-TransducerDownload PDF

Published: 25 Oct 2022, Last Modified: 05 May 2023GENEA Challenge & Workshop 2022 MainproceedingReaders: Everyone
Track: Challenge paper
Team Name: TransGesture
Keywords: gesture generation, speech audio, neural networks, deep learning
Abstract: This paper presents a gesture generation model based on an RNN-transducer, submitted to the GENEA Challenge 2022. The proposed model consists of three neural networks: Encoder, Prediction Network, and Joint Network, which can be jointly trained in an end-to-end manner. We also introduce new loss functions, namely statistical losses, as the additional term to the standard MSE loss to put motion statistics of generated gestures close to the ground truths'. Finally, we show the subjective evaluation results and discuss the results and takeaways from the challenge.
4 Replies