The FineMotion entry to the GENEA Challenge 2023: DeepPhase for conversational gestures generation

Vladislav Korzun, Anna Beloborodova, Arkady Ilin

Keywords: embodied agents, neural networks, gesture generation, social robotics, deep learning, phase manifold

Abstract: This paper describes FineMotion's entry to the GENEA Challenge 2023. We explore the potential of DeepPhase embeddings by adapting neural motion controllers to conversational gesture generation. This is achieved by introducing a recurrent encoder for control features. We additionally use VQ-VAE codebook encoding of gestures to support dyadic setup. The resulting system generates stable realistic motion controllable by audio, text and interlocutor's motion.