DiffuGesture: Generating Human Gesture From Two-person Dialogue With Diffusion Models

Weiyu Zhao, Liangxiao Hu, Shengping Zhang

Keywords: gesture generation, diffusion models, neural networks

Abstract: This paper describes the DiffuGesture entry to the GENEA Challenge 2023. In this paper, we utilize conditional diffusion models to formulate the gesture generation problem. The DiffuGesture system generates human-like gestures from the two-person dialogue scenario, which are responsive to the interlocutor motions and accompany with the input speech. DiffuGesture system is built upon the recent DiffGesture [39]. Specifically, we introduce a lightweight transformer encoder to fuse the temporal relationships between human gestures and multi-modal conditions. Moreover, we adopt implicit classifier-free guidance to trade off between diversity and gesture quality. According to the collective evaluation released by GENEA Challenge 2023, our system demonstrates strong competitiveness in the appropriateness evaluation.