Reconstruction for Generation: Regularizing Motion Diffusion Models with Motion Reconstruction

Yifei Liu, Changxing Ding, Ling Guo, Huaiguang Jiang, Qiong Cao

Keywords: human motion generation; text-to-motion generation; diffusion model

TL;DR: MORGEN improves the motion diffusion models with a self-regularized motion latent space and introduces a novel Reconstructive Error Guidance. These designs enable improved semantic alignment and reduced error accumulation, achieving SOTA performance.

Abstract: Diffusion models have seen widespread adoption for text-driven human motion generation and related tasks due to their impressive generative capabilities and flexibility. However, current motion diffusion models face two major limitations: a representational gap caused by pre-trained text encoders that lack motion-specific information, and error accumulation during the iterative denoising process. This paper introduces MOtion Reconstruction for GENeration (MORGEN) to address these challenges. First, MORGEN leverages a motion latent space as intermediate supervision for text-to-motion generation. To this end, MORGEN co-trains a motion reconstruction branch with two key objective functions: self-regularization to enhance the discrimination of the motion space and motion-centric latent alignment to enable accurate mapping from text to the motion latent space. Second, we propose Reconstructive Error Guidance (REG), a testing-stage guidance mechanism that exploits the diffusion model's inherent self-correction ability to mitigate error accumulation. At each denoising step, REG uses the motion reconstruction branch to reconstruct the previous estimate, reproducing the prior error patterns. By amplifying the residual between the current prediction and the reconstructed estimate, REG highlights the improvements in the current prediction. Extensive experiments demonstrate that MORGEN achieves significant improvements and state-of-the-art performance. Our code will be released.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 1268