Efficient Text-driven Human Motion Generation via Latent Consistency Training

Mengxian Hu, Minghao Zhu, Xun Zhou, Qingqing Yan, Shu Li, Chengju Liu, Qijun Chen

Keywords: motion generation, diffusion, consistency training

Abstract:

Consistency models excel at few-step inference in generative tasks across various scenarios, but typically rely on pre-trained diffusion model distillation, involving additional training costs and performance limitations.
In this paper, we propose a motion latent consistency training framework that learns directly from data rather than distillation for efficient and text-controllable human motion generation.
For representation optimization, we design a motion autoencoder with quantization constraints that enable concise and bounded motion latent representations.
Focusing on conditional generation, we construct a classifier-free guidance (CFG) format with an additional unconditional loss function that extends the CFG technique from the inference phase to the training phase for conditionally guided consistency training.
We further propose a clustering guidance module to provide additional references to the solution distribution at minimal query cost.
By combining these enhancements, we achieve stable and consistent training in non-pixel modality and latent representation spaces for the first time.
Experiments in benchmarks demonstrate that our method significantly outperforms traditional consistency distillation methods with reduced training cost, and enhances the consistency model to perform comparably to state-of-the-art models with lower inference cost.
Our code will be open source.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3445