Video Language Models are Human-Aligned Evaluators for Text to Motion Generation

ICLR 2026 Conference Submission15970 Authors

Keywords: text-driven human motion generation

TL;DR: The paper considers solving the T2M evaluation task by making use of a video language model, and provides a meta-evaluation dataset.

Abstract: Recently, text-to-motion (T2M) has become a basic setting for human motion generation. This work studies the evaluation of alignment between text and generated motion, to credit the reliable use of T2M models. We consider solving the T2M evaluation task by making use of a video language model (VLM). Our basic idea is: render the generated human motion into a skinned video, and then use a VLM for evaluation. To address information loss problem when 3D motion is rendered into 2D video, we develop a method, which ensures reliable evaluation score by analyzing VLM entropy. Our evaluation method, named VeMo, frees T2M evaluation from reliance on motion data while seamlessly leveraging the semantic understanding and reasoning capabilities of advanced VLMs trained on Internet-scale data. To systematically compare the empirical usefulness of different evaluation methods, we manually annotate a meta-evaluation benchmark that includes coarse-grained alignment labels and fine-grained reasons. Extensive experiments and case studies demonstrate the effectiveness of VeMo.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 15970