Evaluating Automatic Hand-Gesture Generation Using Multimodal Corpus Annotations: The Benefits of a Multidisciplinary Approach

Mickaëlla Grondin Verdon, Domitille Caillat, Louis ABEL, Ouni

Abstract: This exploratory study addresses the challenges of evaluating the quality of hand-gesture synthesis. It introduces an interdisciplinary methodology aimed at defining detailed and objective evaluation criteria. The study examines the accuracy and consistency of expert annotations applied on a small dataset combining both natural and synthetic gestures, showing how their comparison can reveal key indicators for assessing communicative efficiency and adequate movement dynamics. The analysis reveals clear differences: communicative gestures are more frequent, shorter, and easier to interpret in natural data, while synthetic gestures are more ambiguous, with less precise annotations and less consistent velocity profiles. These findings support the idea that only an interdisciplinary approach —combining computational modeling with insights from gesture studies in the language sciences— can yield meaningful criteria for evaluating and ultimately improving the quality of synthesized gestures.

Submission Number: 6