Applying Metamorphic Testing for Pose Estimation in the Context of Rugby Analysis: Lessons Learned and Findings

Matias Duran, Will Connors, Thomas Laurent, Ellen Rushe, Anthony Ventresque

Abstract: Analysis of rugby match and training footage is particularly useful for coaches and players to understand and improve their tackling technique, and potentially lower the rate of injuries. Machine learning models (in particular for pose estimation) promise to streamline rugby analysis. However models trained for “general purpose” computer vision tasks, such as pose estimation and object detection, frequently fail as a result of the challenging conditions and significant domain shift that rugby footage presents: high-impact, close-contact play causes problems such as occlusions, motion blur, and unconventional body orientations. It is therefore crucial to understand the specific conditions which cause these systems to fail so they can be prioritised during pre-processing and expensive manual data collection. In this paper we leverage Met-Pose, a metamorphic testing system to understand the specific conditions that cause pose estimation systems to fail. Metamorphic testing is particularly advantageous as this approach side-steps the need for costly, manually labelled data. Our ongoing project on applying pose estimation for rugby analysis employs MediaPipe, a popular, widely used pose estimation system, on rugby broadcast footage. We show how applying metamorphic testing to a sport analytics application can reveal situations that challenge the model without the need for any manual data labelling. For example, our results show that in this context, MediaPipe is particularly sensitive to motion blur and colour loss, but less so to lighting and resolution changes. Furthermore, we show how this process can be adapted to focus on particular aspects of an application by proposing a new metamorphic rule exploring the effect of including or excluding context on MediaPipe’s results. Our results show where MediaPipe struggles in complex, real-world sporting scenarios and also offer concrete insights for improving data augmentation, data collection and system design in sports analytics.