MV-Soccer: Motion-Vector Augmented Instance Segmentation for Soccer Player Tracking

Fahad Majeed, Nauman Ullah Gilal, Khaled A. Al-Thelaya, Yin Yang, Marco Agus, Jens Schneider

Abstract: This work presents a novel real-time detection, instance segmentation, and tracking approach for soccer videos. Unlike conventional methods, we augment video frames by incorporating motion vectors, thus adding valuable shape cues that are not readily present in RGB frames. This facilitates improved foreground/background separation and enhances the ability to distinguish between players, especially in scenarios involving partial occlusion. The proposed framework leverages the Cross-Stage-Partial Network53 (CSPDarknet53) as a backbone, for instance segmentation and integrates motion vectors, coupled with frame differencing. The model is simultaneously trained on two publicly available datasets and a private dataset, SoccerPro, which we created. The reason for simultaneous training is to reduce biases and increase generalization ability. To validate the effectiveness of our approach, we conducted extensive experiments and attained 97% accuracy for the DFL - Bundesliga Data Shootout, 98% on the SoccerNet-Tracking dataset, and an impressive 99% on the SoccerPro (our) dataset.