Tracking-by-Self Detection: A Self-supervised Framework for Multiple Animal Tracking

C. B. Dev Narayan, Fayaz Rahman, Mohib Ullah, Faouzi Alaya Cheikh, Ali Shariq Imran, Christopher Coello, Øyvind Nordbø, G. Santhosh Kumar, Madhu S. Nair

Abstract: Animal tracking is a crucial aspect of animal phenotyping, and industries are using computer vision-based methods to enhance their products. In this paper, we adopt the tracking-by-detection approach and propose a self-supervised framework for multiple animal tracking. Self-supervised learning techniques have recently been employed to train models using unlabeled data and have demonstrated improved accuracy on benchmark datasets. Our proposed framework utilizes an EfficientDet detector that was pre-trained with self-supervised learning using a modified Barlow twins method. The detected animals are associated with tracks using our proposed variant of Deepsort, which utilizes appearance information to improve the detection-to-track association. We trained and tested the framework on a customized dataset from a Norwegian pig farm, which consisted of four test and four train sequences, as well as a detection dataset containing 1674 labelled frames and 3000 unlabeled images for self-supervised learning. To evaluate the performance of our framework, we used standard tracking metrics such as HOTA (Higher order tracking accuracy), MOTA (Multiple object tracking accuracy), and IDF1 (Identification metrics). The implementation of our framework is publicly available at https://github.com/DeVcB13d/Animal_tracking_with_ssl.