Animal Re-Identification via Multiview Spatio-Temporal Track Clustering

Ankit Kumar Upadhyay, Ekaterina Nepovinnykh, S M Rayeed, Aidan Westphal, Lawrence Miao, Julian M. Bain, Jaeseok Kang, Tuomas Eerola, Heikki Kälviäinen, Charles V. Stewart

Abstract: We introduce a novel framework for individual animal identification (re-id) from long image sequences or video that is robust to occlusions, viewpoint variations, and other challenges of in-the-wild imaging. We start by detecting and tracking animals across images in each sequence. Next, we rate detections according to how much distinguishing information they carry, and coarsely sample the most distinguishable, being careful to sample detections showing the left and the right sides of an animal when possible. For each of these samples we compute embeddings using the MiewID algorithm, and we cluster them in embedding space, which allows us to tentatively group sampled detections showing the same individual. Finally, we link clusters across different viewpoints when taken from the same track. This step improves identification accuracy by merging distinct clusters from complementary viewpoints, ensuring that otherwise disjoint detections are recognized as coming from the same animal. After this computation is complete, we flag genuinely new individuals, and we incorporate consistency constraints to correct errors in clustering through manual intervention. We show strong preliminary experimental results, demonstrating near perfect identification accuracy with very little manual verification.