Multi-Object Tracking, Segmentation and Validation in Thermal Images

Mircea Paul Muresan, Radu Danescu, Sergiu Nedevschi

Abstract: In this paper we present a novel multi-object tracking and segmentation approach that works on thermal images and is able to track objects at bounding-box and instance mask levels. Furthermore, we present a novel object validation module, which is necessary because only specific classes of objects are tracked and classifiers and detectors can be subjected to errors such as misclassifications, false detections, erroneous instance masks and missed detections. One of the key difficulties in multi-target tracking is the unknown correspondences between measurements and targets also known as the data association problem. To address this issue the proposed object tracker uses a feature engineering data association approach that exploits multiple features which include structure, appearance, size, context, and motion in the region given by the instance mask. Moreover, an original strategy has been designed for dealing with motion uncertainty, based on optical flow and multiple motion models to better predict the future position of objects in the scene. The proposed method runs in real-time and has been evaluated on an international thermal tracking benchmark showing competitive results.