SCOTT: Shape-Location Combined Tracking with Optimal Transport

Xinye Zheng, Jianbo Ye, James Z. Wang, Jia Li

Abstract: Optimal transport (OT) is a prominent framework for point set registration, that is, to align points in two sets. Point set registration becomes particularly difficult when points are organized into objects and the correspondence among the objects is to be established. The registration of pixels must maintain consistency at the object level despite the possibility of object division, merging, and substantial alteration in size and shape over time. Existing approaches in OT exploit either similarity in shape for the entire set of points or spatial closeness of individual points, but not the two simultaneously. We propose a new weighted Gromov--Wasserstein distance (WGWD) to combine both sources of information. Importantly, we use a bipartite graph partitioning strategy to regularize OT in order to achieve object-level consistency and to enhance computational efficiency. We apply the method to cell tracking, specifically, the task of associating biological cells in consecutive image frames from time-lapse image sequences. We call the system SCOTT Shape-Location COmbined Tracking with Optimal Transport). By establishing a pixel-to-pixel correspondence, our method can effectively detect intricate scenarios including cell division and merging (overlapping). Experiments show that our method achieves high accuracy in tracking the movements of cells and outperforms existing methods in the detection of cell division and merging. Location information is shown to be more useful than shape information, while the combination of the two achieves optimal results.