EgoPoints: Advancing Point Tracking for Egocentric Videos

Ahmad Darkhalil, Rhodri Guerrier, Adam W. Harley, Dima Damen

Abstract: We introduce EgoPoints, a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS [7] evaluation benchmark, we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points, we introduce evaluation metrics that specifically monitor tracking performance on points in-view, out-of-view, and points that require re-identification. We then propose a pipeline to create semi-real sequences, with automatic ground truth. We generate 11 K such sequences by combining dynamic Kubric [17] objects with scene points from EPIC Fields [38]. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences, we improve Co-Tracker [22] across all metrics, including the tracking accuracy $\delta^{\ast}_{avg}$ by 2.7 percentage points and accuracy on ReID sequences ($ReID\delta_{avg}$) by 2.4 points. We also improve $\delta^{\ast}_{avg}$ and $ReID\delta_{avg}$ of PIPs++ [42] by 0.3 and 2.8 respectively.

External IDs:dblp:conf/wacv/DarkhalilGHD25