Integrating Egocentric Videos in Top-View Surveillance Videos: Joint Identification and Temporal Alignment

Shervin Ardeshir, Ali Borji

Abstract: Videos recorded from first person (egocentric) perspective have little visual appearance in common with those from third person perspective, especially with videos captured by top-view surveillance cameras. In this paper, we aim to relate these two sources of information from a surveillance standpoint, namely in terms of identification and temporal alignment. Given an egocentric video and a top-view video, our goals are to: (a) identify the egocentric camera holder in the top-view video (self-identification), (b) identify the humans visible in the content of the egocentric video, within the content of the top-view video (re-identification), and (c) temporally align the two videos. The main challenge is that each of these tasks is highly dependent on the other two. We propose a unified framework to jointly solve all three problems. We evaluate the efficacy of the proposed approach on a publicly available dataset containing a variety of videos recorded in different scenarios.