An Integrated System for Spatio-temporal Summarization of 360-Degrees Videos

Ioannis Kontostathis, Evlampios Apostolidis, Vasileios Mezaris

Abstract: In this work, we present an integrated system for spatio-temporal summarization of 360-degrees videos. The video summary production involves the detection of salient events in the 360-degrees videos, and their synopsis in a concise summary. The analysis relies on state-of-the-art methods for saliency detection in 360-degrees video (ATSal and SST-Sal) and video summarization (CA-SUM). It also contains a camera motion detection mechanism (CMDM) that decides which saliency detection method will be used, as well as a 2D video production component that is responsible for creating a conventional 2D video containing the detected salient events in the 360-degrees video. Quantitative evaluations using two datasets for 360-degrees video saliency detection (VR-EyeTracking, Sports-360) show the accuracy and positive impact of the developed CMDM, and justify our choice to use two different methods for detecting saliency. A qualitative analysis using content from these datasets, gives further insights about the functionality of the CMDM, shows the pros and cons of each used saliency detection method and demonstrates the advanced performance of the trained summarization method against a more conventional approach.