Multi-modality movie scene detection using Kernel Canonical Correlation Analysis

Guangyu Gao, Huadong Ma

Abstract: Scene detection is the fundamental step for efficient accessing and browsing videos. In this paper, we propose to segment movie into scenes which utilizes fused visual and audio features. The movie is first segmented into shots by an accelerating algorithm, and the key frames are extracted later. While feature movies are often filmed in open and dynamic environments using moving cameras and have continuously changing contents, we focus on the association extraction of visual and audio features. Then, based on the Kernel Canonical Correlation Analysis (KCCA), all these features are fused for scene detection. Finally, spatial-temporal coherent shots construct the similarity graph which is partitioned to generate the scene boundaries. We conduct extensive experiments on several movies, and the results show that our approach can efficiently detect the scene boundaries with a satisfactory performance.