PAC-Net: Highlight Your Video via History Preference Modeling

Hang Wang, Penghao Zhou, Chong Zhou, Zhao Zhang, Xing Sun

Abstract: Autonomous highlight detection is crucial for video editing and video browsing on social media platforms. General video highlight detection aims at extracting the most interesting segments from the entire video. However, interest is subjective among different users. A naive solution is to train a model for each user but it is not practical due to the huge training expense. In this work, we propose a Preference-Adaptive Classification (PAC-Net) framework, which can model users’ personalized preferences from their user history. Specifically, we design a Decision Boundary Customizer (DBC) module to dynamically generate the user-adaptive highlight classifier from the preference-related user history. In addition, we introduce Mini-History (Mi-Hi) mechanism to capture more fine-grained user-specific preferences. The final highlight prediction is jointly decided by the user’s multiple preferences. Extensive experiments demonstrate that PAC-Net achieves state-of-the-art performance on the public benchmark dataset, whilst using substantially smaller networks.