Binary feature representation learning for scene retrieval in micro-video

Jie Guo, Xiushan Nie, Muwei Jian, Yilong Yin

Abstract: Micro-video is popular as new social media, and scene retrieval is a useful application in micro-video. At present, few researches focus on scene retrieval in micro-video, and there is a big gap between scene feature and semantics. In order to extract better semantical feature, we propose a combinational fusion method which combines multi-layer neural network and supervised hash learning method. As nonlinear projection, multi-layer neural network fuses multiple modalities by nonlinear transformation, and supervised hash learning method transforms fusion feature by linear projection to binary code for semantics and similarity preservation. We evaluate the proposed method on an actual micro-video dataset crawled from Vine. The experimental results show its superior performance than single multi-modal fusion methods and single hash learning methods.