PKU_WICT at TRECVID 2021: Instance Search Task.

Yuxin Peng, Zhaoda Ye, Junchao Zhang, Hongbo Sun, Dejie Yang, Zhenyu Cui

Abstract: In TRECVID 2021, we participated in all two types of Instance Search (INS) task, ie, automatic way and interactive way. In automatic instance search, a two-stage approach of similarity calculation and re-ranking was applied. In our approach, action and person were recognized by deep neural networks separately, followed by score fusion for instance search. In action-specific recognition, it consisted of four modules, ie, frame-level action recognition, video-level action recognition, object detection and facial expression recognition. In person-specific recognition, face detection, feature extraction and top N query expansion strategy were adopted. In instance score fusion, a re-ranking strategy was applied based on their recognition information, and then the recognition scores of person and action were merged for retrieval. In interactive instance search, the interactive query expansion strategy was conducted to refine the results from automatic search.