A dynamic semantic knowledge graph for zero-shot object detection

Wen Lv, Hongbo Shi, Shuai Tan, Bing Song, Yang Tao

Abstract: Zero-shot object detection (ZSD) learns a mapping relationship between visual space and semantic space; therefore, ZSD can rely on semantic information to identify and localize novel classes. However, due to the variety of images in the same category, ZSD algorithms usually use a fixed semantic embedding resulting in a visual-semantic significant gap. To bridge the gap, a dynamic semantic knowledge graph (DSKG) is proposed based on the visual-semantic relation. First, in order to capture the similarity between semantic information, a semantic knowledge graph is utilized to establish connections between categories. Then, a dynamic semantic reasoning mechanism is introduced to update semantic embedding based on the self-attention mechanism. Finally, experiments show that the DSKG can achieve significant improvements on MS-COCO and PASCAL VOC datasets.