Few-Shot Object Detection Based on Latent Knowledge Representation

Yifeng Cao, Lijuan Duan, Zhaoying Liu, Wenjian Wang, Fangfang Liang

Abstract: Few-shot object detection (FSOD) aims to achieve excellent novel category object detection accuracy with few samples. Most existing researches address this problem by fine-tuning Faster R-CNN, where the model is first trained on the base class set with abundant samples, and then fine-tuned on the novel class set with scarce samples. But in the fine-tuning stage, the connection between the base class set and the novel class set is ignored, which makes it difficult to learn novel classes with scarce samples. To solve this issue, we propose a latent knowledge-based FSOD method, which aims to utilize latent knowledge to build connections between categories. Specifically, first we propose a latent knowledge classifier (LK-Classifier), which realizes object recognition by splitting features through latent knowledge. Then a guidance module is designed to constrain latent knowledge with semantic expression, so as to realize the bridge between base class set and novel class set through latent knowledge. Experimental results show that our method achieves promising results on the FSOD task on the PASCAL VOC and COCO datasets, especially when the number of samples is extremely scarce.