When Few-shot Meets Cross-domain Object Detection: Learning Instance-level Class Prototypes for Knowledge Transfer

Lin Zhang, Bo Zhang, Botian Shi, Jiayuan Fan, Jun Cheng, Tao Chen

Keywords: Domain Adaptive Object Detection, Few-shot Object Detection, Domain Adaptation

Abstract: Adversarial learning is typically used to tackle cross-domain object detection, which transfers a well-trained model from a source domain to a new target domain, assuming that extensive unlabeled training data from the new domain can be easily obtained. However, such an assumption may not hold in many access-constrained target scenarios such as biomedical applications. To this end, we study the Few-Shot Domain Adaptation Object Detection (FSDAOD) problem, where only a few labeled instances from the target domain are available, making it difficult to comprehensively represent the target domain data distribution, and causing the adversarial feature alignment using only a few instances hard to transfer complete knowledge from source to target. Benefiting from the success of prototype-based meta-learning in the few-shot learning community, we propose an Instance-level Prototype learning Network (IPNet) for addressing the FSDAOD problem. The IPNet first develops an Instance-level Prototypical Meta-alignment (IPM) module, which fuses instances from both domains to learn the domain-invariant prototypical representations, for boosting the adaptation model’s discriminability between different classes. Then a Feature-level Spatial Attention Transfer (FSAT) module is further developed, which employs the instance-level prototypes to discriminate various features' salience in one domain to make the model attend to foreground regions, then transfers such attention extraction knowledge to the target images via adversarial learning. Extensive experiments are conducted on several datasets with domain variances, including cross-weather changes, cross-sensor differences, and cross-style variances, and results show the consistent accuracy gains of the IPNet over state-of-the-art methods, e.g., 9.7% mAP increase on Cityscapes-to-FoggyCityscapes setting and 3.0% mAP increase on Sim10k-to-Cityscapes setting.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)