Learning and Adapting Diverse Representations for Cross-domain Few-shot Learning

Ge Liu, Zhongqiang Zhang, Fuhan Cai, Duo Liu, Xiangzhong Fang

Abstract: Conventional few-shot learning (FSL) mainly focuses on transferring knowledge from a single source dataset to a recognition scenario with only a few training samples but still similar to the source domain. In this paper, we consider a more practical FSL setting where multiple semantically different datasets are available to address a wide range of FSL tasks, especially for some recognition scenarios beyond natural images, such as aerial and medical imagery. It can be referred to as multi-source cross-domain FSL. To tackle the problem, we propose a two-stage learning scheme, termed learning and adapting multi-source representations (LAMR). In the first stage, we propose a multi-head network to obtain efficient multi-domain representations, where all source domains share the same backbone except for the last parallel projection layers used for domain specialization. We train the representations in a multitask setting where each in-domain classification task is realized by a cosine classifier. In the second stage, considering that instance discrimination and class discrimination are crucial for robust recognition, we propose two contrastive objectives for adapting the pre-trained representations to be task-specialized on the few-shot data. Careful ablation studies verify that LAMR significantly improves representation transferability, showing consistent performance boosts for FSL. Experiments on the BSCD-FSL benchmarks demonstrate that LAMR can achieve state-of-the-art results, highlighting its versatility and effectiveness for FSL of both natural and specific imaging.