Distillation Excluding Positives for Few-Shot Class-Incremental Learning

Duo Liu, Linglan Zhao, Zhongqiang Zhang, Fuhan Cai, Xiangzhong Fang

Abstract: Few-Shot Class-Incremental Learning (FSCIL) defines a challenging task to continually recognize novel classes with few training data without forgetting old classes. Considering the catastrophic forgetting and overfitting issues, mainstream FSCIL methods resort to obtaining a strong model in the base session and freezing it in incremental sessions. Although prevailing methods perform well in the base classes, they often struggle with poor novel class generalization. To strengthen the representation of these models, this paper focuses on Knowledge Distillation (KD). Since existing KD methods are incompetent for FSCIL and introduce limited improvement, we propose the Distillation Excluding Positives (DEP) method for boosting performance on FSCIL tasks. Specifically, DEP consists of negative relationship distillation and asymmetric self-feature distillation. It can mitigate the over-similarity of intra-class features, leading to a more generalized model. Extensive experiments on three FSCIL benchmarks validate the superiority of DEP over current SOTAs.