Semi-Supervised Few-Shot Incremental Learning with k-Probabilistic Principal Component Analysis

Ke Han, Adrian Barbu

Abstract: This paper introduces a novel method for Semi-Supervised Few-Shot Class Incremental Learning (SSFSCIL) that exhibits virtually no catastrophic forgetting. The method uses a generic feature extractor that was pretrained without supervision on a large image dataset, and a classifier based on a Probabilistic PCA (PPCA) model for each class instead of the standard fully connected layer usually employed as the projection head. The PPCA models are localized around the class means and the models for existing classes are not retrained when new classes are added. The learning algorithm is a modified k-Means that freezes the models on the existing classes and only updates models for the new classes. This makes the approach both computationally efficient and accurate. Extensive experiments on CUB200, CIFAR100, and miniImageNet show the effectiveness of the proposed approach. Additionally, experiments on the ImageNet-1k dataset, which previous methods have avoided due to its size, demonstrate its applicability to large-scale datasets.