Feature Dispersion Adaptation With Pre-Pooling Prototype for Continual Image Classification

Wuxuan Shi, Mang Ye, Wei Yu, Bo Du

Abstract: Catastrophic forgetting, the degradation of knowledge about previously seen classes when learning new concepts from a shifting data stream, is a pitfall faced by neural network learning in open environments. Recent research on continual image classification usually relies on storing samples or prototypes to resist this forgetting. We find that during acquiring knowledge of the new classes, the features of old classes gradually disperse, which leads to confusion of features between classes and makes them difficult to discriminate. Coping with feature dispersion would be a key consideration in resisting catastrophic forgetting, which has been neglected in previous works. To this end, we try to address this issue from two perspectives. First, we propose a dispersing feature generation mechanism, which generates pseudo-features based on the pre-pooling prototypes of the old classes to simulate feature dispersion and remind the classifier to adjust the decision boundary. Second, we design a consistent alignment constraint to alleviate the severity of feature dispersion by maintaining consistency in the hidden states of different depths when aligning the current model with the previous model. Extensive experimental results on various benchmarks show the superiority of our proposed method.

External IDs:doi:10.1109/tmm.2025.3632692