CtF: Mitigating Visual Confusion in Continual Learning Through a Coarse-To-Fine Screening

Zejun Ye, Defeng Zhao, Wentao Zhang, Ruixuan Wang

Abstract: Continual learning requires deep learning models to learn new tasks while retaining knowledge of previous ones. However, the model often catastrophically forgets old knowledge when learning new classes. Particularly in complex scenarios involving visually similar classes, leading to a degraded performance on previously learned similar classes. Inspired by the process of identifying similar classes in human brains, we propose a simple but effective Coarse-screening to Fine-screening (CtF) framework. It initially identifies a small subset of classes that are visually similar. Subsequently in the fine-screening stage, it selects the correct class from the subset by integrating complementary features. In this study, prior knowledge in textual description is employed to complement visual information, which can potentially help model capture subtle distinctions between confused classes. Experimental results on multiple datasets demonstrate CtF can effectively improve continual learning performance of existing methods.