Addressing the Stability-Plasticity Dilemma in Continual Learning through Dynamic Training Strategies

Qingya Sui, Qiong Fu, Yuki Todo, Jun Tang, Shangce Gao

Abstract: Despite significant progress in learning from static data, modern artificial intelligence still faces challenges in continual learning. The stability-plasticity dilemma is a key issue in continual learning. To address this, we propose a novel continual learning model with three dynamic training strategies (DTCL). DTCL integrates adaptive learning rates, experience replay, and dynamic knowledge distillation. These strategies collectively enhance the network's ability to learn new tasks while preserving information from previous tasks. Experimental results on the CIFAR-100 dataset demonstrate that DTCL significantly outper-forms existing baselines in average accuracy. DTCL also demonstrates significant advantages in the classification of colorectal cancer images, highlighting its potential for addressing real-world medical classification challenges. The dynamic training strategies of DTCL effectively balance stability and plasticity, improving continual learning performance.