AUC Maximization in Imbalanced Lifelong Learning

Xiangyu Zhu, Jie Hao, Yunhui Guo, Mingrui Liu

Keywords: Imbalanced Lifelong Learning, AUC Maximization, continual learning

Abstract: Imbalanced data is ubiquitous in machine learning, such as medical or fine-grained image datasets. The existing continual learning methods employ various techniques such as balanced sampling to improve classification accuracy in this setting. However, classification accuracy is not a suitable metric for imbalanced data, and hence these methods may not obtain a good classifier as measured by other metrics (e.g., Area under the ROC Curve). In this paper, we propose a solution to enable efficient imbalanced continual learning by designing an algorithm to effectively maximize one widely used metric in an imbalanced data setting: Area Under the ROC Curve (AUC). We find that simply replacing accuracy with AUC will cause \textit{gradient interference problem} due to the imbalanced data distribution. To address this issue, we propose a new algorithm, namely DIANA, which performs a novel synthesis of model \underline{D}ecoupl\underline{I}ng \underline{AN}d \underline{A}lignment. In particular, the algorithm updates two models simultaneously: one focuses on learning the current knowledge while the other concentrates on reviewing previously-learned knowledge, and the two models gradually align during training.The results show that DIANA achieves state-of-the-art performance on the imbalanced datasets.