Meta-Aggregating Networks for Class-Incremental Learning

Yaoyao Liu, Bernt Schiele, Qianru Sun

Keywords: incremental learning, continual learning, class-incremental learning, meta learning

Abstract: Class-Incremental Learning (CIL) aims to learn a classification model with the number of classes increasing phase-by-phase. The inherent problem in CIL is the stability-plasticity dilemma between the learning of old and new classes, i.e., high-plasticity models easily forget old classes but high-stability models are weak to learn new classes. We alleviate this issue by proposing a novel network architecture called Meta-Aggregating Networks (MANets) in which we explicitly build two residual blocks at each residual level (taking ResNet as the baseline architecture): a stable block and a plastic block. We aggregate the output feature maps from these two blocks and then feed the results to the next-level blocks. We meta-learn the aggregating weights in order to dynamically optimize and balance between two types of blocks, i.e., between stability and plasticity. We conduct extensive experiments on three CIL benchmarks: CIFAR-100, ImageNet-Subset, and ImageNet, and show that many existing CIL methods can be straightforwardly incorporated on the architecture of MANets to boost their performance.

One-sentence Summary: We propose a novel network architecture called Meta-Aggregating Networks (MANets) that can be straightforwardly incorporated into existing Class-Incremental Learning methods to achieve SOTA performance.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2010.05063/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=CQX53VPMDf