AGCN: Augmented Graph Convolutional Network for Lifelong Multi-Label Image Recognition

Kaile Du, Fan Lyu, Fuyuan Hu, Linyan Li, Wei Feng, Fenglei Xu, Qiming Fu

Abstract: The Lifelong Multi-Label (LML) image recognition builds an online class-incremental classifier in a sequential multilabel image recognition data stream. However, training on the data with different Partial Labels may result in more serious Catastrophic Forgetting in old classes. To solve the problem, the study proposes an Augmented Graph Convolutional Network (AGCN)to build an Augmented Correlation Matrix (ACM) across the sequential partial-label tasks and sustain the catastrophic forgetting. First, in ACM, the intra-task relations derive from the hard label statistics, while the inter-task relations further leverage the soft labels from a stored expert network. Then, based on the ACM, AGCN captures label dependencies with dynamic augmented structure and yields effective class representations. Our method is evaluated on two multi-label image benchmarks and the results show that the proposed method is effective for LML image recognition.