Continual learning of context-dependent processing in neural networks

Guanxiong Zeng, Yang Chen, Bo Cui, Shan Yu

Abstract: When neural networks are retrained to solve more than one problem, they tend to forget what they have learned earlier. Here, the authors propose orthogonal weights modification, a method to avoid this so-called catastrophic forgetting problem. Capitalizing on such an ability, a new module is introduced to enable the network to continually learn context-dependent processing.