Continual Learning in the Teacher-Student Setup: Impact of Task Similarity

Sebastian Lee, Sebastian Goldt, Andrew M. Saxe

Abstract: Continual learning{—}the ability to learn many tasks in sequence{—}is critical for artificial learning systems. Yet standard training methods for deep networks often suffer from catastrophic forget...