Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks
Yoshua Bengio, Mehdi Mirza, Ian Goodfellow, Aaron Courville, Xia Da
Dec 24, 2013 (modified: Dec 24, 2013)ICLR 2014 conference submissionreaders: everyone
Decision:submitted, no decision
Abstract:Catastrophic forgetting is a problem faced by many machine learning models and algorithms. When trained on one task, then trained on a second task, many machine learning models 'forget'' how to perform the first task. This is widely believed to be a serious problem for neural networks. Here, we investigate the extent to which the catastrophic forgetting problem occurs for modern neural networks, comparing both established and recent gradient-based training algorithms and activation functions. We also examine the effect of the relationship between the first task and the second task on catastrophic forgetting.
Enter your feedback below and we'll get back to you as soon as possible.