Towards Perpetually Trainable Neural Networks
Primary Area: transfer learning, meta learning, and lifelong learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: deep learning, reinforcement learning, continual learning, plasticity, training dynamics
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
TL;DR: We isolate and address several mechanisms of plasticity loss in neural networks, and integrate these solutions into a training protocol designed to maintain plasticity in nonstationary learning problems.
Abstract: Underpinning the past decades of work on the design, initialization, and optimization of neural networks is a seemingly inoccuous assumption: that the networkis trained on a stationary data distribution. In settings where this assumption is violated, e.g. deep reinforcement learning, learning algorithms become unstableand brittle with respect to hyperparameters and even random seeds. One factor driving this instability is the loss of plasticity, meaning that updating the network’s predictions in response to new information becomes more difficult as training progresses. In this paper, we conduct a thorough analysis of the mehcnaisms of plasticity loss in neural networks trained on nonstationary learning problems, identify solutions to each of these pathologies, and integrate these solutions into a straightforward training protocol designed to maintain plasticity. We validate this approach in a variety of synthetic continual learning tasks, and further demonstrate its effectiveness on naturally arising nonstationarities
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7189
Loading