Efficient Reinforcement Learning by Discovering Neural Pathways

Samin Yeasar Arnob, Riyasat Ohib, Sergey M. Plis, Amy Zhang, Alessandro Sordoni, Doina Precup

Keywords: Energy Efficient AI, Parameter Efficient, Neural Pathways, Continuous Control, Online Reinforcement Learning, Offline Reinforcement Learning, Multitask Reinforcement Learning

TL;DR: To improve energy efficiency and reduce the carbon footprint, we propose Neural Pathway to efficiently use the network parameter space for reinforcement learning.

Abstract: Reinforcement learning (RL) algorithms have been very successful at tackling complex control problems, such as AlphaGo or fusion control. However, current research mainly emphasizes solution quality, often achieved by using large models trained on large amounts of data, and does not account for the financial, environmental, and societal costs associated with developing and deploying such models. Modern neural networks are often overparameterized and a significant number of parameters can be pruned without meaningful loss in performance, resulting in more efficient use of the model's capacity lottery ticket. We present a methodology for identifying sub-networks within a larger network in reinforcement learning (RL). We call such sub-networks, neural pathways. We show empirically that even very small learned sub-networks, using less than 5% of the large network's parameters, can provide very good quality solutions. We also demonstrate the training of multiple pathways within the same networks in a multitask setup, where each pathway is encouraged to tackle a separate task. We evaluate empirically our approach on several continuous control tasks, in both online and offline training

Primary Area: Reinforcement learning

Submission Number: 15093