The Case for Gradual Structured Pruning in Image-based Deep Reinforcement Learning
Keywords: gradual structured pruning, deep reinforcement learning, Procgen benchmark
TL;DR: We demonstrate that group-structured pruning is the preferred neural network pruning method for image-based deep reinforcement learning compared to other pruning techniques.
Abstract: Scaling neural networks in image-based deep reinforcement learning often fails to improve performance. While it was shown that unstructured pruning of scaled networks can unlock performance gains, we find that refining the architecture of the scaled network yields even greater improvements. However, scaled networks in deep reinforcement learning present a practical challenge: the increased computational demands can hinder deployment on embedded devices, as commonly encountered in robotics applications. To address this, we propose a novel gradual group-structured pruning framework that allows performance gains through scaling while maintaining computational efficiency. Our method preserves the network's functional integrity of inter-layer dependencies in groups, such as residual connections, while seamlessly integrating with standard deep reinforcement learning algorithms. Experiments with PPO and DQN show that our approach sustains performance while significantly reducing inference time, making it the preferred approach for resource-limited deployment.
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10988
Loading