Full Network Capacity Framework for Sample-Efficient Deep Reinforcement Learning

Wentao Yang, Xinyue Liu, Yunlong Gao, Wenxin Liang, Linlin Zong, Guanglu Wang, Xianchao Zhang

Keywords: Deep Reinforcement learning; Sample Efficiency; Network Capacity

Abstract: In deep reinforcement learning (DRL), the presence of dormant neurons leads to a significant reduction in network capacity, which results in sub-optimal performance and limited sample efficiency. Existing training techniques, especially those relying on periodic resetting (PR), exacerbate this issue. We propose the Full Network Capacity (FNC) framework based on PR, which consists of two novel modules: Dormant Neuron Reactivation (DNR) and Stable Policy Update (SPU). DNR continuously reactivates dormant neurons, thereby enhancing network capacity. SPU mitigates perturbation from DNR and PR and stabilizes the Q-values for the actor, ensuring smooth training and reliable policy updates. Our experimental evaluations on the Atari 100K and DMControl 100K benchmarks demonstrate the remarkable sample efficiency of FNC. On Atari 100K, FNC achieves a superhuman IQM HNS of 107.3\%, outperforming the previous state-of-the-art method BBF by 13.3\%. On DMControl 100K, FNC excels in 5 out of 6 tasks in terms of episodic return and attains the highest median and mean aggregated scores. FNC not only maximizes network capacity but also provides a practical solution for real-world applications where data collection is costly and time-consuming. Our implementation is publicly accessible at \url{https://github.com/tlyy/FNC}.

Readers: auai.org/UAI/2025/Conference, auai.org/UAI/2025/Conference/Area_Chairs, auai.org/UAI/2025/Conference/Reviewers, auai.org/UAI/2025/Conference/Submission269/Authors, auai.org/UAI/2025/Conference/Submission269/Reproducibility_Reviewers

Submission Number: 269