Optimizing the Neural Architecture of Reinforcement Learning Agents

Nina Mazyavkina, S. Moustafa, Ilya Trofimov, Evgeny Burnaev

Abstract: Reinforcement learning (RL) enjoyed significant progress over the last years. One of the most important steps forward was the wide application of neural networks. However, architectures of these neural networks are quite simple and typically are constructed manually. In this work, we study recently proposed neural architecture search (NAS) methods for optimizing the architecture of RL agents. We create two search spaces for the neural architectures and test two NAS methods: Efficient Neural Architecture Search (ENAS) and Single-Path One-Shot (SPOS). Next, we carry out experiments on the Atari benchmark and conclude that modern NAS methods find architectures of RL agents outperforming a manually selected one.