Revisiting Deep Attention Recurrent Networks

Fernando Fradique Duarte, Nuno Lau, Artur Pereira, Luís Paulo Reis

Abstract: Attention-based agents have had much success in many areas of Artificial Intelligence, such as Deep Reinforcement Learning. This work revisits two such architectures, namely, Deep Attention Recurrent Q-Networks (DARQNs) and Soft Top-Down Spatial Attention (STDA) and explores the similarities between them. More specifically, this work tries to improve the performance of the DARQN architecture by leveraging elements proposed by the STDA architecture, such as the formulation of its attention function which also includes the incorporation of a spatial basis into its computation. The implementation tested, denoted Deep Attention Recurrent Actor-Critic (DARAC), uses the A2C learning algorithm. The results obtained seem to suggest that the performance of DARAC can be improved by the incorporation of some of the techniques proposed in STDA. Overall, DARAC showed competitive results when compared to STDA and slightly better in some of the experiments performed. The Atari 2600 videogame benchmark was the testbed used to perform and validate all the experiments.