Hadamax Encoding: Elevating Performance in Model-Free Atari

Jacob Eeuwe Kooi, Zhao Yang, Vincent Francois-Lavet

Keywords: Model-free, Atari

TL;DR: We proposed a new encoder that achieves the state-of-the-art performance in model-free Atari.

Abstract: Neural network architectures have a large impact in machine learning. In reinforcement learning, network architectures have remained notably simple, as changes often lead to small gains in performance. This work introduces a novel encoder architecture for pixel-based model-free reinforcement learning. The Hadamax (Hadamard max}-pooling) encoder achieves state-of-the-art performance by max-pooling Hadamard products between GELU-activated parallel hidden layers. Based on the recent PQN algorithm, the Hadamax encoder achieves state-of-the-art model-free performance in the Atari-57 benchmark. Specifically, without applying any algorithmic hyperparameter modifications, Hadamax-PQN achieves an 80% performance gain over vanilla PQN and significantly surpasses Rainbow-DQN. For reproducibility, the full code will be available after the author notification.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Jacob_Eeuwe_Kooi1, ~Zhao_Yang3

Track: Regular Track: unpublished work

Submission Number: 117