Maximum a Posteriori Policy Optimisation

Abbas Abdolmaleki, Jost Tobias Springenberg, Yuval Tassa, Remi Munos, Nicolas Heess, Martin Riedmiller

Abstract: We introduce a new algorithm for reinforcement learning called Maximum a-posteriori Policy Optimisation (MPO) based on coordinate ascent on a relative-entropy objective. We show that several existing methods can directly be related to our derivation. We develop two off-policy algorithms and demonstrate that they are competitive with the state-of-the-art in deep reinforcement learning. In particular, for continuous control, our method outperforms existing methods with respect to sample efficiency, premature convergence and robustness to hyperparameter settings.

Keywords: Reinforcement Learning, Variational Inference, Control

Code: [ 3 community implementations](https://paperswithcode.com/paper/?openreview=S1ANxQW0b)

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/maximum-a-posteriori-policy-optimisation/code)