Boosting Trust Region Policy Optimization by Normalizing flows Policy

Yunhao Tang, Shipra Agrawal

Abstract: We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraint, normalizing flows policy can generate samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps avoid bad local optima. We show that normalizing flows policy significantly improves upon factorized Gaussian policy baseline, with both TRPO and ACKTR, especially on tasks with complex dynamics such as Humanoid.

Keywords: Reinforcement Learning, Normalizing Flows

TL;DR: Normalizing flows policy to improve TRPO and ACKTR

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:1809.10326/code)