EMPO: A Clustering-Based On-Policy Algorithm for Offline Reinforcement Learing

Jongeui Park, Myungsik Cho, Youngchul Sung

Keywords: offline reinforcement learning, trajectory clustering, hierarchical reinforcement learning, on-policy reinforcement learning

Abstract: We propose an on-policy algorithm, Expectation–Maximization Policy Optimization (EMPO), for offline reinforcement learning that leverages an EM-based clustering algorithm to recover the behaviour policies used to generate the dataset. By improving each behaviour policy via proximal policy optimization and learning a high-level policy that chooses the optimal cluster at each step, EMPO outperforms existing offline RL algorithms on multiple benchmarks.

Submission Number: 98