MP3: Movement Primitive-Based (Re-)Planning Policy

Hongyi Zhou, Fabian Otto, Onur Celik, Ge Li, Rudolf Lioutikov, Gerhard Neumann

Keywords: Movement Primitives, Reinforcement Learning, Robot Learning

TL;DR: We integrate movement primitives into the deep reinforcement learning framework, enabling the generation of smooth trajectories and effectively learning from sparse and non-Markovian rewards.

Abstract:

We introduce a novel deep reinforcement learning (RL) approach called
Movement Primitive-based Planning Policy (MP3). By integrating movement
primitives (MPs) into the deep RL framework, MP3 enables the generation of
smooth trajectories throughout the whole learning process while effectively
learning from sparse and non-Markovian rewards. Additionally, MP3 maintains the
capability to adapt to changes in the environment during execution. Although
many early successes in robot RL have been achieved by combining RL with
MPs, these approaches are often limited to learning single stroke-based motions,
lacking the ability to adapt to task variations or adjust motions during execution.
Building upon our previous work, which introduced an episode-based RL method
for the non-linear adaptation of MP parameters to different task variations, this
paper extends the approach to incorporating replanning strategies. This allows
adaptation of the MP parameters throughout motion execution, addressing the
lack of online motion adaptation in stochastic domains requiring feedback.

Submission Number: 24