Extracting Strong Policies for Robotics Tasks from Zero-Order Trajectory OptimizersDownload PDF

Sep 28, 2020 (edited Mar 17, 2021)ICLR 2021 PosterReaders: Everyone
  • Keywords: reinforcement learning, zero-order optimization, policy learning, model-based learning, robotics, model predictive control
  • Abstract: Solving high-dimensional, continuous robotic tasks is a challenging optimization problem. Model-based methods that rely on zero-order optimizers like the cross-entropy method (CEM) have so far shown strong performance and are considered state-of-the-art in the model-based reinforcement learning community. However, this success comes at the cost of high computational complexity, being therefore not suitable for real-time control. In this paper, we propose a technique to jointly optimize the trajectory and distill a policy, which is essential for fast execution in real robotic systems. Our method builds upon standard approaches, like guidance cost and dataset aggregation, and introduces a novel adaptive factor which prevents the optimizer from collapsing to the learner's behavior at the beginning of the training. The extracted policies reach unprecedented performance on challenging tasks as making a humanoid stand up and opening a door without reward shaping
  • Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
  • One-sentence Summary: We propose an adaptively guided imitation learning method that is able to extract strong policies for hard robotic tasks from zero-order trajectory optimizers.
12 Replies