Knowledge Transfer using Model-Based Deep Reinforcement Learning

Tlou Boloka, Ndivhuwo Makondo, Benjamin Rosman

Abstract:

Deep reinforcement learning has recently been adopted for robot behavior learning, where robot skills are
acquired and adapted from data generated by the robot while interacting with its environment through a trial-and-error process. Despite this success, most model-free deep reinforcement learning algorithms learn a task-specific policy from a clean
slate and thus suffer from high sample complexity (i.e., they require a significant amount of interaction with the environment to learn reasonable policies and even more to reach convergence). They also suffer from poor initial performance due to executing
a randomly initialized policy in the early stages of learning to obtain experience used to train a policy or value function. Modelbased deep reinforcement learning mitigates these shortcomings. However, it suffers from poor asymptotic performance in contrast
to a model-free approach. In this work, we investigate knowledge transfer from a model-based teacher to a task-specific modelfree learner to alleviate executing a randomly initialized policy in the early stages of learning. Our experiments show that this approach results in better asymptotic performance, enhanced initial performance, improved safety, better action effectiveness, and reduced sample complexity.