User Feedback in Latent Space Robotic Skill Learning

Rok Pahic, Zvezdan Loncarevic, Ales Ude, Bojan Nemec, Andrej Gams

Abstract: In order to operate in everyday human environment, humanoids robots will need to autonomously learn and adapt their actions, using among other reinforcement learning methods (RL). A common challenge in robotic RL is also the generation of appropriate reward functions. A vast body of literature investigates how active human feedback can be introduced into an interactive learning loop, with recent publications showing that user feedback can be used for the RL reward. However, increased complexity of robotic skills in everyday environment also increases their dimensionality, which can practically prevent use of user feedback for the reward, because too many trials are needed. In the paper we present the results of using discretized, user-assigned reward for RL of robotic throwing, with an emphasis on learning in the feature space, i. e., latent space of a deep autoencoder network. Statistical evaluation of a user study with 15 participants, who provided feedback for robotic throwing experiments, show that for certain tasks, RL with discrete user feedback can be effectively applied for robot learning.