Variationally Autoencoded Dynamic Policy Programming for Robotic Cloth Manipulation Planning based on Raw Images

Yoshihisa Tsurumine, Takamitsu Matsubara

Abstract: This paper proposes a framework for model learning and action planning of cloth manipulation tasks based on such high-dimensional and redundant sensor observations as raw pixel images. This framework is called the variationally autoencoded dynamic policy programming (VAE-DPP), which is a combination of variationally autoencoded hidden Markov decision process (VAE-HMDP) and dynamic policy programming (DPP). At first, VAE-HMDP is learned, which consists of two disentangled latent spaces: for the image with VAE and for the hidden Markov decision process (HMDP). Then DPP is applied to the learned HMDP to plan action sequences efficiently. After investigating its effectiveness through a simple simulation of a toy problem, we applied VAE-DPP to two robotic cloth-folding tasks: (1) folding a handkerchief and (2) folding trousers. We successfully generated appropriate actions that reached the target folding state.