Latent Space Curriculum Reinforcement Learning in High-Dimensional Contextual Spaces and Its Application to Robotic Piano Playing

Haruki Abe, Takayuki Osa, Motoki Omura, Jen-Yen Chang, Tatsuya Harada

Abstract: Curriculum reinforcement learning (CRL) enables learning optimal policies in complex tasks such as robotic hand manipulation. However, in tasks with high-dimensional contexts with temporally continuous goals, previous research has encountered issues such as increased computational costs and the inability to create appropriate curricula to facilitate learning. Therefore, this study proposes a novel CRL method that appropriately segments high-dimensional contexts and learns them using a generative model. Additionally, we propose a method to further enhance learning by incorporating difficulty information into the generative model. Finally, we experimentally confirm that our proposed method significantly accelerates learning in complex tasks such as dual-arm dexterous hand tasks, specifically, RoboPianist.