RoboPianist: Dexterous Piano Playing with Deep Reinforcement LearningDownload PDF

Published: 30 Aug 2023, Last Modified: 30 Aug 2023CoRL 2023 PosterReaders: Everyone
Keywords: high-dimensional control, bi-manual dexterity, piano playing, reinforcement learning
TL;DR: We propose a deep RL system, RoboPianist, which can successfully learn to play 150 piano pieces in simulation with 2 anthropomorphic dexterous hands.
Abstract: Developing robotic hands capable of replicating human-like dexterity has been a grand challenge in robotics. While recent works have made impressive progress in the last few years with learning-based approaches, the studied tasks such as in-hand reorientation, relocation and simple tool use fail to capture the breadth of skills exhibited by human hands. To push the boundaries of high-dimensional control, we explore the task of controlling bi-manual anthropomorphic robotic hands to play the piano, which is targeted at testing high spatial and temporal precision, coordination, and planning, all with an underactuated system frequently making-and-breaking contacts. We propose a deep RL system, RoboPianist, which can successfully learn a diverse repertoire of 150 piano pieces. The resulting policies exhibit highly dexterous behaviors, and produce visually and acoustically pleasing performances. Website featuring videos, code, and datasets is available at https://robopianist.github.io/.
Student First Author: yes
Supplementary Material: zip
Instructions: I have read the instructions for authors (https://corl2023.org/instructions-for-authors/)
0 Replies

Loading