From Tabula Rasa to Emergent Abilities: Discovering Robot Skills via Reset-Free Unsupervised Quality-Diversity

Luca Grillotti, Maxence Faldor, Antoine Cully

Track: full paper

Keywords: Quality-Diversity, Reinforcement Learning, Reset-Free Learning, Robotics

TL;DR: We present URSA, a method enabling robots to autonomously discover and learn their abilities in a reset-free manner without extensive human guidance.

Abstract: Autonomous skill discovery in robotics seeks to enable robots to acquire diverse behaviors without explicit human guidance. However, learning such behaviors directly in the real world remains challenging due to the high number of interactions required. Existing approaches typically rely either on learning in simulated environments before real-world deployment, or on carefully designed heuristics. While the former face challenges when transferring to real robots due to the reality gap, the latter may require domain expertise to design effective heuristics. The recent algorithm Quality-Diversity Actor-Critic (QDAC) has shown promise in discovering diverse high-performing behaviors, yet its application to reset-free robotics remains limited due to safety concerns and the requirement for skills to be manually defined beforehand. Here, we propose Unsupervised Reset-free Skill Acquisition (URSA), an extension of QDAC that enables robots to autonomously discover and master diverse skills directly in reset-free environments, without prior knowledge of the skill space. URSA uses a novel skill sampling method and safety constraints to continuously discover diverse robot behaviors while maintaining safe operation. URSA manages to discover diverse velocity and unsupervised skills on a Unitree A1 quadruped robot in simulation. These results establish a new framework for reset-free robot learning that enables continuous skill discovery with a small amount of human intervention, representing a significant step toward more autonomous and adaptable robotic systems.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Presenter: ~Luca_Grillotti1

Format: Yes, the presenting author will attend in person if this work is accepted to the workshop.

Funding: No, the presenting author of this submission does *not* fall under ICLR’s funding aims, or has sufficient alternate funding.

Submission Number: 49