Learning Progress-Guided LLM Goal Generation for Autotelic Skill Learning

Guillaume Pourcel, Grgur Kovač, Thomas Carta, Cédric Colas, Pierre-Yves Oudeyer

Keywords: learning progress, curriculum learning, reinforcement learning, large language models, goal generation, open-ended learning, exploration

Abstract: Reinforcement learning agents typically operate within fixed goal spaces, which limits the breadth of skills they can acquire. Large language models promise to overcome this constraint through dynamic goal generation, yet prompting them for merely *interesting* goals rarely produces effective curricula. We evaluate open-ended curricula using two key dimensions — *learnability* and *diversity* — and show that competence-based LLM approaches generate goals that appear promising but drive limited genuine learning progress. Our method instead optimizes goal generation directly for learning progress and consistently outperforms competence-based baselines on both learnability and diversity. In the *Crafter* domain, this leads agents to acquire diverse, challenging, and practically useful skills in the absence of extrinsic rewards.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 24693