Representation-Driven Exploration for Long-Horizon Manipulation

Andrea Protopapa, Davide Buoso, Francesca Pistilli, Giuseppe Averta

Keywords: long-horizon manipulation, learning from demonstrations, reinforcement learning, intrinsic exploration, representation learning, graph-based reward learning

TL;DR: Learned representations can improve exploration in long-horizon manipulation: latent novelty encourages diverse behaviors, while graph-based rewards capture stage-level task progress.

Abstract: Exploration is a central challenge in long-horizon robot manipulation, where sparse rewards and high-dimensional observations make policy learning inefficient. We study a representation-driven perspective based on two complementary mechanisms. The first uses embedding-space novelty as an intrinsic signal that encourages visitation of semantically diverse states. The second uses object-centric graph abstractions to learn stage-aware rewards that reflect progress through multi-step tasks. Together, these components show how structured latent representations can make exploration more informative and better aligned with task semantics. Experiments indicate that embedding-based exploration improves latent-space coverage and downstream policy learning, while graph-based reward learning provides interpretable step-like transitions. These findings suggest a promising mechanism toward hierarchical exploration strategies that combine novelty and abstraction for long-horizon robot learning.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 9