Following Ancestral Footsteps: Co-Designing Agent Morphology and Behaviour with Self-Imitation Learning

Published: 24 Jun 2024, Last Modified: 07 Jul 2024EARL 2024 PosterEveryoneRevisionsBibTeXCC BY 4.0
Keywords: Co-Adaptation, Reinforcement Learning, Self-Imitation Learning, Co-Design
TL;DR: When co-adapting agent morphology and behaviour it helps to imitate the behaviour of previous morphologies!
Abstract: In this paper we consider the problem of co-adapting the body and behaviour of agents, a long-standing research problem in the community of evolutionary robotics. Previous work has largely focused on the development of methods exploiting massive parallelization of agent evaluations with large population sizes, a paradigm which is not applicable to the real world. More recent data-efficient approaches utilizing reinforcement learning can suffer from distributional shifts to transition dynamics as well as to states and action spaces when experiencing new body morphologies. In this work, we propose a new co-adaptation method combining reinforcement learning and State-Aligned Self-Imitation Learning. We show that the integration of a self-imitation signal improves data-efficiency, behavioural recovery for unseen designs and performance convergence.
Submission Number: 4
Loading