Following Ancestral Footsteps: Co-Designing Morphology and Behaviour with Self-Imitation Learning

Sergio Hernández-Gutiérrez, Ville Kyrki, Kevin Sebastian Luck

Keywords: Inverse Reinforcement Learning, Co-Design, Evolutionary Robotics, Imitation Learning

TL;DR: Self-imitation learning is applicable to co-design and helps to increase the data efficiency when co-adapting morphology and behaviour.

Abstract: In this paper we consider the problem of co-adapting the body and behaviour of agents, a long-standing research problem in the community of evolutionary robotics. Previous work has largely focused on the development of methods exploiting massive parallelization of agent evaluations with large population sizes, a paradigm which is not applicable to the real world. More recent data-efficient approaches utilizing reinforcement learning can suffer from distributional shifts in transition dynamics as well as in state and action spaces when experiencing new body morphologies. In this work, we propose a new co-adaptation method combining reinforcement learning and State-Aligned Self-Imitation Learning. We show that the integration of a self-imitation signal improves the data-efficiency of the co-adaptation process as well as the behavioural recovery when adapting morphological parameters.

Submission Number: 80