ECoDe: A Sample-Efficient Method for Co-Design of Robotic Agents

Kishan Reddy Nagiredla, Buddhika Laknath Semage, Arun Kumar A V, Thommen George Karimpanal, Santu Rana

Keywords: Co-Design, Reinforcement Learning, Machine Learning, Robotics

Abstract:

Co-designing autonomous robotic agents involves simultaneously optimizing the
controller and the agent’s physical design. Its inherent bi-level optimization formulation necessitates an outer loop design
optimization driven by an inner loop control optimization. This
can be challenging when the design space is large and each
design evaluation involves a data-intensive reinforcement learning
process for control optimization. To improve the sample efficiency of co-design,
we propose a multi-fidelity-based exploration strategy in
which we tie the controllers learned across the design spaces
through a universal policy learner for warm-starting subsequent
controller learning problems. Experiments performed on a wide
range of agent design problems demonstrate the superiority of
our method compared to baselines. Additionally, analysis
of the optimized designs shows interesting design alterations
including design simplifications and non-intuitive alterations.

Submission Number: 2