Graph-Structured Reinforcement Learning for Controlling a Transformable-Wheel Robot

Haoqi Tang, James Clinton, Anthony J. Clark, Neil Chulani

Keywords: Transformable-Wheel Robot, Reinforcement Learning, Graph Neural Network

Abstract:

Transformable-wheel robots occupy a middle
ground between conventional wheeled and legged systems: they
can roll efficiently on even terrain, yet reconfigure to better han-
dle obstacles. In this work, we study reinforcement learning for
controlling a transformable-wheel robot and investigate whether
a graph-structured actor-critic policy provides an advantage rela-
tive to a flat multilayer perceptron (MLP). Our policy represents
the robot as one body node and four corner nodes, which encodes
the platform’s symmetry and relational structure. We find in
our experiments that graph-based policies improve early training
behavior by helping the agent avoid certain local failure modes.
Our results show that consistent forward locomotion can be
learned and that both graph-based and MLP policies can perform
well under the present task formulation, yet the graph-based
policy is able to escape obstacle-relative local basins faster. We
also identify the deep connection between reward task design
and graph feature design for similar morphology dependent
navigation tasks.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 18