Navigation control of mobile robots using graph neural network and reinforcement learning with fuzzy reward

Feng Zhang, Chengbin Xuan, Hak-Keung Lam, Shun-Hung Tsai

Abstract: Automatic mobile robot navigation controller design has been researched for decades whereas suffer from its parameters dependence. Reinforcement learning (RL) provides a learning-based solution for solving navigation problem without tuning controller's parameters which dependent plant models. Graph neural network (GNN) has further extended the success of RL by extracting hidden spatial and temporal information from the non-Euclidean data structure and achieved higher training efficiency. However, most of the existing GNN-RL controllers are dependent on abundant structural information rigorously. This requirement is difficult to be satisfied in mobile robot navigation control due to its scarce structural information. We addressed this issue and extended GNN's application to scenarios lacks abundant structural information by proposing a spatial-temporal graph neural network (ST-GNN), whose graph is built by nodes containing the information of different timestamps. By synthesizing the ST-GNN with the reinforcement learning (RL) method, we proposed a ST-GNN based RL controller to achieve mobile robot navigation control. Furthermore, a fuzzy reward system is proposed to speed up the learning. Our experiment results suggest that ST-GNN based controller can obtain a better performance compared with the traditional multi-layer perceptron (MLP) based RL controller and the advantage of fuzzy reward is also verified.