Distributed Deep Reinforcement Learning for Ackermann Multi-Robot Formation: A Weighted Multi-Objective Optimization

Wenjian Zhong, Bangquan Xie, Yanzhou Li, Lucia Cascone, Yongkang Lu, Shenghuang He

Abstract: The integration of Internet of Things (IoT) with the multi-robot formation problem has been successfully applied in autonomous vehicles, unmanned aerial vehicles (UAVs), and industrial robotics, enhancing robot collaboration in complex tasks and increasing their autonomy when interacting with uncertain environments. However, extending formation strategies to Ackermann-steering robots remains challenging due to the non-holonomic motion constraints inherent in their kinematics. To address this, we propose the Ackermann Multi-Objective Deep Reinforcement Learning (AM-DRL) framework, a distributed deep reinforcement learning (DDRL) approach specifically tailored for Ackermann-type dynamics. First, we design a novel reward function that incorporates projection, distance, alignment, curvature, and deceleration terms, allowing more effective policy learning under real-world driving constraints. Second, we introduce a dynamic weighted parameter-sharing mechanism that prioritizes agent contributions during model updates based on individual performance, thus accelerating convergence and reducing the risk of local optima. Third, we formulate multi-objective policy optimization (MOPO) using distinct actor-critic networks to enhance policy differentiation across heterogeneous agents. Experimental results show that, compared to baseline algorithms without weighted parameter sharing, the proposed method reduces the average distance error by 16.65% and improves formation integrity by 35.14%, demonstrating superior convergence, trajectory smoothness, and robustness.

External IDs:doi:10.1109/tce.2025.3603259