A Hybrid Adaptive Dynamic Programming for Optimal Tracking Control of USVs

Shan Xue, Ning Zhao, Weidong Zhang, Biao Luo, Derong Liu

Abstract: This article presents an efficient method for solving the optimal tracking control policy of unmanned surface vehicles (USVs) using a hybrid adaptive dynamic programming (ADP) approach. This approach integrates data-driven integral reinforcement learning (IRL) and dynamic event-driven (DED) mechanisms into the solution of the control policy of the established augmented system while obtaining both the feedforward and feedback components of the tracking controller. For the USV model and the reference trajectory, an augmented system is established, and the tracking Hamilton-Jacobi–Bellman (HJB) equation is derived based on IRL, aiming to fully utilize system data information and reduce model dependency. For the solution of the tracking HJB equation, the DED-based controller update rule is used to further reduce the burden of network transmission. In implementing the ADP method, the DED experience replay-based weight update rule is utilized to recycle data resources. Experiments show that compared with the static event-driven (SED) approach, the DED approach reduces the sample size by 78% and increases the average interval by about four times.