Prescribed Performance Robust Approximate Optimal Tracking Control via Stackelberg Game

Junkai Tan, Shuangsi Xue, Huan Li, Zihang Guo, Hui Cao, Dongyu Li

Abstract: Real-world applications of nonlinear systems tracking control are always challenging due to the existence of uncertainties and disturbances. To design a robust optimal tracking controller for uncertain nonlinear systems with disturbances and actuator saturation, this paper investigates the prescribed performance robust optimal tracking control problem. A prescribed performance mechanism is constructed to convert the dynamics of tracking error into transformed error dynamics, which keeps the system’s operating states within specific bounds, ensuring tracking with predefined error constraints. For the optimal tracking controller design, an optimal index is established to optimize the performance of tracking control, and a robust optimal index is established to optimize the disturbance effect on the tracking error. To achieve robust optimal tracking control that minimizes both optimal and robust optimal indexes, a Stackelberg game is constructed, which provides a hierarchical game structure for the optimal controller and the worst disturbance. The robust optimal controller is approximated online using reinforcement learning techniques. An actor-critic-identifier algorithm is designed to approximate the optimal value function, optimal controller, and drifted system parameters. Lyapunov theory is utilized to analyze the closed-loop system’s stability. To demonstrate the effectiveness of the proposed robust optimal control method, two numerical simulations and a hardware experiment on a quadcopter system are conducted. The experiment results demonstrate that our method successfully achieves prescribed performance tracking control when actuators are saturated and disturbances are present. Note to Practitioners—In this paper, the probelm of mixed $H_{2}/H_{\infty }$ prescribed-performance optimal tracking control for nonlinear systems with input saturation is investigated. To constrain the operating states of the system within certain bounds, the prescribed performance transformation is designed to achieve tracking with predefined error constraints. For the optimal controller design, the $H_{2}$ index is established to minimize the optimal tracking performance, and the $H_{\infty }$ index is designed to minimize the disturbance effect on the tracking error. A Stackelberg-based non-zero sum game between the optimal controller and the worst disturbance is established to design the mixed $H_{2}/H_{\infty }$ optimal tracking controller. The designed optimal controller is approximated online using reinforcement learning. Effectiveness of the proposed method is demonstrated by two numerical simulations and a hardware experiment on a quadcopter system. Based on the proposed high-performance controller, engineers can design a high-performance robust optimal tracking controller for uncertain nonlinear systems with extreme conditions of disturbances and actuator saturation.