Multi-UAV Cooperative Pursuit Strategy with Limited Visual Field in Urban Airspace: A Multi-Agent Reinforcement Learning Approach

Zhe Peng, Guohua Wu, Biao Luo, Ling Wang

Abstract: The application of multiple unmanned aerial vehicles (UAVs) for the pursuit and capture of unauthorized UAVs has emerged as a novel approach to ensuring the safety of urban airspace. However, pursuit UAVs necessitate the utilization of their own sensors to proactively gather information from the unauthorized UAV. Considering the restricted sensing range of sensors, this paper proposes a multi-UAV with limited visual field pursuit-evasion (MUV-PE) problem. Each pursuer has a visual field characterized by limited perception distance and viewing angle, potentially obstructed by buildings. Only when the unauthorized UAV, i.e., the evader, enters the visual field of any pursuer can its position be acquired. The objective of the pursuers is to capture the evader as soon as possible without collision. To address this problem, we propose the normalizing flow actor with graph attention critic (NAGC) algorithm, a multi-agent reinforcement learning (MARL) approach. NAGC executes normalizing flows to augment the flexibility of policy network, enabling the agent to sample actions from more intricate distributions rather than common distributions. To enhance the capability of simultaneously comprehending spatial relationships among multiple UAVs and environmental obstacles, NAGC integrates the “obstacle-target” graph attention networks, significantly aiding pursuers in supporting search or pursuit activities. Extensive experiments conducted in a high-precision simulator validate the promising performance of the NAGC algorithm.