Deep Reinforcement Learning Agents With Collective Situational Awareness for Beyond Visual Range Air Combat

Joao P. A. Dantas, Felipe L. L. Medeiros, Adrisson R. Samersla, Pedro L. R. Botelho, Vitor C. F. Gomes, Samara R. Silva, Yuri D. Ferreira, Alessandro O. Arantes, Marcia R. C. Aquino, Marcos R. O. A. Maximo

Abstract: This work explores Beyond Visual Range (BVR) air combat simulations, focusing on two-versus-two scenarios involving autonomous agents. The engagement phase in BVR combat presents complex and unpredictable situations, as it is difficult to anticipate the behavior of opposing aircraft and the outcomes of tactical decisions, especially in multi-agent settings. A promising approach is the use of Deep Reinforcement Learning (DRL), which enables agents to learn from dynamic environments. According to fighter pilots, collective situational awareness, defined as understanding the spatial distribution and orientation of allies and opponents, is essential for executing coordinated tactical maneuvers. The main contribution of this work is AsaGym, a library for developing and training DRL-based fighter agents in BVR scenarios. A case study demonstrates its use, applying a reward function that promotes coordination based on collective situational awareness, and compares different DRL algorithms to assess their ability to foster cooperative behavior. The results highlight DRL’s potential to address the complexities of modern air combat and support the development of more adaptive and effective tactics in multi-agent BVR scenarios.

External IDs:doi:10.1109/access.2025.3597199