A 3D Modeling Framework for the Application of Unmanned Aircraft Systems Integration

Berat Mert Albaba, Negin Musavi, Yildiray Yildiz

Abstract: Predicting the outcomes of integrating Unmanned Aerial Systems (UAS) into the National Airspace System (NAS) is a complex problem that is required to be addressed by simulation studies before allowing the routine access of UAS into the NAS. This paper focuses on providing a 3-dimensional (3D) simulation framework developed using a game-theoretical methodology to evaluate integration concepts using scenarios where manned and unmanned air vehicles co-exist. In the proposed method, the human pilot interactive decision-making process is incorporated into airspace models, which can fill the gap in the literature where the pilot behavior is generally assumed to be known a priori. The proposed human pilot behavior is modeled using the dynamic level-k reasoning concept and approximate reinforcement learning. Level-k reasoning concept is a notion in game theory and is based on the assumption that humans have various decision-making levels. In this study, Neural Fitted Q Iteration, which is an approximate reinforcement learning method, is used to model time-extended decisions of pilots with 3D maneuvers. An analysis of UAS integration is conducted using an example 3D scenario in the presence of manned aircraft and fully autonomous UAS equipped with sense and avoid algorithms.