Adaptive Multi-Agent Reinforcement Learning Solver for Tactical Conflict Resolution in Diverse Urban Airspace Configurations

This paper discusses the development of an adaptive Deep Reinforcement Learning solver, designed for centralised onground Tactical Conflict Resolution and applied within diverse in-cruise configurations of urban airspace. Our approach utilises a Multi-agent Reinforcement Learning algorithm with a shared policy framework, employing a Proximal Policy Optimisation baseline. The solver aims to ensure tactical conflict resolution through centralised decision-making and distributed instructions via speed calibration and flight level assignment. Each operation adheres to a designated flight plan and Operational Volume tolerances. Our methodology first enhances the solver's adaptability by interpreting a broad spectrum of in-cruise conflict configurations. This is achieved through a cautious observation and action space design, a normalisation strategy, and a Recurrent Neural Network that generalises conflict resolution for any number of intruders. We also apply existing standards from Airborne Collision Avoidance Systems to evaluate the safety performance of our Machine Learning-based solver, addressing the gap in Machine Learning model evaluation as a conventional system. We explore eight case studies of varying difficulty to investigate the safety impact of specific conflict configurations, from single to multiple intersections, considering shared routes or individual operation volumes, among other factors. Finally, we develop a ninth case study that maps all possible configurations under cruise conditions. We explore different trained models and assess their compatibility with other case studies, demonstrating the solver's adaptability in performing Tactical Conflict Resolution tasks for operations in urban airspace.