Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace

Rodolphe Fremond; Yan Xu; Gokhan Inalhan

doi:10.1109/DASC55683.2022.9925730

Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace

Rodolphe Fremond, Yan Xu, Gokhan Inalhan

Cranfield University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

The present paper formalises the development of a Multi-agent Reinforcement Learning (MARL) solver for U-space Service Providers (USSPs) supporting the tactical conflict resolution and exhibited in the Air Mobility Urban - Large Experimental Demonstration (AMU-LED) project. It relies on an Advantage Actor Critic (A2C) model with a Proximal Policy Optimisation (PPO) learning baseline. The application of the autonomous system is demonstrated under a synthetic (with live and virtual) air/unmanned traffic management (ATM/UTM) environment. The Unmanned Aircraft Systems (UASs) are flying in cruise phase at low altitudes, whose respective flight plan generates intersections for enforcing a high collision frequency. The study adopts a step-wise complexity approach of scenarios that confront two agents' observation methods and showcases a practical case of tactical conflict resolution. The experiments show encouraging deconfliction performance with promising prospects for seeing a such solver deployed.

Original language	English
Title of host publication	2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665486071
DOIs	https://doi.org/10.1109/DASC55683.2022.9925730
Publication status	Published - 2022
Externally published	Yes
Event	41st IEEE/AIAA Digital Avionics Systems Conference, DASC 2022 - Portsmouth, United States Duration: 18 Sept 2022 → 22 Sept 2022

Publication series

Name	AIAA/IEEE Digital Avionics Systems Conference - Proceedings
Volume	2022-September
ISSN (Print)	2155-7195
ISSN (Electronic)	2155-7209

Conference

Conference	41st IEEE/AIAA Digital Avionics Systems Conference, DASC 2022
Country/Territory	United States
City	Portsmouth
Period	18/09/22 → 22/09/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

Actor Critic
Multi-Agent System
Proximal Policy Optimization
Reinforcement Learning
U-space Service Providers
Urban Air Mobility

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/DASC55683.2022.9925730

Cite this

Fremond, R., Xu, Y., & Inalhan, G. (2022). Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace. In 2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings (AIAA/IEEE Digital Avionics Systems Conference - Proceedings; Vol. 2022-September). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DASC55683.2022.9925730

Fremond, Rodolphe ; Xu, Yan ; Inalhan, Gokhan. / Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace. 2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. (AIAA/IEEE Digital Avionics Systems Conference - Proceedings).

@inproceedings{5a0d6967023c4d749d559a07e97e8523,

title = "Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace",

abstract = "The present paper formalises the development of a Multi-agent Reinforcement Learning (MARL) solver for U-space Service Providers (USSPs) supporting the tactical conflict resolution and exhibited in the Air Mobility Urban - Large Experimental Demonstration (AMU-LED) project. It relies on an Advantage Actor Critic (A2C) model with a Proximal Policy Optimisation (PPO) learning baseline. The application of the autonomous system is demonstrated under a synthetic (with live and virtual) air/unmanned traffic management (ATM/UTM) environment. The Unmanned Aircraft Systems (UASs) are flying in cruise phase at low altitudes, whose respective flight plan generates intersections for enforcing a high collision frequency. The study adopts a step-wise complexity approach of scenarios that confront two agents' observation methods and showcases a practical case of tactical conflict resolution. The experiments show encouraging deconfliction performance with promising prospects for seeing a such solver deployed.",

keywords = "Actor Critic, Multi-Agent System, Proximal Policy Optimization, Reinforcement Learning, U-space Service Providers, Urban Air Mobility",

author = "Rodolphe Fremond and Yan Xu and Gokhan Inalhan",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 41st IEEE/AIAA Digital Avionics Systems Conference, DASC 2022 ; Conference date: 18-09-2022 Through 22-09-2022",

year = "2022",

doi = "10.1109/DASC55683.2022.9925730",

language = "English",

series = "AIAA/IEEE Digital Avionics Systems Conference - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings",

address = "United States",

}

Fremond, R, Xu, Y & Inalhan, G 2022, Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace. in 2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings. AIAA/IEEE Digital Avionics Systems Conference - Proceedings, vol. 2022-September, Institute of Electrical and Electronics Engineers Inc., 41st IEEE/AIAA Digital Avionics Systems Conference, DASC 2022, Portsmouth, United States, 18/09/22. https://doi.org/10.1109/DASC55683.2022.9925730

Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace. / Fremond, Rodolphe; Xu, Yan; Inalhan, Gokhan.
2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. (AIAA/IEEE Digital Avionics Systems Conference - Proceedings; Vol. 2022-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace

AU - Fremond, Rodolphe

AU - Xu, Yan

AU - Inalhan, Gokhan

PY - 2022

Y1 - 2022

N2 - The present paper formalises the development of a Multi-agent Reinforcement Learning (MARL) solver for U-space Service Providers (USSPs) supporting the tactical conflict resolution and exhibited in the Air Mobility Urban - Large Experimental Demonstration (AMU-LED) project. It relies on an Advantage Actor Critic (A2C) model with a Proximal Policy Optimisation (PPO) learning baseline. The application of the autonomous system is demonstrated under a synthetic (with live and virtual) air/unmanned traffic management (ATM/UTM) environment. The Unmanned Aircraft Systems (UASs) are flying in cruise phase at low altitudes, whose respective flight plan generates intersections for enforcing a high collision frequency. The study adopts a step-wise complexity approach of scenarios that confront two agents' observation methods and showcases a practical case of tactical conflict resolution. The experiments show encouraging deconfliction performance with promising prospects for seeing a such solver deployed.

AB - The present paper formalises the development of a Multi-agent Reinforcement Learning (MARL) solver for U-space Service Providers (USSPs) supporting the tactical conflict resolution and exhibited in the Air Mobility Urban - Large Experimental Demonstration (AMU-LED) project. It relies on an Advantage Actor Critic (A2C) model with a Proximal Policy Optimisation (PPO) learning baseline. The application of the autonomous system is demonstrated under a synthetic (with live and virtual) air/unmanned traffic management (ATM/UTM) environment. The Unmanned Aircraft Systems (UASs) are flying in cruise phase at low altitudes, whose respective flight plan generates intersections for enforcing a high collision frequency. The study adopts a step-wise complexity approach of scenarios that confront two agents' observation methods and showcases a practical case of tactical conflict resolution. The experiments show encouraging deconfliction performance with promising prospects for seeing a such solver deployed.

KW - Actor Critic

KW - Multi-Agent System

KW - Proximal Policy Optimization

KW - Reinforcement Learning

KW - U-space Service Providers

KW - Urban Air Mobility

UR - https://www.scopus.com/pages/publications/85141939246

U2 - 10.1109/DASC55683.2022.9925730

DO - 10.1109/DASC55683.2022.9925730

M3 - Conference contribution

AN - SCOPUS:85141939246

T3 - AIAA/IEEE Digital Avionics Systems Conference - Proceedings

BT - 2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 41st IEEE/AIAA Digital Avionics Systems Conference, DASC 2022

Y2 - 18 September 2022 through 22 September 2022

ER -

Fremond R, Xu Y, Inalhan G. Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace. In 2022 IEEE/AIAA 41st Digital Avionics Systems Conference, DASC 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2022. (AIAA/IEEE Digital Avionics Systems Conference - Proceedings). doi: 10.1109/DASC55683.2022.9925730

Application of an autonomous multi-agent system using Proximal Policy Optimisation for tactical deconfliction within the urban airspace

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this