Cooperative Planning for an Unmanned Combat Aerial Vehicle Fleet Using Reinforcement Learning

Burak Yuksek, Mustafa Umut Demirezen, Gokhan Inalhan, Antonios Tsourdos

Research output: Contribution to journalArticlepeer-review

10 Citations (Scopus)


In this study, reinforcement learning (RL)-based centralized path planning is performed for an unmanned combat aerial vehicle(UCAV) fleet in a human-madehostile environment. The proposed method provides a novel approach in which closing speed and approximate time-to-go terms are used in the reward function to obtain cooperative motion while ensuring no-fly-zones (NFZs) and time-of-arrival constraints. Proximal policy optimization (PPO) algorithm is used in the training phase of the RL agent. System performance is evaluated in two different cases. In case 1, the warfare environment contains only the target area, and simultaneous arrival is desired to obtain the saturated attack effect. In case 2, the warfare environment contains NFZs in addition to the target area and the standard saturated attack and collision avoidance requirements. Particle swarm optimization (PSO)-based cooperative path planning algorithm is implemented as the baseline method, and it is compared with the proposed algorithm in terms of execution time and developed performance metrics. Monte Carlo simulation studies are performed to evaluate the system performance. According to the simulation results, the proposed system is able to generate feasible flight paths in real-time while considering the physical and operational constraints such as acceleration limits, NFZ restrictions, simultaneous arrival, and collision avoidance requirements. In that respect, the approach provides a novel and computationally efficient method for solving the large-scale cooperative path planning for UCAV fleets.

Original languageEnglish
Pages (from-to)739-750
Number of pages12
JournalJournal of Aerospace Information Systems
Issue number10
Publication statusPublished - 2021
Externally publishedYes

Bibliographical note

Publisher Copyright:
© 2021 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.


This work is supported in part by the Engineering and Physical Sciences Research Council (Grant No. EP/V026763/1).

FundersFunder number
Engineering and Physical Sciences Research CouncilEP/V026763/1


    Dive into the research topics of 'Cooperative Planning for an Unmanned Combat Aerial Vehicle Fleet Using Reinforcement Learning'. Together they form a unique fingerprint.

    Cite this