An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat

Yasin Baykal; Baris Baspinar

doi:10.1109/DASC58513.2023.10311295

An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat

Yasin Baykal^*, Baris Baspinar

^*Corresponding author for this work

Department of Aeronautical Engineering

Istanbul Technical University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Citations (Scopus)

Abstract

This paper presents an evolutionary reinforcement learning approach based on Deep Q Networks to address the maneuver decision challenge of unmanned aerial vehicles (UAV) in short-range aerial combat. The proposed approach aims to improve the UAVs' autonomous maneuver decision process and generate a robust policy against alternative enemy strategies. The training process involves parallel training of multiple workers, evaluation of models at regular intervals, selection of the best model, testing the best model against enemy policies, and updating the pool of enemy strategies. The proposed method continuously improves the trained models and generates more robust policies with higher win rates than standard reinforcement learning techniques or k-level learning approaches.

Original language	English
Title of host publication	DASC 2023 - Digital Avionics Systems Conference, Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9798350333572
DOIs	https://doi.org/10.1109/DASC58513.2023.10311295
Publication status	Published - 2023
Event	42nd IEEE/AIAA Digital Avionics Systems Conference, DASC 2023 - Barcelona, Spain Duration: 1 Oct 2023 → 5 Oct 2023

Publication series

Name	AIAA/IEEE Digital Avionics Systems Conference - Proceedings
ISSN (Print)	2155-7195
ISSN (Electronic)	2155-7209

Conference

Conference	42nd IEEE/AIAA Digital Avionics Systems Conference, DASC 2023
Country/Territory	Spain
City	Barcelona
Period	1/10/23 → 5/10/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

air combat
decision-making
reinforcement learning

Access to Document

10.1109/DASC58513.2023.10311295

Cite this

Baykal, Y., & Baspinar, B. (2023). An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat. In DASC 2023 - Digital Avionics Systems Conference, Proceedings (AIAA/IEEE Digital Avionics Systems Conference - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/DASC58513.2023.10311295

@inproceedings{eb50a2fe27ec406bb32f5b3f65f5eab4,

title = "An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat",

abstract = "This paper presents an evolutionary reinforcement learning approach based on Deep Q Networks to address the maneuver decision challenge of unmanned aerial vehicles (UAV) in short-range aerial combat. The proposed approach aims to improve the UAVs' autonomous maneuver decision process and generate a robust policy against alternative enemy strategies. The training process involves parallel training of multiple workers, evaluation of models at regular intervals, selection of the best model, testing the best model against enemy policies, and updating the pool of enemy strategies. The proposed method continuously improves the trained models and generates more robust policies with higher win rates than standard reinforcement learning techniques or k-level learning approaches.",

keywords = "air combat, decision-making, reinforcement learning",

author = "Yasin Baykal and Baris Baspinar",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 42nd IEEE/AIAA Digital Avionics Systems Conference, DASC 2023 ; Conference date: 01-10-2023 Through 05-10-2023",

year = "2023",

doi = "10.1109/DASC58513.2023.10311295",

language = "English",

series = "AIAA/IEEE Digital Avionics Systems Conference - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "DASC 2023 - Digital Avionics Systems Conference, Proceedings",

address = "United States",

}

Baykal, Y & Baspinar, B 2023, An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat. in DASC 2023 - Digital Avionics Systems Conference, Proceedings. AIAA/IEEE Digital Avionics Systems Conference - Proceedings, Institute of Electrical and Electronics Engineers Inc., 42nd IEEE/AIAA Digital Avionics Systems Conference, DASC 2023, Barcelona, Spain, 1/10/23. https://doi.org/10.1109/DASC58513.2023.10311295

An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat. / Baykal, Yasin; Baspinar, Baris.
DASC 2023 - Digital Avionics Systems Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. (AIAA/IEEE Digital Avionics Systems Conference - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat

AU - Baykal, Yasin

AU - Baspinar, Baris

PY - 2023

Y1 - 2023

N2 - This paper presents an evolutionary reinforcement learning approach based on Deep Q Networks to address the maneuver decision challenge of unmanned aerial vehicles (UAV) in short-range aerial combat. The proposed approach aims to improve the UAVs' autonomous maneuver decision process and generate a robust policy against alternative enemy strategies. The training process involves parallel training of multiple workers, evaluation of models at regular intervals, selection of the best model, testing the best model against enemy policies, and updating the pool of enemy strategies. The proposed method continuously improves the trained models and generates more robust policies with higher win rates than standard reinforcement learning techniques or k-level learning approaches.

AB - This paper presents an evolutionary reinforcement learning approach based on Deep Q Networks to address the maneuver decision challenge of unmanned aerial vehicles (UAV) in short-range aerial combat. The proposed approach aims to improve the UAVs' autonomous maneuver decision process and generate a robust policy against alternative enemy strategies. The training process involves parallel training of multiple workers, evaluation of models at regular intervals, selection of the best model, testing the best model against enemy policies, and updating the pool of enemy strategies. The proposed method continuously improves the trained models and generates more robust policies with higher win rates than standard reinforcement learning techniques or k-level learning approaches.

KW - air combat

KW - decision-making

KW - reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85178655602&partnerID=8YFLogxK

U2 - 10.1109/DASC58513.2023.10311295

DO - 10.1109/DASC58513.2023.10311295

M3 - Conference contribution

AN - SCOPUS:85178655602

T3 - AIAA/IEEE Digital Avionics Systems Conference - Proceedings

BT - DASC 2023 - Digital Avionics Systems Conference, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 42nd IEEE/AIAA Digital Avionics Systems Conference, DASC 2023

Y2 - 1 October 2023 through 5 October 2023

ER -

An Evolutionary Reinforcement Learning Approach for Autonomous Maneuver Decision in One-to-One Short-Range Air Combat

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this