Reinforcement Learning Based Autonomous Air Combat with Energy Budgets

Fighter pilots may become commanders in the air in the future. Modern jet fighter aircraft have different capabilities to command the field by using various equipment. Additionally, manned and unmanned teams can be composed to increase air dominance since the human capacity is limited for long flight times for missions. When this happens, the pilots can command their unmanned wing-mans during the mission. To reach these kinds of scenarios, more tasks need to be realized autonomously to dominate the airfield with the hybrid unmanned fleet. Air combat is one of the most important and challenging tasks for fighter pilots. Due to the complexity of the problem, most of the time the air combat missions need to be realized by human pilots due to the lack of unmanned aircraft's capability. Increasing the autonomy level for this specific problem may be beneficial for armies. Therefore, the air combat mission is mostly studied for many years to solve the problem from several approaches, either pilot assistance systems or fully autonomous missions. Additionally, strong improvements in both computer technology and artificial intelligence have been experienced. The number of problems that have been solved by using artificial networks is also increasing. It is thought that similar approaches can be used to solve an autonomous air combat problem. This article aims to develop an agent that will preserve the specific energy of the aircraft while being successful in air combat missions using artificial intelligence methods. Three different agents with different reinforcement learning-based algorithms (DDPG, SAC, and PPO) are studied for the task. The agents have trained to succeed in the air combat mission in custom-generated simulation infrastructure. The novel training process is explained in detail. The performances of the agents have been assessed with the simulations performed in different scenarios. The results have shown the effectiveness of the algorithms.