Abstract
In this work, we propose a self-improving artificial intelligence system to enhance the safety performance of reinforcement learning (RL)-based autonomous driving (AD) agents using black-box verification methods. RL algorithms have become popular in AD applications in recent years. However, the performance of existing RL algorithms heavily depends on the diversity of training scenarios. A lack of safety-critical scenarios during the training phase could result in poor generalization performance in real-world driving applications. We propose a novel framework in which the weaknesses of the training set are explored through black-box verification methods. After discovering AD failure scenarios, the RL agent's training is re-initiated via transfer learning to improve the performance of previously unsafe scenarios. Simulation results demonstrate that our approach efficiently discovers safety failures of action decisions in RL-based adaptive cruise control (ACC) applications and significantly reduces the number of vehicle collisions through iterative applications of our method. The source code is publicly available at https://github.com/data-and-decision-lab/self-improving-RL.
Original language | English |
---|---|
Title of host publication | Proceedings - ICRA 2023 |
Subtitle of host publication | IEEE International Conference on Robotics and Automation |
Publisher | Institute of Electrical and Electronics Engineers Inc. |
Pages | 5631-5637 |
Number of pages | 7 |
ISBN (Electronic) | 9798350323658 |
DOIs | |
Publication status | Published - 2023 |
Event | 2023 IEEE International Conference on Robotics and Automation, ICRA 2023 - London, United Kingdom Duration: 29 May 2023 → 2 Jun 2023 |
Publication series
Name | Proceedings - IEEE International Conference on Robotics and Automation |
---|---|
Volume | 2023-May |
ISSN (Print) | 1050-4729 |
Conference
Conference | 2023 IEEE International Conference on Robotics and Automation, ICRA 2023 |
---|---|
Country/Territory | United Kingdom |
City | London |
Period | 29/05/23 → 2/06/23 |
Bibliographical note
Publisher Copyright:© 2023 IEEE.
Funding
ACKNOWLEDGEMENT This work is supported by Istanbul Technical University BAP Grant NO: MOA-2019-42321. We gratefully thank Eatron Technologies for their technical support.
Funders | Funder number |
---|---|
Istanbul Teknik Üniversitesi | MOA-2019-42321 |
Keywords
- Autonomous Driving
- Black-Box Verification
- Deep Reinforcement Learning