A Real-World Reinforcement Learning Framework for Safe and Human-Like Tactical Decision-Making

Lane-change decision-making for vehicles is a challenging task for many reasons, including traffic rules, safety, and the stochastic nature of driving. Because of its success in solving complex problems, deep reinforcement learning (DRL) has been suggested for addressing these issues. However, the studies on DRL to date have gone no further than validation in simulation and failed to address what are arguably the most critical issues, namely, the mismatch between simulation and reality, human-likeness, and safety. This paper introduces a real-world DRL framework for decision-making to design safe and human-like agents that can operate in the real world without extra tuning. We propose a new learning paradigm for DRL integrated with Real2Sim transfer, which comprises training, validation, and testing phases. The approach involves two simulator environments with different levels of fidelity, which are parameterized via real-world data. Within the framework, a large amount of randomized experience is generated with a low-fidelity simulator, whereupon the learned skills are validated regularly in a high-fidelity simulator to avoid overfitting. Finally, in the testing phase, the agent is examined concerning safety and human-like decision-making. Extensive simulation and real-world evaluations show the superiority of the proposed approach. To the best of the authors' knowledge, this is the first application of DRL lane-changing policy in the real world.