Monodepth-Based Object Detection and Depth Sensing for Vehicle Vision Systems

For the appropriate deployment of digital twin technologies aimed at enhancing road safety, it is essential to detect surrounding objects and precisely estimate their distances. In autonomous vehicles, this is typically accomplished using 2D cameras, stereo cameras, 3D cameras, or expensive distance sensors. Measuring the distances of detected objects in real-time can significantly contribute to fully autonomous driving. However, estimating distances using 2D monocular cameras is quite challenging. This paper employs two separate artificial neural networks for object detection and the estimation of object distances in monocular camera images. In addition, a new algorithm is applied to obtain a more robust distance, known as the Weighted Distance Estimator Algorithm. Models that operate independently and in parallel use You Only Look Once for object detection. Simultaneously, a Monodepth2 model, based on the U -Net architecture, is used to perceive the estimated distances. U-Net is an encoder-decoder network with skip connections. The developed application running on Nvidia Jetson Xavier NX demonstrates real-time processing performance smoothly at 12 FPS. This application showcases the robust performance of an embedded device that can be utilized in autonomous vehicle systems. It shows that reliable object detection and distance estimation in autonomous vehicles are possible even using 2D monocular cameras, with a Mean Absolute Error of 1.51 meters.