On real-time semantic segmentation with comprehensive off-road datasets for enhanced terrain classification

This paper introduces a novel approach to dataset annotation across 13 diverse classes for terrain classification. The method is applied to an established set of terrain-related datasets, and a new dataset by the authors’ team referred to as CranfieldTerra. These datasets were trained using 18 distinct neural network (NN) architectures, and based on overall test accuracy, precision, recall, and Intersection over Union (IoU) scores, the accuracy of each label, and training time, the effectiveness of these models was evaluated. Furthermore, the methodology has been systematically validated and tested in real-time using a platform called the Husky-A200. This comprehensive evaluation ensures the reliability and accuracy of the methodology under practical conditions. An innovative real-time switch model is introduced that dynamically selects the most appropriate neural network from the set of pre-trained models based on the calculated environmental density rate and the presence of specific features like ’Person, House, Vegetation Area, and Mud Area’ class counts. This approach significantly enhances the adaptability and performance of real-time semantic segmentation in varied environmental conditions, leading to more efficient terrain classification.