Road Weather Condition Recognition via Fusing Images and Upsampled Point Cloud Reflection Intensities

The recognition of road weather conditions can provide environmental information for autonomous driving systems, enabling them to develop safe driving strategies. Previous studies have utilized cameras or radars to identify road weather conditions, but they were unable to detect specific dry, wet, or snowy road areas. Some methods based on image segmentation can detect these areas, but their performance is limited by the similar features exhibited by dry and wet roads. In this article, we propose a novel method for segmenting dry, wet, and snowy road areas based on the fusion of images and point cloud reflection intensities. By fusing the point cloud reflection intensity, we enhance the distinction between different road types, thereby improving the accuracy of the segmentation. Furthermore, we address the challenge posed by sparser point cloud data compared to image pixels by introducing a point cloud reflection intensity upsampling method. Unlike other interpolation methods, our proposed method relies on the physical principle that an object's reflectivity is correlated with its color and texture features, which makes the point cloud reflection intensity obtained by upsampling more reliable. To evaluate the performance of our method, we conducted experiments on a self-collected dataset that contains over 20 000 images. More than 4000 images were annotated at the pixel level, and each of them was accompanied by the corresponding point cloud data. The experimental results demonstrate that our proposed algorithm achieves an impressive mean intersection over union (mIoU) of 87.94%. Compared to image segmentation algorithms that do not fuse point clouds, our algorithm achieves an approximately 3% improvement in mIoU, substantiating its effectiveness.