T-UNet: A Novel TC-Based Point Cloud Super-Resolution Model for Mechanical LiDAR

Mechanical LiDAR is one of the most crucial perception sensors for autonomous vehicles. However, the vertical angular resolution of low-cost multi-beam LiDAR is small, limiting the perception and movement range of mobile agents. This paper presents a novel temporal convolutional (TC)-based U-Net model for point cloud super-resolution, which can optimize the point cloud of low-cost LiDAR based on fusing spatiotemporal features of the point cloud. We project the 3D point cloud on a 2D image plane and extend a U-Net convolutional neural network model with a temporal convolutional (TC) module for processing consecutive frames. Each time the model generates one dense/up-sampled image from low-end LiDAR consecutive frames and projects it back into the 3D space as the final result. Considering the intrinsic noise of LiDAR, the structural similarity index measure (SSIM) is introduced as the loss function. Experiments are carried out on both datasets generated by the CARLA simulator and a small-scale dataset collected from actual road conditions with a local vehicle platform. Results show that the proposed model achieves a high peak signal to noise ratio (PSNR). It means the T-UNet model can effectively upsample the sparse point cloud of low-cost LiDAR to a dense point cloud which is almost indistinguishable from the high-end LiDAR point cloud. The source code can be accessed at https://github.com/donkeyofking/lidar-sr.git