3D Object Detection with Twin-Surface Depth Completion and Pseudo-LiDAR Grid Fusion

Most 3D detection methods relying solely on LiDAR are inevitably affected by the sparsity of point clouds. Given that color images can provide abundant additional details to enrich LiDAR data, many studies have explored the fusion of LiDAR points and color images to improve 3D object detection accuracy. However, these approaches have not fully addressed the issue of LiDAR point sparsity, and different data representations between images and point clouds pose challenges for effective fusion, often leading to suboptimal performance. This work proposes a novel 3D object detection framework, which integrates two key components: pseudo point generation and object detection by fusing pseudo and LiDAR points. For the first component, we employ a twin surface estimation method to realize depth completion, which applies a fuzzy model based on the binary ambiguity hypothesis, resulting in the estimation (foreground and background surfaces) of twin surfaces. By fusing these twin surfaces with appropriate weighting, this work generates a dense depth map to produce pseudo-points enriching a sparse LiDAR point cloud. For the second component, this work introduces a fusion strategy called 3D grid-wise focused fusion, which effectively combines the pseudo-point cloud and the original LiDAR point cloud for object detection. We evaluate our approach with a benchmark KITTI dataset, which outperforms traditional 3D object detection methods relying solely on single sensor.