Multistage Fusion Approach of Lidar and Camera for Vehicle-Infrastructure Cooperative Object Detection

We propose VICOD, a multistage fusion approach of lidar and camera for vehicle-infrastructure cooperative object detection. The proposed network takes the vehicle-side point cloud and image data with the infrastructure-side point cloud data as inputs, after extracting features, the vehicle-side detection results were generated through the region proposal network and the second-stage detection network, and the infrastructure-side detection results were generated through the region proposal network and the detection head. The detection boxes fusion network performs the match and combination of the vehicle-side and infrastructure-side detection results to generate fused detection boxes, which improves detection accuracy and expands the sensing range of the vehicle side. We compare the results with the benchmark provided by the DAIR-V2X dataset [1] we used, which shows that our scheme significantly improves the average precision of vehicle-infrastructure cooperative object detection. Besides, because we use only point cloud data on the infrastructure side for late fusion, the infrastructure-to-vehicle data transmission cost and time delay will be lower compared to using vehicle-side and road-side sensors for early fusion and feature-level fusion.