Cross Fusion of Point Cloud and Learned Image for Loop Closure Detection

Loop closure detection (LCD) plays a crucial role in simultaneous localization and mapping (SLAM) systems to eliminate accumulated odometry drifts as the map is built, and using multi-modal information can improve the accuracy and robustness of this system compared to single sensor. However, traditional fusion methods often require sophisticated space alignment of the sensors, which places high requirements on hardware equipment. Moreover, these methods fuse the point cloud and image in a weak manner, such as concentrating two kinds of features using calibration information, which makes they cannot take full use of the multi-modal information. In this letter, we propose a method for fusing asymmetric point clouds and images to detect loops. Bird's Eye View (BEV) of point cloud and image achieve information interaction and associations through learnable modules, rather than through hard intrinsic and extrinsic matrices to perform cross-fusion. Multi-information is fused in BEV grids, then the features of BEV key points are enhanced. Through experiments, it is found that under the training of this fusion strategy, the image information can be encoded into the BEV processing modules, and ultimately the performance of the method can be improved without images. The proposed method is evaluated on KITTI and KITTI-360, and the results demonstrate the state-of-the-art performance and remarkable efficiency.