Self-supervised Multi-view Stereo via View Synthesis Representation Consistency

Self-supervised Multi-view Stereo technique aiming at the reconstruction of 3D models from 2D images has made great progress. However, existing methods are mainly based on the premise that the corresponding pixels between different views have similar characteristics. However, in the actual scene, we often suffer from the interference of the occlusion area and non-Lambert surface, which reduces the quality of the depth map obtained by depth estimation, and also affects the accuracy and completeness of the final generated point cloud model. In this paper, we propose a new self-supervised framework that uses joint feature consistency and view synthesis representation consistency to construct the self-supervised signal. Additionally, we add data joint representation augmentation mechanism branch to capture the similarity degree of corresponding pixel points between images, so as to improve the unsatisfactory situation faced in the process of large-scale real depth acquisition. The results of experiments on DTU dataset show that our proposed method has good performance, even better than some supervised methods. Furthermore, the results of experiments on Tanks&Temples dataset prove that it also has good generalization ability.