Free-Viewpoint Multi-View Face Reconstruction Using Deep Learning Method

Accurately recovering the three-dimensional shape of a face from a two-dimensional image is a challenging task with many applications. Although some progress has been made in research based on 3D morphable models, most current methods mainly focus on using a single image for reconstruction. The limited information contained in a single image inevitably limits the effectiveness of face reconstruction. This study proposes a deep learning-based multi-view reconstruction method using 3D morphable models. Without the need for real faces, only weakly supervised learning with multiple face images is used to obtain accurate facial shapes under free view conditions. On the basis of single-view, we have developed an improved shape aggregation method. By using information from different images for facial depth estimation and weighted shape aggregation, the accuracy of the reconstruction is improved. This method allows for fast and accurate 3D face modeling. Compared with the real face obtained from 3D scanning, the average error is 1.51 ± 0.23 mm. The results show that the method achieves good reconstruction accuracy and verifies the effectiveness of the multi-view aggregation strategy.