3D Photography with One-shot Portrait Relighting

3D photography is a fascinating way to synthesize novel views from limited captured views by using image-based rendering techniques. However, due to the lack of consideration on the light condition, the existing methods cannot achieve vivid results for augmented reality systems and virtual reality systems. In this paper, we present a physical-based framework that explicitly models 3D photography with relighting from a one-shot portrait. Instead of directly rendering new views, we first propose a facial albedo extraction network (FAENet) for synthesizing new views under different light conditions. In order to render more realistic reflected light, we come up with a solution for accurate mesh reconstruction through fine-grained portrait depth estimation. By taking advantage of these two technical components, our method is capable of generating novel views with different lighting conditions, which can faithfully deliver realistic rendered results. Extensive experiments show the proposed method can achieve better visual results.