High-fidelity view synthesis for light field 3D display

Light field display is a glasses-free 3D display that samples and reconstructs the light field guided by ray optics. Generating a high-quality and artifact-free synthetic image for light field displays from a sparse set of input views is a significant challenge, primarily due to disocclusion holes created during view synthesis. To address this issue, a high-fidelity view synthesis method is proposed using a two-stage hybrid framework. In the first stage, a dense set of virtual views is synthesized from sparse RGB-D inputs using forward warping, followed by a multi-view fusion process to resolve partial disocclusions. Crucially, this stage precisely identifies unrecoverable full disocclusion regions, generating an adaptive mask that assigns the subsequent hole filling task to the specialized refinement network in the second stage. In the second stage, a geometry-aware generative model, termed the Coherent Multi-view Refinement Network (CMVR-Net), is proposed to perform generative inpainting within the masked regions identified in the previous stage. By processing a stack of adjacent views simultaneously, the CMVR-Net leverages local light field context to generate content that respects spatial-angular coherence. This approach guarantees geometric consistency across views, effectively eliminating the artifacts associated with conventional per-view inpainting and producing stable visual results. To facilitate robust training and evaluation, a comprehensive synthetic dataset featuring diverse 3D scenes rendered with randomized convergent camera configurations is constructed. This dataset effectively addresses the scarcity of publicly available resources for convergent camera arrays. Experiments demonstrate that the proposed method significantly outperforms state-of-the-art approaches, achieving superior quantitative metrics in hole regions and producing visually consistent and artifact-free results on a light field 3D display prototype.