SDP-GS: Sparse-view Gaussian splatting via segmentation-aware depth priors

The 3D Gaussian Splatting (3DGS) algorithm has demonstrated impressive performance in recent sparse view synthesis tasks. However, existing sparse view synthesis algorithms based on depth priors do not fully address outliers in monocular depth priors, and inaccurate depth supervision further impedes the learning of the scene's 3D structure in sparse view synthesis. Inspired by the neurological hypothesis of 3D spatial cognition, which posits that 3D spatial cognition consists of subcomponents of spatial cognition and their relative positions, we propose sparse-view Gaussian splatting via segmentation-aware depth priors (SDP-GS). A segmentation-level depth refinement module is first introduced, which combines sparse approximate depth ground truth with dense monocular depth estimations to yield more accurate and reliable depth labels. Experimental results confirm a strong correlation between the accuracy of depth refinement and the segmentation regions. Next, segmentation-aware Gaussian splatting is utilized to simultaneously perform scene rendering and acquire labels for synthesized viewpoints, providing segmentation information for segmentation-level correlation loss in pseudo viewpoints. Additionally, based on reliable depth labels and segmentation masks, depth-consistent constraints are applied to supervise the rendering depth information for both known and pseudo viewpoints. Results on public datasets, like LLFF, and Mip-NeRF360, show that the method improves edge detail clarity and achieves competitive performance in novel view synthesis under sparse viewpoint configurations. Project page: https://github.com/dengyangyan/SDP-GS.