GSBrief: A Globally Consistent Descriptor with 3D Gaussian Splatting for Visual Localization

Visual localization is a critical component in a wide range of applications. Recent advancements in scene representation, particularly the use of 3D Gaussian Splatting (3D GS), have introduced promising opportunities for enhancing localization pipelines. However, effectively leveraging the features of 3D GS while maintaining full integration of texture, geometric context, and global consistency remains a significant challenge. In this paper, we introduce GSBrief, a novel, globally consistent descriptor designed specifically for 3D GS based visual localization. GSBrief captures scene features through a structured extraction process that seamlessly incorporates both texture and geometry information. The resulting descriptors are engineered to be invariant to scale, position, and rotation, ensuring robust performance across a wide range of conditions. Building on GSBrief, we propose GSBriefNet, a regression network designed to predict GSBrief descriptors, which is based on the Swin Transformer architecture. GSBriefNet employs a Siamese network design to enforce global consistency and simultaneously regresses point maps to recover the ground truth scale. The predicted GSBrief descriptors can be directly applied to tasks such as relative camera pose estimation, relocalization, and Simultaneous Localization and Mapping (SLAM). We demonstrate the effectiveness of our approach through experiments on benchmark datasets, including ScanNet, 7 Scenes, Cambridge Landmarks, TUM RGB-D and Bonn. The results show that our method achieves state-of-the-art performance across all three tasks, providing a robust solution for visual localization.