SparKLoc: Sparse Key-Gaussian Matching for 3D Gaussian-Based Visual Localization

Abstract

3D Gaussian Splatting (3DGS) has emerged as an effective scene representation for visual localization, but most existing methods rely on embedding keypoint descriptors into Gaussian primitives, leading to high memory overhead and requiring joint optimization of geometry and features. We propose SparKLoc, a visual localization framework that directly matches 2D image features with 3D Gaussian primitives, eliminating the need for descriptor distillation. Our method operates on a pre-built 3DGS model, fully decoupling scene reconstruction from feature learning. Furthermore, we introduce a sparse selection strategy that extracts a compact set of discriminative key-Gaussians for matching, reducing the memory footprint significantly compared to the keypoint-distillation approaches. We train a cross-modal attention matcher that establishes 2D-3D correspondences between patchwise image features and learned 3D primitive features. Camera pose is first estimated via PnP + RANSAC based on the correspondence set and the estimate is subsequently refined by image-to-image feature matching against a rendered view. Experiments on standard visual localization benchmarks demonstrate that SparKLoc outperforms the prior 3DGS-based feature-distillation approaches with substantially improved memory efficiency.