Improved 3D Gaussian splatting integrating VGGT and LiDAR: Application in visual relocalization for UAV inspection of large aircraft

Abstract

For high-precision unmanned aerial vehicle (UAV) visual relocalization in large-scale aircraft inspection, this paper proposes an improved 3DGS method integrating visual geometry grounded transformer (VGGT) with LiDAR and designs a coarse-to-fine two-stage image matching strategy to tackle the challenge. First, the low-rank adaptation technique is adopted to fine-tune the segment anything model (SAM3), and a dedicated dataset with refined aircraft structural features is constructed to achieve high-precision foreground segmentation of aircraft targets, which eliminates environmental interference for 3D mapping. Second, point clouds generated by VGGT are registered and fused with LiDAR point clouds, providing 3DGS with initialized data possessing real physical scales. Meanwhile, a smooth L1 depth loss is introduced to build a joint photometric–geometric optimization framework, so as to enhance the geometric consistency of the model. Finally, the pose estimation problem is converted into an image similarity matching problem. Coarse localization is realized through sparse sampling on multi-layer concentric hemispheres combined with dense matching via RoMa, while precise localization is completed by fusing feature distance, structural similarity index measure, and peak signal-to-noise ratio. Experimental results show that the improved SAM3 segmentation model achieves superior performance. The proposed improved 3DGS outperforms mainstream methods, including the original 3DGS, in rendering metrics and training efficiency. In addition, the presented pose estimation method achieves higher localization accuracy than HLoc, NeRF-Loc, and other comparative approaches in both indoor and outdoor scenarios. This method ensures high-precision and real-time mapping and localization, provides an effective scheme for aviation UAV intelligent inspection, and broadens the application of 3DGS in industrial automation.