Efficient Incremental Optimization for 3-D Gaussian Splatting From Mobile Camera Videos With Unknown Poses
TL;DR
This paper proposes a framework for efficient incremental optimization of 3D Gaussian Splatting models and achieves a 24% relative improvement in SSIM with just 120 seconds of additional optimization on the Mip-NeRF360 dataset.
Abstract
Recent advances in 3D reconstruction using mobile cameras have expanded their potential applications beyond traditional domains such as surveying and virtual reality, extending to a wide range of industries. In particular, 3D Gaussian Splatting (3DGS), which can generate photorealistic novel view synthesis from video captured with off-the-shelf RGB cameras, shows promise for industrial use cases involving mobile camera systems that collect images in real time. However, pipelines based on offline Structure-from-Motion (SfM), e.g., COLMAP, are computationally expensive and thus limit practical deployment. While neural network–based acceleration methods have emerged, they are typically limited to processing a small number of input frames, constraining both reconstruction accuracy and spatial coverage. This paper proposes a framework for efficient incremental optimization of 3DGS models. Our framework enables fast and effective fine-tuning by adaptively adjusting the camera poses for additional image frames based on the relationship between L1 and SSIM rendering losses used for 3DGS optimization. Applied to rapidly initialized 3DGS models, our approach achieves a 24% relative improvement in SSIM with just 120 seconds of additional optimization on the Mip-NeRF360 dataset.