Mapeo semántico de vocabulario abierto mediante 3D Gaussian Splatting

Abstract

Open-vocabulary semantic maps are key to the autonomy of mobile robots in complex, unstructured environments. Compared to traditional techniques, open-vocabulary semantic maps represent a significant improvement, as they are not limited by a predefined set of categories. For their construction, geometric representations based on 3D Gaussian Splatting enriched with low-level semantic descriptors have gained popularity. However, robotic operations demand high-level labels, such as textual descriptions and categories. This work presents an evolution of OpenSplat3D that integrates Large Vision-Language Models (LVLMs) for the description and categorization of object instances, whose consistency is validated within the latent space of Vision-Language Models (VLMs). Additionally, the geometric reconstruction has been optimized through dense initialization and the use of a depth-based loss function. Both proposals are validated using the Replica and ScanNet datasets, yielding accurate, semantically enriched maps.