When Maps Are Incomplete: Evaluating Vision-Language-Guided Navigation Against Nav2

Abstract

Real-world mobile robots may not always have access to complete or accurate prior maps. In such cases, vision-language models (VLMs), which combine visual perception with semantic reasoning, offer a promising way to interpret environmental changes and revise navigation decisions. This study develops and evaluates two VLM-guided navigation architectures under incomplete-map conditions. The first uses a double-VLM structure for global route reasoning and local subgoal-region selection. The second uses a single-global VLM planner that generates rolling waypoint plans for Nav2 execution. Both preserve Nav2 for localization, costmap updates, and low-level execution. Three Isaac Sim environments were used: an office with hidden furniture, a maze with an omitted wall, and a warehouse with blocked aisles. For each environment, a true map generated an oracle path, while trials used an incomplete test map. Results show that Nav2 performed best when hidden obstacles were detected early, achieving 100% success in the office, but struggled after committing to blocked routes. The single-global VLM planner achieved 100% success in the maze, while the double-VLM planner showed blocked-route recognition but suffered from local execution failures. These findings suggest that VLMs are most useful for high-level route reinterpretation, while geometric planners remain essential for local execution.