Look, Then Guide: Training-Free Decoupling of Visual Goal Recognition from Navigation in Aerial Vision-language Navigation

Abstract

Vision-language models (VLMs) show strong visual understanding for aerial navigation, but their action generation remains unreliable. We study this gap in the context of orbit-search-based navigation, where an aerial agent circles a reference landmark while a VLM detects a language-described goal. We find that the VLM recognizes goals with stable accuracy across configurations, yet the subsequent approach phase—where the VLM must navigate toward the recognized goal—succeeds only 11.3% of the time. The failure is not a detection problem but a navigation problem: the visual approach prompt uses a format absent from the training data, and no amount of prompt engineering restores performance. We propose Geometric Waypoint Navigation, which eliminates VLM-based navigation after detection. The aerial agent moves toward the detected direction using the same deterministic waypoint-following controller already proven in the orbit phase, requiring no additional training or parameters. On the CityNav test unseen split, success rate increases from 35.5% to 49.0%, substantially surpassing all published baselines including GeoNav (25.9%), HTNav (21.7%), and FlightGPT (21.2%). Detection rates remain exactly constant at 31.0%, confirming that gains originate entirely from improved navigation rather than improved perception. Controlled experiments show that geometric waypoint following outperforms VLM Replay by 8.0 percentage points, and a fixed 30m approach distance outperforms VLM-estimated adaptive distances by 3.0 points. Our findings suggest a broader design principle: when a learned model serves as a detector in an embodied pipeline, separating perceptual decisions from motor execution can substantially improve reliability.