From Language to 3D Search Goal: A Lightweight Closed-Loop Framework for Language-Guided UAV Target Search

Abstract

A key problem in language-guided UAV target search is how to transform a language-referred target in the current observation into an executable spatial goal. Existing methods either predict actions directly or introduce relatively heavy mapping, memory, or planning modules, making the intermediate link between semantic grounding and spatial execution difficult to examine in isolation. In this paper, we present a lightweight closed-loop framework for language-guided UAV target search and reaching. Given a natural-language instruction, an RGB image, a depth map, and the UAV pose, the system first localizes a 2D target with a vision-language model, then recovers a 3D search goal in the world coordinate system using depth cues and camera geometry, and finally executes point-to-point flight toward the recovered goal. Rather than addressing obstacle avoidance, global mapping, cooperative coverage, or complex trajectory optimization, we focus on validating whether semantic target grounding, explicit 3D search-goal recovery, and flight execution can form an effective perception-to-execution loop. Preliminary AirSim results show that observation-consistent 3D search-goal recovery yields more stable target-search execution than both an image-plane heuristic baseline and a fixed-depth recovery baseline. These results suggest that explicit 3D search goals provide a practical and interpretable bridge between semantic grounding and spatial execution.