A System-Oriented Lightweight Vision-Based Framework for Real-Time UAV Aerial Surveillance
Abstract
Reliable detection of small uncrewed aerial vehicles (UAVs) is essential for low-altitude airspace surveillance, but remains challenging in practical onboard and edge-side systems due to long-range observation, tiny target scale, complex backgrounds, and strict constraints on computation, memory, latency, and power consumption. To address these challenges, this article proposes SPhantomNet, a system-oriented lightweight vision-based UAV detection framework designed for real-time aerial surveillance under resource-constrained deployment conditions. Instead of relying on isolated lightweight module substitution, SPhantomNet is built upon a coordinated high-resolution-preserving design. A P2/P3/P4 feature-fusion topology is introduced to reduce fine-grained spatial information loss caused by excessive downsampling. A Shifted Pinwheel Convolution (SPConv) module is designed to enlarge the effective local spatial aggregation range through directional shifted sampling, improving contextual representation for distant UAV targets. In addition, GhostDynamicConv combines inexpensive Ghost feature generation with input-dependent convolutional responses to enhance adaptive feature representation while controlling model complexity. Extensive experiments on the DUT Anti-UAV benchmark show that SPhantomNet achieves 0.934 mAP50, 0.630 mAP50-95, and 0.535 AP-Small with a model size of only 4.3 MB, outperforming representative YOLO-family baselines, non-YOLO lightweight detectors, anchor-free detectors, and the transformer-based LW-DETR baseline. Multidataset evaluations on MAV-VID and UAV-Eagle further verify the robustness of the proposed framework across different UAV sensing scenarios. Moreover, an ONNX/TensorRT-compatible deployment variant, SPhantomNet (Deploy), is evaluated on an NVIDIA Jetson AGX Orin platform. The deployment results demonstrate real-time inference using a TensorRT FP16 engine, stable thermal behavior, and practical system-level power consumption under embedded hardware constraints. These results indicate that SPhantomNet provides an effective accuracy–efficiency tradeoff for real-time small-UAV detection in resource-constrained low-altitude surveillance systems.