Spectral–Spatiotemporal Prompting for Multispectral UAV Tracking

Abstract

UAV tracking often faces critical challenges such as target appearance variations and poor lighting conditions. Compared to RGB images, multispectral images (MSIs) possess rich spatial–spectral information, enabling trackers to better handle scenarios involving illumination variations and similar colors. Existing multispectral (MS) tracking methods typically rely on full fine-tuning of RGB-based models and inadequately exploit temporal information, leading to suboptimal performance due to limited training data. To address these issues, we propose a unified spectral–spatiotemporal prompting (SSTP) for MS UAV tracking, named SSTP. We design a dual-granularity spatial–spectral fusion ( $\mathbf {DGS^{2}F}$ ) module to extract and fuse coarse-grained spatial features and fine-grained spectral features, enabling more comprehensive utilization of detailed spectral information. Furthermore, we introduce hierarchical temporal modeling, where short-term memory prompt (SMP) enhances the backbone network for joint spectral–spatiotemporal feature extraction, while Mamba-based long-term memory prompt (MLMP) facilitates long-range motion sequence modeling. Extensive experiments on the MS tracking benchmarks demonstrate that our method achieves state-of-the-art performance while maintaining real-time efficiency, with separate training and evaluation conducted on each benchmark, achieving an AUC of 69.2% on the MUST dataset and leading performance on both the HOT2020 and HOT2024 datasets. The code will be available at https://github.com/YXYbit/SSTP