DHMD: Harnessing the Power of State-Space Models for Advanced Vehicle-Aware RGB–T Object Detection in Spatial–Frequency Domain

Abstract

Red-green-blue–thermal (RGB–T) multimodal object detection has become an effective paradigm for robust remote sensing perception by exploiting the complementary characteristics of visible (VI) and thermal imagery. However, existing convolutional neural network (CNN)- and Transformer-based RGB–T detectors still suffer from several limitations, including insufficient global context modeling or high computational cost, modality heterogeneity and cross-modal feature misalignment, inadequate spatial–frequency feature interaction, and weak perception of small vehicles in complex remote sensing scenes. To address these challenges, this article proposes a novel dual-head Mamba detector (DHMD), an accuracy-oriented RGB–T detector that achieves a favorable tradeoff between detection accuracy and computational efficiency for remote sensing vehicle detection. The proposed method focuses on two fundamental aspects: cross-modal discrepancy modeling and task-aware vehicle representation. First, to address the semantic misalignment and spatial–frequency representation discrepancy between RGB and thermal modalities, DHMD establishes a unified cross-modal representation learning strategy. A dual-Mamba backbone is employed to efficiently capture long-range dependencies with linear computational complexity, while an align-fuse module for semantic flow (AFMSF) alleviates cross-modal semantic misalignment through bimodal semantic enhancement, learnable semantic flow field prediction, and deformable alignment-based feature fusion. Meanwhile, a space–frequency interactive mixer (SFIM) jointly models spatial structures and frequency-domain information to strengthen spatial–frequency feature coupling and enhance the representation of small vehicles. Second, to address the small-scale, densely distributed, and geometrically structured characteristics of remote sensing vehicles, a task-aware vehicle representation strategy is developed by incorporating vehicle-aware geometric priors into adaptive multiscale feature fusion through a rectangular self-adaptive calibration adaptive spatial feature fusion (RSC-ASFF) module, thereby improving the robustness of vehicle perception in complex scenes. Extensive experiments on the DroneVehicle, vehicle detection in aerial imagery (VEDAI), and M3FD datasets demonstrate that DHMD achieves state-of-the-art detection performance, obtaining mAP50 scores of 0.834, 0.679, and 0.848, respectively. These results validate the effectiveness and efficiency of the proposed framework for red-green-blue–thermal (RGB–T) remote sensing vehicle detection.