Skip to content
Open access

Research on Multimodal UAV Object Tracking Based on a Dual-Branch Interactive Fusion Encoder

2026 · Computers, Materials & Continua · 0 citations · 31 references

Abstract

: Unmanned Aerial Vehicle (UAV) target tracking is essential for security surveillance and search-and-rescue missions. However, existing methods primarily rely on single-modal visible light imagery, leading to high missed-detection rates and poor robustness in complex environments such as nighttime, fog, and occlusion. While multimodal fusion offers mitigation, current approaches are constrained by coarse cross-modal feature alignment, high computational complexity, and challenges in suppressing modality bias. To address these issues, this study proposes a novel multimodal UAV target tracking framework integrating a Dual-Branch Interactive Multimodal Detection Network (DBIMDet) with the ByteTrack tracker. Built upon the Real-Time Detection Transformer (RT-DETR), the network employs parameter-decoupled Residual Network-18 backbones to extract multi-scale visible and infrared features. It introduces a Cross-Modal Interactive Fusion Encoder (CMIFE) utilizing Polarized Linear Attention for efficient global modeling, and incorporates a Spatially-Enhanced Feed-forward Network (SEFN) to inject spatial priors. Additionally, Modality-Aware Bypass Adaptive (Mona) and Dynamic Hyperbolic Tangent (DyT) normalization are employed to mitigate modality bias, while a Multimodal Collaborative Loss (MCLoss) guides the enhancement of modality consistency and complementarity. Extensive experiments on the DroneVehicle dataset demonstrate that, among the compared methods evaluated in this study (MM-Det, RSDet, UA-CMDET, and IVD-NET), the proposed method achieves the best performance on this benchmark, with an mAP50 of 84.33% and an mAP50-95 of 60.26%. Real-world nighttime video evaluations further validate its adaptability under the evaluated challenging illumination conditions. The framework provides a robust solution through efficient fusion and bias suppression, ensuring reliable UAV tracking in challenging conditions.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.