Efficient Lightweight Multimodal Object Detection With Adaptive Gated Cross-Modal Interaction
Abstract
Multimodal object detection that integrates visible-spectrum red-green-blue (RGB) and infrared (IR) imagery has become increasingly important for robust perception in challenging conditions such as low illumination, cluttered backgrounds, and small targets, especially in uncrewed aerial vehicle (UAV) and edge-computing scenarios. However, many existing RGB–IR detection frameworks rely on heavy backbones and fixed or aggressive fusion strategies, which introduce redundant computation and uncontrolled cross-modal noise, limiting their applicability to real-time and resource-constrained platforms. To address these limitations, this article proposes an efficient and lightweight RGB–IR dual-stream detection framework that emphasizes adaptive cross-modal interaction. The proposed method adopts a dual-stream backbone to preserve modality-specific representations and introduces adaptive gated cross-modal interaction (AGCMI) at early stages to enable learnable gated feature interaction, regulating information exchange between RGB and IR branches for effective noise suppression. At later stages, a weight-guided aggregation (WGA) module performs two-stage adaptive fusion with a residual-dominant design, promoting stable performance with minimal parameter overhead while reducing the risk of performance degradation. In addition, a lightweight partial-channel C2f (PC2f) backbone variant is employed to further reduce parameters and computational overhead. Experiments conducted on the DroneVehicle dataset demonstrate that the proposed model achieves 81.6% mAP on the test set with only 4.1M parameters. Compared with representative state-of-the-art (SOTA) multimodal detectors, the proposed approach attains competitive accuracy while offering a substantial reduction in model size and improved efficiency. These results demonstrate that the proposed RGB–IR dual-stream detection framework achieves an effective balance between detection performance and deployment efficiency, suggesting strong potential for real-time multimodal perception tasks on resource-constrained platforms.