Mining Heterogeneous Advantages via Feature Purification and Recombination for Multimodal Object Detection

Abstract

Visible-infrared (IR) image fusion constitutes a pivotal paradigm for achieving all-weather robust object detection. However, its performance gains are frequently impeded by cross-modal semantic heterogeneity. This issue degrades representation quality in two primary ways. First, heterogeneous noises (e.g., thermal artifacts and shadows) induce severe feature contamination. Second, modality-specific advantages are often diluted. Due to heterogeneous feature representations, targets that are salient in only a single modality are frequently suppressed by invalid information from the other, ultimately leading to missed detections. Addressing these issues, we propose a novel framework for mining heterogeneous advantages via feature purification and recombination for multimodal object detection, termed FPR-Net. To this end, a cross-adversarial heterogeneous noise suppression (CAHN) mechanism is designed to facilitate feature purification. By mining the untainted semantic advantages of the complementary modality, CAHN dynamically guides the noise-corrupted modality to achieve robust semantic clarity. Furthermore, a saliency-guided feature recombination (SGFR) strategy is proposed to mitigate the dilution of modality-specific advantages. SGFR uses local response deviation to construct spatial saliency priors and enhance task-relevant components. By disentangling and recombining these features, it systematically mines heterogeneous advantages. Extensive experiments on four public datasets demonstrate that FPR-Net achieves significant performance gains in both remote sensing and natural scenes. In particular, FPR-Net achieves an $\text {mAP}_{50}$ of 81.5% on the DroneVehicle dataset and 83.0% on the VEDAI dataset.