DMFNet: Phase-Enhanced Multi-Level Deformable Fusion for Multimodal Remote Sensing Classification

Abstract

The collaborative classification of hyperspectral images (HSI) with a complementary modality such as light detection and ranging (LiDAR) or synthetic aperture radar (SAR), which is known as multi-source remote sensing image classification (MRSIC), can combine rich spectral information with structural cues and is helpful for the recognition of complex ground objects. However, effectively incorporating these structural cues into multi-level spectral-spatial features and modeling their spatial relationships remain challenging under limited training samples. To address these problems, a multi-level convolutional collaborative classification network based on the DefMamba fusion mechanism (DMFNet) is proposed in this paper. First, monogenic phase congruency (MPC) responses are added to the complementary modality to enhance its structural information. Then, a dual-branch feature extraction module is constructed to derive multi-level features from HSI and the enhanced complementary modality and aggregate them into a joint representation. Finally, a DefMamba fusion module applies deformable selective scanning to the aggregated features, so as to adaptively model spatial dependencies according to the fused content. Experiments on three public multimodal remote sensing datasets, covering both HSI–LiDAR and HSI–SAR settings, demonstrate that DMFNet achieves the highest mean overall accuracy among the compared methods with 10 labeled training samples per class, with the largest improvement on the HSI-SAR scene. Additional experiments on Houston 2018 further evaluate its classification performance on a large-scale scene.