Spatial–Spectral Multiscale Mamba Fusion Network for Hyperspectral and LiDAR Data Classification

Abstract

Joint classification of multimodal remote sensing data, such as hyperspectral image (HSI) and light detection and ranging (LiDAR), is important for Earth observation. Early deep learning-based methods usually adopt a single network to extract the features of HSI and LiDAR (HSI-LiDAR) data, respectively. However, this kind of single-network architecture has limited ability to represent complex features, which may deliver a suboptimal classification result. Moreover, many existing hybrid network frameworks often rely on simple feature fusion operations, such as concatenation, cascading, or summation, and fail to explicitly model the complex complementary relationships across modalities. To address this issue, this article proposes a spatial–spectral multiscale Mamba (S2MSMamba) fusion network for HSI-LiDAR data classification. In the proposed framework, the Mamba branch models the global long-range dependencies with linear computational complexity, while the convolutional neural network (CNN) branch extracts local spatial textures and edge information. S2MSMamba jointly exploits the complementary strengths of Mamba and CNN by effectively mining the global long-range dependencies and the local spatial structure features of ground objects. In addition, a multiscale hierarchical alignment mechanism is developed to address scale variations of ground objects. Moreover, an adaptive decision-level fusion strategy is further adopted to integrate unimodal feature to refine classification performance. Experiments conducted on the Houston, MUUFL, and Trento benchmark datasets show that the proposed S2MSMamba outperforms existing methods on several evaluation metrics, demonstrating its effectiveness for land-cover classification The proposed method and the data used in this article are available at https://github.com/binzhao1018/S2MSMamba.