Hierarchical attention fusion network with dynamic coordinate and group-aware enhancement for remote sensing semantic segmentation

Abstract

Remote sensing image segmentation is challenged by large object scale variations, blurred boundaries, and complex backgrounds. To address these issues, we propose HAFNet, a hierarchical attention fusion network built upon U-Net with collaborative innovations in feature extraction, cross-layer fusion, and feature enhancement. First, we embed a Dynamic Gated Coordinate Attention (DGCA) module in the encoder. It incorporates coordinate information to capture direction-aware long-range dependencies and uses a dynamic gating mechanism to adaptively adjust attention weights along vertical and horizontal directions, improving geometric modeling of elongated roads and irregular water bodies. Second, a Multi-scale Dynamic Calibration Fusion Module (MDCFM) bridges the semantic gap between adjacent encoder layers. It enhances salient regions via spatial attention, generates bidirectional weight matrices for crosscalibration between low-level details and high-level semantics, and applies DGCA for direction-aware enhancement, mitigating semantic mismatch in feature fusion. Finally, a Group-aware Dynamic Focusing Module (GDFM) is introduced in the decoder. It partitions features along channels, applies CBAM independently to each group to capture multi-directional edge information, and employs dynamic weight recalibration to strengthen key regions while preserving feature integrity, enabling precise boundary reconstruction. Experiments on ISPRS Potsdam and LoveDA datasets show that HAFNet achieves superior mIoU and F1 scores over existing methods, particularly in small object segmentation and boundary refinement, demonstrating the effectiveness of the collaborative design of the three modules.