HFER-SAM: Boundary-Aware Segment Anything Model via High-Frequency Enhanced and Residual Injection for Building Extraction

Abstract

Automatic building extraction from high-resolution remote sensing imagery is vital for urban applications. Although the Segment Anything Model (SAM) offers powerful semantic capabilities, its direct application faces severe challenges. The inherent low-pass filtering of Vision Transformers typically causes blurred building edges, distorted right angles, and severe adhesion among dense clusters. To address these bottlenecks, we propose a Boundary-Aware Segment Anything Model via High-Frequency Enhanced and Residual Injection (HFER-SAM). First, a Multi-scale Semantic Adaptation Module (MSAM) uses domain adapters and coordinate attention to recover spatial priors for precise domain alignment. Second, a High-Frequency Residual Injection Module (HFRIM) employs a lightweight CNN to extract roof textures and edge gradients. These are injected as residual terms into the semantic stream to explicitly sharpen physical boundaries. Finally, a Boundary-Constrained Dual-Task Head (BC-DTH) utilizes morphological supervision to sever erroneous connections between adjacent buildings and resolve target adhesion. Experiments on the WHU and Inria datasets reveal that HFER-SAM achieves an Intersection over Union of 91.53% and 82.06%, respectively. Our method significantly outperforms existing state-of-the-art models in both accuracy and boundary fidelity, providing a robust avenue for high-precision building mapping.