Aware Mahalanobis Convolution and Polygon Structure Enhancement-Based Deep Network for Building Detection in Remote Sensing Images

Abstract

As fundamental elements of urban landscapes, accurate building detection from high-resolution remote sensing imagery is an essential application of remote sensing information interpretation. However, this task remains challenging due to significant variations in building orientation, scale, and geometric shape. Existing methods rely on fixed kernel-based convolutions, struggle to adapt to diverse spatial scales, and fail to effectively model the intrinsic structures of buildings. To address these challenges, we propose aware Mahalanobis convolution (AMC)-polygon structure enhancement (PSE)-Net, an end-to-end building detection network. To better capture buildings with diverse orientations and sizes, the AMC module is presented, which utilizes a content-aware Mahalanobis transformation to automatically adjust the convolution kernels in both the orientation and scale domains based on image content, thereby effectively capturing architectural features across multiple directions and scales. To characterize the intrinsic geometric priors of regular building shapes, the PSE module is then developed, which integrates a self-supervised collinearity-preserving loss to inherit geometric priors and an Earth Mover’s Distance (EMD)-based vertex regression loss to enhance consistency of detection results. In addition, to effectively mitigate semantic confusion between buildings and visually similar ground objects, such as roads and parking areas, the hierarchical spatial–semantic fusion (HSSF) head is designed to integrate multiple levels of semantic features to achieve robust detection. Experiments on the WHU dataset, the Inria Aerial Image Labeling dataset, and the Massachusetts Building dataset demonstrate that AMC-PSE-Net achieves 91.95% intersection over union (IoU) and 95.81% $F1$ -score on the WHU dataset, outperforming state-of-the-art methods.