Class Distribution-Aware Adversarial Training for Semantic Segmentation of Imbalanced Remote Sensing Imagery

Abstract

Class imbalance is a fundamental challenge in semantic segmentation of remote sensing imagery, causing deep convolutional networks to systematically neglect minority categories such as vehicles and small water bodies. To address this, we propose a class distribution-aware adversarial training framework that explicitly penalizes the model’s bias toward dominant classes at the representation level. The framework comprises two key components. First, a Hierarchical Attention U-Net (HAU-Net) integrates a Parallel Hybrid Attention Module (PHAM) at multiple scales, combining spatial and channel attention to amplify feature responses for small and sparsely distributed objects. Second, a distribution discriminator is trained to distinguish the predicted per-class probability distribution from a balanced uniform prior; through a minimax game, the segmentation network is forced to produce class-balanced predictions without manual loss re-weighting. Extensive experiments on the ISPRS Potsdam and Vaihingen datasets demonstrate that our method achieves an overall mIoU of 73.82% and delivers substantial improvements on severely under-represented minority classes, including a +12.11% IoU gain on vehicles and a +19.26% IoU gain on clutter/background, compared to our non-adversarial baseline. Ablation studies confirm that both the hierarchical attention design and the adversarial training contribute independently to these improvements. Our framework offers an effective solution to class imbalance in remote sensing segmentation, with the potential to be integrated into diverse segmentation architectures.