CLIP-guided structure-semantic collaborative adversarial erasure for weakly supervised semantic segmentation

Abstract

Adversarial erasing (AE) is commonly used to expand class activation maps (CAM) in Weakly Supervised Semantic Segmentation (WSSS), but existing AE methods struggle with semantic drift, background over-activation, and boundary blur due to limited structural awareness. We propose a CAM optimization method that combines cross-modal semantic constraints and structure-guided modeling within AE. For semantics, a CLIP-based Semantic Distribution Consistency (SDC) module aligns class response distributions between original and reconstructed images, using category texts as anchors to stabilize discriminative cues and suppress background expansion. For structure, a Locally Structure-aware Hybrid Gated Pyramid Pooling (LS-HGPP) module captures local inconsistencies across adjacent scales to produce pixel-wise gating, enhancing fine-scale features at boundaries while preserving coarse context, improving structural completeness and boundary clarity. Experiments on PASCAL VOC 2012 and MS-COCO show our method achieves competitive results, outperforming several mainstream methods including ACR on VOC 2012.