Semantic-Driven Adversarial Reconstruction Learning for Open-Set Recognition in Remote Sensing Imagery

Abstract

Open-Set Recognition (OSR) in Remote Sensing Scene Images (RSSIs) is severely hindered by complex backgrounds, which obscure the generative failures of unknown classes in traditional reconstruction-based methods. To address this, we propose a Semantic-Driven Adversarial Reconstruction (SDAR) framework that shifts the OSR paradigm to targeted semantic verification. A two-stage training protocol is utilized: the encoder is first trained via classification to extract high-level semantic features, after which the decoder is unfrozen for joint training to achieve semantic-level reconstruction. Driven by these semantics, a Class Activation Map (CAM) is used to dynamically mask only the core semantic elements, and a background-agnostic loss forces the model to exclusively reconstruct these critical parts rather than complex backgrounds. Consequently, the model solely masters the generative essence of known semantics; when presented with unknown categories lacking these identified semantic rules, it inevitably fails to reconstruct their masked cores, yielding a highly discriminative metric for OSR.