OpenGPL: Multimodal Open-Set Recognition via Dual-Level Pseudo-Unknown Generation and Positive–Negative Prompt Learning
The multimodal fusion of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has advanced remote sensing (RS) classification. However, existing methods are mainly based on the closed-set assumption and are thus less effective in real-world open-set scenarios where unknown categories may appear during inference, due to the lack of representative unknown samples and the semantic gap across modalities. To address this, we propose OpenGPL, a novel multimodal open-set recognition (OSR) framework that integrates contrastive language-image pretraining (CLIP) with generative models. First, to tackle the problem of missing unknown samples, we propose a dual-level pseudo-unknown generation strategy. At the pixel level, we employ a diffusion model with a multimetric filtering mechanism to synthesize high-quality pseudo-unknown samples. At the feature level, we introduce a normalizing flow to sample features near the decision boundary, thereby tightening the feature distribution of known classes via an outlier exposure (OE)-inspired strategy. Second, to bridge the multimodal semantic gap, we design a positive–negative prompt learning mechanism. By maximizing the similarity between features and positive prompts while pushing them away from negative prompts, this scheme explicitly constructs a more discriminative semantic space, effectively enlarging the boundary between known and unknown categories. Finally, we adopt a dual-verification mechanism that fuses energy scores with similarities to negative prompts to accurately reject unknown classes. Comprehensive experiments conducted on three benchmark HSI-LiDAR datasets demonstrate that the OpenGPL achieves competitive performance in recognizing known classes and detecting unknown categories in open-set scenarios.