Yunsong Li

TMA-Fed: Text-Driven Modality-Aware Federated Learning for Multi-Satellite Remote Sensing

OpenGPL: Multimodal Open-Set Recognition via Dual-Level Pseudo-Unknown Generation and Positive–Negative Prompt Learning

The multimodal fusion of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has advanced remote sensing (RS) classification. However, existing methods are mainly based on the closed-set assumption and are thus less effective in real-world open-set scenarios where unknown categories may appear during inference, due to the lack of representative unknown samples and the semantic gap across modalities. To address this, we propose OpenGPL, a novel multimodal open-set recognition (OSR) framework that integrates contrastive language-image pretraining (CLIP) with generative models. First, to tackle the problem of missing unknown samples, we propose a dual-level pseudo-unknown generation strategy. At the pixel level, we employ a diffusion model with a multimetric filtering mechanism to synthesize high-quality pseudo-unknown samples. At the feature level, we introduce a normalizing flow to sample features near the decision boundary, thereby tightening the feature distribution of known classes via an outlier exposure (OE)-inspired strategy. Second, to bridge the multimodal semantic gap, we design a positive–negative prompt learning mechanism. By maximizing the similarity between features and positive prompts while pushing them away from negative prompts, this scheme explicitly constructs a more discriminative semantic space, effectively enlarging the boundary between known and unknown categories. Finally, we adopt a dual-verification mechanism that fuses energy scores with similarities to negative prompts to accurately reject unknown classes. Comprehensive experiments conducted on three benchmark HSI-LiDAR datasets demonstrate that the OpenGPL achieves competitive performance in recognizing known classes and detecting unknown categories in open-set scenarios.

UHFusion: Prompt-Guided Multi-Task Learning Network for Hyperspectral Image Fusion via Multi-Type Mixture-of-Experts

Hyperspectral image (HSI) fusion aims to generate high-resolution HSI by integrating low-resolution hyperspectral data with auxiliary high-resolution sources (e.g., panchromatic (PAN), RGB, or MSI). While recent deep learning-based HSI fusion approaches have achieved promising results, they are typically designed for specific modality pairs and struggle to generalize across diverse fusion settings, leading to redundant architectures and limited adaptability. To overcome these limitations, we propose UHFusion, a prompt-guided multi-task learning (MTL) network for HSI fusion, which enables flexible multi-task adaptation within a single framework. Particularly, UHFusion introduces two key components: 1) a multi-type mixture-of-experts (MTMoEs) that decomposes HSI fusion into complementary spatial, spectral, and spatial–spectral reasoning processes, and dynamically composes expert responses conditioned on the auxiliary modality, allowing the model to selectively emphasize task-relevant spatial structures or spectral characteristics without architectural modification; and 2) a prompt-guided multi-task adaptation module (PMAM), which encodes modality-specific priors into learnable prompt embeddings and leverages a task relation graph to perform task-conditioned feature modulation, thereby improving the spatial–spectral fusion quality across diverse fusion tasks. Extensive experiments demonstrate that UHFusion achieves superior performance and generalization across multiple HSI fusion tasks and modalities, providing an effective all-in-one solution for HSI reconstruction. The code is available at: https://github.com/Jiahuiqu/UHFusion

Planetary Scene Classification via a Novel Zero-Shot Hierarchical State-Space Model

Planetary scene classification plays a fundamental role in geomorphological analysis and autonomous exploration missions. However, planetary terrains exhibit high intraclass structural variability, and their analysis relies on an extremely limited set of annotated samples, making exhaustive premission labeling impractical. Therefore, recognition systems should be able to identify unseen classes with few (and sometimes no) training examples. This naturally motivates the adoption of the zero-shot learning (ZSL) paradigm for planetary scene classification. Existing solutions either require large-scale pretraining or employ heterogeneous and attention-intensive designs, limiting their practicality in data-scarce and resource-constrained planetary environments. To address these issues, we propose HiL-SSM, a hierarchical interactive linear state-space modeling framework for zero-shot planetary scene classification. It employs a unified, attention-free architecture based on structured state-space models (SSMs), enabling joint optimization of visual representation learning and semantic alignment within a single backbone framework. Importantly, it does not rely on large-scale pretraining. A hierarchical stage-wise interaction mechanism is introduced to progressively refine visual–semantic correspondence across multiple representation levels, enabling stronger alignment between geomorphological structures and semantic descriptors. Experiments on the ZSMars dataset demonstrate that the proposed framework achieves favorable classification performance under multiple seen/unseen splits while balancing computational complexity and accuracy.

Towards Efficient Embodied Reasoning: Mixture-of-Depth Compute Allocation for Vision-Language-Action Model

Vision-Language-Action (VLA) model plays a crucial role in embodied decision making. While practical deployment requires fast inference under limited onboard computation, a full forward pass through the vision-language model makes such deployment challenging. To address this issue, existing methods typically employ lightweight techniques to compress the backbone. However, these information-lossy methods degrade spatial representations for action generation. In contrast, rate-distortion principles aim to reduce computation while retaining control-sufficient information. Inspired by this insight, we introduce Effective-Edge Flow, an action-aligned attribution measure that quantifies the marginal contribution of token interactions across network depth. This analysis reveals a consistent depth asymmetry, with visual evidence dominating early layers and linguistic reasoning sustaining task-relevant influence into deeper layers. Building on this structure, we propose MoDeVLA, the first rate-distortion driven efficient VLA model that performs token-wise depth allocation via Mixture-of-Depth Conditioning and integrates shallow visual-spatial with deep textual-logical features for action conditioning. Extensive real-robot evaluations across 20 tasks and multiple embodiments demonstrate that MoDeVLA preserves task performance while reducing latency by about 38% and FLOPs by 86% on edge device NVIDIA Jetson Orin, highlighting its strong ability for embodied systems deployment.

Towards Efficient Embodied Reasoning: Mixture-of-Depth Compute Allocation for Vision-Language-Action Model

Vision-Language-Action (VLA) model plays a crucial role in embodied decision making. While practical deployment requires fast inference under limited onboard computation, a full forward pass through the vision-language model makes such deployment challenging. To address this issue, existing methods typically employ lightweight techniques to compress the backbone. However, these information-lossy methods degrade spatial representations for action generation. In contrast, rate-distortion principles aim to reduce computation while retaining control-sufficient information. Inspired by this insight, we introduce Effective-Edge Flow, an action-aligned attribution measure that quantifies the marginal contribution of token interactions across network depth. This analysis reveals a consistent depth asymmetry, with visual evidence dominating early layers and linguistic reasoning sustaining task-relevant influence into deeper layers. Building on this structure, we propose MoDeVLA, the first rate-distortion driven efficient VLA model that performs token-wise depth allocation via Mixture-of-Depth Conditioning and integrates shallow visual-spatial with deep textual-logical features for action conditioning. Extensive real-robot evaluations across 20 tasks and multiple embodiments demonstrate that MoDeVLA preserves task performance while reducing latency by about 38% and FLOPs by 86% on edge device NVIDIA Jetson Orin, highlighting its strong ability for embodied systems deployment.