Skip to content

MGRL-RSCC: Multi-Granularity Reward Reinforcement Learning for Fine-Grained Remote Sensing Change Captioning

Aug 2026 · 0 citations · 55 references
Computer Science

TL;DR

Extensive experiments on multiple public RSCC benchmark datasets demonstrate that the proposed MGRL-RSCC effectively mitigates exposure bias and conservative generation problems in traditional autoregressive methods.

Abstract

Remote Sensing Change Captioning (RSCC), which aims to generate accurate and detailed linguistic descriptions of ground object variations from bi-temporal remote sensing images, is a critical and challenging task in intelligent remote sensing interpretation. The mainstream autoregressive training paradigm faces severe exposure bias and train-test distribution mismatch, resulting in cumulative generation errors. They tend to produce conservative and template-fixed captions while ignoring subtle scene change details. To address these challenges, this paper proposes a novel multi-granularity reward reinforcement learning paradigm, termed MGRL-RSCC. Specifically, we first leverage a CNN and hierarchical self-attention module to extract and enhance visual features from bi-temporal remote sensing images. A Transformer decoder is then utilized to complete visual-to-linguistic translation. Different from existing methods, we design a dual-decoding strategy and a two-stage joint optimization scheme, which combines token-level supervised learning via greedy decoding and multi-granularity reward-driven self-critical reinforcement learning via sampling decoding. We further construct three complementary reward functions covering linguistic fluency, change state consistency, and structural-semantic relevance to comprehensively optimize caption quality and alleviate false and missing change descriptions. Extensive experiments on multiple public RSCC benchmark datasets demonstrate that the proposed MGRL-RSCC effectively mitigates exposure bias and conservative generation problems in traditional autoregressive methods. The source code and pre-trained models will be released on https://github.com/Event-AHU/MGRL-RSCC

View source

Similar papers

Conference Oct 2026

CA-HLD: context-adaptive cropping with hidden-layer distillation for SAR pretraining

Self-supervised learning (SSL) has become a practical paradigm for learning transferable synthetic aperture radar (SAR) representations from abundant unlabeled acquisitions. However, SAR pretraining based on self-distillation with no labels (DINO) leaves two aspects of the training signal insufficiently addressed: rand...

Yan Tang, Shao-Ming Zhang · 0 citations
Open access Oct 2026

HyMTAS-GBM: Integrating Semantic Convolutions and Multi-Head Attention with Gradient Boosting for Robust Object Detection

Object detection in complex scenes requires models that can maintain precise spatial attention and reliable query representations. However, existing CNN and DETR-based detectors often exhibit diffuse cross-attention, weak query initialization, and limited robustness in cluttered or dynamic environments. This paper prop...

Rahul Mandlik, Avinash Sharma · 0 citations
Sep 2026

Toward Lightweight Fine-Tuning of YOLO via Guided Optimization for Cross-Domain Remote Sensing Object Detection

RMoE-YOLO is presented, a lightweight fine-tuning framework that adapts a pretrained YOLOv8n detector to new remote sensing domains under tight label and compute budgets, and improves low-label cross-domain detection while remaining efficient.

Wen Liu, Pei-Pei Yan, Ming-Lai Chen et al. · 0 citations
Preprint Oct 2026

Omni-RS: Unifying Multi-Task Understanding and Generation in Remote Sensing

Practical remote sensing applications often require both understanding and generation capabilities for the same geographic scene. However, most existing methods address either understanding or generation in isolation, while existing unified remote sensing models'generative ability is limited to text-to-image. To bridge...

Pan Yin, Shuai Yang, Yuan-Hao Liu et al. · 0 citations

PoE-Fuse: Precision-Weighted Expert Fusion for Bi-Temporal Change Understanding

Bi-temporal change understanding, which localizes and characterizes what changed between two satellite images, is central to disaster response and environmental monitoring, spanning change detection, building localization, and damage assessment. Strong vision-language models address these tasks, but adapting them typic...

Haruki Watase, Shunya Nagashima, T. Nishimura · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 28, 2026

Making machines that can listen

Professor Paris Smaragdis develops computers with superhuman hearing for applications like speech recognition, music understanding, and environmental monitoring.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.