Electro-Optical Response-Aware Learning for PAN-Sharpening of Satellite Images

Abstract

Most deep-learning-based panchromatic (PAN)-sharpening methods are trained under simplified and fixed degradation models, typically assuming an isotropic Gaussian point spread function (PSF). However, real electro-optical (EO) satellite systems exhibit scene- and acquisition-dependent blur characteristics caused by diffraction, optical aberrations, platform motion, and intersensor misalignment, which results in a domain gap between synthetic training data and real-world observations. To narrow this gap, we analyze satellite star images and observe that the effective blur is often anisotropic and exhibits extended non-Gaussian residuals. Based on this observation, we propose a PSF prediction network (PSFPN) that estimates a scene-specific PSF consisting of an anisotropic Gaussian component and a residual term. The predicted PSFs quantitatively reveal sensor-dependent EO responses: the mean anisotropy ratios are 1.012, 1.310, and 1.041 for Korea Multi-Purpose Satellite-3A (KOMPSAT-3A), WorldView-II, and QuickBird, respectively. We then use the predicted PSFs to generate physically consistent training data for existing PAN-sharpening networks without changing their inference architectures. To evaluate the effectiveness of our PSFPN, we conducted comparative experiments on four deep-learning-based PAN-sharpening methods trained with fixed PSF-based and PSFPN-based degradation processes. Extensive experiments on multiple satellite datasets show that the PAN-sharpening methods trained with PSFPN-based degradation outperform those trained with fixed PSF-based degradation in both reduced- and full-resolution evaluations, while also enhancing geometric consistency and reducing misregistration-related artifacts. In particular, under the PSFPN-based degradation test protocol, the average peak signal-to-noise ratio (PSNR) values of the four deep-learning-based PAN-sharpening methods trained with our PSFPN-based degradation are higher than those trained with a fixed PSF by 0.6970, 0.9158, and 1.0432 dB on the KOMPSAT-3A, WorldView-II, and QuickBird datasets, respectively. Under the random anisotropic Gaussian-based degradation test protocol, the corresponding average PSNR improvements on the KOMPSAT-3A, WorldView-II, and QuickBird datasets are 0.5058, 0.4900, and 0.5037 dB, respectively, without any additional inference cost.