Physical-consistency-guided deep unfolding for metalens-based snapshot spectral imaging

Abstract

End-to-end metalens-based snapshot spectral imaging jointly optimizes the optical encoder and computational decoder, offering a promising approach for compact and efficient hyperspectral acquisition. However, existing reconstruction methods often provide limited consistency with the known forward imaging model and insufficient modeling of the global frequency-domain dependencies introduced by diffraction-coded convolution, limiting reconstruction accuracy and crossdataset generalization. To address these issues, we propose a physical-consistency-guided deep unfolding method for metalens-based snapshot spectral imaging. The proposed framework embeds the metalens point spread functions and camera spectral response functions into the unfolding reconstruction process, and introduces a physical consistency loss composed of three forward-model consistency terms: PSF-encoding consistency, SRF-projection consistency, and measurement consistency. In addition, we design a spatial–Fourier hybrid prior module as the data-driven prior module in the deep unfolding network. The spatial branch enhances local textures and edge structures, while the Fourier branch captures long-range dependencies and global frequency-domain characteristics introduced by diffraction-coded convolution; the resulting spatial and Fourier features are adaptively fused through an attention mechanism to enhance spectral–spatial feature representation. Trained on the ICVL dataset and evaluated on the NTIRE 2022 HSI dataset, the proposed framework achieves the best reconstruction performance among representative purely deep-learning-based methods for metalens snapshot spectral imaging. This metalens-based snapshot spectral imaging framework advances the development of miniaturized and high-speed spectral imaging systems.