Structure-Based Feature Representation for Robust Multi-Modal Image Matching

Abstract

Multi-modal image matching (MIM) remains a challenging problem due to nonlinear radiometric variations and geometric distortions across heterogeneous sensors. This paper proposes a robust feature-based matching framework that reduces reliance on intensity information while enhancing structural representation. The filter with local normalization is applied to transform the input images into a common intermediate domain. A block-based strategy is then employed to enforce a uniform spatial distribution of keypoints using the ORB (Oriented FAST and Rotated BRIEF) detector. To further suppress intensity variations and improve discriminability, a novel Max-Index-based HOG (MIHOG) is developed. This descriptor integrates multi-scale feature representations and encodes dominant structural information through discrete max-index mapping. Finally, correspondences are established using a brute-force matching strategy. Extensive experiments are conducted on two multi-modal datasets covering eight diverse scenarios. The proposed method achieves an average NCM of 224.52, RMSE of 3.4788, and SR of 92%. MIHOG obtains the highest NCM on 4/8 test scenarios and improves the average NCM by 18.3% compared with the second-best method. Meanwhile, it maintains competitive computational efficiency, with an average running time of 10.20s. These results demonstrate that MIHOG can provide dense and reliable correspondences under complex cross-modal radiometric and geometric variations.