Learning Robust 3D Gaussian Representations via Discrepancy-Aware Infrared and Visible Modality Fusion

Abstract

Infrared and visible image fusion (IVIF) combines robust infrared sensing with the rich structural details of visible imagery. Existing IVIF methods are predominantly developed in 2D image space, limiting them to fixed viewpoints and hindering explicit modeling of modality discrepancies at the scene level. Although recent works introduce multimodal cues into 3D representations, they focus primarily on cross-modality rendering conflicts or modality-dominant reconstruction rather than fusion-oriented selective preservation and suppression. We therefore propose Discrepancy-Aware Fusion with 3D Gaussian Splatting (DAF-GS), which learns robust 3D Gaussian representations through discrepancy-aware infrared and visible modality fusion. Specifically, a discrepancy-aware modality-guided cue mechanism estimates modality dominance from current-view responses and long-term modality dependency statistics, and incorporates the resulting cue into pixel-wise color rendering for explicit discrepancy-aware optimization. A primitive-wise detail compensation module further restores complementary fine-grained details weakened during selective suppression. Extensive experiments demonstrate the robustness of the learned representations and clear advantages over 2D fusion methods for novel-view synthesis across multiple datasets. The code is available at https://github.com/RollingPlain/DAF-GS.