Robust traffic scene recognition via bidirectional cross-attention-based vision-map fusion

Abstract

Accurate traffic scene recognition serves as a critical foundation for decision-making and safe driving in autonomous driving and intelligent transportation systems. Existing methods mostly rely on single visual data vulnerable to environmental variations, or vision-LiDAR fusion schemes with insufficient capacity to represent road topology and traffic semantic information, limiting recognition accuracy and robustness. To address these limitations, this paper proposes an end-to-end traffic scene recognition network based on the fusion of monocular camera images and corresponding road map top-down view data. We design a learnable cross-view spatial alignment module to eliminate perspective discrepancy, and a bidirectional cross-attention fusion module to enable deep bidirectional interaction between visual semantic and map topology features. Experiments on a self-built dataset covering five typical traffic scenes show that the proposed method achieves an overall recognition accuracy of 92.6%, outperforming the best single-input baseline Swin-Tiny by 3.3%. Ablation studies further validate the effectiveness of each core module.