A Reliable Target Recognition Method for Complex Scenarios Based on Multimodal Sensor Information Fusion

Abstract

The harsh conditions of underground coal mines necessitate enhancing the reliability and adaptability of mobile robot perception systems within operational scenarios. Addressing core challenges such as LiDAR point cloud failure due to dust and water vapor interference, alongside visual camera imaging distortion under low-illumination conditions, we propose a 3-D object detection method based on Radar and LiDAR fusion enhanced with target velocity and Radar Cross Section (RCS) information, named VR-LRFusion, in which the network takes both Radar and LiDAR data as inputs. To fully leverage the strengths of Radar in terms of velocity and RCS data, velocity encoders and RCS encoders are introduced, and Radar features are enhanced separately through a velocity-RCS attention mechanism. Subsequently, to address the sparsity of the Radar point cloud, the fusion module employs a cross-attention mechanism to integrate LiDAR features with the enhanced Radar features, thereby improving the network’s ability to detect moving obstacles. Experimental validation on the nuScenes public dataset demonstrates that, compared to existing methods, our approach achieves improvements of 2.93% and 2.42% in mean average precision (mAP) and NDS metrics, respectively. VR-LRFusion exhibits robust detection performance in dynamic object detection. At the same time, we have constructed a dataset of underground coal mine environments, and our algorithm has demonstrated exceptional adaptability to this environment.