Robust Multi-Sensor Fusion Architecture Tested on CARLA Sim

Abstract

This paper presents a multi-sensor fusion architecture with camera data segmentation. Robust perception is a fundamental prerequisite for the safety of autonomous vehicles, particularly in dynamic environments and varying weather conditions. Multi-sensor fusion approaches, integrating camera and LiDAR data, have emerged as the reference solution for 3D object detection, thanks to the complementary information provided by each modality. However, most existing work validates their architectures on static benchmarks such as the KITTI dataset, which do not allow for the evaluation of the system's robustness under controlled and reproducible variations in environmental conditions. In this work, we propose to deploy and evaluate a multi-sensor fusion pipeline in the CARLA nearrealistic simulator, which offers a dynamic, configurable, and physically realistic environment that faithfully reproduces realworld driving conditions. The adopted architecture is based on a fusion at the intermediate representation level, combining features from the camera, after segmentation of the raw data, and from LiDAR. The results obtained show that the simulation in CARLA constitutes a complementary and rigorous evaluation framework, bridging the gap between laboratory validation and deployment in real-world conditions.