RAO++: Realistic Real-time Multi-vehicle Collaboration on Asynchronous Sensors

Abstract

Cooperative perception enables connected autonomous vehicles to extend their sensing range and overcome occlusions by exchanging sensor data. However, its real-world deployment is hindered by asynchronous sensor streams and inaccurate localization of occluded regions. This work presents RAO++, a real-time cooperative perception system that merges asynchronous sensor data from different vehicles through our novel designs of motion-compensated occupancy flow prediction, on-demand data sharing, with a variety of system optimizations to improve the accuracy and coverage of the perception system. Our comprehensive evaluation, including real-world and emulation experiments under diverse LiDAR configurations, shows that RAO++ outperforms asynchronous-unaware methods by more than 34% in perception coverage and by up to 14% in perception accuracy. Moreover, RAO++ reduces latency by 1.2–3.5× and communication overhead by 45–70% compared with the state-of-the-art asynchronous-aware baseline, while maintaining comparable detection accuracy. Finally, RAO++ demonstrates a practical data overhead of 12.8 KB per frame, enabling deployment under realistic bandwidth constraints.