Multi-Task Learning-Based Multimodal Beam Prediction for V2I Communications in Dynamic Environments
Abstract
Millimeter-wave (mmWave) beamforming is essential for achieving ultra-reliable low-latency communications in vehicular networks, and its performance critically relies on accurate beam prediction. While recent deep learning based beam prediction methods leverage out-of-band sensing data to eliminate the dependence on channel state information, they exhibit significant performance degradation in dynamic vehicle-to-infrastructure (V2I) scenarios. The key challenge lies in the environmental dynamics, such as low-light conditions, intermittent occlusions, and rapidly varying multipath propagation, which compromise the reliability of sensors. To address this issue, we propose a multimodal beam prediction scheme based on multi-task learning. The proposed scheme employs a cross-modal gated fusion module to adaptively integrate multimodal features from cropped RGB regions of interest extracted via object detection, LiDAR point clouds, and global navigation satellite system real-time kinematics data to predict the optimal beam index. In addition, we introduce a physical constraint correction mechanism to enhance the multi-task learning strategy, which jointly optimizes beam prediction, blockage prediction, and reflection intensity prediction. Experimental evaluations on real-world V2I datasets demonstrate that the proposed scheme achieves top-5 accuracies of 94.02% and 91.36% in daytime and low-light scenarios, significantly improving robustness and reliability in dynamic environments.