Multimodal Learning for V2V Beam Prediction: Do Large Language Models Always Help?

Abstract

Accurate future beam prediction is essential for millimeter-wave (mmWave) vehicle-to-vehicle (V2V) communications under highly dynamic mobility. This paper studies dualmodal V2V future beam prediction using camera and GPS inputs. In particular, it focuses on whether large language models (LLMs) consistently help improve prediction performance across different sensing conditions. To this end, we develop and compare two complementary frameworks: a Stage-Wise Fusion Model (SFM) and an LLM-Based Fusion Model (LFM). SFM introduces stage-wise GPS-visual fusion into an EfficientNet-B3 backbone, enabling geometric information to interact with visual features throughout the feature hierarchy. LFM builds on a pretrained Qwen2.5-1.5B backbone with LoRA fine-tuning, and performs beam prediction by modeling a unified sequence of geometric prompt tokens, GPS tokens, dual-view visual tokens, and a learnable query token. Experimental results on the DeepSense 6G V2V dataset show that LFM exhibits stronger scenario dependence and mainly serves to probe the achievable performance upper bound, whereas SFM maintains more stable cross-scenario performance with a more favorable tradeoff among robustness, complexity, and deployment practicality.