The model incorporates three core components: spatiotemporal embedding module, spatiotemporal fusion module, and LLM backbone, which adopts a differentiated parameter adaptation strategy to balance training efficiency and traffic data adaptability.
Abstract
Traffic flow prediction is a core supporting technology for intelligent transportation systems. It uses historical data to infer future traffic dynamics in specific areas, thereby helping to alleviate congestion and improve resource allocation efficiency. Traditional neural networks struggle to break through accuracy limits due to their reliance on singular feature modeling, while large language models (LLMs) suffer from insufficient capture of spatial topological information and mining spatiotemporal correlation. This study proposes a Dynamic Fusion Large Language Model (DF-LLM) for traffic flow prediction. The model incorporates three core components: spatiotemporal embedding module, spatiotemporal fusion module, and LLM backbone. The spatiotemporal embedding module enables synergistic representation of multi-scale spatiotemporal features. The spatiotemporal fusion module integrates spatial topology and dynamic dependencies via graph convolution. The LLM backbone adopts a differentiated parameter adaptation strategy to balance training efficiency and traffic data adaptability. Additionally, it introduces a context aggregation attention module to strengthens global dependencies. More importantly, the LLM backbone takes the residual connections to mitigate the gradient vanishing in deep networks. Experiments show that DF-LLM has achieved better performance by comparing the metrics on all the four datasets.
The rapid growth in traffic volumes has increased the demand for traffic-flow forecasting models with stronger prediction capability. Traditional methods that rely on local feature extraction and static spatial graph construction can no longer fully meet these requirements. To address the short- and long-term fluctuati...
Hai Yan, Rui-Dong Jian, Cheng-Cheng Wang et al.· Algorithms· 0 citations
This paper aims to propose attention-based dynamic graph convolutional recurrent neural network (ADGCRNN) for highway traffic flow prediction, which outperforms state-of-the-art baseline models and realizes multiresolution temporal fusion via self-attention.
Wei-Long Ding, Rui-Zhi Xue, Qi Yu et al.· International Journal of Web...· 0 citations
A node-tokenized GPT-2 framework is proposed and attention adaptation and residual prediction mechanisms for traffic flow forecasting are investigated, indicating that residual prediction improves forecasting accuracy, whereas simply modifying the attention structure does not necessarily lead to reliable modeling of tr...
Ming-Zhu Gao, Ruo-Han Ning· International Conference on...· 0 citations
An enhanced Dynamic Spatiotemporal Residual Network (DST-ResNet) framework for network-scale traffic speed prediction is proposed, which employs a multi-scale grid partitioning strategy to segment urban road networks at varying levels of granularity, enabling precise predictions at both local and global scales.
The results demonstrate the effectiveness of temporally informed data-driven learning for 10-min-ahead traffic forecasting within a spatially structured traffic-data framework and indicate complementary strengths between the two leading models.
Alvin V. Modales, Yu-Shiuan Tsai, Mark Rennel D. Molato et al.· Applied Sciences· 0 citations
Overall, DH-STGCN provides a flexible input-conditioned hierarchical representation for multistep traffic flow prediction, and Controlled hierarchy comparisons favor the window-conditioned assignment over fixed-uniform, static-hard, globally shared, and alternative differentiable assignments.
Jinghao Hu, Yan He, Run-Kui Li et al.· Applied Sciences· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 13, 2026