A Review of Transformer Models for Time Series Forecasting
TL;DR
This review compares three representative Transformer-based models and indicates that model complexity is not the sole determinant of forecasting performance and that look-back windows and experimental protocols also affect model comparisons.
Abstract
Long-term time series forecasting is important in energy scheduling, traffic management, meteorological monitoring, and industrial operations. However, conventional statistical models and recurrent neural networks have limitations in modeling long-range dependencies, complex periodic patterns, and multivariate relationships. This review compares three representative Transformer-based models—Informer, Autoformer, and PatchTST—through literature synthesis and comparative analysis. Informer reduces long-sequence computation through sparse attention, Autoformer strengthens periodic modeling through series decomposition and Auto-Correlation, and PatchTST improves input representation through patch-based tokenization. Public results on the Electricity dataset show that PatchTST outperforms earlier Transformer-based models, while DLinear achieves comparable errors. These findings indicate that model complexity is not the sole determinant of forecasting performance and that look-back windows and experimental protocols also affect model comparisons. The review further discusses computational cost, non-stationarity, and cross-variable modeling.