A Multi-Dimensional Benchmark of Large Language Models for Time Series Forecasting

Abstract

Large Language Models (LLMs) have recently been applied to time series forecasting via prompt-based inference, yet their evaluation remains limited by potential benchmark contamination and an overemphasis on accuracy. We introduce a systematic benchmark that evaluates LLMs across accuracy, efficiency, cost, and environmental impact. Our framework spans seven datasets from diverse domains and incorporates a Continuous Wavelet Transform (CWT)-based strategy to select informative temporal segments under context-length constraints. We evaluate several LLMs ranging from 0.6B to 1T parameters, covering multiple model families, prompting strategies, and data representations. Results show that LLMs are competitive zero-shot forecasters across several datasets, although forecasting performance and efficiency vary substantially across model families. Performance is highly sensitive to prompt design and data representation, while larger models do not consistently achieve better forecasting accuracy. These findings highlight that LLMs can serve as practical general-purpose forecasters when efficiency and sustainability are considered alongside accuracy. Code and benchmark resources are available at: Repository