A Composite Forecast Evaluation Framework for Univariate Time Series Forecasting: Evidence from Gold, CPI, and VIX

Abstract

Time series forecasting plays an important role in economic and financial analysis; however, forecasting performance is often evaluated using single metrics, potentially leading to inconsistent model rankings and incomplete assessment. To address this limitation, this study proposes a Composite Forecast Evaluation Score (CFES) that integrates Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), and Directional Accuracy (DA) into a unified forecasting evaluation framework. The proposed framework was applied to compare five univariate forecasting models—Naive, Seasonal Naive, Drift, ARIMA/SARIMA, and Support Vector Regression (SVR)—using monthly Gold, Consumer Price Index (CPI), and Volatility Index (VIX) data from January 2010 to December 2024, with 2010–2023 used for training and 2024 reserved for testing. Forecast robustness was assessed through one-step-ahead rolling-origin forecasting, coefficient of variation (CV)-based stability analysis, and Friedman–Nemenyi statistical testing. Results indicate that forecasting performance is dataset-dependent, with Drift achieving the best performance for Gold, ARIMA/SARIMA for CPI, and SVR for the highly volatile VIX dataset based on CFES. Statistical analysis further revealed significant forecasting differences among models for Gold and CPI, whereas no significant difference was observed for VIX. These findings provide preliminary empirical support for CFES as a multidimensional framework for forecasting evaluation across heterogeneous economic and financial time series.