From Forecasting Accuracy to Trading Profitability: Evaluating Sequence Models for Stock Price Prediction

Abstract

Accurate stock price forecasting remains a challenging problem due to the noisy, nonlinear, and non-stationary characteristics of financial time series. Although recent advances in deep learning have improved predictive capabilities, most prior studies evaluate forecasting models primarily using statistical error metrics, with limited consideration of their practical value in trading and investment decision-making. This creates a gap between predictive performance and real-world economic utility. This study proposes a decision-oriented evaluation framework for multi-step stock price forecasting that jointly assesses predictive accuracy and trading profitability within a unified experimental setting. Using data from 91 ASX 100 stocks after data cleaning, with a testing period spanning 2019–2020, several deep learning architectures, including Multi-Layer Perceptron (MLP), Gated Recurrent Unit (GRU), Seq2Seq, and attention-based sequence models, are systematically compared under identical training and trading conditions. The results show that the Seq2Seq model achieved the best overall performance, obtaining the lowest average MAPE of 0.0293 and the highest ROI of 23.2%, while the attention-based model achieved a similar MAPE of 0.0294 but a lower ROI of 12.4%. Although differences in forecasting accuracy were relatively small, the Seq2Seq model achieved the highest observed trading profitability and generated a higher observed return than a passive market benchmark under the proposed evaluation framework. These findings suggest that evaluation based solely on prediction accuracy may not fully capture the practical value of forecasting models.