Machine learning-driven financial time series forecasting: application of a feature engineering-enhanced RF model in A-share energy stocks

Abstract

Amid the accelerating integration of AI and energy finance, existing random forest (RF) based stock forecasting studies often neglect the dual impact of redundant feature interference and ensemble redundancy on prediction stability. This study proposes a dual-path optimized RF framework for closing price forecasting of 6 A-share energy leaders (242 daily samples per stock in 2024). Different from conventional single-step feature selection, we introduce a two-stage feature screening mechanism combining Spearman rank correlation and permutation feature importance, and further embed ensemble pruning to remove low-contribution sub-trees from the RF pool. Validated via nested 5-fold cross-validation, the optimized framework achieves an average R² of 0.986, with 85% of predictions falling within a 2% deviation range. Comparative experiments show the proposed framework reduces RMSE by 12.7% compared to baseline RF, and outperforms standalone LSTM by 18.3% in out-of-sample stability. The framework demonstrates strong generalization capability across different market cap segments of the energy sector, providing a more robust tool for energy finance quantitative analysis.