Is Transformer all you need? Answer is no to Building Energy Forecasting in High Volatility

Abstract

Modeling and predicting building energy consumption is crucial for addressing energy efficiency issues in buildings and tackling the challenges posed by urban expansion and so on. Precisely forecasting energy usage at a smaller time interval opens up possibilities to address a wider range of issues in several domains, including urban planning and the energy market. The Transformer has been widely utilized in time series forecasting. The Transformer has been criticized to have limited scalability regarding the long dependency among long sequences and to lose temporal information. Some MLP models can also achieve comparable performance as Transformers. Motivated by these findings, we wonder if recurrent neural networks (RNNs) based models can also match or even outperform the Transformers and MLP models in time series forecasting. This study compares several Transformers, MLP and GRU-based models trained on power consumption data in Massachusetts, showing that the Transformer and MLP model achieve comparable performance and the GRU-based model outperforms the other types of models.