FEDA: Fast Exponential Decay Approach for Multivariate Time Series Forecasting

Abstract

Multivariate time series forecasting that uses past observations to predict future data has been widely used in various application. Numerous methods have been developed, including deep learning based approaches. One of the most successful models is Mamba, which is based the state space models (SSMs). However, Mamba and its variants still face the challenges of long training times owing to its nonlinear operations. In this paper, we proposed a Fast Exponential Decay Approach (FEDA) to improve the training efficiency of Mamba. We have observed that the hidden states in the model are linear combinations of past inputs. Therefore, we attempted to use a fully connected layer to generate all the hidden states. Although a fully connected layer does not have a recurrent structure, based on the intuitive idea in multivariate time series forecasting that the influence of observations diminishes over time, we multiplied the weights of the fully connected layer by a time-dependent coefficient to emulate a recurrence-like behavior. We evaluated our model on various datasets, and the results show that our model achieves the performance comparable to the state of the art models, while significantly improving their training efficiency. More specifically, it reduces training time by 20% to 80% depending on the channels of the dataset.