Hybrid ARIMA-LSTM Model for Anomaly Detection in Streamed Time-Series Data

Abstract

The exponential growth of data generated by wireless sensor networks (WSNs), Internet of Things (IoT) devices, and real-time monitoring systems has heightened the need for accurate and scalable anomaly detection techniques in streamed time-series data. Traditional statistical models, including ARIMA, provide advantages such as interpretability and reduced computational demands; however, they exhibit limitations when handling non-linear dynamics. In contrast, deep learning models, exemplified by LSTM, demonstrate robust performance capabilities but face challenges related to transparency and higher computational costs. This study proposes a hybrid Auto Regressive Integrated Moving Average (ARIMA) and Long Short Term Memory (LSTM) networks to enhance anomaly detection in real-time streaming environments. The proposed method integrates ARIMA to model linear patterns and LSTM to learn the residual non-linear components. The Residuals from ARIMA are transformed into an LSTM model trained to forecast error corrections. Anomaly detection is implemented using exponential moving average model by monitoring deviations from expected values using a threshold mechanism. Extensive experiments were conducted using misconfiguration of CPU utilization in AWS Auto Scaling Groups (ASG) dataset. The hybrid model demonstrated superior performance compared to the standalone ARIMA and LSTM models in most of the evaluated scenarios. It achieved an accuracy of 99%, precision of 99%, recall of 99% and F1 score of 98% on the full dataset. The ARIMA-LSTM hybrid framework offers a robust performance and solution characterized by interpretability, scalability, and accuracy that make it an ideal system for real-time, high-stakes anomaly detection in dynamic streaming environments.