From detection to repair: an interactive framework for reliable IoT data streams

Abstract

The purpose of this study is to address the challenges of repairing errors in real-time data streams generated by Internet of Things (IoT) devices, which benefit various sectors including healthcare, business, and industry. Although these data streams are valuable, IoT readings often contain errors that lead to unreliable analysis and flawed decisions. Traditional IoT data repairing techniques rely primarily on batch processing methods, such as rule-based filtering, which introduce latency and cannot effectively handle real-time streaming data. Furthermore, these conventional approaches typically remove all anomalies without identifying their underlying causes, which can result in the loss of critical insights. Compounding these issues, the computational demands of real-time processing present significant obstacles, and the dynamic nature of data streams makes anomaly repair especially difficult. The study proposes a real-time anomaly repair model for structured IoT data streams. The model classifies and repairs detected anomalies automatically. The proposed model uses statistical measurements and machine learning techniques to repair anomalies. The authors evaluated the model using four data sets, demonstrating improved data quality in real-time data streams by correctly assigning repair techniques to the detected and classified anomalies. By eliminating manual intervention and triggering repair only upon anomaly detection, the proposed method reduces decision delays and avoids unnecessary computational overhead, making it well-suited for efficiently handling anomalies in real-time data streams. This research offers an automated framework to repair anomalies in real-time data streams by applying one of three actions: delete, keep or replace, selected according to anomaly classification and type. Also, it integrates a repair toolset that uses statistical measurements and machine learning techniques to provide multiple replacement options for anomaly repair.