Design and Implementation of a Scalable FPGA-Based Architecture for CluStream Data Stream Processing

Abstract

The growth of global connectivity, driven by the deployment of 5G networks and the expansion of the Internet of Things, has significantly increased the volume of data continuously flowing into electronic systems for processing. Due to their dynamic and unbounded nature, these data cannot be fully stored for later analysis, which demands solutions capable of processing and analyzing them in real time. In this context, the data stream paradigm emerges, where the statistical distribution of the incoming data may vary over time, a phenomenon known as concept drift. To address this challenge, this work integrates unsupervised machine learning techniques for advanced clustering, applies FPGA-based hardware design, and supports machine learning algorithms programmed in HLS through a novel methodology for designing scalable implementations, referred to as feature scaling. Based on this approach, a hardware implementation inspired by the online phase of the CluStream algorithm is proposed for real-world real-time data stream analysis. This phase consists of incrementally summarizing incoming data into micro-clusters by continuously updating their statistical properties, enabling efficient real-time processing without storing the entire dataset. The developed implementation enables tracking the temporal evolution of data by generating and dynamically updating micro-clusters, with changes reflected as variations in their statistical descriptors. Furthermore, the presented framework incorporates a scalable approach that allows adjustment of key system parameters, such as the number of data features, without requiring a complete redesign of the implementation. Experimental results conducted on the ZCU104 platform—which integrates an ARM Cortex-A53 processor running PYNQ—demonstrate that the proposed implementation successfully reproduces the expected behavior of CluStream. Performance evaluations reveal that the FPGA architecture achieves peak speedup values of up to $16.95\times $ during the initialization phase, which is executed with a set of 300 initial data samples, and $10.47\times $ during the single-sample update phase compared to the ARM processor, yielding maximum processing throughputs of 5784.42 KB/s and 357.20 KB/s, respectively. These metrics validate the framework’s scalability and its advantages in temporal stability and adaptability to dynamic, multi-feature data streams.