Logfusion: A Lightweight Preprocessing Pipeline for Anomaly Detection
TL;DR
LogFusion is presented, a lightweight preprocessing pipeline that fuses contextual embeddings of log messages with engineered behavioural indicators to produce machine learning-ready features for unsupervised anomaly detection algorithms.
Abstract
: The increasing frequency and sophistication of cyber-attacks continue to expose vulnerabilities in modern digital infrastructures. System logs provide a rich source of evidence for detecting such threats, but raw logs are noisy, heterogeneous, and imbalanced, making them inconvenient for direct machine learning applications. This paper presents LogFusion, a lightweight preprocessing pipeline that fuses contextual embeddings of log messages with engineered behavioural indicators (e.g., temporal windows, event rarity, and log-level dynamics) to produce machine learning-ready features for unsupervised anomaly detection algorithms. Evaluated on three public benchmark datasets (HDFS, BGL, and OpenStack), LogFusion increases anomaly detection accuracy from 51 – 61% (raw Bag-of-Words baseline) to 84 – 98% across all detectors, with consistently high recall (>0.95 on HDFS and OpenStack). On HDFS, LogFusion-OCSVM achieves 87% F1-score, outperforming LogBERT (82.32%) and other deep learning baselines. Runtime profiling confirms end-to-end processing under 13 seconds on CPU-only hardware, demonstrating practical feasibility for resource-constrained deployments in SMEs and IoT/edge environments.