Research on attack knowledge graph construction and APT attack chain intelligent reconstruction system oriented to massive data

Abstract

Advanced Persistent Threat (APT) is featured by organized operations, long-term latency, strong concealment, and continuous evolution of tactics and techniques. It has become the core cybersecurity threat faced by national critical information infrastructure, energy and financial industries, as well as large government and enterprise intranets. Traditional cybersecurity defense mostly relies on single-device data sources, static feature rule matching and single-point anomaly detection modes, which fail to adapt to the massive, multi-source, heterogeneous and high-speed security big data scenarios composed of full network traffic, terminal EDR logs, cloud audit records, system operation and maintenance logs, and open-source threat intelligence. Such traditional modes generally suffer from prominent problems including data overload and redundancy, isolated alarms across multiple devices, lack of attack correlation, lagging threat analysis and judgment, and high manual analysis costs. To solve the above bottlenecks, this paper designs and implements an intelligent reconstruction system for attack knowledge graph construction and APT attack chain oriented to massive data. Firstly, a stream-batch integrated architecture is adopted to build an intelligent processing pipeline for multi-source security data. With Apache Flink as the core engine for real-time computing, it completes unified access, protocol parsing, standardized normalization, semantic enrichment, and online construction of spatio-temporal entity relationships for multi-source heterogeneous data. Secondly, a fine-grained cybersecurity ontology is extended based on the MITRE ATT&CK framework. Combined with the Transformer pre-trained model, automatic extraction of entities and relationships from unstructured threat intelligence texts is realized. Time decay and confidence propagation algorithms are introduced to achieve dynamic iterative updating of the knowledge graph. Furthermore, a multi-head graph attention network inference model is constructed. Evidence subgraphs are established through alarm semantic mapping, and the encoder-decoder structure is utilized to mine spatio-temporal correlations and implicit links of attacks, realizing automatic reasoning and optimal path scoring of APT attack chains. Experimental results show that the system can stably support TB-level daily streaming data throughput, the F1-score of knowledge graph construction reaches 0.89, and the F1-score of APT attack chain reconstruction is as high as 0.923, which is over 18% higher than traditional rule-based and static knowledge graph methods. This research constructs an intelligent analysis paradigm integrating data-driven perception and knowledge-driven cognition, providing theoretical support and engineering implementation reference for intelligent security operation, APT threat hunting and traceability analysis.