LLM Based Static Malware Analysis for Windows Portable Executables

Abstract

In this work we introduce a novel application of Large Language Models (LLMs) towards the classification of malicious Windows Portable Executable (WinPE) files. Rather than fine-tuning an LLM based on the raw binary of the samples serving as the corpus, in this case we leverage our in depth knowledge of the underlying structure of the WinPE format to apply a two phased approach to translate each file into human readable format. For our studies we focus on the header portion of the files in order to reduce the required time for the fine-tuning process, performing Parameter Efficient Fine Tuning (PEFT) applying a QLoRA adapter to a light-weight baseline model. In addition, we deploy our fine-tuned model according to Software Defined Networking (SDN) principles. The model is containerized, with constraints applied in terms of the amount of CPU, memory, and networking resources that are available. In addition, the model is deployed as a virtualized LLM (vLLM) adhering to a server-client architecture. We benchmark the fine-tuned model against the SoReL-20M dataset and compare results in terms of both accuracy, including the macro-averaged F1 score, and latency, including the Time-to-First-Token (TTFT). By fine-tuning the model to understand the language of the Windows Portable Executable format, we demonstrate a significant improvement in macro-averaged F1 score, up to an absolute difference of 50 percentage points over generic LLMs, while maintaining comparable latency.