Open access

A hybrid machine learning framework for dynamic malware classification using CNN, LSTM, and Random Forest models

Karthick Ganapathy

2026 · Journal of Computer and Forensic Sciences · 0 citations · 8 references

TL;DR

The findings indicate that combining deep representation learning with ensemble classification can improve dynamic malware detection and reduce dependence on static signatures.

Abstract

Malware continues to evolve through polymorphic, metamorphic, and file-less techniques, reducing the effectiveness of signature-based detection. This study proposes a hybrid dynamic malware classification framework that combines Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and a Random Forest (RF) classifier. Dynamic behavioural traces are transformed into machine-readable features using Term Frequency-Inverse Document Frequency (TF-IDF), Count Vectorization, and Word2Vec embeddings. CNN and LSTM models are used as feature extractors to capture local and sequential behavioural patterns, while RF performs the final classification to improve robustness and interpretability. The reported experimental evaluation shows that the embedding-based hybrid CNN-RF configuration achieved the best performance, reaching 98.21% accuracy. The findings indicate that combining deep representation learning with ensemble classification can improve dynamic malware detection and reduce dependence on static signatures. The proposed framework is therefore a promising approach for adaptive malware classification and future security monitoring applications.

DOI Semantic Scholar

Read PDF

Similar papers

Open access Jun 2026

A Hybrid CNN–LSTM Framework for Behavioral Malware Detection and Dynamic YARA Rule Generation

Results show that the fusion of deep learning-based behavioral analysis with external contextual intelligence is an effective and deployable malware detection solution which supports dynamic YARA rule generation.

Hamid Talib Zaidan, Jumana Waleed, Ruaa Azzah Suhail · Journal of Al-Qadisiyah for... · 0 citations

Conference Jul 2026

A Comparative Analysis of Machine Learning and Deep Learning Methods for Malware Detection in Portable Executable Files

Malware severely threatens cybersecurity through Portable Executable (PE) files. In this study, the performances of machine learning and deep learning models for malware detection are compared using static analysis features extracted from PE files. Tree-based ensemble models like Random Forest, XGBoost, LightGBM, and C...

Gokay Mutlu, Neşe Rihani, E. Bayazit · Signal Processing and Commun... · 0 citations

#explainable ai Open access Aug 2026

DEVELOPMENT OF A HYBRID NN–CNN DEEP LEARNING FRAMEWORK FOR INTELLIGENT MALWARE DETECTION, FAMILY CLASSIFICATION, AND VARIANT IDENTIFICATION

A Hybrid Neural Network–Convolutional Neural Network (NN–CNN) Deep Learning Framework for malware detection, malware-family classification, and malware-variant identification and considers two important issues in practical malware detection: model explainability and generalization to previously unseen malware.

Chioma Grace Nwankwo, B. C. Amanze, Ikechukwu Amaefule · World Journal of Advanced Re... · 0 citations

Open access Aug 2026

Intelligent malware detection on Android smartphones via a hybrid approach using gradient boosting and convolutional neural network

Evaluation using metrics such as accuracy, precision, F1 score, and false positive rate indicates that CNN-GBM outperforms existing deep learning models, and enhancements stem from the effective integration of CNN feature extraction with GBM’s boosting capabilities.

C. Chimeleze, Norziana Jamil, Z. M. Zain et al. · Scientific Reports · 0 citations

Open access Jul 2026

EXPLAINABLE MACHINE LEARNING-BASED MALWARE DETECTION USING PORTABLE EXECUTABLE STRUCTURAL FEATURES AND HYBRID VOTING ENSEMBLE

The findings demonstrate the potential of using ensemble learning in conjunction with explainable artificial intelligence techniques to enhance malware detection systems and cybersecurity applications.

Haider Ali Muften · Journal for Technology and S... · 0 citations

Open access 2026

Hybrid Convolutional-Hopfield Neural Networks for Image-Based Malware Classification

A novel hybrid CNN-Hopfield Neural Network (CNN-HNN) framework that replaces traditional dense classification heads with continuous Modern Hopfield associative memory layers is proposed, achieving top-tier accuracy and reducing classification-head parameter counts while reducing floating-point operations (FLOPs) and ac...

Uma Kannan, Rajendran Swamidurai · International journal of res... · 0 citations