Skip to content
Open access

A Deep Learning Framework for Three-Dimensional Malware Image Classification

Jun 2026 · Applied Sciences · Vol 16, pp. 6434 · 0 citations · 23 references

TL;DR

It is shown that multi-dimensional visualization of malware achieves improved classification performance, and the combination of three-dimensional malware visualization, deep learning, and genetic optimization is promising for the development of future intelligent malware detection tools.

Abstract

The rapid growth of sophisticated malware, including polymorphic, metamorphic, and zero-day threats, has made traditional signature-based and heuristic detection methods increasingly insufficient in modern desktop computing environments. As cyber threats continue to evolve in both complexity and scale, the demand for intelligent and adaptive malware detection mechanisms capable of identifying previously unseen attacks has become more critical than ever. In this study, we propose a novel deep learning framework for three-dimensional malware image classification that utilizes visual representation learning to improve malware detection performance. The proposed framework converts raw malware binaries into three-dimensional grayscale and RGB image representations, allowing hidden structural and spatial patterns within malware samples to be analyzed more effectively. By transforming malware data into multi-dimensional visual forms, the proposed system facilitates the process of automatically learning hierarchical features by CNN through multi-dimensional visualization of malware binary codes. In addition, an optimization technique using Genetic Algorithms is implemented within this architecture to improve classification performance and stability. The proposed evolutionary algorithm performs an effective search process within the large parameter space of 3D-CNN, leading to the identification of models that facilitate learning. It is shown that multi-dimensional visualization of malware achieves improved classification performance. It can be concluded that the combination of three-dimensional malware visualization, deep learning, and genetic optimization is promising for the development of future intelligent malware detection tools.

Read PDF

Similar papers

Open access 2026

A Multi-Modal Deep Learning Framework for Robust Polymorphic Malware Detection

: Modern malware is increasingly employing polymorphism, packing, and metamorphism to evade traditional signature-based detection. Because of this, there is an urgency to have more reliable classification systems. Visual malware analysis, where binaries are converted into grayscale images, has demonstrated potential in revealing structural patterns of malware family classification. However, recent methods mostly rely on single-stream, lightweight Convolutional Neural Networks (CNNs). These models have a major blind spot. The visual representation textures can be heavily obscured without changing the underlying malicious code, causing severe performance drops on newer or even rare malware classes. This paper presents a Hybrid Multi-Modal Deep Learning framework to fix this vulnerability. The proposed dual-stream architecture uses image recognition via EfficientNetB0 alongside metadata analysis using 1D-convolutional byte embeddings. This paper evaluated the framework on the modern MalwareVision-2025 dataset (approximately 125,000 samples) and the legacy Malimg dataset (9339 samples). On MalwareVision-2025, the model reached a weighted accuracy of 88.44% and achieved 100% benign recall on the evaluated split. The testing across both datasets shows that combining visual and structural features reduces modality collapse. This creates a much stronger system compared to using either input type on its own. In particular, the hybrid approach improves detection performance on deeply hidden contemporary threats, including the AveMariaRAT and CobaltStrike families.

Phil Steadman, Paul Jenkins, Rajkumar Singh Rathore et al. · 0 citations
#explainable ai Open access Aug 2026

DEVELOPMENT OF A HYBRID NN–CNN DEEP LEARNING FRAMEWORK FOR INTELLIGENT MALWARE DETECTION, FAMILY CLASSIFICATION, AND VARIANT IDENTIFICATION

A Hybrid Neural Network–Convolutional Neural Network (NN–CNN) Deep Learning Framework for malware detection, malware-family classification, and malware-variant identification and considers two important issues in practical malware detection: model explainability and generalization to previously unseen malware.

Chioma Grace Nwankwo, B. C. Amanze, Ikechukwu Amaefule · 0 citations
Review Aug 2026

A Survey on Deep Learning Approaches for Malware Detection and Classification

Malware is a serious threat in the cybersecurity area because of its dynamic nature, the variety of malware families, stealth, propagation and the capability of evading traditional security products. Therefore, proper malware detection and classification are crucial for detecting malicious software and for securing computer systems from unauthorized access and data stealing, and for disrupting systems. This study covers all the bases when it comes to deep learning approaches for malware detection and classification. It covers the principles, different forms of malware, how to detect deep learning malware, how to represent data, obtaining features, and applications. The traditional detection methods are described with their drawbacks, namely based on signature, behavioral and heuristic methods. The report also delves into the methodologies used by deep learning to classify malware, namely CNNs and Bidirectional Long Short-Term Memory (BiLSTM) networks. BiLSTM models excel at learning sequential features from code-or behavior-related data, whereas CNN-based representation learning approaches excel at learning spatial features from malware representations. Moreover, the various detection techniques (static, dynamic and hybrid) are discussed so that their role in malware analysis can be understood. The survey identifies the current challenges and gaps in research and emphasizes the need for strong, scalable and adaptive deep-learning models to combat new malware threats and enhance cybersecurity protection.

Manjula Jain · 0 citations
Open access Jul 2026

Hybrid noise-resistant technique for malware classification.

Visualization-based malware detection has recently gained significant attention for binary and multiclass malware classification using machine learning and deep learning techniques. However, existing visualization-based frameworks still face several important limitations, including insufficient robustness evaluation, limited cross-dataset validation, restricted malware diversity, and difficulty distinguishing visually similar and noise-sensitive malware families. In many cases, the visual similarity between malware classes and the presence of perturbations negatively affect feature extraction quality, leading to degraded classification performance and reduced generalization capability. To address these challenges, this study proposes a novel hybrid malware representation framework that integrates Convolutional Autoencoder (CAE)-based latent structural learning with Local Binary Pattern (LBP)-based texture feature extraction for robust malware classification. To the best of our knowledge, this study represents one of the first comprehensive investigations of hybrid latent-texture representation learning within a memory-forensics malware visualization setting while jointly addressing robustness, perturbation resilience, scalability, and cross-dataset generalization through a unified evaluation framework. The proposed framework combines global hierarchical representations learned through CAE with fine-grained local texture descriptors extracted using LBP to improve the discrimination of visually similar malware families and enhance robustness against noisy visualization conditions. The extracted features are subsequently evaluated using multiple machine learning classifiers, where XGBoost achieved the highest performance with an accuracy of 99.90%, precision of 99.79%, recall of 99.92%, and F1-score of 99.85%. To comprehensively evaluate the proposed framework, extensive experiments are conducted using both a memory-forensics malware dataset and the large-scale BODMAS dataset containing 134,435 PE malware samples spanning 581 malware families. The experimental evaluation incorporates cross-validation, ablation analysis, robustness assessment under multiple perturbation conditions, and feature-space visualization analysis. The results demonstrate that the proposed CAE+LBP framework consistently outperforms standalone feature extraction approaches and conventional end-to-end CNN models while maintaining strong robustness and cross-dataset generalization capability across diverse malware distributions and noisy conditions.

Syed Shakir Hameed Shah, Syed Shakir Hameed Shah, Ahmed Hamed et al. · 0 citations
Preprint Jul 2026

HilEnT: Hilbert, Entropy Transformed Image Based Malware Detection

With the increasing threat of malware across various software related domains, malware detection and classification is critical to determine the response actions. Different strategies have been adopted to address the challenge of malware detection. With the advent of deep learning techniques, malware detection using image processing has garnered research attention. In this work, we proposed a novel malware binary to image transformation technique HilEnT based on a combination of Hilbert curve-based transformation of malware binary and the entropy feature comparison of malware file with benign and malware classes. Three grayscale images produced during this process are combined to form a three-channel colored image which is then used for malware detection using machine learning techniques. We performed supervised binary and multiclass classification to evaluate the effectiveness of our proposed HilEnT. We also evaluated a few-shot learning technique to assess the robustness of our proposed HilEnT in a practical setting where the number of available class samples is limited. Furthermore, we investigated the benefits of combination of Histogram of Oriented Gradients and Principal Component Analysis for time performance improvements through feature reduction techniques. We evaluated our proposed methodology on four datasets: Dike, Michael Lester Dataset, Microsoft BIG 2015 and a self-collected dataset, and achieved the state-of-the-art results.

Rahul Kale, Thesath Wijayasiri, K. Fok et al. · 0 citations