Isolation Forest-Based Anomaly Detection In APK Files

Abstract

The quick advancement of smartphones and Android apps has created a high increase in malicious applications which potentially can steal sensitive information, track user activity, or cause damage to the equipment. Traditional Isolation Forest-based detectors rely on treating all the data as a single global distribution and analyzing the particular features independently, but this limits their chance to detect the anomalous patterns relevant to the local area as well as the interaction among features such as permissions and calls. Thus, the paper presents the Hybrid Clustered Isolation Forest-Factorization Machine Framework for unsupervised Android malware detection. Initially, the framework uses MiniBatch K-Means to sort this application into groups with similar behavior. Then, cluster-based models of Isolation Forest are created for identifying local anomaly patterns that will be missed with the help of the only global model. The proposed pipeline incorporates missing-value imputation, RobustScaler-based feature scaling, frequency-based categorical encoding, and statistical feature selection (mutual information and ANOVA) to ensure robustness on the high-dimensional, imbalanced static features extracted from the Drebin dataset (7,255 samples, 216 features). Unlike conventional Isolation Forest, which is confined to global, feature-independent anomaly scoring, the proposed hybrid design jointly models local behavioral structure and feature interactions while remaining largely unsupervised. Experimental results show that the standalone Factorization Machine branch achieved an accuracy of 92.14% and a ROC-AUC of 95.95%, while the complete Hybrid The ensemble model showed great results with an accuracy value of 95.24%, precision equal to 95.07%, recall equal to 98.92%, and F1-score equal to 96.96%, the values which are higher than the performances of the isolated Clustered Isolation Forest and Factorization Machine models.