Multi-Source Generalization-Aware Phishing URL Detection Using Calibrated Stacked Ensemble and False-Positive Control
Phishing remains a persistent cybersecurity threat, with over 1.3 million attacks reported in a single quarter of 2023. Despite strong benchmark performance, many machineand deep-learning models exhibit limited deployment reliability because they are evaluated using balanced, single-source datasets with randomized splits. This paper addresses this gap in two phases. First, it presents a controlled multi-source empirical study of four baseline phishing URL detection models–Logistic Regression, Support Vector Machine, Random Forest, and XGBoost–under four increasingly realistic evaluation conditions, including cross-source generalization, temporal drift, and class imbalance. Second, it introduces GAFPNet (Generalization-Aware and False-Positive Controlled Framework Network), a five-module stacked-ensemble framework. GAFPNet uses dataset-neutral lexical and structural URL feature extraction, SMOTE-based imbalance correction, Platt scaling for ensemble calibration, and a tunable false-positive control scheme. Experiments using a consolidated 13,000-sample set from PhishTank, OpenPhish, and Tranco Top-Sites show that baseline accuracy decreases by 13.16 to 20.22 percentage points in cross-source testing and by 2.41 to 8.14 percentage points in standard testing. GAFPNet achieves 99.12% accuracy, a 98.97% F1-score, an AUC-ROC of 0.9943, an MCC of 0.988, and a false positive rate of 0.74%, outperforming the evaluated baselines in all four scenarios. An ablation study confirms the contribution of each module. These results position GAFPNet as a generalization-aware and deployment-oriented phishing URL classifier for real-time filtering applications.