Leakage-Aware Fake-News Screening for Nigerian Political Claims: Temporal and Domain Evaluation of a Context-Aware Ensemble

Abstract

Automatic fake-news classifiers can learn publisher, recurring topics, and other factors unrelated to factual veracity. In this paper, a context-aware classifier is evaluated under temporal and event grouping for Nigerian political and election-related claims. The global training set included a corpus of 29,269 non-synthetic fact-check claims descriptions and a manually verified supplement of 135 Nigerian fact-check claims collected from 46 fact-check web pages and 75 event groups. Verdicts were classified as REAL if they were unequivocally TRUE, CORRECT, or ACCURATE and classified as FAKE if they were FALSE, INCORRECT, or FAKE. The global backbone used a combination of word and character TF-IDF feature sets with Logistic Regression classifier, calibrated Linear Support Vector Machine, Random Forest, Gradient Boosting, and validation-weighted soft voting. A separate context head using a Logistic Regression model was trained from grouped out-of-fold predictions on 101 claims with dates of 2023-2025 and added to the global ensemble as a blend. All 34 claims from 2026 were reserved for final testing. Contextual blend achieved 0.6765 accuracy, 0.6825 balanced accuracy, 0.6762 macro F1-score, and 0.7368 ROC-AUC, and the REAl and FAKE recall of 0.6316 and 0.7333, respectively. The global ensemble achieved 0.6947 balanced accuracy and 0.6964 macro F1-score in the same test. The difference in balanced accuracies was −0.0123, and its 95% event-group bootstrap confidence interval was (−0.1399; 0.0947), with an exact p-value of 1.000.