Skip to content
Open access

FakeDiverse a curated multi-source news corpus for context-aware fake news detection using BERT and DeBERTa

Jul 2026 · Scientific Reports · 0 citations

TL;DR

This study examines the effectiveness of two transformer-based architectures—BERT and DeBERTa—for identifying fake news using only textual information from headlines and article bodies and achieves strong performance on FakeDiverse corpus, demonstrating the need for enhanced generalization strategies as well as domain adaptation.

Abstract

The rapid growth of online media has accelerated the circulation of misinformation, posing serious risks to public trust, social stability, and democratic processes. Detecting fake news remains a challenging task, particularly because traditional machine learning approaches such as Support Vector Machines and Naïve Bayes rely on shallow linguistic cues and often fail to recognize sarcasm, implicit tone, or evolving writing styles. Earlier deep learning models, including CNNs and LSTMs, provide modest improvements but continue to struggle with long text and shifting topical contexts. This study examines the effectiveness of two transformer-based architectures—BERT and DeBERTa—for identifying fake news using only textual information from headlines and article bodies. To enhance generalization, news articles from ten publicly available datasets are consolidated into a single corpus namely FakeDiverse, that exposes the models to a broader spectrum of linguistic patterns. MinHash similarity along with Locality Sensitive Hashing algorithm has been employed to evaluate the lexical redundancy within the FakeDiverse corpus as a post hoc analysis. Class imbalance between real and fake news is addressed through a weighted cross-entropy loss function, ensuring that minority-class instances are adequately represented during training. FakeDiverse dataset is split in the ratio of 80:20 for training and testing, and subsequently the models BERT and DeBERTA are trained on the training set and evaluated using the test sets. Experimental results demonstrate that both the models capture contextual nuances effectively, with BERT achieving 98% accuracy and DeBERTa reaching 99%. The findings demonstrate that BERT and DeBERTa achieves strong performance on FakeDiverse corpus, and cross-dataset evaluation shows that models have challenges to generalize unseen data distribution thereby highlighting the need for enhanced generalization strategies as well as domain adaptation.

Read PDF

Similar papers

Open access Jul 2026

Fake News Identification Using Hybrid Transformer Ensemble Approach

A hybrid transformer-based ensemble model for automated fake news identification using the FakeNewsNet dataset is proposed and Experimental results show that the ensemble model achieves an accuracy of approximately 93%, outperforming the individual constituent models.

M. E. Babu, G. Sukanya · 0 citations
Open access Aug 2026

Multilingual Fake News Detection Using Machine Learning with Contextual-Based Feature Extraction

Fake news has become a major challenge in today’s digital environment, particularly in languages where labeled data is limited. Most existing research has primarily focused on English due to the easy availability of annotated datasets, whereas low-resource languages such as Bengali remain underexplored. This study presents a multilingual approach for fake news detection using machine learning with contextual-based feature extraction. The proposed method integrates n-gram techniques with sentence-level contextual embeddings to capture both word-level patterns and semantic meaning. Since labeled data is not available for the Bengali dataset, a translation-based strategy is employed, followed by a pseudo-labeling process to assign labels automatically. The models are trained on English news titles and subsequently evaluated on both English and Bengali datasets to examine their cross-lingual effectiveness. The experimental findings indicate that ensemble-based classifiers such as Random Forest and Gradient Boosting achieve reliable performance across both languages. In some cases, the results for Bengali data are comparable or slightly better than those for English. The study demonstrates that effective fake news detection is possible in low-resource languages using short text data without relying on manually labeled datasets. The proposed approach provides a simple and efficient solution for multilingual fake news detection in data-scarce environments.

Nikita Garg, Pritam Singh Negi · 0 citations
Open access Aug 2026

Fake News Detection Using Machine Learning and LLM Embeddings: A Comparative Study of TF-IDF and BERT Representations on the Welfake Dataset

The rapid growth of social media and online news platforms has significantly increased the spread of misinformation and fake news. Fake news can influence public opinion, create social unrest, and negatively impact decision-making processes. Therefore, developing intelligent systems capable of automatically identifying fake news has become an important research area in Artificial Intelligence and Natural Language Processing (NLP). The proposed work presents a Fake News Detection System that combines traditional machine learning techniques with Large Language Model (LLM) embeddings to improve classification performance. The proposed system utilizes textual news data from benchmark datasets such as WELFake and LIAR. Initially, the news articles undergo pre-processing steps including text cleaning, normalization, and removal of irrelevant characters. Two different feature extraction approaches are employed: Term Frequency-Inverse Document Frequency (TF-IDF) and contextual embeddings generated using the pre-trained BERT (Bidirectional Encoder Representations from Transformers) model. The extracted features are used to train and evaluate multiple Machine learning classifiers, including Logistic Regression, Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), and XGBoost. Performance evaluation is carried out using standard metrics such as Accuracy, Precision, Recall, F1-Score, and Area Under the ROC Curve (AUC-ROC). Comparative analysis is performed to assess the effectiveness of traditional TF-IDF features versus BERT-based semantic embeddings. Experimental results demonstrate that BERT embeddings capture contextual and semantic information more effectively than conventional text representations, leading to improved fake news classification accuracy. The system also provides visualization tools such as confusion matrices, ROC curves, and performance comparison charts for comprehensive evaluation. The proposed framework highlights the potential of integrating transformer-based language models with classical machine learning algorithms to build robust and scalable fake news detection systems. Our work contributes to combating misinformation by providing an efficient and automated approach for identifying deceptive news content in digital media environments.

Umme Noor Us Saqa, Sreenivasa B. R. · 0 citations
Conference Jul 2026

Enhancing Fake News Detection using a BERT–BiLSTM Hybrid Framework

The rapid proliferation of digitally distributed news has made large-scale automated verification an urgent research priority, as deceptive content spreads across social platforms faster than manual reviewers can evaluate it. This work introduces a two-stage deep learning pipeline in which a pretrained BERT encoder generates rich contextual token embeddings that are subsequently processed by a Bidirectional Long Short-Term Memory (BiLSTM) network, enabling the classifier to exploit both sentence-level semantics and document-level narrative flow simultaneously. Experiments on the WELFake benchmark [1] following a standardised cleaning pipeline—lowercasing, URL stripping, tokenisation, stop-word removal, and WordNet lemmatisation— yielded 98.7 % accuracy and an F1-score of 0.986. This surpasses a BERT-only baseline by 3.1 percentage points and exceeds several previously published state-of-the-art results. The gains confirm that coupling transformer-based semantic representations with recurrent sequential modelling produces a measurable and reproducible improvement in misinformation detection. Planned extensions include multilingual evaluation and knowledge-distilled encoder replacements targeting real-time throughput constraints.

Ardra P Namboodiri, Archa P S, Honey Mol O · 0 citations
Open access Jul 2026

Synthesizing Political Context Understanding with AI Techniques to Improve Fake News Detection Performance across Regions

Fake news is spreading quickly on the internet, which is very bad for society and the security of the government. The significant issue that was talked about in the paper was the creation of automatic systems that can detect fake news better and adapt to various areas. The dataset used in the study is the LIAR dataset, which is a standard set of various political statements labeled with varying degrees of truthfulness. Text is also cleaned up, tokenized, and represented with existing trained word embeddings such as GloVe and Word2Vec as a step in data preparation. To identify complex trends in the text, most language and contextual features are removed, such as syntactic, semantic, and sentiment-based ones. The primary contribution of this study is a way of grouping various features into one representation. A set of models is subjected to performance tests, and it includes Random Forest, Naive Bayes, Convolutional Neural Network (CNN), Autoencoder, and a proposed Hybrid CNN-Autoencoder architecture. The hybrid model performs the most, having the greatest precision and the most equalized classification scores. Comparative analysis demonstrates that the combination of deep learning and knowledge of the environment significantly enhances the level of detection in domains. It is a flexible AI-based system that can work in the context of language and political differences and is a big step forward in searching for fake information automatically.

Pundlik Dattatray Jadhav, R. K. Shukla · 0 citations