Implementation of Transfer Learning for Automatic Summarization in Research Article Synthesis
Made Hanindia Prami SwariPuji Lestari TariganGusti Eka YuliastutiAgung Mustika Rizki
Firza Prima Aditiawan
Jul 2026· JOIV: International Journal on Informatics Visualization· 0 citations
TL;DR
The findings indicate that BERT-based extractive summarization can support preliminary literature screening, but further improvement is needed through stronger baseline comparison, human evaluation, and redundancy-aware optimization.
Abstract
The increasing number of scientific publications has made literature screening more time-consuming, particularly for researchers who need to identify the main contribution of an article before reading the full text. This study develops an extractive summarization model for Indonesian scientific articles using a BERT-based transfer learning approach. The proposed method represents each sentence with contextual embeddings and selects relevant sentences based on their similarity to the document representation, while applying a redundancy threshold to reduce redundancy. A curated corpus of Indonesian research articles was used for model development and evaluation. The generated summaries were evaluated using ROUGE-1, ROUGE-2, ROUGE-L, and ROUGE-Lsum, with the article abstract used as the reference summary. The experimental results show that the proposed model achieved a ROUGE-1 score of approximately 33%, indicating it retained important keywords and central information from the source documents. However, the lower ROUGE-2 score suggests that the model still has limitations in preserving phrase-level continuity and sentence coherence. Qualitative analysis also shows that the model can capture the main ideas of scientific articles, although some methodological details and contextual information are occasionally omitted. These findings indicate that BERT-based extractive summarization can support preliminary literature screening, but further improvement is needed through stronger baseline comparison, human evaluation, and redundancy-aware optimization.
The results show that BART achieves the best performance with an ROUGE-2 F1-score of 0.40664, while T5 demonstrates superior grammatical acceptability, achieving 93.36%, but BART achieves a very near performance to T5.
Emad Nabil· Islamic University Journal o...· 0 citations
: The surge in online video data presents the imperative requirement of summarizing it for information compression, efficient understanding, and to support decision-making. Recent research studies have predominantly utilized structured input data for summarization. This paper explores the current state-of-the-art models in summarizing unstructured, conversational, and long transcript data. We further apply a transfer learning approach using a fine-tuned BART model, pre-trained on the SAMSum dataset. Hyperparameter tuning and selective layer freezing are applied to optimise model performance. This study focuses on abstractive summarization of long video transcript data. Our methodology integrates ChatGPT to generate reference abstractive summaries from long transcript data. A semi-automated pipeline using TextRank is proposed for reference summary generation. The proposed fine-tuned model shows a significant increase in ROUGE scores over the baseline model. The findings suggest that the transfer learning approach is effective for abstractive summarization of long transcript data in real-world conversational domains.
Deepali Fernandes, Rupali S. Wagh· Journal of Computer Science· 0 citations
This paper introduces MAAD, a high-quality, carefully constructed and curated by the authors large-scale Arabic dataset for abstractive news summarisation. The authors selected a high-quality subset of 50,000 articles from the dataset Original, which contains 602,792 articles. To maintain the quality, diversity, and training suitability of the subset, the subset underwent a multi-stage preprocessing pipeline involving noise removal, duplicate filtering, linguistic normalisation, and expert validation. The experimental evaluation was executed in two phases. In the first phase, three transformer-based models (ArabicT5, AraBART, and mT5) were evaluated on a controlled subset of 1,110 articles to establish fair baseline comparisons among models, where ArabicT5 achieved the best performance (ROUGE-1: 23.64, ROUGE-2: 11.82, ROUGE-L: 22.10). In the second phase, ArabicT5-base was trained on all 50,000 articles to evaluate scalability, achieving substantially improved results of 68.4, 52.3, and 64.1, respectively, with a BLEU score of 58.7. The findings emphasise the significance of scale, effective preprocessing, and the benefits of Arabic-specific pretraining on the quality of summarisation. Moreover, a human evaluation on 500 randomly sampled instances verified fluency and adequacy scores of 4.86 and 4.35, respectively, with a strong inter-annotator agreement (Cohen's Kappa: 0.78 and 0.74). Overall, the findings indicate that MAAD is a reliable and scalable dataset with strong potential to serve as a benchmark for Arabic abstractive summarisation and to support the development of robust transformer-based models.
M. Al-Nahari, Ayedh Abdulaziz Mohsen, Nada Abdu Al-Humidi et al.· 2026 6th International Confe...· 0 citations
AI-based text summarization plays a critical role in managing the rapidly increasing volume of textual information by enabling efficient information retrieval and concise knowledge representation. Conventional summarization approaches often rely on heuristic rules and surface-level features, limiting their ability to capture contextual relevance and semantic coherence in long documents. To overcome these limitations, this study develops a deep learning–based extractive text summarization framework using a hybrid Bidirectional Encoder Representations from Transformers (BERT) combined with Bidirectional Long Short-Term Memory (BiLSTM) architecture. The model leverages BERT’s bidirectional transformer encoder to generate rich contextual sentence embeddings and employs a BiLSTM network to model inter-sentence dependencies and document-level structure, enabling effective identification of summary-worthy sentences while preserving original semantic content. The framework is evaluated using the BBC News Summary dataset, which contains professionally written news articles and corresponding human reference summaries. Performance is assessed through sentence-level classification metrics and ROUGE-based evaluation. Experimental results demonstrate strong effectiveness, achieving an overall accuracy of 98.5%, with balanced precision (98.7%), recall (98.2%), and F1-score (98.45%), indicating reliable sentence importance classification. In addition, the model attains competitive ROUGE ((Recall-Oriented Understudy for Gisting Evaluation) scores (ROUGE-1 = 0.56, ROUGE-2 = 0.33, and ROUGE-L = 0.52), confirming substantial overlap with human summaries. These results highlight the robustness, generalization capability, and practical applicability of the suggested BERT–BiLSTM structure for automatic text summarization and knowledge representation.
Dr. RakeshKumar ER, S. Ramachandran, Ayushi Khandal et al.· 2026 7th International Confe...· 0 citations
The proposed framework using Bidirectional Long Short-Term Memory with a hypergraph and a dominating set mechanism proves to be an efficient approach to automatic summarization and has the potential to be applied in journalism, healthcare, legal analysis, and digital content management.
Pradeepa Sampath, S. Subashini, V. Shanmuganathan et al.· Artificial Intelligence and...· 0 citations
The study proposed an attentive autoencoder framework that integrated a multihead self-attention mechanism for contextual representation, an importance scoring network for keyword ranking, and a dual-loss optimization function combining reconstruction loss and sparsity loss to enable accurate keyword identification without manual annotation.
Lei Peng· Journal of Cases on Informat...· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
