Temporal and linguistic enrichment for abstractive text summarization using transformer models

Abstract

Abstractive text summarization is a critical yet challenging task in natural language processing, requiring models to generate concise, coherent, and factually accurate summaries. Existing transformer-based architectures, such as BART and T5, often underperform in scenarios where language semantics evolve over time or when lexical diversity is limited. These limitations result in summaries that lack contextual relevance and semantic generalization, especially in dynamic domains such as news reporting. In this work, we present a lightweight hybrid abstractive summarization model that enhances temporal awareness and linguistic flexibility. The model integrates synonym-based augmentation and time-sensitive embeddings into a base transformer encoder-decoder architecture, enabling the generation of temporally aligned and semantically enriched summaries. To further improve reliability, we introduce a factual alignment evaluation module that assesses entity-level consistency using named entity recognition. We conduct extensive experiments on two benchmark datasets, XSum and CNN/Daily Mail, and evaluate the model using a wide range of metrics, including ROUGE, BLEU, METEOR, BERTScore, and factual consistency checks. The ablation study confirms the individual contributions of temporal modeling and synonym enrichment, with the combined model achieving up to 8.3% ROUGE-1 and 7.9% METEOR improvements over the baseline. The model demonstrates practical applicability for summarization systems requiring both temporal relevance and semantic robustness, while maintaining CPU efficiency and interpretability. Implications for downstream tasks like news trend analysis are also explored.