Evaluation of the BERT model for text semantic similarity

Abstract

Text semantic similarity calculation is a core technology in the field of natural language processing, and it is crucial in scenarios such as question answering systems and information retrieval. Traditional methods based on keyword matching or static word vectors are difficult to capture deep semantics of text and cannot effectively solve the problem of lexical ambiguity. This study aims to systematically evaluate the performance and advantages of the BERT model in text semantic similarity calculation. By constructing a multi-source dataset containing academic literature, online text, and standard evaluation data, and after preprocessing such as text cleaning and word segmentation standardization, a pre-training + finetuning training paradigm was adopted, and the model was trained by setting appropriate hyperparameters. The F1 score and correlation coefficient were used as core indicators to conduct comparative experiments with traditional models such as Word2Vec and GloVe. The results show that the BERT model achieves F1 scores of 0.84 and 0.91 on the CNSE and CNSS datasets, respectively, which are significantly better than traditional models, and it can effectively utilize contextual information to eliminate ambiguity and capture deep semantic relationships. This study validates the effectiveness of the BERT model in semantic similarity calculation, providing more accurate technical support for related application scenarios, and laying the foundation for subsequent model optimization and lightweighting research.