Adaptive BERT for Semantic Alignment and Intelligent Triage of Multilingual Inquiries in Cross-Border E-Commerce

Abstract

Currently, in the field of cross-border e-commerce, there are problems such as insufficient adaptation of general pre-training models to domain features, significant differences in the semantic gap between English-Chinese cross-language and English-Russian scenarios, low training efficiency and high deployment costs when implementing complex models. In this regard, the study proposes a domain-adaptive Multilingual Bidirectional Encoder Representations from Transformers (MBERT) model that integrates a Domain Adaptation Layer (DAL) and a Cross-Language Alignment Module (CLAM). This study first performs domain adaptation optimization through pre-training a Domain-Distinguish Task (DDT) BERT model. On this basis, MBERT is used to provide the model with a cross-language semantic representation, and the DDT mechanism and Retrieval-Augmented Generation (RAG) technology are finally integrated. The experimental results showed that, on the SemEval-2016 attribute-level emotion annotation data set, the proposed model achieved average accuracies of 82.2%, 83.2%, and 88.0% in the English-Chinese, English-Russian, and English-Spanish scenarios, respectively, demonstrating the best performance. Pre-training took 6.3 h and converged in 16 rounds. Inference took 19ms/sentence and occupied 9GB of video memory. Adding DAL alone increased the F1 value in the English-Spanish scene by 4.1%, and adding CLAM alone increased the English-Chinese accuracy by 4.2%.