A Bilingual Academic Chatbot Based on Semantic Retrieval Using mBERT

Abstract

This study proposes a bilingual academic chatbot based on a semantic retrieval approach using the Multilingual BERT (mBERT) transformer architecture to support academic information services in higher education. The dataset was constructed from official academic information at Garut Institute of Technology, including new student admissions, academic calendars, institutional profiles, and lecturer and staff data. The data were organized in a bilingual question–and–answer format in Indonesian and English. The mBERT model was fine-tuned using a Sentence-BERT framework to generate sentence embeddings for semantic retrieval tasks, with MultipleNegativesRankingLoss applied during training. Model performance was evaluated using BERTScore to measure semantic similarity between chatbot responses and human reference answers. Experimental results show that the fine-tuned model outperformed the base model, achieving an average F1-score improvement from 0.7638 to 0.8152 for Indonesian and from 0.7556 to 0.8005 for English. The results also demonstrate more stable score distributions, indicating consistent semantic performance. The optimized model was subsequently integrated into a web-based prototype to enable real-time bilingual academic question answering. These findings confirm that combining mBERT with semantic retrieval effectively enhances the relevance and contextual accuracy of chatbot responses, thereby supporting digital transformation and improving the efficiency of academic services in higher education.