XLM-R for Robust Cross-Lingual and Multilingual Sentiment Analysis of Code-Mixed E-Commerce Product Reviews

Abstract

The rapid expansion of global e-commerce platforms has led to a substantial increase in user-generated reviews written in multiple languages and code-mixed forms. These multilingual and cross-lingual reviews reflect diverse consumer opinions; however, linguistic heterogeneity introduces challenges in accurately capturing sentiment across different language settings. Conventional monolingual models often struggle to generalize in such environments, leading to inconsistent sentiment classification performance. This study investigates the effectiveness of transformer-based and large language models for multilingual and cross-lingual sentiment analysis. To ensure a comprehensive evaluation, this study compares conventional transformer models (BERT-base and RoBERTa), multilingual pre-trained models (mBERT and XLM-R), and a large language model (GPT 3.5). The experimental framework involves fine-tuning and evaluating each model on two datasets, MARC and BanglishRev, using standard classification metrics, including accuracy, precision, recall, and F1-score. The results indicate that multilingual transformer architectures outperform conventional and general-purpose models in linguistically diverse settings. XLM-R consistently achieves the highest performance across both datasets, reaching 0.97 accuracy, while maintaining balanced precision and recall. mBERT also demonstrates competitive results, whereas GPT 3.5 shows comparatively lower performance without task-specific fine-tuning.