Rating Prediction in Brazilian Portuguese: From Classical Features to Large Language Models
TL;DR
This paper consolidates results from six interconnected studies investigating rating prediction and rating-text inconsistency detection and provides practical guidelines for model selection based on accuracy requirements, training data availability, and cost constraints.
Abstract
Online reviews play a crucial role in e-commerce, yet research on rating prediction for Brazilian Portuguese remains limited. This paper consolidates results from six interconnected studies investigating rating prediction and rating-text inconsistency detection. We evaluate approaches spanning classical machine learning with 58 textual features, BERT-based models, and ten large language models in zero-shot settings. Results show that BERTimbau achieves the best performance among fine-tuned models (MAE 0.56, RMSE 0.91), while DeepSeek and ChatGPT-4o lead among Large Language Models (LLMs) (RMSE 0.93). We also extend the analysis to a multilingual context with emoji signals across 13 European languages. For inconsistency detection, we find that LLM reliability varies substantially: ChatGPT-o3 shows low consistency across runs (κ = 0.18), while DeepSeek-3.2 achieves near-perfect agreement (κ > 0.95) with F1-score above 97%. Our findings provide practical guidelines for model selection based on accuracy requirements, training data availability, and cost constraints.