Convolutional Enhancement of Indobert for Sentiment Analysis in E-Commerce Reviews
Abstract
Customer reviews are among the clearest signals of product and service quality on e-commerce platforms, yet automatically spotting non-positive ones is difficult once positive feedback dominates the data by a wide margin. This paper examines whether adding a convolutional layer on top of a finetuned IndoBERT model improves detection of non-positive sentiment in heavily imbalanced Indonesian e-commerce reviews. Using 57,981 cleaned Tokopedia product reviews with a positive-to-non-positive ratio of roughly 36:1, we built and compared two binary sentiment classifiers under identical preprocessing, splitting, and class-weighting protocols: a baseline that classifies directly from IndoBERT's pooled [CLS] representation, and IndoBERT-CNN, which instead routes the full sequence of hidden states through three parallel one-dimensional convolutional filters (kernel sizes 2, 3, and 4) before prediction. Both models were tuned via grid search and trained with class-weighted cross-entropy loss after undersampling the majority class. On a held-out test set preserving the original imbalance, IndoBERT-CNN reached a macro F1-score of 0.7730 against the baseline's 0.7627, with accuracy at 96.53% and 96.31% respectively, alongside consistent gains in non-positive-class precision and F1-score. Retraining the best configuration of each scenario across five independent random seeds confirmed the macro F1-score gap as statistically significant (paired t-test, $\mathbf{p}=\mathbf{0. 0 2 1 9})$ rather than a one-off result of a lucky initialization. These findings indicate that local n-gram feature extraction via small convolutional kernels offers a real, though modest, benefit for minority-sentiment detection, even as non-positive precision remains constrained by the severity of the class imbalance.