Evaluating Audience Perception in Indonesian Animation: Comparative Aspect-Based Sentiment Analysis Using Random Forest and IndoBERT
TL;DR
Findings confirm the superiority of pre-trained Transformer architectures for complex multi-class ABSA tasks in low-resource Indonesian social media text.
Abstract
The rapid expansion of the Indonesian animation industry has sparked vibrant public discourse on social media, yet existing research primarily focuses on binary or overall sentiment rather than fine-grained aspect-level evaluations. This study presents an aspect-based sentiment analysis (ABSA) comparing a traditional machine learning baseline (Random Forest with TF-IDF and SMOTE) against a deep learning model (fine-tuned IndoBERT) on 2,759 tweets discussing domestic animated films Jumbo and Merah Putih: One For All (MPOA). An 11-class joint aspect-sentiment classification scheme was established using a semi-automated annotation pipeline, achieving almost perfect inter-annotator agreement (Cohen’s Kappa κ = 0.8616). Experimental results demonstrate that fine-tuned IndoBERT significantly outperformed the Random Forest baseline, increasing accuracy from 40.16% to 51.64% (+11.48 percentage points) and weighted F1-score from 36.60% to 45.69% (+24.8% relative gain). Corpus-wide aspect distribution revealed that general film evaluation dominates social media discussion (95.9%), followed by animation quality (1.9%) and storyline (1.6%). Furthermore, sentiment analysis uncovered contrasting reception patterns: Jumbo attained overwhelmingly positive net sentiment (+63.8%), whereas MPOA faced severe negative backlash (−46.2%) driven by technical animation critiques and public controversies. Despite overall performance constraints caused by severe class imbalance and limited training data, these findings confirm the superiority of pre-trained Transformer architectures for complex multi-class ABSA tasks in low-resource Indonesian social media text.