An explainable AfroXLMR approach for multi-label emotion classification of Amharic social media text with dataset release.

Abstract

Emotion detection from social media is crucial for understanding human emotions across languages. However, for low-resourced languages such as Amharic, the lack of annotated data makes this task challenging. Additionally, most current models use black-box methods that obscure whether predictions rely on linguistically meaningful cues. To address these gaps, this study proposes a multi-label emotion classification model for Amharic by fine-tuning AfroXLMR. To enhance transparency, we integrate explainable artificial intelligence (XAI) into the framework. We compiled and annotated a new dataset of 22,000 unique social media comments across eight emotion categories for training, validation, and testing. The data was split into 80% for training, 10% for validation, and 10% for testing. The proposed model achieved a recall of 87% and a Hamming loss of 0.08. To interpret its predictions, we applied Local Interpretable Model-agnostic Explanations (LIME). We also evaluated the model against several state-of-the-art baselines, including XLM-R base, mBART, BiLSTM, LSTM, CNN, and AfriBERTa. The results show that our approach outperformed each baseline, achieving F1-score improvements of 5% over XLM-R base, 3% over mBART, 5% over BiLSTM, 7% over LSTM, 9% over CNN, and 2% over AfriBERTa. Bootstrapped statistical significance testing confirms that these improvements are robust and not attributable to random variation. In conclusion, the fine-tuned AfroXLMR model demonstrates promising performance in Amharic multi-label emotion classification. Building on this success, next steps could involve exploring more advanced fine-tuning strategies and expanding our datasets to strengthen both performance and the model's ability to generalize across diverse Amharic contexts.