Beyond Classification: A Multilingual mBERT Framework for Sentiment Analysis, Zero-Shot Categorization, and Severity Assessment on social media
TL;DR
The fine-tuned model is applied for sentiment analysis of tweets from the preprocessed TweetEval dataset, and it is shown that cross-language transfer learning without any preliminary training on multilingual sentiment datasets provides quite satisfactory performance.
Abstract
Considering the growth of multilingual user made content within social-media platforms, there is an urgent need for developing scalable, language-agnostic approaches for their analysis. Within this paper, we analyze mBERT's performance in sentiment classification in a binary setting as well as the possibility of performing transfer learning between languages. Specifically, the fine-tuned model is applied for sentiment analysis of tweets from the preprocessed TweetEval dataset, obtaining 79.2% of accuracy and 74.7% of the F1 score. It is shown that cross-language transfer learning without any preliminary training on multilingual sentiment datasets provides quite satisfactory performance. However, a more complex approach can be used, which consists of applying filtering of negative sentiments, categorization of subcategories through a sentence transformer with zero-shot settings, and grouping the resulting data in several major categories to obtain severity scores according to frequency thresholds. The application of the sentiment classification with transformers in combination with issue prioritization makes it possible to develop an end-to-end approach to structuring multilingual social media content.