HTA-RoBERTa: An Hybrid Triple-Attention Framework for Context-Aware Sentiment Classification

Abstract

Large scale unstructured text data is one of the sources of information where the classification of sentiment is an important feature for extracting meaningful views and emotional orientations. But the absence of it, noisy characters, irrelevant symbols, and the high frequency of stop words can have a negative impact on the representation of features and the classification. The aim of this study is to present an optimized hybrid triple-attention-based RoBERTa framework for sentiment classification that is both accurate and robust. The text data is preprocessed systematically, which includes detecting and handling missing data, removing noise (meaningless characters and symbols), text normalization, and stop-word removal. The cleaned text is then fed into RoBERTa to produce rich contextualized word representations. A hybrid triple-attention mechanism is introduced to further capture the complementary semantic dependency at both the token and contextual and feature levels, for better identification of sentiment bearing information. In addition, an optimization approach is used to find appropriate model parameters and enhance convergence and generalization ability. The proposed system is tested using the standard sentiment set of databases and performance measurement used are accuracy, precision, recall, F1 Score, and AUC. The proposed framework classify sentiment with better performance than the conventional machine learning and baseline transformer-based approaches.