LEXF-ATT-XLM: A hybrid lexicon-enhanced attention model for hate speech detection in low-resource language Roman Urdu
TL;DR
A novel hybrid deep learning architecture that synergizes contextual language modeling with explicit domain knowledge and integrates a domain specific Roman Urdu hate lexicon as an auxiliary regression supervision signal within a multi task learning framework to guide the model’s focus.
Abstract
Hate speech detection in low-resource and informally written languages remains a significant challenge due to the lack of annotated corpora, orthographic variability, and complex code-mixing. Roman Urdu a non-standardized variant of Urdu written in the Latin script exemplifies these linguistic hurdles. In this paper, we propose LEXF-ATT-XLM, a novel hybrid deep learning architecture that synergizes contextual language modeling with explicit domain knowledge. Our model leverages XLM-RoBERTa for deep contextual embeddings, passed through a two-layer Bidirectional Gated Recurrent Unit (BiGRU) and a multi-head attention mechanism to capture both sequential and salient linguistic patterns. Crucially, the pooled representations are fused via a learnable gating layer (Linear + tanh) before final classification. Furthermore, we integrate a domain specific Roman Urdu hate lexicon as an auxiliary regression supervision signal within a multi task learning framework to guide the model’s focus. Evaluated on the RU-HSD-30K dataset using 3-fold stratified cross-validation, the proposed model achieves an average accuracy of 88.83% and a weighted F1 score of 88.83%, with fold wise weighted F1-scores of 88.58%, 89.19%, and 88.71%. Extensive ablation studies confirm that the lexicon-guided auxiliary supervision significantly enhances the model’s ability to handle lexical variations, negations, and informal spelling. These findings demonstrate the robust effectiveness of our approach in addressing the unique linguistic challenges of Roman Urdu hate speech detection.