Hybrid Contextual Transformer Framework for Depression Detection in Multilingual Social Media

Abstract

This paper presents an experimental study of contextual transformer-based architectures for depression detection from conversations on social media data in the eRisk 2025 dataset by the CLEF Lab. The study addresses two tasks: (1) Depressive Symptom Relevance Detection from conversational posts and (2) Multilingual Depression risk classification. For the first task, different approaches of contextual input configurations (PRE-TEXT-POST) along with domain-specific transformer models (MentalBERT and MentalRoBERTa), probability calibration strategies, a Context-Aware Weighted Fusion (CAWF) mechanism, and multi-seed ensemble methods are thoroughly examined. The results show that MentalBERT, when used with TEXT+POST context, yields the best performance among single models, although the averaging ensemble method leads to further improvement in prediction stability. For the second task, a Hybrid XLM-RoBERTa + CNN + BiLSTM model is proposed for depression detection in multilingual settings with a highly skewed class distribution. The findings show that transformer architectures which are well-calibrated perform better than proposed Context-Aware Weighted Fusion (CAWF) mechanism for Task 1 and balanced evaluation metrics remain important for imbalanced mental-health datasets for Task 2.