Domain-informed density extraction for robust mental health classification of long social media posts
TL;DR
The findings demonstrate that a leakage-safe evaluation protocol is essential for producing credible results in mental health classification from long social media posts and show that domain-informed density extraction provides a robust and practical text representation, with its greatest benefits emerging when models are unable to process the entire post.
Abstract
Background Recent advances in machine learning and natural language processing (NLP) have enabled the early identification of mental disorders from social media content. Among such platforms, Reddit is characterized by linguistically rich, long-form narratives in which users describe their psychological symptoms, emotions, and personal experiences. However, these datasets present significant challenges for text classification because the posts are lengthy, voluminous, and class-imbalanced. In this study, we revisit the concept of a segmentation-based, expression-weighted representation that emphasizes clinically relevant language while suppressing irrelevant content. During this investigation, we identify a critical limitation in conventional preprocessing pipelines—specifically, the application of chunking and oversampling prior to the train–test split—which introduces data leakage and consequently inflates reported performance. To address this issue, we propose a leakage-safe evaluation protocol together with a domain-informed density extraction method that identifies clinically dense passages using a lexicon derived exclusively from the training data. Results The proposed method was evaluated using logistic regression, linear SVM, XGBoost, fastText, RNN, and TextCNN, together with a transformer-based baseline (MentalBERT), all under a leakage-safe evaluation protocol. After eliminating data leakage, the apparent advantage of naive weighted chunking disappeared, and the headline macro-F1 score decreased from approximately 0.96 to 0.67. In contrast, the proposed domain-informed density extraction method consistently outperformed fixed-window chunking in five of the six models and matched or exceeded the full-text baseline in several cases. Statistically significant improvements were observed for logistic regression, RNN, and fastText, despite using only a fraction of the input text. For the context-limited transformer, density-based selection significantly outperformed naive truncation (macro-F1 + 0.023, 95% CI [+0.006, +0.041], p = 0.008). Conclusion Our findings demonstrate that a leakage-safe evaluation protocol is essential for producing credible results in mental health classification from long social media posts. They further show that domain-informed density extraction provides a robust and practical text representation, with its greatest benefits emerging when models are unable to process the entire post. By selectively preserving clinically informative content while reducing input length, the proposed approach offers an effective and computationally efficient alternative to increasingly complex model architectures.