The Effect of Hyperparameter Settings on BERT-Based Detection of Mental Health Support Needs on X (Twitter)

Abstract

Mental health issues, particularly indications of the need for professional support, have become increasingly prominent in the digital era, requiring effective and efficient detection methods. This study develops a text classification model to identify indications of the need for professional mental health support from Indonesian-language posts on the X platform. The dataset consists of 40,000 scraped tweets collected from July 2023 to July 2025, followed by preprocessing (noise removal, slang and bilingual term normalization, and stopword removal) and automatic labeling using TF-IDF scoring and rule-based cue detection. The dataset was split using stratified sampling into train/validation/test sets (70%/20%/10%), with a label distribution of 73% non-needs_help and 27% needs_help. The model was built by fine-tuning BERT (bert-base-uncased) for binary classification (needs_help, non-needs_help) and evaluated using accuracy, precision, recall, and F1-scores (macro and weighted). Hyperparameter experiments across 12 combinations (learning rate 2e-5, 3e-5, 5e-5; batch size 16, 32; epochs 3, 5) show that the best configuration is learning rate 5e-5, batch size 32, and 3 epochs, achieving an accuracy of 0.8608 and a macro F1-score of 0.8017 (macro precision 0.8585; macro recall 0.7733). Epoch-wise stability analysis indicates a consistent decrease in evaluation loss up to epoch 3, while macro F1 peaks at epoch 2 and remains stable at epoch 3, confirming that hyperparameter selection affects both performance and training stability when fine-tuning BERT on informal social media text.