Feature Representation Effects in Transformer Model for Speech-Based Depression Detection

Abstract

Speech-based depression detection has increasingly explored self-supervised speech representations; however, their effectiveness in low-resource clinical and non-English settings remains insufficiently understood. This study investigates the impact of feature representation on depression detection using a fixed Transformer architecture to isolate representation effects. We compare handcrafted acoustic features, comprising clinically motivated prosodic and spectral descriptors, against a feature fusion representation that combines these extracted features with Wav2Vec2 embeddings. Experiments are conducted on two linguistically distinct datasets, a Malay clinical speech corpus and the DAIC-WOZ dataset, using stratified five-fold cross-validation with F1-score as the primary evaluation metric. Results show that handcrafted acoustic features consistently outperform the feature fusion representation across both datasets, achieving F1-scores of approximately 0.75 (Malay) and 0.60 (DAIC-WOZ), compared to 0.61 and 0.46 for the fusion setting. The fusion approach does not improve performance and in some cases leads to degradation, indicating limited complementarity between handcrafted acoustic features and self-supervised speech embeddings under simple integration. These findings suggest that in low-resource clinical speech settings, carefully extracted acoustic features remain highly competitive, and naive feature fusion with high-dimensional self-supervised representations does not necessarily yield performance improvements. The study highlights the importance of representation selection in clinical speech modelling and provides empirical evidence on the limitations of direct feature fusion for depression detection tasks.