Robust Multimodal Depression Detection via Targeted Fine-Tuning of RoBERTa and Wav2Vec2

Abstract

The proposed paper presents a multimodal deeplearning approach to detect Depression from Clinical Interview data through transcript analysis and Acoustic Feature extraction. This was implemented by freezing both encoder models, RoBERTa (for Transcript Analysis) and Wav2Vec2 (for Acoustic Feature Extraction) except for the last two layers, followed by creating projection layers of 256 units and a Joint classifier w ith Dropout of 0.4. The dataset from DAIC-WOZ consisting of 189 participants was taken and a filtering b ased on missing and corrupted data was done which produces 142 valid samples. These samples were used to develop the final model through 3-fold stratified cross-validation w ith Class Weighting (Pos Weight=2.38), Gradient Clipping, and Early Stopping protocols in place. The final results produced a Mean accuracy of 88.80% (±8.30%), Mean F1 score of 83.60% (±10.80%) and a Mean Area Under the Curve (AUC) score of 91.60% (±10.00%); whereas the best fold accuracy achieved was 95.70% and a 0.993 AUC with convergence achieved in 10 epochs. Collectively, the study validates that multimodal fusion through purposefully targeted fine-tuning to resolve issues due to overfitting and class imbalances provides an effective and reliable automated method for screening individuals for Depression.