Multimodal Deep Learning for Mental Health Assessment: A Comprehensive Survey of Stress, Emotion, and Depression Detection

Abstract

Stress has developed a huge issue worldwide, impacting people's mental and physical health. In this context, the need for precise and timely stress detection in healthcare, workplace settings, and customized wellness monitoring has revealed the drawbacks of traditional stress detection methods, such as subjective evaluations and custom-designed feature engineering, which struggle with the issues of bias, limited generalization, and inaccurate or outdated stress metrics. In recent years, deep learning rising to the spotlight for stress detection, owing to its capability to learn hierarchical illustrations automatically from multimodal data. This survey aims to structure current state-of-the-art stress detection frameworks based on deep learning methods regarding the model architectures, data modalities, benchmark datasets, and evaluation methodologies. Different approaches such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), hybrid models, and transformer-based models are thoroughly explored. Moreover, analysis of multimodal techniques combining physiological signals, speech, and behavioral data is performed, and it is shown that the multimodal approach is more robust and better able to detect stress. The survey emphasizes the fact that, as a rule, hybrid and transformer-based models perform better than conventional and single-modality ones. The main problems are the difficulty of getting good data, cross-domain generalization, interpreting the result, and computational difficulty. Future directions focus on the use of large-scale multimodal data, lightweight real-time models, and explainable artificial intelligence techniques. In summary, this research provides a brief and structured summary of the various deep learning approaches for stress detection that would be beneficial for researchers working in affective computing.