A Multimodal Deep Learning Based Framework for Early and Accurate Diagnosis of Depression Using Electroencephalography and Event‐Related Potential Signals
TL;DR
This is the first study to integrate EEG and ERP data in a multimodal fashion using three feature sets, achieving a high accuracy of 91% with GRU‐based classification and indicates that the combination of multimodal features and deep learning can help improve the accuracy of depression diagnosis and develop effective assistive tools in this field.
Abstract
Major depressive disorder significantly impacts both physical and mental health, leading to decreased emotional and cognitive functioning. The limitations of traditional methods for diagnosing depression—such as reliance on patient cooperation, subjective bias, and low accuracy—have prompted researchers to explore alternative diagnostic tools. This study presents a multimodal approach aimed at enhancing the accuracy of depression diagnosis by utilizing Electroencephalography (EEG) and Event‐Related Potential (ERP) Signals. The multimodal approach which incorporates different knowledge from different data modalities can lead to more accurate and stable diagnosis while providing a tool for early detection. For the analysis of the mentioned modalities, three key feature sets including, spatial, time‐frequency, and spectral features are extracted. Node2vec algorithm is used to extract spatial information from a graph representation of the EEG signals from different electrodes. Cross wavelet transformation (XWT) combined with a pre‐trained AlexNet was employed to extract time‐frequency features, while power spectral density (PSD) features were processed by a temporal‐separable convolutional network (TSCN) to extract spectral features. These features were then classified separately using Gated Recurrent Unit (GRU) and Transformer models to compare their effectiveness in depression diagnosis. The data which is experimented in this article is extracted from the Multi‐modal Open Dataset for Mental‐Disorder Analysis (MODMA) public database and includes signals from 3 and 128 electrodes. The results show that the GRU model outperforms Transformer and achieves k‐fold cross‐validation accuracies of 70.91%, 81.33%, 75.33% and 91% in different datasets. To the best of our knowledge, this is the first study to integrate EEG and ERP data in a multimodal fashion using three feature sets, achieving a high accuracy of 91% with GRU‐based classification. These results indicate that the combination of multimodal features and deep learning can help improve the accuracy of depression diagnosis and develop effective assistive tools in this field.