EEG Based Emotion Recognition with Hybrid CNN-LSTM Deep Learning Model

Abstract

Emotion recognition using deep learning has gained importance across various domains because of its ability to provide interaction between computer and human beings. It helps the system in understanding and responding to emotions effectively. It has wide range of applications like healthcare, education, customer service, autonomous system and so on. EEG is used for the emotion recognition as it helps in direct monitoring of the brain activity. When CNN and LSTM techniques are used separately for emotion recognition, they have their own drawbacks. CNN provides limited temporal modelling and has overfitting issues on datasets that are small. The LSTM provides slow training, has convergence issues and holds difficulty in capturing the spatial features. To overcome these issues, the proposed work integrates both these techniques to form a hybrid model. CNN is used to effectively capture the spatial features from the EEG signals and identifies the patterns present in the brain activities. The LSTM network analyses the temporal dependencies in the data. This combined approach paves path for comprehensive extraction of feature which is then processed by an emotion classifier to categorize the emotional states that have been detected. An experiment was conducted by utilizing images and videos as stimulus and the brain activities of the subjects were monitored with the help of EEG electrodes. The steps involved in the proposed work includes signal collection and processing, labelling emotions, extraction of features and classification. The emotion states are of four types. They are EAEV (Elevated Arousal, Elevated Valence), EASV (Elevated Arousal, Suppressed Valence), SASV (Suppressed Arousal, Suppressed Valence), SAEV (Suppressed Arousal, Elevated Valence. The results obtained have been compared to the other existing approaches and have been analyzed. It has been proved that the proposed model gives better results in terms of the various performance metrices.