A Comparative Review of Deep Learning Architectures for Emotion Recognition with Experimental Validation in Clinical Behavioral Monitoring

Abstract

—Deep learning has significantly advanced emotion recognition across visual, physiological, and multimodal data. Nevertheless, selecting appropriate architectures for real-world deployment remains challenging due to variations in dataset characteristics, computational constraints, and application requirements. This paper presents a comparative analysis of major deep learning architectures for emotion recognition, including Convolutional Neural Networks (CNNs), ResNet, MobileNet, Long Short-Term Memory (LSTM), Bidirectional LSTM, and Vision Transformers (ViTs), highlighting their strengths and limitations across heterogeneous data environments. Building upon the comparative review, an experimental study was conducted to examine the feasibility of a hybrid deep learning framework for detecting clinically observable affective states from temporally evolving facial video data in palliative care settings. The proposed approach integrates spatial and temporal representations and is evaluated using Leave-One-Video-Out validation to simulate real-world deployment on previously unseen individuals. Experimental results demonstrate stable learning behavior and meaningful behavioral prediction patterns despite moderate numerical accuracy caused by patient variability and dataset imbalance. Analysis indicates that hybrid architectures are effective in distinguishing stable versus expressive behavioral conditions, supporting the interpretation of affect recognition as behavioral condition awareness rather than categorical emotion classification. The study provides practical insights into architecture selection and demonstrates that hybrid deep learning systems can support continuous, non-intrusive behavioral monitoring as assistive analytical tools across healthcare and related application domains.