Self-Attention-Based VGG16 Approach for Sign Language Gesture Recognition in Inclusive Education

Abstract

Sign language recognition (SLR) plays an important role in improving communication between deaf or hard-of-hearing individuals and the hearing community and has promising applications in inclusive education. However, conventional convolutional neural network (CNN)-based models mainly focus on local feature extraction and may fail to effectively capture global spatial dependencies, especially when recognizing visually similar static gestures. To address this limitation, this paper proposes a self-attention-based VGG16 framework for static sign language gesture recognition of alphabetic (A–Z) and numeric (0–9) hand signs. The proposed approach integrates a multi-head self-attention (MHSA) mechanism into a pretrained VGG16 architecture in order to enhance global feature representation while preserving effective local feature extraction. Experiments conducted on a 37-class static sign language dataset show that the proposed model outperforms the baseline VGG16 architecture, achieving a test accuracy of 99.87%. The obtained results confirm the effectiveness of self-attention in improving recognition performance and prediction stability for static sign language gestures. These findings suggest that the proposed framework can serve as a promising building block for intelligent assistive technologies supporting inclusive educational environments.