Attention-Enhanced 1D Convolutional Neural Network Transformer for Skeleton-Based Sign Language Recognition

Abstract

This study proposes a Sign Language Recognition (SLR) method based on the analysis of temporal sequences derived from hand skeletal data. The input functions consist of hand-marked coordinates obtained from the video and normalized to a fixed length of the sequence by interpolation. The velocity is introduced together with position data to increase the feature representation and effectively expand the capacity of the feature space. The proposed architecture is a hybrid model that combines a 1D convolutional neural network with a Transformer encoder. The convolution block is responsible for extracting local time patterns, while the Transformer attention mechanism allows for modeling long-term time dependencies within frames. In addition, a channel attention module is introduced to improve the ability to adaptively isolate and detect the most relevant features. To increase the generalization ability of the model and the stability of training, several regularization and optimization strategies were adopted, including time data expansion, label smoothing, mixing regularization, and a single-loop learning rate scheduling policy. In the experiment, the evaluation has been carried out using skeletal representations and dynamic recognition of gestures to confirm the effectiveness of the proposed approach. The model demonstrates strong and reliable performance and shows considerable potential for integration into automatic Sign Language Translation (SLT) systems and advanced human-computer interaction applications.