Context-Aware Lightweight Indonesian Sign Language Recognition Using LSTM and IndoBERT
Abstract
Communication barriers between deaf and hearing individuals remain due to the lack of affordable and computationally efficient assistive technologies, especially for Indonesian Sign Language (Sistem Isyarat Bahasa Indonesia, SIBI) in low-resource educational settings. Existing approaches often rely on computationally intensive architectures or large word-level datasets, making them less suitable for deployment in low-resource environments. This study proposed a novel context-aware SIBI recognition framework that effectively balanced recognition performance and computational efficiency. The system operated primarily at the alphabet level and provided assistive word prediction without requiring large word-level sign datasets. The proposed method used keypoint-based visual feature extraction combined with a long short-term memory (LSTM) network to capture temporal gesture patterns. A Trie-based prefix search was used to generate candidate words, followed by contextual refinement using the Indonesian version of bidirectional encoder representations from transformers (IndoBERT) to produce meaningful word predictions. This design enabled efficient temporal modeling and lightweight contextual processing within a central processing unit (CPU)-friendly architecture suitable for low-resource environments. Experimental results showed that the model achieved an inference accuracy of 93.59%, with a precision of 94.38%, recall of 93.59%, and an F1 score of 92.83%, while maintaining near real-time performance on CPU-only hardware. Statistical evaluation through analysis of variance (ANOVA) showed that the model performance was stable and McNemar’s test demonstrated that the contextual modeling component brought a statistically significant performance improvement. These findings demonstrate that accurate, context-aware, and computationally efficient SIBI recognition can be achieved on low-cost hardware, supporting practical deployment in real-world assistive communication scenarios.