SignAura: An AI-Based Real-Time Recognition System for Sri Lankan Sign Language

Abstract

The communication barrier experienced by deaf and hard-of-hearing in Sri Lanka is still a serious issue in society since there are few interpreters with Sri Lankan Sign Language (SLSL) and no automated translation technologies. The present paper introduces SignAura, an artificial intelligence-driven real-time SLSL sign-recognition system that provides multilingual lexical text and synthesized speech. The system uses computer vision and a fully connected neural network to identify isolated SLSL signs from live video streams.A fully connected neural-network classifier was used instead of the CNN-LSTM architecture, with 21 hand landmark keypoints extracted per frame using MediaPipe and represented as 63 numerical features.The final processed dataset contained 38,210 frame-level records, including 30,568 training records and 5,731 held-out testing records, covering 381 sign classes. During model training, 10% of the training data was used as an internal validation subset.The final model achieved a test accuracy of 80.27%, with macro precision of 80.63%, macro recall of 80.36%, and macro F1-score of 79.70%, while the test loss was 3.3375. A confusion matrix was generated from the held-out test predictions to examine class-level classification behaviour and identify frequently confused sign classes. A controlled frame-rate benchmark was not conducted as part of the final evaluation; therefore, a specific FPS value is not reported. The system performs isolated sign-level recognition and provides corresponding multilingual lexical output rather than complete grammatical sign-language translation. The current implementation does not fully incorporate non-manual linguistic features such as facial expressions, mouth movements, and body posture. The results reveal that it is possible to implement lightweight AI-based assistive technologies to support inclusive and scalable communication support based on the context of Sri Lanka.