A dual-architecture deep learning pipeline for real-time high-accuracy Arabic sign language recognition

Abstract

This paper presents a dual-architecture deep learning pipeline for real-time Arabic Sign Language (ArSL) recognition, designed to enhance communication accessibility for the Deaf and Hard of Hearing community. The current system recognises only isolated static Arabic letters; extending it to dynamic gestures and continuous signing is beyond the scope of this work and is identified as a key direction for future research. The proposed system integrates MediaPipe-based hand landmark detection with background removal and data augmentation, processing both static images and live video streams to translate isolated gestures into letters of the alphabet. We systematically compare two classification approaches under identical preprocessing and statistical evaluation protocols: (1) a fine-tuned ResNet18 achieving a mean accuracy of 97.8 ± 0.3% (95% CI [97.2%, 98.4%]), and (2) a hybrid architecture combining EfficientNet-B2 feature extraction with a Random Forest classifier, achieving 98.9 ± 0.2% (95% CI [98.5%, 99.3%]) on a diverse dataset of 7857 labelled RGB images from over 200 participants. This constitutes the first rigorous statistical comparison of these two paradigms specifically for ArSL recognition. Real-time deployment on consumer hardware achieves 19–22 FPS with per-frame latencies of 45–52 ms and no perceptible lag, demonstrating practical utility. Error analysis identifies anatomically similar letter pairs (Sad/Dad, Teh/Theh) as primary challenges. This work provides an accurate, efficient tool for real-world assistive technology applications.