Development of an Augmented Reality and AI-Based SIBI Recognition Application for Deaf Students at a Special School
TL;DR
The findings indicate that the developed application is valid, practical, and capable of supporting SIBI learning in preliminary classroom implementation, and should be interpreted within the scope of a limited trial involving a small sample size and the absence of a control group.
Abstract
This study aimed to develop and evaluate an Android-based learning application that integrates Augmented Reality (AR) and artificial intelligence to support Indonesian Sign Language System (SIBI) learning for deaf students. The study employed the Research and Development (R&D) method using the Four-D (4D) model consisting of Define, Design, Develop, and Disseminate stages. The gesture recognition module combined MediaPipe for hand landmark detection with MobileNetV2 for gesture classification, achieving an approximate recognition accuracy of 89 %. The model was trained using 80% of the training data and 20% of the testing data. Expert validation involved two content experts and two media experts, while practicality testing involved 12 respondents (nine deaf students and three teachers). Effectiveness was evaluated using a one-group pretest-posttest design involving nine deaf students across four learning sessions. Data were collected through observations, interviews, documentation, questionnaires, and learning achievement assessments. The developed application obtained an overall expert validation score of 4.67 (Very Valid). The software quality evaluation based on the selected ISO/IEC 25010 characteristics showed 100% functional suitability, 80% portability, and 100% compatibility. The practicality testing produced an average score of 4.63 (Very Practical). The effectiveness evaluation indicated a significant improvement in learning outcomes (p = 0.001), with a moderate N-gain (0.56) and 77.78% classical learning mastery. These findings indicate that the developed application is valid, practical, and capable of supporting SIBI learning in preliminary classroom implementation. Nevertheless, the findings should be interpreted within the scope of a limited trial involving a small sample size and the absence of a control group.