Sign Language to Text and Audio Converter

Abstract

Communication between sign-language users and people who do not understand sign language is difficult when an interpreter is not available. This paper proposes the Sign Language to Text and Audio Converter, a low-cost, camera-based assistive system that recognizes static hand gestures for English alphabet and number signs and converts them into readable text and spoken audio. In the proposed design, a live webcam feed is processed with OpenCV and MediaPipe to detect and isolate the hand region. The extracted region is resized, converted to grayscale and normalized, and then classified by a Convolutional Neural Network (CNN) built with TensorFlow/Keras and trained on a labelled sign-language image dataset. Recognized symbols are buffered into words and sentences, displayed in a simple interface, and converted into speech by a text-to-speech (TTS) engine, so that the user can choose text or voice output. The system needs only a standard webcam and avoids sensor gloves or special hardware. It is organized into four modules: study and dataset preparation, hand detection and preprocessing, CNN training and classification, and text/audio output with testing. This paper presents the architecture, methodology and planned evaluation of the system; the implementation is in progress, and no experimental results are claimed.