Real-Time Static Hand-Gesture-to-Text Conversion Using MediaPipe Landmarks and Classical Machine Learning
Abstract
This paper presents a low-cost, browser-based system that converts static hand gestures into text in real time using only a standard webcam. Hand landmarks are extracted with MediaPipe Hands, converted into a 42-dimensional wrist-relative, scale-normalized feature vector, and classified into 36 classes (digits 0–9 and letters a–z). A hold-to-confirm state machine with a short voting filter turns the per-frame predictions into words and sentences, and the application is deployed as a Streamlit web service using WebRTC. On a custom landmark dataset of 3,263 samples, a Random Forest classifier reaches 95.9% accuracy and 0.952 macro F1 on a stratified 20% hold-out set (majority-class baseline: 4.6%). Repeated stratified cross-validation on the training partition gives 94.2 ± 1.0% accuracy, and Random Forest, k -nearest neighbours and RBF-SVM classifiers differ by about one percentage point after tuning. Classifier-only inference takes a median of 2.8 ms per sample on a laptop CPU. The evaluation is row-level and not signer-independent; this limitation, together with confusable classes such as 0/o and 6/w, is analysed and directions for signer-independent evaluation are given.