Gesture-Based Computer Interface Using Hand Gestures and Eye Blink Detection

Abstract

Conventional human-computer interaction relies on physical input devices such as a mouse and keyboard, which is not always convenient, hygienic, or accessible to every user. This paper presents a touchless, webcam-based computer interface that allows a user to control common mouse and navigation functions using hand gestures and eye blinks, without any dedicated sensor or wearable hardware. The system captures live video using OpenCV and processes each frame with Media Pipe Hands and Media Pipe Face Mesh to obtain twenty-one hand landmarks and dense facial landmarks in real time. The hand landmarks are used to move the on-screen cursor, and to recognise pinch, fist, two-finger, and multi-finger gestures that are mapped to clicking, dragging, scrolling, zooming, volume adjustment and swipe-based page navigation. The facial landmarks around each eye are used to compute the Eye Aspect Ratio (EAR), which is thresholder and timed to distinguish short and long blinks of the left and right eye, each of which is mapped to a distinct mouse action. Detected gestures and blink events are translated into system-level mouse and keyboard commands using PyAutoGUI. An on-screen heads-up display further allows the user to enable or disable individual interaction modes at run time through a pinch-based menu. The proposed system demonstrates that a standard webcam, combined with pretrained landmark-detection models, is sufficient to build a functional, low-cost, touchless computer interface with applicability to accessibility-oriented and hygienic human-computer interaction scenarios