Multimodal User Authentication via Keystroke-Triggered Fusion of Glove-Based Finger Pressure and Hand Kinematics
TL;DR
A per-modality ablation finds the fused model significantly better than both single-modality models at every smoothing window, and a per-modality ablation finds the fused model significantly better than both single-modality models at every smoothing window.
Abstract
Keystroke dynamics constitute a cost-effective behavioral biometric, but their timing features degrade under environmental and hardware variation. We replace these features with 19 dimensions of finger pressure and hand kinematics, sampled around each keystroke by a custom data glove–ten piezoresistive pressure sensors and a nine-axis inertial measurement unit (IMU). The two modalities are encoded by separate convolutional neural network–long short-term memory (CNN-LSTM) branches and joined only at the embedding. We collected typing data from ten participants, one enrolled as the genuine user. Training uses a desktop keyboard and evaluation a laptop keyboard recorded on a later day; the held-out impostor is excluded from training, and each of the nine takes that role in turn (Leave-One-Impostor-Out). Under this condition the method attains a mean Equal Error Rate (EER) of $0.53~\pm ~0.42$ % after three consecutive keystrokes, about 0.77 s of typing at the measured median pace, and $0.18~\pm ~0.22$ % after five. Results depend strongly on which impostor is held out. A per-modality ablation finds the fused model significantly better than both single-modality models at every smoothing window. The pressure-level channels carry that advantage and have to be scale-matched to deliver it, by separate encoders or by standardization indifferently.