A Real-Time FMCW Radar Hand Gesture Recognition System Based on TA-VMRNN

Abstract

To address the limitations of vision-based gesture recognition regarding illumination dependence and privacy concerns, as well as the parameter redundancy of traditional radar deep learning models, this article proposes a real-time frequency-modulated continuous-wave (FMCW) radar gesture recognition system based on a lightweight TA-VMRNN architecture. Range, doppler, and angle features are extracted from radar signals to construct range-time (RTM), doppler-time (DTM), and angle-time (ATM) maps as network inputs. In terms of feature processing, the proposed TA-VMRNN architecture adopts Triplet Attention at the front end and incorporates a VMRNN spatiotemporal memory network at the back end to achieve radar feature extraction and fusion. Experimental results demonstrate that the proposed model achieves a recognition accuracy of 99.24% on a handwritten gesture dataset containing digits 0–9, while the number of parameters (Params) and floating-point operations (FLOPs) are only 0.465 M and 0.056 G, respectively. Furthermore, in leave-one-subject-out (LOSO) cross-validation, the average accuracies for offline and real-time recognition reach 97.62% and 97.37%, respectively. In a discrete grid spatial test, the system obtains an average recognition accuracy of 96.45%. The proposed end-to-end system provides a highly viable and practical solution for human–computer interaction (HCI) scenarios under constrained computing resources.