STFusion-Net: A Lightweight Dual-Path Spatiotemporal Fusion Framework for Efficient Event-Based Gesture Recognition on DVS128
Abstract
Dynamic Vision Sensors (DVS) encode visual information as asynchronous brightness change events, enabling microsecond-level temporal resolution, high-dynamic range and power efficiency, making them particularly attractive for computationally constrained embedded systems. The prior art shows that most works tackle the problem by focusing on either accuracy or efficiency, while failing to combine explicit spatiotemporal decomposition, attention refinement and learned fusion, all within a light-weight architecture. In this paper we propose STFusion-Net, a dual path convolutional-recurrent architecture, which decouples motion structure and spatial relations (via depthwise-separable CNN), and motion dynamics (via bi-directional GRU), and performs channel-temporal attention refinement followed by a learned fusion. Our method is extensively evaluated on the IBM DVS128 gesture dataset under a particularly challenging subject-independent protocol (20 train, 3 validation, 6 test subjects leaving 288 test samples). STFusion-Net achieves 89.93 ±0.31% accuracy and 89.86 ±0.36% macro F1 with only 0.35M parameters, 18.60 M FLOPs and 2.70 ms GPU latency (370 FPS model throughput). Compared to the best performing competing approach (CNN+Attention), STFusion-Net achieves an increase of +1.46 pp in accuracy while reducing both parameters by 27.1% and GPU latency by 18.2%. We perform a comprehensive ablation analysis including component-wise contribution, representations, attention variants, and temporal-slice sensitivity studies. These results establish STFusion-Net as a principled light-weight baseline for event-based gesture recognition.