SonoActNet: Wearable Ultrasound-Based Human Action Recognition for Rehabilitation Monitoring

Abstract

Rehabilitation exercises are now more often performed outside of the clinic, and the ability to assess the quality of movement reliably using sensors is critical. Wearable ultrasound has the potential to be useful in this context as it can directly monitor the activation and deformation of the muscle rather than relying solely on the external movement. By contrast, many current rehabilitation monitoring pipelines rely primarily on inertial sensing or surface electromyography (EMG), which are indirect measures of muscle activity and are prone to degradation when actions look kinematically similar or when there are interindividual differences in anatomy and action style. Conventional CNN- and RNN-based models also have the disadvantage of only capturing a small fraction of the temporal structure and frequently fuse modalities in a trivial manner. In this work, we introduce a temporal Transformer framework, called SonoActNet, which extracts multi-scale ultrasound features, applies crossframe attention and contrastive pretraining to model long-range muscle dynamics, and fuses IMUs to provide complementary kinematic context. We test the approach on three rehabilitation oriented datasets, namely SonoRehab-24 (clinic), SonoRehabHome (home), and Ultrasound-ActionLite (public) with crosssubject, cross-session and domain-shift protocols. SonoActNet achieves up to 95.8% accuracy and 95.2% F1-score and consistently outperforms CNN baseline with ultrasound only, inertialonly approaches, and traditional multimodal fusion methods. The findings suggest that the SonoActNet is a viable model for wearable ultrasound for real-time rehabilitation monitoring.