Fine-grained wrist activity recognition can support applications such as procedural step guidance and context-aware assistance, yet acquiring labeled data for every new task, user, and activity granularity remains a bottleneck. We present TransfHAR, a self-supervised wrist IMU framework for on-demand, fine-grained activity recognition by learning transferable motion priors from global, unlabeled activities. We show that self-supervised pretraining on coarse wrist IMU activities (e.g., sitting, walking, exercise) learns motion structure rich enough to transfer to fine-grained manipulative, gestural, and procedural activities (e.g., snapping, stirring, waving) that are absent from pretraining. We implement TransfHAR as a real-time smartwatch application that lets users define and expand their own activity set for personalized recognition from only a few demonstrations. Across three offline cross-dataset evaluations, TransfHAR matches or exceeds fully supervised baselines that use complete label sets with equal or additional sensor channels, by 6.2 balanced-accuracy points on average. In an in-lab study with 10 participants each performing seven novel wrist activities, TransfHAR reaches 86.7% balanced accuracy across participants with five examples per class and 90.4% when updated from a single one-minute recording per class. These results indicate that broad self-supervised wrist pretraining provides an effective foundation for on-demand fine-grained activity recognition.
Aidan Bradshaw, Riku Arakawa, Xin Liu et al.· 0 citations
Recent years have witnessed outstanding success of deep learning in sensor-based human activity recognition (HAR), spanning a wide range of real-world applications like healthcare management, fitness tracking, and fall detection. However, sensor data annotation scarcity still remains a main challenge unresolved, which requires human annotators to take a long-term and tedious observation to segment and timestamp sensor samples meticulously, hampering the wide use of deep learning models, especially in few-shot HAR scenario. To handle such issue, this paper introduces a cross-modal data augmentation, by exploiting activity label text as key words to search for activity-related images to construct an augmented dataset. On this basis, a new cross-modal feature adapter is designed, which fine-tunes a pre-trained CLIP image encoder to effectively align image-sensor pairs. Through a learnable residual ratio, it may adaptively blend the old knowledge inherited from the original zero-shot CLIP with the new knowledge adapted from few-shot training samples, which makes training converges faster whilst forming a streamlined time series sensor encoder. Extensive experiments and ablation studies are performed on three public HAR benchmarks. The experimental results demonstrate that the proposed method outperforms existing state-of-the-art HAR baselines under all few-shot scenarios. A practical on-device inference latency is provided.
Xin Liu, Lei Zhang, Wenbo Huang et al.· IEEE journal of biomedical a...· 0 citations