An open-set hybrid CNN-TCN network for human action recognition with unknown action rejection in collaborative robot workspaces

Abstract

Introduction Open-set recognition capability is becoming necessary for collaborative robots because actual shop-floor dynamics do not always correspond to the fixed action classes used during training. Misclassification of unknown human motions may lead to inappropriate robot responses, whereas reliable rejection enables safer supervisory control. Methods An open-set human action recognition framework is developed by combining a spatial Convolutional Neural Network encoder with a Temporal Convolutional Network-based temporal aggregation module and an unknown-action rejection gate based on open-set scoring. The framework is evaluated on the NTU RGB+D 120 dataset using cross-subject and cross-view protocols with held-out action classes. Results The proposed framework achieves 93.20% and 91.00% top-1 accuracy under cross-subject and cross-view evaluation, respectively, with corresponding Macro-F1 scores of 88.40% and 86.20%. For open-set detection, it achieves an AUROC of 0.938 and an AUPR of 0.750, with a 5.00% false acceptance rate for unknown actions and a 91.82% true known-action acceptance rate. The system operates at 15.0 ms per window and 66 fps on an NVIDIA Jetson Orin NX 16 GB, with a model size of 42 MB. Discussion The results demonstrate that calibrated unknown-action rejection can improve safety-aware human action recognition while retaining practical real-time performance for collaborative robot workspaces.