Motion-Aware Graph Convolutional Network for Topology-Enhanced Skeleton-Based Action Recognition

Abstract

Skeleton-based action recognition has achieved significant progress through spatio-temporal graph convolutional networks. However, existing topology-enhanced methods treat all skeletal joints uniformly, failing to emphasize task-relevant joints for coordination-level hand-centric actions. Moreover, the graph topology is defined by fixed spatial proximity or generic channel-wise refinement, ignoring the fact that joints engaged in correlated motion patterns carry stronger discriminative signals. Therefore, we propose a Motion-Aware Spatio-Temporal Graph Convolutional Network (MA-STGCN) with a unified motion-aware framework consisting of two tightly coupled modules. First, a Motion-Aware Joint Attention (MJA) module is proposed, enabling the model to dynamically emphasize joints with salient motion for different action categories. Then, a Motion-Correlated Graph Refinement (MCR) module constructs a sample-specific inter-joint motion correlation matrix and uses it to adaptively refine the graph adjacency, strengthening connections between joints that move in coordinated patterns. Experiments are performed on the NTU RGB+D 60/120 datasets, our method achieves 94.2% and 91.3% accuracy on the bone stream. Comprehensive ablation studies validate the effectiveness of each component.