SSTMamba: A Skeleton Guided Spatio-Temporal State Space Model for 3D Human Pose Estimation

Abstract

Monocular 3D human pose estimation aims to recover the positions of human joints in 3D space from 2D images or videos. Existing Transformer-based methods usually rely on spatial or temporal self-attention mechanisms to obtain global context, but their computational cost grows rapidly with sequence length, which limits the efficiency of long sequence pose modeling. Mamba architecture based on state space models (SSM) has provided a near-linear alternative for long sequence modeling. However, existing SSM-based methods mostly do not sufficiently reflect the structured dependencies among human joints, especially anatomical priors such as kinematic chains and left-right symmetry. To address the above problems, this paper proposes SSTMamba, a skeleton guided spatio-temporal state space model. Specifically, the model contains a Skeleton Path Spatial Mamba module(SPSM), which models spatial correlations among joints within a single frame by dividing human joints into kinematic topology paths and bilateral symmetry paths; and a Phase shifted Strided Temporal Mamba module(PSTM), which reduces the temporal modeling length through strided scanning and uses phase complementary sampling offsets to model cross-frame temporal correlations of each joint. SSTMamba follows the sequence-to-sequence(seq2seq) frame-wise prediction mode. While avoiding the construction of dense self- attention matrices, it realizes structural prior guided spatial modeling and efficient long range temporal modeling. Experiments on Human3.6M show that SSTMamba achieves competitive 3D pose estimation performance on the basis of near-linear computational complexity.