DiMMPose: A Diffusion-Mamba Hybrid Framework with Multi-Prompt for Efficient and Robust 3D Human Pose Estimation

Abstract

Monocular 3D Human Pose Estimation (3D HPE) typically adopts a two-stage approach: estimating 2D joint positions from images and then lifting them to 3D coordinates, effectively reducing dataset bias inherent in direct methods. However, current lifting techniques face two key challenges: many Transformer-based methods rely on attention-based or staged spatial–temporal modeling, which can limit efficient long-range frame-joint reasoning, while diffusion models support probabilistic modeling of pose uncertainty but remain sensitive to joint-coordinate noise. We propose DiMMPose, a diffusion-based framework enhanced by Mamba’s state-space model for robust and efficient 3D pose estimation. Its denoising process consists of two coordinated modules. The Spatiotemporal Mamba Block (STMB) serves as the core feature extraction module, employing internal Pose Mamba components with bidirectional state propagation and linear complexity to efficiently model long-range frame-joint dependencies. STMB further refines these features through Spatiotemporal Scan and Merge, which traverses the same skeleton tokens in complementary frame-joint orders and fuses the resulting representations. The Multi-Prompt Mamba Denoiser (MPMD) combines structured prompts encoded by LongCLIP with learnable prompt representations to provide anatomical and motion-related guidance during denoising. DiMMPose achieves an average MPJPE of 28.9 mm on Human3.6M under the DET setting, with action-specific errors of 21.2 mm for Walking and 22.0 mm for WalkTogether. It improves over FinePOSE by 3.0 mm, reduces inference latency by 57.1%, and achieves 23.0 mm MPJPE on MPI-INF-3DHP (N = 243).