A Review of Deep Learning and Synthetic Data Generation for Human Pose Estimation in Autonomous Robot Systems

Abstract

Human pose estimation has become an important perception capability for autonomous robotic systems, enabling machines to interpret human body posture and motion in real time. This paper presents an overview of deep learning methods for 2D and 3D human pose estimation, with an emphasis on their applicability to autonomous robotic systems that require accurate, reliable, and computationally efficient human perception. First, the fundamental body representation models and main pose estimation paradigms are described, including single-person and multi-person approaches, as well as skeleton-based and model-based 3D estimation. Representative CNN-based, transformer-based, and emerging state-space model architectures are then reviewed and compared according to commonly used benchmark metrics. The role of synthetic data generation is examined as a means of addressing the limited availability and diversity of accurately annotated training data, especially for 3D pose estimation. Simulation-based rendering pipelines, parametric body models, motion-capture data, generative approaches, and fusion-based methods are discussed as techniques for addressing challenges related to body shape, pose and motion variation, appearance diversity, camera viewpoints, lighting conditions, occlusions, and accurate ground-truth annotations. The paper also highlights the synthetic-to-real domain gap as a key limitation that must be considered when applying synthetic datasets to real-world robotic systems. Finally, future research directions are discussed, including the use of generative AI, synthetic data generation, and multimodal sensing such as RF/WiFi to support more reliable human perception in autonomous robotic environments.