Partial Human Perception from a Robot's Egocentric Visual Perspective: A Benchmark for Body-Part Visibility Prediction

Abstract

Invariant human perception from visual input is essential during human-robot interaction. Although multiple approaches exist, such as face recognition or human pose estimation, these methods frequently fail in real-world interactive settings, where robots typically operate from close-range view-points that result in partial observations via camera recording caused by occlusion, truncation, and motion. This study investigates human perception under these challenging conditions using a body-part visibility prediction task. To this end, we analyzed a novel egocentric dataset, collected with a humanoid robot (Nao) during social interactions. The dataset consists of 6,563 annotated frames labeled for the visibility of the head, torso, arms, and legs. A set of visual representation models is benchmarked, including general image, pose estimation, and body parsing models, as well as end-to-end baselines. We used Macro-F1 score as a performance metric and computational cost regarding inference time to evaluate the models. The results indicate that the representations are the primary determinant, with the RTMW-X model showing the best performance with its raw embeddings as representations. The analysis further highlights the trade-off between accuracy and computational cost, which is particularly relevant for resource-constrained robots in human-robot interaction applications. Additionally, differences in difficulty of prediction between specific body regions, such as the head and arms, are discussed to provide guidance for deploying perception models in human-robot interaction.