An Interpretable Vision-Language Framework for Evaluating the Uncanny Valley Effect of XR Humanoid Characters

Abstract

As AI-generated humanoid characters are increasingly used in virtual, augmented, and mixed reality applications, evaluating the Uncanny Valley Effect (UVE) is crucial for immersive user experience. Existing evaluation methods map visual features to affective scores, offering limited interpretability regarding which visual cues are associated with affinity judgments. Among the theoretical perspectives proposed to explain the UVE, perceptual conflict provides a visual-cue-oriented perspective for analyzing whether local-feature realism supports a coherent overall human-likeness impression and how this is reflected in affinity judgments, yet this perspective is rarely incorporated into interpretable UVE assessment. Thus, we propose UVE-Perception Chain-of-Thought (UVE-PCoT), a vision-language framework for interpretable UVE evaluation from a perceptual-conflict-oriented perspective. UVE-PCoT organizes assessment through a structured perceptual decomposition, including assessments of overall human-likeness, local-feature realism, perceptual conflict, and affinity. To provide supervision, we construct UVE-R, a structured rationale dataset with image-grounded, rating-consistent rationales linking visual cue observations, cue-level inconsistency analysis, and affinity judgments. Results show that UVE-PCoT improves affinity prediction and cue-level explanation over general-purpose multimodal large language models and ablations. Our approach operationalizes this perceptual-conflict-oriented perspective into an interpretable framework, advancing UVE evaluation from black-box scoring to explanatory analysis and providing cue-level insights for XR character assessment and revision.