A systematic review of emotional nonverbal cues in embodied virtual agents for virtual environments

Abstract

Embodied virtual agents (EVAs) are increasingly deployed in immersive virtual environments, where their nonverbal cues serve as a primary channel for affective communication. However, systematic evidence on how these nonverbal cues shape users’ emotional perception in virtual environments remains limited and dispersed across the literature. This paper presents a systematic literature review from a human-perception perspective to examine EVAs’ nonverbal cues in virtual environments and their effects on users’ affective perception. Searches were conducted in Scopus, the ACM Digital Library, IEEE Xplore, Web of Science, and SpringerLink, yielding 1066 records. Following the PRISMA guidelines, 52 studies met the inclusion criteria. The results indicate that facial expressions, posture and gaze are the most commonly used cues that influence users’ affective outcomes, with postural cues often demonstrating greater effectiveness in virtual environments. EVAs’ nonverbal expressions shape users’ affective perception across multiple dimensions, including their emotional responses, perceived social affect, and sense of social presence. More importantly, the immersive properties of virtual environments amplify the emotional contagion effects of these cues, while also increasing users’ sensitivity to subtle inconsistencies. Furthermore, the effectiveness of EVAs’ nonverbal cues in shaping users’ affective perceptions is context-dependent, with differing priorities across application domains. Advances in nonverbal generation for EVAs, from rule-based systems to generative AI, have progressively improved fidelity and affective perception, although they may also give rise to the uncanny valley effect. This review summarizes that EVAs’ nonverbal emotional cues substantially shape users’ affective perception in virtual environments, underscoring the importance of coherent multimodal cue design and context-appropriate generation methods for creating emotionally effective virtual agents.