The Role of Gestures in the Perception of Creative One-Word References Produced by Humans and Virtual Agents

Abstract

Gestures are fundamental in human communication, but their role in creative language use is underexplored. Prior research indicates that co-speech gestures can support and enhance creative thinking, yet their influence on how listeners perceive novel words is not well understood. Virtual agents offer a scalable and experimentally controlled platform for investigating the production and perception of creative speech and gestures. We present a study that examined whether gestures influence the perception of referring expressions consisting entirely of novel words, and whether these effects differ between human and VA presenters. Two mixed-design experiments manipulated presenter type (human vs. VA) and gesture behavior condition (no gestures, redundant action/object-related, semantically supplementary). Results showed no strong effect of specific gesture or presenter conditions on comprehension. However, gestures improved perceived communication quality and showed a positive trend for enhancing perceived originality and effectiveness, two core dimensions of creativity. Furthermore, we found that higher visual imagery ability was associated with higher task performance and hence better comprehension of creative verbal or multimodal references. Our findings demonstrate that the integration of gesture research with virtual agents, individual differences, and concepts of creativity offers a promising direction for advancing the theoretical as well as methodological understanding of both human-human and human-agent communication.