Abstract Human communication is inherently multimodal, relying on signals conveyed through speech, vocal tone and facial movements. While gestures and vocal cues are known to influence how speech is understood, the role of facial movements remains poorly specified. Here, we test whether dynamic facial movements shape how spoken responses are interpreted by systematically modulating listener inferences of speaker confidence and doubt. Using a data-driven reverse-correlation approach and a generative model of facial behaviour, we manipulated speakers’ upper-face movements while holding verbal content constant. On each trial, participants (n = 44; UK English and Mandarin Chinese speakers) rated the confidence of speakers responding to questions (e.g. ‘Would they make a good leader?’) on a five-point scale across 1800 trials. We find that opposing facial movements bias these inferences: eye closure and brow raising are associated with confidence, whereas brow lowering and eye tightening are associated with doubt. Signal amplitude further shapes these inferences, primarily by intensifying perceived doubt for doubt-associated movements. These mappings are largely shared between the two cultural groups, indicating common signal–inference relationships alongside differences in the weighting of specific cues. Together, these findings show that facial movements provide graded social signals that systematically shape how spoken responses are interpreted.
People differ not only in what they express facially during conversation, but also in how their facial behavior unfolds over time. This person-specific temporal structure remains underexplored computationally. Prior work has largely focused on controlled settings with limited numbers of participants, and many facial re...
Masoumeh Chapariniya, Pierre Vuillecard, J. Odobez et al.· Proceedings of the 28th Inte...· 0 citations
Recognizing emotions from faces and voices is fundamental to successful social interaction, yet how the brain represents emotional information across sensory systems remains unclear. Here, we systematically mapped the contribution of individually-defined face- and voice-selective cortical regions in 24 human participan...
Stefania Mattioni, Federica Falagiarda, Tanisha Arya et al.· Journal of Neuroscience· 0 citations
During face perception, observers preferentially attend to features that signal emotional expressions (e.g. the eye region of angry and the mouth of happy faces), reflecting their importance for inferring others' emotional states. Since previous research has demonstrated these effects primarily in highly controlled lab...
Matthias Gamer, Lisa-Marie Krause, D. Gromer et al.· British Journal of Psycholog...· 0 citations
Face-based social evaluations are ubiquitous and consequential, yet it remains unclear how these evaluations are represented in spontaneous physiological responses. Across four studies (222 participants: 142 females, 57 males, 23 unreported; 503 neurons), we used a data-driven modeling framework to examine how facial a...
An-Qi Mao, Run-Nan Cao, Sai Sun et al.· Journal of Neuroscience· 0 citations
Gesture serves as a foundational modality for conveying event structure, providing critical visual information about transitivity and participant roles. Sign languages integrate manual (hand-based) and non-manual (face, torso, head) articulations to encode complex linguistic information. Among these, mouth movements pr...
J. Krebs, R. Wilbur, D. Roehm et al.· Brain and Language· 1 citation
Facial and vocal behaviors jointly convey affective and communicative information, yet their cross-modal organization remains difficult to characterize quantitatively. We propose a framework for characterizing facial-vocal organization by aligning facial and vocal dynamics using canonical time warping. Multivariate fac...
Yumi Shikauchi, Naoe Taiga, L. Puhlmann et al.· bioRxiv· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.