Perceived and Felt Emotion in Music and Dance

Abstract

Perceiving and feeling emotion from nonverbal signals (e.g., facial expressions) is a considerable part of human social interactions. A common approach to studying such nonverbal interactions is to use music for auditory cues and dance for embodied gestures, given their capacity to communicate emotions. Yet most investigations examine facial expressions, auditory cues, or embodied gestures in isolation. Acknowledging the multimodal nature of artistic performances, this study tested combined display modes of nonverbal cues to examine how each mode shapes both perceived expressive effectiveness and felt intensity. We manipulated stimuli from a live-recorded music and dance performance to yield four conditions: Face-only (gestures masked), Gesture-only (face masked), Music-only, and All-modalities (face, gesture, and music as recorded). Participants (N = 165) were randomly assigned to one condition and rated, on 0–100 scales, 1) how well each emotion was expressed (perceived expressive effectiveness); and 2) how strongly they felt each emotion (felt intensity) for six basic emotions. We refer to this contrast as the emotion locus (perceived vs. felt). Aggregated results showed perceived expressive effectiveness ratings were approximately 7 points higher than felt intensity ratings, and mixed-effects models confirmed a reliable main effect of emotion locus. The All-modalities condition did not consistently outperform unimodal displays; effects depended on the target emotion (e.g., modality-specific advantages for fear and attenuation of felt anger in the All-modalities condition). Contrary to expectation, adding more channels is not inherently better. These findings underscore the need for emotion-specific analyses of modality combinations in future research.