A Multimodal Latent State-Space Framework Reveals Spontaneous Interpersonal Synchrony Dynamics

Abstract

Interpersonal synchrony reflects the temporal coordination of behavioural, physiological, and neural processes between interacting individuals. Existing synchrony-analysis methods quantify signal similarity but provide limited insight into recurring coordination states and their temporal organization across modalities. We propose a multimodal latent state-space framework that integrates EEG, body movement, facial expression, and eye-gaze signals into a unified dyadic divergence representation. Following dimensionality reduction, Gaussian mixture modelling identifies recurring synchrony states, whose temporal organization is characterized by measures of occurrence, persistence, and long-range stability. The framework was evaluated using a publicly available dataset of 23 dyads who passively observed one another under Near and Far Vision conditions without communication or an explicit shared task. Five latent synchrony states were identified, spanning highly synchronized to highly desynchronized interaction patterns. Near Vision interactions exhibited greater occupancy of the highest synchrony state (22.0% vs. 17.3%), longer persistence (d = 0.59, p =.004), and greater long-range stability (d = 0.56, p =.007). Surrogate analyses further showed a substantial disruption of the recovered state architecture when interpersonal temporal dependencies were removed. These findings demonstrate that recurring multimodal synchrony states can emerge under minimally constrained social conditions and highlight latent state-space modelling as a complementary approach for characterizing dynamic interpersonal coordination.