True Consensus from Subtleties - Detecting Cognitive Synchrony in Two-person Interactions Based on Multimodal Physiological and Behavioral Signals

Abstract

We study the problem of detecting whether consensus in a dyadic interaction reflects genuine information integration or a superficial agreement. While recent studies have linked interpersonal physiological synchrony to teamwork and decision quality, current multimedia datasets and models provide limited support for modeling consensus authenticity in conflict-driven interactions. To address this gap, we introduce CODEC (COnsensus DEtection in Conflict), a dyadic multimodal dataset collected with a hidden-profile paradigm that naturally induces disagreement, negotiation, and consensus formation. The dataset includes synchronized wearable physiological and behavioral streams from 13 dyads, together with fine-grained temporal annotations of interaction states. We also present a dynamics-informed late-fusion model that combines subject-aware prompt tuning, cross-modal attention guided by a cross-recurrence (CRQA) prior, and supervised contrastive learning to classify true versus false consensus from continuous multimodal segments. Under leave-one-group-out evaluation, the proposed model achieves a weighted F1 score of 0.85 and consistently improves over early-fusion and sequential baselines. Our results suggest that multimodal physiological-behavioral signals offer a promising direction for analyzing consensus quality beyond overt audiovisual behavior alone. The dataset can be accessed through https://github.com/CODEC-MM/Dataset.