A Reproducible Dependence-Aware Multilevel Framework for Repeated Multimodal Data in Human–Computer Interaction
Abstract
Repeated multimodal experiments in human–computer interaction (HCI) generate hierarchically and temporally structured data, making analyses that treat sensor-derived observations as independent vulnerable to invalid inference. The primary contribution of this study is a reproducible, dependence-aware multilevel framework that integrates five connected modules: synchronization and quality assessment, participant-by-phase feature construction, longitudinal inference, within-person multimodal integration, and design-calibrated simulation of pseudoreplication. The framework preserves the participant as the independent experimental unit while accommodating outcome-specific transformations, dependence structures, diagnostics, multiplicity-controlled comparisons, and exploratory moderation analyses. Its implementation is demonstrated using a publicly available stress–recovery dataset comprising 35 participants and 175 participant-phase observations. The empirical application identified substantial within-person dependence across outcomes (intraclass correlations: 0.646–0.984), heterogeneous behavioral and physiological trajectories, no robust resilience moderation after false-discovery-rate correction, and a dominant within-person principal component explaining 30.3% of multimodal variance. As a complementary methodological evaluation, simulation under a true null effect produced a false-positive rate of 30.8% for naive ordinary least squares, compared with 6.2% for a random-intercept mixed model. The empirical findings serve to demonstrate the framework rather than provide definitive population-level conclusions about stress resilience. By connecting data-quality decisions, feature construction, dependence-aware modeling, multiplicity control, within-person integration, and simulation within an auditable process, the framework provides a transferable route from repeated multimodal streams to interpretable and statistically defensible evidence in HCI research.