Context-dependent effects of multimodal explanatory interfaces on passenger workload and situation awareness in fully autonomous driving.

Abstract

This study investigated how multimodal explanatory interfaces affect passenger workload, situation awareness, trust, and XAI evaluation in fully autonomous driving. A 3 × 2 × 2 within-subject simulator experiment manipulated NDRT modality (radio, reading, video), auditory prompt type (spoken explanation vs. alert tone), and visual text presentation (text vs. no text). Responses were assessed using fNIRS, eye tracking, HRV, NASA-TLX, and subjective measures. Explanatory effectiveness was context dependent: no-text explanations were more favorable during radio engagement, whereas text-supported explanations were more favorable during video engagement. Auditory-prompt effects also varied by task state, supporting context-aware explanation design matched to passenger task occupancy.