Evaluating Multimodal Explainable AI in Ambiguous Driving Scenarios: Effects on User Trust and Satisfaction
TL;DR
Overall, multimodal explanations appear beneficial, particularly during initial use, but should adapt to user needs and context, but should adapt to user needs and context.
Abstract
This paper investigates how different Explainable Artificial Intelligence (XAI) modality combinations affect users in fully automated vehicles across different types of ambiguous driving scenarios (ADS). In an online study (N = 123), three explanation modality combinations were compared: visual-textual (VT), visual-auditory (VA), and visual-textual-auditory (VTA). VT resulted in significantly lower situational trust and significantly higher cognitive load than VA and VTA. Explanation satisfaction increased significantly from VT to VA and was highest for VTA. While baseline levels of trust, satisfaction, and cognitive load differed between ADS categories, no significant modality–scenario interaction was found, indicating that modality effects generalize across forms of ambiguity. Based on these results, five design guidelines for XAI in fully automated vehicles were derived. Overall, multimodal explanations appear beneficial, particularly during initial use, but should adapt to user needs and context.