Skip to content
#explainable ai Book Open access

Evaluating Multimodal Explainable AI in Ambiguous Driving Scenarios: Effects on User Trust and Satisfaction

Sep 2026 · Adjunct Proceedings of the 18th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · 0 citations · 20 references

TL;DR

Overall, multimodal explanations appear beneficial, particularly during initial use, but should adapt to user needs and context, but should adapt to user needs and context.

Abstract

This paper investigates how different Explainable Artificial Intelligence (XAI) modality combinations affect users in fully automated vehicles across different types of ambiguous driving scenarios (ADS). In an online study (N = 123), three explanation modality combinations were compared: visual-textual (VT), visual-auditory (VA), and visual-textual-auditory (VTA). VT resulted in significantly lower situational trust and significantly higher cognitive load than VA and VTA. Explanation satisfaction increased significantly from VT to VA and was highest for VTA. While baseline levels of trust, satisfaction, and cognitive load differed between ADS categories, no significant modality–scenario interaction was found, indicating that modality effects generalize across forms of ambiguity. Based on these results, five design guidelines for XAI in fully automated vehicles were derived. Overall, multimodal explanations appear beneficial, particularly during initial use, but should adapt to user needs and context.

Read PDF

Similar papers

Book Open access Sep 2026

Exploring the Design Space of Adaptive Explanations in Automated Vehicles

Explanations in automated vehicles have been proposed to support users’ understanding, trust, and acceptance of automated driving behavior. However, many explanation interfaces remain static interventions, even though users’ information needs may vary depending on driving context, familiarity, perceived criticality, an...

Philipp Asteriou, Ambika Shahu, Philipp Wintersberger · 0 citations
#artificial intelligence Review Sep 2026

Automatic multimodal UX improvement recommendations from LLM agent user simulations

AMUSER is presented, a multimodal framework which simulates user behaviour and automatically generates prioritised UX improvement recommendations from resulting data and suggests an asymmetric role of multimodality: visual access during simulation improves recommendations through richer traces, while providing visual i...

Anurupa Chowdhury, Bin Wu, Hossein A. Rahmani et al. · 0 citations
Sep 2026

Context-dependent effects of multimodal explanatory interfaces on passenger workload and situation awareness in fully autonomous driving.

This study investigated how multimodal explanatory interfaces affect passenger workload, situation awareness, trust, and XAI evaluation in fully autonomous driving. A 3 × 2 × 2 within-subject simulator experiment manipulated NDRT modality (radio, reading, video), auditory prompt type (spoken explanation vs. alert tone)...

Qing-Xing Qu, Zhong-Yang Xiao, Yi-Ru Zhao et al. · 0 citations
Book Open access Sep 2026

The Personality Behind the Wheel: Conversational Agent Design as a Strategic Asset for Automotive Brands

Results suggest that functional reliability and voice interaction quality form the primary acceptance baseline, while visual representation serves as a secondary and strategically differentiating layer for in-car conversational agents.

Esther Carolina Kaehne, Ignacio J. Alvarez · 0 citations
Book Open access Sep 2026

Dialog with trucks - A driving simulator study exploring situationally-aware conversational agents

Current AI-based conversational agents (CAs) largely lack situational awareness, limiting their ability to understand driving context, driver state, and immediate driver needs. Consequently, the use of CA may increase drivers’ cognitive load and impair safety and user experience. This study investigates the role of sit...

Johanna Vännström, Andreas Absér, Jacob Weilandt et al. · 0 citations
#explainable ai Book Open access Sep 2026

From Assistance to Affective Co-Driving: Novice Drivers’ Emotional Needs and Design Expectations for In-Vehicle AI Agents

HACoD is proposed, a Human–Agent Emotional Co-Driving Framework that links driver emotional needs, human–agent collaboration patterns, and affective agent design elements and highlights the need for transparent data boundaries and real-world validation.

Wei Gong, Ming-Yuan Zhang, Mei-Cheng Liu et al. · 0 citations

Related blog posts

GPT-Lab Nov 13, 2026

TenderIntell at ICSOB 2026: Three Days in Gothenburg

At ICSOB 2026 in Gothenburg, we presented TenderIntell, our work on multi-agent support for tendering. This post looks back at the conference, the idea behind a panel of three AI reviewers who score each proposal independently, and why evaluating agents reliably is the question we keep coming back to. The post TenderIntell at ICSOB 2026: Three Days in Gothenburg appeared first on GPT-Lab.

MIT News · Artificial Intelligence Nov 13, 2026

Helping robots fit into the human workplace

Founded by Anthony Jules ’92, SM ’92 and Professor Emeritus Rodney Brooks, Robust AI’s robots are designed to collaborate with humans in factories and warehouses.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.