From Explanations to Reflective Questions in Human-AI Decision-Making

Abstract

Many decision support systems (DSS) provide predictions, recommendations, and, increasingly, explanations. Supporting human-AI decision-making with context-specific questions, however, remains largely unexplored. Questions can stimulate reflection and critical thinking, thereby introducing productive friction in the decision-making process and potentially reducing overreliance on DSS. This paper presents a proof-of-concept for generating data-driven questions based on a DSS prediction and its corresponding explanation, i.e., feature contribution, using a local language model. We illustrate our method using a realistic example from the medical field. In informal discussions (n = 2), gathering views on the possible usefulness of questions in decision-making, the clinicians mentioned that the generated questions have potential to help them reconsider the prediction and consider alternative options. Our proof-of-concept informs the design of human-AI interactions aimed at promoting the cognitive engagement of decision-makers and mitigating overreliance on DSS by shifting the focus from explanations to questions.