How Contextual Information in Physical Environments Shapes Decisions with AI

Abstract

AI assistants in augmented reality may advise users about parts of the physical world they cannot directly observe. We ask whether contextual information from the situation helps users make better decisions with fallible AI. In an AR visual- search study (N = 32), we varied target prevalence and scene visibility while participants judged whether a key was present, with or without AI assistance. Lowering target prevalence and revealing part of the desk together improved observed AI-assisted accuracy, but our analysis did not demonstrate that this gain came from participants making better use of context. Context also changed which answers seemed plausible: when keys were rare, participants became more skeptical of Present suggestions, but this shift applied to both correct and incorrect suggestions rather than improving selective acceptance of correct advice. Revealing part of the desk also increased decision time, with little evidence of better AI use. Overall, contextual information can shape AI- assisted decisions and increase decision effort without necessarily helping users judge when the AI is correct.