AI Aversion: Do People Trust and Accept Artificial Intelligence Risk Calculator Recommendations?

INTRODUCTION Although medical risk calculators are increasingly being used for risk prediction in various contexts, prior research in this area suggests that people may be averse to recommendations from these tools. Moreover, given the rise of artificial intelligence in health care, aversion toward AI may manifest toward risk calculators that use AI/machine learning models. Recent work has suggested that one potential way to combat AI aversion is through explainable AI (XAI), which can make underlying models more transparent.

Methods

The current study investigated whether these factors would affect public trust, acceptance, and comfort related to recommendations from a risk calculator. Participants were randomized into a 2 (calculator type: statistical vs AI) × 2(model explanation: explained vs not explained) × 2(evaluability: risk reference table vs no table) between-subjects experimental design. They read a hypothetical scenario and received a calculator output with a risk estimate and recommendation before completing measures of trust, comfort, and acceptance.

Results

Analyses revealed main effects of calculator type and explanation: participants who received the AI-based calculator outputs were less likely to trust and be comfortable with the calculator recommendation. Participants who received an explanation of the underlying model were more likely to trust and accept the calculator recommendation. However, there was no effect of XAI on trust, acceptance, and comfort or any other significant interactions.

Conclusion

The current study serves as a starting point for research on trust and acceptance of AI-based risk calculators. Our findings support the AI aversion hypothesis and suggest that there needs to be more work to identify how best to explain AI-based calculators to foster trust and comfort more specifically.