Open access
Preparing AI chatbots to respond to patient distress and suicidality in high-risk healthcare settings
Joanna M. Streck
Dallas T. Swendeman
W. Comulada
Xian Y. Ho
Julia Cannistraro
G. Pachas
D. Alves-Back
W. Li
Jonathan L. Jackson
Delta-Marie Lewis
Namrata Tomar
David Warren
D. Levy
Lillian Gelberg
Medicine
Abstract
Generative artificial intelligence (AI) chatbots are conversational agents that simulate human-like conversations in real-time, including behavioral health support, thus presenting challenges for monitoring and intervening on suicide risk. There is little information on how to address suicide risk in chatbot deployments, despite calls and concerns. We describe suicide safety monitoring features for a chatbot implemented in a high-risk clinical setting and make recommendations for chatbot design alongside human-monitoring and triage.