Patient Refusal of LLM-Assisted Care: Autonomy in an AI-Enabled Health System

Abstract

As large language models (LLMs) become embedded in clinical decision support, clinicians and institutions will increasingly encounter patients who object to their use. At present, LLMs should be regarded as adjunctive, clinician-supervised tools rather than substitutes for professional clinical judgment or autonomous diagnostic and treatment decision-makers. This paper reasons that refusal of LLM-assisted care should be considered meaningful only when patients receive transparent disclosure, understand how the technology contributes to their care, and can identify which forms of LLM involvement may reasonably be avoided. The ethical basis for such refusal extends beyond autonomy to include data protection, human oversight, accountability, and justified concerns about model performance in underrepresented populations. However, a patient’s interest in avoiding LLM involvement does not necessarily create an unlimited institutional obligation to maintain a fully parallel non-AI pathway. Clinicians and healthcare organizations must therefore distinguish between protecting patients from unwanted technological involvement and requiring institutions to provide additional resources or alternative workflows. Clear governance is needed to define when LLM use is optional, avoidable, or integral to care, and to ensure that patient preferences are addressed consistently without compromising safety, fairness, privacy, or professional accountability.