Comparative evaluation of large language models for patient-facing information in clear aligner therapy

Abstract

Patients considering clear aligner therapy should understand treatment requirements, limitations, and the importance of professional supervision. As many patients now seek orthodontic information from artificial intelligence (AI) chatbots, this study evaluated how effectively these systems communicate patient-facing information related to clear aligner therapy. Twenty common patient-facing questions about clear aligner therapy were developed using conversational language. Three large language models (LLM), ChatGPT 5.2, Claude Sonnet 4.5, and Gemini Flash 3, generated responses using default settings. Three orthodontic specialists, blinded to model identity, rated responses for quality and empathy using 5-point Likert scales. Readability was assessed using the Flesch–Kincaid Grade Level. Inter-rater reliability was evaluated using intraclass correlation coefficients (2,1). Model performance was compared using repeated-measures analysis of variance with Bonferroni-adjusted post hoc testing. Inter-rater reliability ranged from fair to good. Significant differences were observed among models for quality (F[2,38] = 11.43; p < 0.001) and empathy (F[2,38] = 3.92; p = 0.028), while readability did not differ significantly ( p = 0.321). Gemini Flash 3 achieved the highest mean scores for both quality (4.57 ± 0.42) and empathy (4.28 ± 0.51). All models consistently encouraged professional orthodontic consultation and avoided recommending unsupervised treatment. Large language models can provide appropriate patient-facing information about clear aligner therapy, though meaningful differences in communication quality and empathy exist. Clinical oversight remains essential when integrating AI tools into orthodontic patient education.