Patient Education in Hand Surgery: A Blinded Comparison of GPT-5.5 and Claude Opus 4.8

Abstract

Patients increasingly consult conversational artificial intelligence about hand and wrist conditions, but the performance of proprietary large language models remains unclear. We compared the reliability, overall quality, and readability of patient-facing texts generated by GPT-5.5 and Claude Opus 4.8. Thirty hand surgery topics were selected from orthopedic education resources and the authors' outpatient practice. Each model received the same prompt and generated one response per topic (60 texts). Two fellowship-trained, certified hand surgeons independently evaluated source-masked outputs. DISCERN assessed reliability and treatment information, the Global Quality Score (GQS) overall quality, and the Flesch-Kincaid Grade Level (FKGL) and Flesch Reading Ease (FRE) readability. Platforms were compared with the Mann-Whitney U test and interrater agreement with the intraclass correlation coefficient (ICC). Median DISCERN scores were 66.5 for GPT-5.5 and 63.5 for Claude Opus 4.8 (p = 0.143); median GQS was 4.0 for both (p = 0.389). Claude Opus 4.8 had a lower median FKGL (8.75 vs 10.4; p < 0.001) and higher median FRE (50.35 vs 43.6; p = 0.009). ICCs (95% CIs) were 0.755 (0.589-0.853) for DISCERN and 0.676 (0.458-0.807) for GQS. Expert-rated reliability and overall quality did not differ significantly, whereas Claude Opus 4.8 produced more favorable formula-based readability scores; median FKGL for both models nevertheless remained above the recommended sixth-grade level. Because FKGL and FRE quantify surface linguistic complexity rather than understanding, these findings describe written outputs only; they do not establish better patient comprehension, clinical effectiveness, factual equivalence, or safety.