Expert-Rated Documentary Quality of AI-Assisted Hospital Discharge Reports: A Retrospective Paired Comparison with Physician-Written Reports

Abstract

Background/Objectives: The hospital discharge report is a critical document for care continuity that generates a substantial administrative burden for clinicians. Generative artificial intelligence (AI) offers the potential to reduce this burden while improving documentary quality. This study aims to compare, under real-world conditions with a GDPR-oriented architecture based on prior local anonymisation, the quality of AI-assisted discharge reports (IAIA) against those drafted by the responsible physician (INF). Methods: A retrospective, paired, expert-evaluation study was conducted at a Spanish university hospital. One hundred and twenty consecutive clinical cases from nine departments were included (240 reports total). Each case was independently evaluated by one of ten primary care physicians using a structured rubric covering 13 clinical dimensions (ordinal scale 1–3) and a global rating scale (1–10). The Wilcoxon signed-rank test was applied to all paired comparisons; effect size was estimated using the paired rank-biserial correlation (r). Results: IAIA achieved a significantly higher overall mean rating than INF (8.14 vs. 7.30 out of 10; p < 0.0001; r ≈ 0.76, large effect). IAIA was nominally superior in 9 of 13 clinical dimensions; after Bonferroni correction for the 13 per-dimension comparisons, six of these differences remained statistically significant, with the largest gains in family history, principal diagnosis hierarchy, and structured listing of secondary diagnoses. INF retained an advantage only in allergies and intolerances (2.69 vs. 2.46; p = 0.002), where IAIA tended to use generic formulas. Three dimensions showed no significant difference (prior treatment, physical examination, procedures). Conclusions: AI-assisted discharge reports received higher expert-rated documentary quality scores in a non-blinded paired evaluation across most evaluated dimensions. The physician-written report retained an advantage only in the safety-critical allergy domain, where allergy information must not be inferred by the model but sourced from verified structured fields or explicitly flagged as pending physician validation, supporting the need for a supervised hybrid model in which AI generates the initial draft while the clinician mandatorily validates sensitive content. Prior local anonymisation constitutes a GDPR-oriented approach to generative AI deployment in European hospital settings, substantially reducing the risk of disclosure of identifiable clinical information.