Comparative performance of Gemini and DeepSeek large language models in surgical decision support for mastoidectomy in chronic otitis media with cholesteatoma

Abstract

To compare the clinical accuracy of two state-of-the-art Large Language Models (LLMs), Gemini Pro v3.1 and DeepSeek v3.2, in providing surgical decision support and postoperative management for mastoidectomy using expert-validated real patient data. A retrospective analysis included 121 mastoidectomy cases performed for chronic otitis media with cholesteatoma. Clinical case scenarios, including otoscopy, pure-tone audiometry, and radiological findings, were presented to both models. AI-generated responses were evaluated using a 100-point scoring system developed by four otolaryngologists in two domains: surgical planning and postoperative management. Paired-sample t-tests compared model performance, while Mann–Whitney U and Kruskal-Wallis tests assessed the effects of clinical variables. No significant difference was found between Gemini and DeepSeek in surgical planning (76.07% vs. 73.97%, p  = 0.258). However, Gemini significantly outperformed DeepSeek in postoperative management (67.85% vs. 62.11%, p  = 0.001). Both models achieved significantly higher scores in surgical planning than postoperative care. DeepSeek demonstrated significantly higher surgical accuracy in patients with comorbidities ( p  = 0.004) and postoperative complications ( p  < 0.001). Surgical approach significantly influenced DeepSeek’s performance ( p  = 0.001), with better results in microscopic than endoscopic cases. Both LLMs performed well in surgical planning but were less effective in postoperative management. Gemini demonstrated superior performance in postoperative recommendations, whereas DeepSeek benefited from clearer indicators of disease severity. These findings support the potential of LLMs as adjunctive tools in otologic surgical decision-making while emphasizing the need for expert clinical oversight.